ETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Examiner’s Note
Examiner has cited particular paragraphs/columns and line numbers or figures in the references as applied to the claims below for convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations with the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, in preparing the responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. Applicant is reminded that the Examiner is entitled to give the broadest reasonable interpretation to the language of the claims. Furthermore, the Examiner is not limited to the Applicant’s definition which is not specifically set forth in the claims.
Status of Application
The list of claims 1-6 is pending in this application. In the claim set filed 12/23/2025:
Claim(s) 1, 5 and 6 is/are the independent claim(s) observed in the application.
Claim(s) 1, 5 and 6 has/have been indicated as amended.
Claim(s) 2-4 has/have been indicated as originally presented.
Response to Arguments
With respect to Applicant’s remarks filed on 12/23/2025; Applicant's “Amendments and Remarks” have been fully considered. Applicant’s remarks will be addressed in sequential order as they were presented.
With respect to the Title Objection, Applicant’s “Amendments and Remarks” have been fully considered and are persuasive. Therefore, the Title Objection has/have been withdrawn.
With respect to the rejection(s) of claim(s) 1-6 under 35 U.S.C. § 102(a)(1) and 35 U.S.C. § 103, Applicant’s “Amendments and Remarks” have been fully considered and are persuasive. Therefore, the rejection(s) of claim(s) 1-6 under 35 U.S.C. § 102(a)(1) and 35 U.S.C. § 103 has/have been withdrawn.
Office Note: Due to applicant’s amendments, further claim rejections appear on the record as stated in the Final Office Action below.
Final Office Action
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102(e), (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claim(s) 1 and 4-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (United States Patent Publication 2023/0033243 A1) in view of Musk et al. (United States Patent 2020/0265247 A1), referenced as Yang and Musk, respectively, moving forward.
With respect to claim 1, Yang discloses:
“A vehicle control device comprising a processor configured to: perform autonomous driving control of a host vehicle” [Yang; "In some embodiments, a non-transitory computer-readable medium may be provided, having non-transitory computer-readable instructions encoded thereon that, when executed by a processor, causes the processor to: generate an object detection representation of a candidate object based on sensor data representing a captured image of an environment surrounding the vehicle; determine, based on the object detection representation, whether the candidate object is an outlier; and in response to determining the candidate object is not an outlier: validate the candidate object as an object; determine a proximity distance of the object to the vehicle; and output the proximity distance of the object;" ¶: 0008;
"In some embodiments, the output of outlier rejection model 322 may be utilized in performing autonomous or semi-autonomous parking of vehicle 101 and/or in performing autonomous or semi-autonomous navigation of vehicle 101, e.g., determining an object is present in a vicinity of vehicle 101 (or that the object is associated with a false detection) and performing or recommending suitable action based on the determination. In some embodiments, separate training images may be used for training the model with respect to a parking scenario as compared to a scenario in which vehicle 101 is navigating a road or performing off-road navigation;" Fig. 3; ¶: 0036; See also: ¶: 0024, 0039];
“and estimate whether a vehicle-to-vehicle distance of the host vehicle and a preceding vehicle is suitable, shorter than the suitable vehicle-to-vehicle distance, or longer than the suitable vehicle-to-vehicle distance by using trained machine learning model based on an image including the preceding vehicle captured by a monocular camera mounted on the host vehicle” [Yang; "In some embodiments, a plurality of image sensors configured to generate the sensor data based on captured images of the environment surrounding the vehicle. The plurality of image sensors may be respectively included in a plurality of monocular fisheye cameras;" ¶: 0004;
"In response to determining the candidate object is (i) within the range of interest and (ii) not a false detection of an object, the candidate object may be validated as a detected object, and processing may continue to object-to-vehicle distance estimation model 324. Model 324 may be employed to monitor the environment surrounding vehicle 101 and estimate or predict a distance between the validated detected object and vehicle 101. Model 324 may be trained using training images of surrounding images of vehicle 101 annotated with an indication of a distance between vehicle 101 and the object for the particular training image, and may output a prediction as to whether the validated detected object is within a predefined range of vehicle 101 (e.g., 8 feet or 10 feet, or any other suitable distance). In some embodiments, at 326, an upper bound and a lower bound of any suitable distance values may be compared to the output of model 324 in determining a distance of the object from the vehicle associated with cameras having captured images of a surrounding area of the vehicle. In some embodiments, the lower bound may be, e.g., 4 feet, or any other suitable distance. In some embodiments, post-processing techniques may be employed (e.g., Non-maximum Suppression (NMS), or any other suitable technique, or any combination thereof). The processing described in connection with FIG. 3 may be performed on each captured individually or collectively, such that one or more outputs associated with the image data from the plurality of images may be fused together to facilitate the monitoring of the object of interest around vehicle 101 within the range of interest. Accordingly, accurate object-to-vehicle distance estimation may be realized with reliable detection and little false alarm, object proximity may be monitored with good range around vehicle 101;" Fig. 3; ¶: 0039; See also: ¶: 0038 ,0040, 0042 0046];
“wherein the machine learning model is generated by performing machine learning using teacher data images including a preceding vehicle for learning captured from a vehicle for capturing teacher data images and annotation information added to the teacher data images, the annotation information includes information showing any of the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being a suitable vehicle-to-vehicle distance, the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being shorter than the suitable vehicle-to-vehicle distance, and the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being longer than the suitable vehicle-to-vehicle distance” [Yang; "For example, outlier rejection model 322 may be trained using training images in which an object is within 8 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle), and object-to-vehicle distance estimation model 324 may be trained on using training images in which an object is within 10 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle). In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn that a particular class of object (e.g., a vehicle or a person) being of a certain size and at a certain location corresponds to a particular distance from vehicle 101. In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn a correlation between box size height (e.g., pixelwise), distance (e.g., pixelwise) from center of box, and other parameters, with a distance between an object and a vehicle;" Fig. 3; ¶: 0038; See also: ¶: 0006, 0039, 0040, 0042 0046];
“the processor is configured to output result of estimation showing any of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being the suitable vehicle-to-vehicle distance, the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being shorter than the suitable vehicle-to-vehicle distance, and the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being longer than the suitable vehicle-to-vehicle distance” [Yang; "In response to determining the candidate object is (i) within the range of interest and (ii) not a false detection of an object, the candidate object may be validated as a detected object, and processing may continue to object-to-vehicle distance estimation model 324. Model 324 may be employed to monitor the environment surrounding vehicle 101 and estimate or predict a distance between the validated detected object and vehicle 101. Model 324 may be trained using training images of surrounding images of vehicle 101 annotated with an indication of a distance between vehicle 101 and the object for the particular training image, and may output a prediction as to whether the validated detected object is within a predefined range of vehicle 101 (e.g., 8 feet or 10 feet, or any other suitable distance). In some embodiments, at 326, an upper bound and a lower bound of any suitable distance values may be compared to the output of model 324 in determining a distance of the object from the vehicle associated with cameras having captured images of a surrounding area of the vehicle. In some embodiments, the lower bound may be, e.g., 4 feet, or any other suitable distance. In some embodiments, post-processing techniques may be employed (e.g., Non-maximum Suppression (NMS), or any other suitable technique, or any combination thereof). The processing described in connection with FIG. 3 may be performed on each captured individually or collectively, such that one or more outputs associated with the image data from the plurality of images may be fused together to facilitate the monitoring of the object of interest around vehicle 101 within the range of interest. Accordingly, accurate object-to-vehicle distance estimation may be realized with reliable detection and little false alarm, object proximity may be monitored with good range around vehicle 101;" Fig. 3; ¶: 0039; See also: ¶: 0038 ,0040, 0042 0046];
“and perform the autonomous driving control of the host vehicle based on the result of estimation” [Yang; "In some embodiments, the output of outlier rejection model 322 may be utilized in performing autonomous or semi-autonomous parking of vehicle 101 and/or in performing autonomous or semi-autonomous navigation of vehicle 101, e.g., determining an object is present in a vicinity of vehicle 101 (or that the object is associated with a false detection) and performing or recommending suitable action based on the determination. In some embodiments, separate training images may be used for training the model with respect to a parking scenario as compared to a scenario in which vehicle 101 is navigating a road or performing off-road navigation;" Fig. 3; ¶: 0036].
And while Yang discloses: “the estimation of whether the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle is suitable, shorter than the suitable vehicle-to-vehicle distance, or longer than the suitable vehicle-to-vehicle distance” [Yang; "In some embodiments, training data may be divided into multiple groups. For example, outlier rejection model 322 may be trained using training images in which an object is within 8 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle), and object-to-vehicle distance estimation model 324 may be trained on using training images in which an object is within 10 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle). In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn that a particular class of object (e.g., a vehicle or a person) being of a certain size and at a certain location corresponds to a particular distance from vehicle 101. In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn a correlation between box size height (e.g., pixelwise), distance (e.g., pixelwise) from center of box, and other parameters, with a distance between an object and a vehicle;" ¶: 0038; See also: ¶: 0039],
Yang does not specifically state: “without using a distance measurement sensor or without using a calculated numerical value of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle.”
Musk, which is in the same field of invention of vehicle control systems and methods using image recognition to identify vehicle distance, teaches: “without using a distance measurement sensor or without using a calculated numerical value of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle” [Musk; "The trained machine learning model can be deployed to vehicles for accurately predicting object properties, such as distance, direction, and velocity, using only vision data. For example, once the machine learning model has been trained to be able to determine an object distance using images of a camera without a need of a dedicated distance sensor, it may become no longer necessary to include a dedicated distance sensor in an autonomous driving vehicle;" ¶: 0011;
"At 309, the trained machine learning model is applied. For example, the machine learning model trained at 303 is applied to sensor data received at 307. In some embodiments, the application of the model is performed by an AI processor such as AI processor 109 of FIG. 1 using a deep learning network such as deep learning network 107 of FIG. 1. In various embodiments, by applying the trained machine learning model, one or more object properties such as an object distance, direction, and/or velocity are predicted from image data. For example, different objects are identified in the image data and an object distance and direction for each identified object are inferred using the trained machine learning model;" Fig. 3; ¶: 0044].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the vehicle control system and method for controlling an autonomous vehicle using a trained machine learning model that performs image recognition of proximate objects as disclosed by Yang to incorporate the teachings regarding performing object recognition using a trained machine learning model that does not require dedicated distance sensors in order to operate accurately as taught by Musk with a reasonable expectation of success. By combining these inventions, the outcome is a control system and method for controlling an autonomous vehicle that is more robust in its ability to automatically generate highly accurate machine learning results for predicting object properties without requiring human intervention [Musk; ¶: 0011].
With respect to claim 4, Yang discloses: “wherein the annotation information includes information showing a type of the preceding vehicle for learning, and the annotation information is added to a teacher data image when annotation of the teacher data image is performed” [Yang; "In some embodiments, one or more machine learning models 214 may be trained to learn patterns and features associated with certain classes of objects, e.g., a person, a car, a bus, a motorcycle, a train, a bicycle, background, etc. In some embodiments, such machine learning models may be trained to learn patterns and features associated with sub-classes (e.g., a sedan, a minivan, a truck, or any other suitable sub-class) of a class (e.g., cars, or any other suitable class). Classification may be carried out by machine learning model 214 comprising one or more machine learning models, such as, for example, a CNN trained to receive input images of objects surrounding a vehicle (e.g., where the image may be annotated with any suitable bounding shape relative to an object and/or a distance from vehicle to object annotation and/or a class of object annotation), and output likelihoods that these vehicles correspond to particular vehicle categories. Such CNNs may be trained on training data sets containing images of vehicles manually tagged with their particular vehicle types. In some embodiments, any combination of the following classes may be employed in training and/or evaluating the model (e.g., background, airplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, dining table, dog, horse, motorbike, motorcycle, person, potted plant, sheep, sofa, train, TV monitor, truck, stop sign, traffic light, traffic sign, motor, or any other suitable class, or any combination thereof). In some embodiments, a confidence score may be output along with the prediction of a class to which an identified object belongs (e.g., 86% probability that an identified object is a human being, or any other suitable probability). In some embodiments, the confidence score may be compared to a predefined threshold confidence score in determining how to classify the object;" ¶: 0032; See also: ¶: 0038, 0046, 0052, 0058].
With respect to claim 5, Yang discloses:
“A vehicle control method comprising: performing autonomous driving control of a host vehicle” [Yang; "In some embodiments, the output of outlier rejection model 322 may be utilized in performing autonomous or semi-autonomous parking of vehicle 101 and/or in performing autonomous or semi-autonomous navigation of vehicle 101, e.g., determining an object is present in a vicinity of vehicle 101 (or that the object is associated with a false detection) and performing or recommending suitable action based on the determination. In some embodiments, separate training images may be used for training the model with respect to a parking scenario as compared to a scenario in which vehicle 101 is navigating a road or performing off-road navigation;" Fig. 3; ¶: 0036; See also: ¶: 0024, 0039];
“and estimating whether a vehicle-to-vehicle distance of the host vehicle and a preceding vehicle is suitable, shorter than the suitable vehicle-to-vehicle distance, or longer than the suitable vehicle-to-vehicle distance by using trained machine learning model based on an image including the preceding vehicle captured by a monocular camera mounted on the host vehicle” [Yang; "In some embodiments, a plurality of image sensors configured to generate the sensor data based on captured images of the environment surrounding the vehicle. The plurality of image sensors may be respectively included in a plurality of monocular fisheye cameras;" ¶: 0004;
"In response to determining the candidate object is (i) within the range of interest and (ii) not a false detection of an object, the candidate object may be validated as a detected object, and processing may continue to object-to-vehicle distance estimation model 324. Model 324 may be employed to monitor the environment surrounding vehicle 101 and estimate or predict a distance between the validated detected object and vehicle 101. Model 324 may be trained using training images of surrounding images of vehicle 101 annotated with an indication of a distance between vehicle 101 and the object for the particular training image, and may output a prediction as to whether the validated detected object is within a predefined range of vehicle 101 (e.g., 8 feet or 10 feet, or any other suitable distance). In some embodiments, at 326, an upper bound and a lower bound of any suitable distance values may be compared to the output of model 324 in determining a distance of the object from the vehicle associated with cameras having captured images of a surrounding area of the vehicle. In some embodiments, the lower bound may be, e.g., 4 feet, or any other suitable distance. In some embodiments, post-processing techniques may be employed (e.g., Non-maximum Suppression (NMS), or any other suitable technique, or any combination thereof). The processing described in connection with FIG. 3 may be performed on each captured individually or collectively, such that one or more outputs associated with the image data from the plurality of images may be fused together to facilitate the monitoring of the object of interest around vehicle 101 within the range of interest. Accordingly, accurate object-to-vehicle distance estimation may be realized with reliable detection and little false alarm, object proximity may be monitored with good range around vehicle 101;" Fig. 3; ¶: 0039; See also: ¶: 0038 ,0040, 0042 0046];
“wherein the machine learning model is generated by performing machine learning using teacher data images including a preceding vehicle for learning captured from a vehicle for capturing teacher data images and annotation information added to the teacher data images the annotation information includes information showing any of the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being a suitable vehicle-to-vehicle distance, the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being shorter than the suitable vehicle-to-vehicle distance, and the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being longer than the suitable vehicle-to-vehicle distance” [Yang; "For example, outlier rejection model 322 may be trained using training images in which an object is within 8 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle), and object-to-vehicle distance estimation model 324 may be trained on using training images in which an object is within 10 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle). In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn that a particular class of object (e.g., a vehicle or a person) being of a certain size and at a certain location corresponds to a particular distance from vehicle 101. In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn a correlation between box size height (e.g., pixelwise), distance (e.g., pixelwise) from center of box, and other parameters, with a distance between an object and a vehicle;" Fig. 3; ¶: 0038; See also: ¶: 0006, 0039, 0040, 0042 0046];
“in estimation of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle, result of estimation showing any of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being the suitable vehicle-to-vehicle distance, the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being shorter than the suitable vehicle-to-vehicle distance, and the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being longer than the suitable vehicle-to-vehicle distance is output” [Yang; "In response to determining the candidate object is (i) within the range of interest and (ii) not a false detection of an object, the candidate object may be validated as a detected object, and processing may continue to object-to-vehicle distance estimation model 324. Model 324 may be employed to monitor the environment surrounding vehicle 101 and estimate or predict a distance between the validated detected object and vehicle 101. Model 324 may be trained using training images of surrounding images of vehicle 101 annotated with an indication of a distance between vehicle 101 and the object for the particular training image, and may output a prediction as to whether the validated detected object is within a predefined range of vehicle 101 (e.g., 8 feet or 10 feet, or any other suitable distance). In some embodiments, at 326, an upper bound and a lower bound of any suitable distance values may be compared to the output of model 324 in determining a distance of the object from the vehicle associated with cameras having captured images of a surrounding area of the vehicle. In some embodiments, the lower bound may be, e.g., 4 feet, or any other suitable distance. In some embodiments, post-processing techniques may be employed (e.g., Non-maximum Suppression (NMS), or any other suitable technique, or any combination thereof). The processing described in connection with FIG. 3 may be performed on each captured individually or collectively, such that one or more outputs associated with the image data from the plurality of images may be fused together to facilitate the monitoring of the object of interest around vehicle 101 within the range of interest. Accordingly, accurate object-to-vehicle distance estimation may be realized with reliable detection and little false alarm, object proximity may be monitored with good range around vehicle 101;" Fig. 3; ¶: 0039; See also: ¶: 0038 ,0040, 0042 0046];
“and the autonomous driving control of the host vehicle is performed based on the result of estimation output in the estimation of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle” [Yang; "In some embodiments, the output of outlier rejection model 322 may be utilized in performing autonomous or semi-autonomous parking of vehicle 101 and/or in performing autonomous or semi-autonomous navigation of vehicle 101, e.g., determining an object is present in a vicinity of vehicle 101 (or that the object is associated with a false detection) and performing or recommending suitable action based on the determination. In some embodiments, separate training images may be used for training the model with respect to a parking scenario as compared to a scenario in which vehicle 101 is navigating a road or performing off-road navigation;" Fig. 3; ¶: 0036].
And while Yang discloses: “the estimation of whether the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle is suitable, shorter than the suitable vehicle-to-vehicle distance, or longer than the suitable vehicle-to-vehicle distance” [Yang; "In some embodiments, training data may be divided into multiple groups. For example, outlier rejection model 322 may be trained using training images in which an object is within 8 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle), and object-to-vehicle distance estimation model 324 may be trained on using training images in which an object is within 10 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle). In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn that a particular class of object (e.g., a vehicle or a person) being of a certain size and at a certain location corresponds to a particular distance from vehicle 101. In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn a correlation between box size height (e.g., pixelwise), distance (e.g., pixelwise) from center of box, and other parameters, with a distance between an object and a vehicle;" ¶: 0038; See also: ¶: 0039],
Yang does not specifically state: “without using a distance measurement sensor or without using a calculated numerical value of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle.”
Musk teaches: “without using a distance measurement sensor or without using a calculated numerical value of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle” [Musk; "The trained machine learning model can be deployed to vehicles for accurately predicting object properties, such as distance, direction, and velocity, using only vision data. For example, once the machine learning model has been trained to be able to determine an object distance using images of a camera without a need of a dedicated distance sensor, it may become no longer necessary to include a dedicated distance sensor in an autonomous driving vehicle;" ¶: 0011;
"At 309, the trained machine learning model is applied. For example, the machine learning model trained at 303 is applied to sensor data received at 307. In some embodiments, the application of the model is performed by an AI processor such as AI processor 109 of FIG. 1 using a deep learning network such as deep learning network 107 of FIG. 1. In various embodiments, by applying the trained machine learning model, one or more object properties such as an object distance, direction, and/or velocity are predicted from image data. For example, different objects are identified in the image data and an object distance and direction for each identified object are inferred using the trained machine learning model;" Fig. 3; ¶: 0044].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the vehicle control system and method for controlling an autonomous vehicle using a trained machine learning model that performs image recognition of proximate objects as disclosed by Yang to incorporate the teachings regarding performing object recognition using a trained machine learning model that does not require dedicated distance sensors in order to operate accurately as taught by Musk with a reasonable expectation of success. By combining these inventions, the outcome is a control system and method for controlling an autonomous vehicle that is more robust in its ability to automatically generate highly accurate machine learning results for predicting object properties without requiring human intervention [Musk; ¶: 0011].
With respect to claim 6, Yang discloses:
“A non-transitory recording medium having recorded thereon a computer program for causing a processor to execute a process comprising: performing autonomous driving control of a host vehicle” [Yang; "In some embodiments, a non-transitory computer-readable medium may be provided, having non-transitory computer-readable instructions encoded thereon that, when executed by a processor, causes the processor to: generate an object detection representation of a candidate object based on sensor data representing a captured image of an environment surrounding the vehicle; determine, based on the object detection representation, whether the candidate object is an outlier; and in response to determining the candidate object is not an outlier: validate the candidate object as an object; determine a proximity distance of the object to the vehicle; and output the proximity distance of the object;" ¶: 0008;
"In some embodiments, the output of outlier rejection model 322 may be utilized in performing autonomous or semi-autonomous parking of vehicle 101 and/or in performing autonomous or semi-autonomous navigation of vehicle 101, e.g., determining an object is present in a vicinity of vehicle 101 (or that the object is associated with a false detection) and performing or recommending suitable action based on the determination. In some embodiments, separate training images may be used for training the model with respect to a parking scenario as compared to a scenario in which vehicle 101 is navigating a road or performing off-road navigation;" Fig. 3; ¶: 0036; See also: ¶: 0024, 0039];
“and estimating whether a vehicle-to-vehicle distance of the host vehicle and a preceding vehicle is suitable, shorter than the suitable vehicle-to-vehicle distance, or longer than the suitable vehicle-to-vehicle distance by using trained machine learning model based on an image including the preceding vehicle captured by a monocular camera mounted on the host vehicle” [Yang; "In some embodiments, a plurality of image sensors configured to generate the sensor data based on captured images of the environment surrounding the vehicle. The plurality of image sensors may be respectively included in a plurality of monocular fisheye cameras;" ¶: 0004;
"In response to determining the candidate object is (i) within the range of interest and (ii) not a false detection of an object, the candidate object may be validated as a detected object, and processing may continue to object-to-vehicle distance estimation model 324. Model 324 may be employed to monitor the environment surrounding vehicle 101 and estimate or predict a distance between the validated detected object and vehicle 101. Model 324 may be trained using training images of surrounding images of vehicle 101 annotated with an indication of a distance between vehicle 101 and the object for the particular training image, and may output a prediction as to whether the validated detected object is within a predefined range of vehicle 101 (e.g., 8 feet or 10 feet, or any other suitable distance). In some embodiments, at 326, an upper bound and a lower bound of any suitable distance values may be compared to the output of model 324 in determining a distance of the object from the vehicle associated with cameras having captured images of a surrounding area of the vehicle. In some embodiments, the lower bound may be, e.g., 4 feet, or any other suitable distance. In some embodiments, post-processing techniques may be employed (e.g., Non-maximum Suppression (NMS), or any other suitable technique, or any combination thereof). The processing described in connection with FIG. 3 may be performed on each captured individually or collectively, such that one or more outputs associated with the image data from the plurality of images may be fused together to facilitate the monitoring of the object of interest around vehicle 101 within the range of interest. Accordingly, accurate object-to-vehicle distance estimation may be realized with reliable detection and little false alarm, object proximity may be monitored with good range around vehicle 101;" Fig. 3; ¶: 0039; See also: ¶: 0038 ,0040, 0042 0046];
“wherein the machine learning model is generated by performing machine learning using teacher data images including a preceding vehicle for learning captured from a vehicle for capturing teacher data images and annotation information added to the teacher data images, the annotation information includes information showing any of the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being a suitable vehicle-to-vehicle distance, the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being shorter than the suitable vehicle-to-vehicle distance, and the vehicle-to-vehicle distance between the vehicle for capturing the teacher data images and the preceding vehicle for learning being longer than the suitable vehicle-to-vehicle distance” [Yang; "For example, outlier rejection model 322 may be trained using training images in which an object is within 8 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle), and object-to-vehicle distance estimation model 324 may be trained on using training images in which an object is within 10 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle). In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn that a particular class of object (e.g., a vehicle or a person) being of a certain size and at a certain location corresponds to a particular distance from vehicle 101. In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn a correlation between box size height (e.g., pixelwise), distance (e.g., pixelwise) from center of box, and other parameters, with a distance between an object and a vehicle;" Fig. 3; ¶: 0038; See also: ¶: 0006, 0039, 0040, 0042 0046];
“in estimation of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle, result of estimation showing any of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being the suitable vehicle-to-vehicle distance, the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being shorter than the suitable vehicle-to-vehicle distance, and the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle being longer than the suitable vehicle-to-vehicle distance is output” [Yang; "In response to determining the candidate object is (i) within the range of interest and (ii) not a false detection of an object, the candidate object may be validated as a detected object, and processing may continue to object-to-vehicle distance estimation model 324. Model 324 may be employed to monitor the environment surrounding vehicle 101 and estimate or predict a distance between the validated detected object and vehicle 101. Model 324 may be trained using training images of surrounding images of vehicle 101 annotated with an indication of a distance between vehicle 101 and the object for the particular training image, and may output a prediction as to whether the validated detected object is within a predefined range of vehicle 101 (e.g., 8 feet or 10 feet, or any other suitable distance). In some embodiments, at 326, an upper bound and a lower bound of any suitable distance values may be compared to the output of model 324 in determining a distance of the object from the vehicle associated with cameras having captured images of a surrounding area of the vehicle. In some embodiments, the lower bound may be, e.g., 4 feet, or any other suitable distance. In some embodiments, post-processing techniques may be employed (e.g., Non-maximum Suppression (NMS), or any other suitable technique, or any combination thereof). The processing described in connection with FIG. 3 may be performed on each captured individually or collectively, such that one or more outputs associated with the image data from the plurality of images may be fused together to facilitate the monitoring of the object of interest around vehicle 101 within the range of interest. Accordingly, accurate object-to-vehicle distance estimation may be realized with reliable detection and little false alarm, object proximity may be monitored with good range around vehicle 101;" Fig. 3; ¶: 0039; See also: ¶: 0038 ,0040, 0042 0046];
“and the autonomous driving control of the host vehicle is performed based on the result of estimation output in the estimation of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle” [Yang; "In some embodiments, the output of outlier rejection model 322 may be utilized in performing autonomous or semi-autonomous parking of vehicle 101 and/or in performing autonomous or semi-autonomous navigation of vehicle 101, e.g., determining an object is present in a vicinity of vehicle 101 (or that the object is associated with a false detection) and performing or recommending suitable action based on the determination. In some embodiments, separate training images may be used for training the model with respect to a parking scenario as compared to a scenario in which vehicle 101 is navigating a road or performing off-road navigation;" Fig. 3; ¶: 0036].
And while Yang discloses: “the estimation of whether the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle is suitable, shorter than the suitable vehicle-to-vehicle distance, or longer than the suitable vehicle-to-vehicle distance” [Yang; "In some embodiments, training data may be divided into multiple groups. For example, outlier rejection model 322 may be trained using training images in which an object is within 8 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle), and object-to-vehicle distance estimation model 324 may be trained on using training images in which an object is within 10 feet from a vehicle (e.g., manually tagged to indicate a distance from the object to the vehicle). In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn that a particular class of object (e.g., a vehicle or a person) being of a certain size and at a certain location corresponds to a particular distance from vehicle 101. In some embodiments, outlier rejection model 322 and/or object-to-vehicle distance estimation model 324 may be trained to learn a correlation between box size height (e.g., pixelwise), distance (e.g., pixelwise) from center of box, and other parameters, with a distance between an object and a vehicle;" ¶: 0038; See also: ¶: 0039],
Yang does not specifically state: “without using a distance measurement sensor or without using a calculated numerical value of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle.”
Musk teaches: “without using a distance measurement sensor or without using a calculated numerical value of the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle” [Musk; "The trained machine learning model can be deployed to vehicles for accurately predicting object properties, such as distance, direction, and velocity, using only vision data. For example, once the machine learning model has been trained to be able to determine an object distance using images of a camera without a need of a dedicated distance sensor, it may become no longer necessary to include a dedicated distance sensor in an autonomous driving vehicle;" ¶: 0011;
"At 309, the trained machine learning model is applied. For example, the machine learning model trained at 303 is applied to sensor data received at 307. In some embodiments, the application of the model is performed by an AI processor such as AI processor 109 of FIG. 1 using a deep learning network such as deep learning network 107 of FIG. 1. In various embodiments, by applying the trained machine learning model, one or more object properties such as an object distance, direction, and/or velocity are predicted from image data. For example, different objects are identified in the image data and an object distance and direction for each identified object are inferred using the trained machine learning model;" Fig. 3; ¶: 0044].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the vehicle control system and method for controlling an autonomous vehicle using a trained machine learning model that performs image recognition of proximate objects as disclosed by Yang to incorporate the teachings regarding performing object recognition using a trained machine learning model that does not require dedicated distance sensors in order to operate accurately as taught by Musk with a reasonable expectation of success. By combining these inventions, the outcome is a control system and method for controlling an autonomous vehicle that is more robust in its ability to automatically generate highly accurate machine learning results for predicting object properties without requiring human intervention [Musk; ¶: 0011].
Claim(s) 2 and 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Musk and Zhao et al. (United States Patent 2020/0324766 A1), referenced as Zhao moving forward.
With respect to claim 2, Yang does not specifically state: “wherein the processor is configured to perform the autonomous driving control of the host vehicle so that the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle becomes longer up to the suitable vehicle-to-vehicle distance when the result of estimation showing the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle is shorter than the suitable vehicle-to-vehicle distance is output and the processor is configured to perform the autonomous driving control of the host vehicle so that the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle becomes shorter down to the suitable vehicle-to-vehicle distance when the result of estimation showing the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle is longer than the suitable vehicle-to-vehicle distance is output.”
Zhao, which is in the same field of invention of control systems/methods for controlling autonomous vehicles based on the proximity of surrounding objects and/or vehicles, teaches: “wherein the processor is configured to perform the autonomous driving control of the host vehicle so that the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle becomes longer up to the suitable vehicle-to-vehicle distance when the result of estimation showing the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle is shorter than the suitable vehicle-to-vehicle distance is output and the processor is configured to perform the autonomous driving control of the host vehicle so that the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle becomes shorter down to the suitable vehicle-to-vehicle distance when the result of estimation showing the vehicle-to-vehicle distance of the host vehicle and the preceding vehicle is longer than the suitable vehicle-to-vehicle distance is output” [Zhao; "The following gap is defined as a desired following gap range in one embodiment, wherein the desired following gap range has hysteresis in the form of a desired minimum following gap and a desired maximum following gap, both of which are distance values. The desired minimum following gap may be selected based upon vehicle stopping distance and related factors in order to maintain a minimum safe following distance. The desired maximum following gap may be selected based upon traffic density and related factors to maintain vehicle progress in traffic flow. Values for the desired minimum following gap and the desired maximum following gap may be vehicle-specific and determined in relation to vehicle speed, vehicle load, road surface conditions, weather conditions, etc. Values for the desired minimum following gap and the desired maximum following gap may be predetermined and stored as a calibrated array in a memory device of the controller 15 for access during vehicle operation. Values for the desired minimum following gap and the desired maximum following gap may be dynamically adjusted based upon changes in the vehicle speed, vehicle load, road surface conditions, weather conditions, etc." ¶: 0049; ¶: 0040, 0041, 0047, 0051, 0061].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the vehicle control system and method for controlling an autonomous vehicle using a trained machine learning model that performs image recognition of proximate objects as disclosed by Yang to incorporate the teachings regarding controlling a vehicle to maintain an inter-vehicle distance comprising of a minimum following gap and maximum following gap that may be dynamically adjusted based on a plurality of conditions as taught by Zhao with a reasonable expectation of success. By combining these inventions, the outcome is a control system and method for controlling an autonomous vehicle that is more robust in its ability to operate an adaptive cruise control for maintaining longitudinal distances to hand changes in road gradients to prevent unnecessary accelerations and decelerations and gain energy efficiency [Zhao; ¶: 0005, 0023, 0048].
With respect to claim 3, while Yang discloses: “the preceding vehicle for learning includes a vehicle traveling in a different lane from a lane which the vehicle for capturing the teacher data images is traveling in” [Yang; In at least the paragraphs and figures cited, Yang discloses an example training image in which a vehicle, denoted 401 in Fig. 4, travelling in an opposite travel direction lane to the host vehicle is considered during the training of the disclosed machine learning model; Fig. 4; ¶: 0040],
Yang does not specifically state: “wherein the preceding vehicle is traveling in the same lane as a lane which the host vehicle is traveling in.”
Zhao teaches: “wherein the preceding vehicle is traveling in the same lane as a lane which the host vehicle is traveling in” [Zhao; "Cruise control is an on-vehicle system that controls operation of a prime mover in response to an operator command to operate a vehicle at a set vehicle speed. Adaptive cruise control is a refinement of a cruise control system that incorporates extra-vehicle monitoring systems to monitor sensed objects that are in a trajectory of the vehicle, e.g., monitoring speeds of proximal vehicles that are in a lane of travel of the vehicle. An adaptive cruise control system is a form of a longitudinal motion control system;" ¶: 0003;
"One form of an adaptive cruise control system operates by controlling a vehicle at a set vehicle speed while taking into account speed of one or more lead vehicle(s), in order to maintain a desired gap between the lead vehicle and the subject vehicle;" ¶: 0004].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the vehicle control system and method for controlling an autonomous vehicle using a trained machine learning model that performs image recognition of proximate objects as disclosed by Yang to incorporate the teachings regarding controlling a vehicle to maintain an inter-vehicle distance comprising of a minimum following gap and maximum following gap that may be dynamically adjusted based on a plurality of conditions as taught by Zhao with a reasonable expectation of success. By combining these inventions, the outcome is a control system and method for controlling an autonomous vehicle that is more robust in its ability to operate an adaptive cruise control for maintaining longitudinal distances to hand changes in road gradients to prevent unnecessary accelerations and decelerations and gain energy efficiency [Zhao; ¶: 0005, 0023, 0048].
Prior Art (Not relied upon)
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure can be found in the attached form 892.
TANIGAWA et al. (United States Patent Publication 2018/0373943 A1) discloses: A moving body detecting method for, by means of at least one computer, detecting a target moving body that is a moving body which possibly constitutes an obstacle to running of a target vehicle includes acquiring a photographed image that is generated by photographing with a camera situated on board the target vehicle. Next, the photographed image is inputted as input data into a recognition model for recognizing a moving body in an image taken of the moving body, type information indicating a type of the moving body, and position information indicating that one of a plurality of positions including a sidewalk and a roadway in which the moving body is present. Then, the target moving body in the photographed image is detected by acquiring the type information and position information of the moving body in the photographed image as outputted from the recognition model.
Sujan et al. (United States Patent Publication 2020/0387167 A1) discloses: An electronic control system is configured to control operation of a platoon including a plurality of vehicles. The electronic control system may be configured one or more of operate each of the vehicles to provide operation emulating the lowest non-platooning vehicle performance capability among the plurality of vehicles of the platoon, operate an individualized predictive cruise control (IPCC) process for each of the vehicles and a corresponding supervisory safety process for the platoon, and operate a cooperative predictive cruise control (CPCC) process for each of the vehicles and a corresponding supervisory safety process for the platoon.
Kim et al. (United States Patent Publication 2023/0273033 A1) discloses: A method for measuring an inter-vehicle distance includes acquiring a driving image photographed by a photographing device of a first vehicle which is being driven; detecting a second vehicle from the acquired driving image; detecting first feature points of a second vehicle region in a first frame corresponding to a frame in which the second frame is detected before a frame in which the second vehicle is not detected among a plurality of frames constituting the driving image, when the second vehicle is not detected from the driving image; detecting second feature points in a second frame corresponding to a current frame by tracking the detected first feature points; calculating a feature point change value between the first feature points and the second feature points; and calculating an inter-vehicle distance from the photographing device of the first vehicle to the second vehicle based on the calculated feature point change value.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAMI N BEDEWI whose telephone number is (571)272-5753. The examiner can normally be reached Monday - Thursday - 6:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott A. Browne can be reached on (571-270-0151). The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.N.B./Examiner, Art Unit 3666C
/SCOTT A BROWNE/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights