DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 01/22/2026 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Objections
Claim(s) 1 is/are objected to because of the following informalities:
In re claim 1; “and a millimeter wave radar without the LiDAR” seems to contain a typographical error. It is suggested that the claim language be changed to recite; “and a millimeter wave radar without a LiDAR”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Jeong Woo (KR-20210070853-A), Golomedv (US-20200210715-A1), and Hori US-20030160866-A1 in view of Stiller US-20020072869-A1.
1. (Currently amended) Lee Jeong Woo (KR-20210070853-A) discloses
A method for developing an autonomous driving vehicle [0008-11] The present invention for solving the above problems utilizes predictive information of a pre-learned deep learning model provided from an infrastructure system for autonomous driving so that an autonomous vehicle can safely find a route… an infrastructure sensor-based deep learning model and a vehicle sensor-based deep learning model learned from another autonomous vehicle including
an external [0055] sensor [0053-55] The data processing module 120 processes the sensor data input from the vehicle sensor 11 installed in the autonomous vehicle 10 through the internal communication module 110 to obtain training data (or training data), and input the transformed training data (learning data) to the data learning module 130… be obtained from, for example, a camera (RGB camera, stereo camera, infrared camera), an ultrasonic sensor, a radar sensor, a lidar sensor, and the like
****,
Golomedv (US-20200210715-A1) discloses in a similar invention field of endeavor, a consideration for a method for training machine learning object detection to include sensors which include “…a number, a type, and a detection performance as a sensor specification…”;
(Golomedv [0015] It is contemplated that the present technology may be employed with different type of sensors mounted on vehicles, such as, but not limited, to cameras, LIDAR sensors and RADAR sensors and in different physical settings, where an MLA has been trained for object recognition or detection from data acquired in a first physical setting and may be exposed to data acquired in a second physical setting, where the representation of features of the objects are at least in part different… [0311] … MLA that has been previously trained to detect objects and features in sensor data having been acquired in standard conditions (i.e. conditions of initial training data) to detect the same objects and features in sensor data having been acquired in sub-standard conditions and having, to some extent, different representations of the objects and features. It is contemplated that the present technology may be employed on data acquired by one or more sensors, such as, but not limited, to cameras, LIDAR sensors and RADAR sensors for example.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a number, a type, and a detection performance as a sensor specification with a reasonable expectation for success, as taught by Golomedv, for the benefit of [0312] …expanding a range of technical solutions for addressing a particular technical problem, namely improving performance of a machine learning algorithm to classify objects in sensor data by re-training the machine learning algorithm to classify objects in sensor data having been acquired in conditions inferior to the initial training conditions. Such an approach may allow saving resources used by an electronic device.
wherein the autonomous driving vehicle autonomous vehicle 10 includes
an autonomous driving system configured to apply a machine learning model to at least a recognition process to recognize an external environment [0024] In one example, the vehicle terminal device 100 generates (constructs) a vehicle sensor-based deep learning model (M1) for autonomous driving by itself, and uses sensor data obtained from the vehicle sensor to generate the deep learning model (M1), the autonomous driving process including recognition of the surrounding environment, determination of the route to be driven, and vehicle control is performed based on a detection result of the external sensor [0055] Sensor data includes all kinds of data required for autonomous driving. The sensor data may be, for example, data (or surrounding environment data) related to the surrounding environment of an alley or irregular road section that the autonomous driving vehicle 10 currently passes. As the surrounding environment data, it may be data (hereinafter, obstacle data) obtained by detecting an obstacle existing in front of the autonomous driving vehicle 10 . Such obstacle data may be obtained from, for example, a camera (RGB camera, stereo camera, infrared camera), an ultrasonic sensor, a radar sensor, a lidar sensor, and the like and
a planning process to plan a route using a result of the recognition process among the recognition process [0024] In one example, the vehicle terminal device 100 generates (constructs) a vehicle sensor-based deep learning model (M1) for autonomous driving by itself, and uses sensor data obtained from the vehicle sensor to generate the deep learning model (M1), the autonomous driving process including recognition of the surrounding environment, determination of the route to be driven, and vehicle control is performed, the planning process, and
a vehicle control process to perform autonomous driving control based on the route [0024] …the autonomous driving process including recognition of the surrounding environment, determination of the route to be driven, and vehicle control is performed,
the machine learning model being configured to output the route [0077] the executed deep learning model uses the sensor data processed by the data processing module (120 in FIG. 2) as input to predict path information for autonomous driving and/or vehicle control information of the autonomous driving vehicle, etc. create Such predicted path information and/or vehicle control information is transmitted to the electronic control device 13 that controls autonomous driving of the autonomous driving vehicle 10 through an internal communication module ( 110 in FIG. 2 ) to be utilized for autonomous driving. by taking the detection result of the external sensor as an input [0024] … the vehicle terminal device 100 generates (constructs) a vehicle sensor-based deep learning model (M1) for autonomous driving by itself, and uses sensor data obtained from the vehicle sensor to generate the deep learning model (M1),
the method comprising:
a first step of training the machine learning model [0077] by adapting at least the recognition process and the planning process [0024] using a first autonomous driving vehicle [0032] vehicle A including the external sensor [0024, 0053, 0055, 0062]
****; and
Golomedv (US-20200210715-A1) discloses in a similar invention field of endeavor, a consideration for a method for training machine learning object detection to include sensors which include “…having a first sensor specification…”;
(Golomedv [0015] It is contemplated that the present technology may be employed with different type of sensors mounted on vehicles, such as, but not limited, to cameras, LIDAR sensors and RADAR sensors and in different physical settings, where an MLA has been trained for object recognition or detection from data acquired in a first physical setting and may be exposed to data acquired in a second physical setting, where the representation of features of the objects are at least in part different… [0311] … MLA that has been previously trained to detect objects and features in sensor data having been acquired in standard conditions (i.e. conditions of initial training data) to detect the same objects and features in sensor data having been acquired in sub-standard conditions and having, to some extent, different representations of the objects and features. It is contemplated that the present technology may be employed on data acquired by one or more sensors, such as, but not limited, to cameras, LIDAR sensors and RADAR sensors for example.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a number, a type, and a detection performance as a sensor specification with a reasonable expectation for success, as taught by Golomedv, for the benefit of [0312] …expanding a range of technical solutions for addressing a particular technical problem, namely improving performance of a machine learning algorithm to classify objects in sensor data by re-training the machine learning algorithm to classify objects in sensor data having been acquired in conditions inferior to the initial training conditions. Such an approach may allow saving resources used by an electronic device.
a second step of applying the machine learning model trained in the first step [0011-13] a vehicle terminal device according to an aspect of the present invention is a communication module for receiving, …and a vehicle sensor-based deep learning model learned from another autonomous vehicle… the deep learning model based on the vehicle sensor learned from the first autonomous driving vehicle to a second [0032] vehicle B autonomous driving vehicle [0032-39] that autonomous vehicles A and B sequentially enter an alleyway or an irregular road… vehicle A learns the sensor data obtained from the vehicle sensor that detects the environment of the irregular road, and a vehicle sensor-based deep learning model (M1) is created… detects the autonomous driving vehicle B that has entered the irregular road… the deep learning models M1 and M2 stored in the database are transmitted to the vehicle terminal device of the autonomous vehicle B including the external sensor having a second sensor specification [0055, 0045] the vehicle terminal device of the autonomous vehicle B receives the sensor data obtained from the vehicle sensor as an input, and the route or control information predicted through the corresponding models (deep learning models M1 and M2) received from the infrastructure system 400 At the same time, the sensor data collected until the autonomous vehicle B exits the relevant road is input to retrain the corresponding models (deep learning models M1 and M2).
****.
Golomedv (US-20200210715-A1) discloses in a similar invention field of endeavor, a consideration for a method for training machine learning object detection to include sensors which include “…a second sensor specification that is inferior to the first sensor specification…”;
(Golomedv [0311] … MLA that has been previously trained to detect objects and features in sensor data having been acquired in standard conditions (i.e. conditions of initial training data) to detect the same objects and features in sensor data having been acquired in sub-standard conditions and having, to some extent, different representations of the objects and features. It is contemplated that the present technology may be employed on data acquired by one or more sensors, such as, but not limited, to cameras, LIDAR sensors and RADAR sensors for example.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a second sensor specification that is inferior to the first sensor specification with a reasonable expectation for success, as taught by Golomedv, for the benefit of [0312] …expanding a range of technical solutions for addressing a particular technical problem, namely improving performance of a machine learning algorithm to classify objects in sensor data by re-training the machine learning algorithm to classify objects in sensor data having been acquired in conditions inferior to the initial training conditions. Such an approach may allow saving resources used by an electronic device.
the first sensor specification includes an external camera group, a LiDAR, and **** radar [0055] Sensor data includes all kinds of data… As the surrounding environment data, it may be data (hereinafter, obstacle data) obtained by detecting an obstacle existing in front of the autonomous driving vehicle 10 . Such obstacle data may be obtained from, for example, a camera (RGB camera, stereo camera, infrared camera), an ultrasonic sensor, a radar sensor, a lidar sensor, and the like,
****.
Hori US-20030160866-A1 discloses in a similar invention field of endeavor, a consideration for obstacle detection using sensors comprising “…a millimeter wave radar”;
(Hori [claim 5.] The obstacle detection device for a vehicle according to claim 2, wherein the radar ranging is conducted with a millimeter wave radar.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a millimeter wave radar with a reasonable expectation for success, as taught by Hori, for the benefit of a radar ranging portion 14 is preferably a millimeter wave radar, which is relatively unaffected by weather conditions such as rain and fog, but may also be a laser radar [0023].
Golomedv (US-20200210715-A1) discloses in a similar invention field of endeavor, a consideration for a method for training machine learning object detection to include sensors which include “…and the second sensor specification includes an external camera and **** radar…”;
(Golomedv [0015] It is contemplated that the present technology may be employed with different type of sensors mounted on vehicles, such as, but not limited, to cameras, LIDAR sensors and RADAR sensors and in different physical settings, where an MLA has been trained for object recognition or detection from data acquired in a first physical setting and may be exposed to data acquired in a second physical setting, where the representation of features of the objects are at least in part different )
(Golomedv [0311] … MLA that has been previously trained to detect objects and features in sensor data having been acquired in standard conditions (i.e. conditions of initial training data) to detect the same objects and features in sensor data having been acquired in sub-standard conditions and having, to some extent, different representations of the objects and features. It is contemplated that the present technology may be employed on data acquired by one or more sensors, such as, but not limited, to cameras, LIDAR sensors and RADAR sensors for example.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a second sensor specification with a reasonable expectation for success, as taught by Golomedv, for the benefit of [0312] …expanding a range of technical solutions for addressing a particular technical problem, namely improving performance of a machine learning algorithm to classify objects in sensor data by re-training the machine learning algorithm to classify objects in sensor data having been acquired in conditions inferior to the initial training conditions. Such an approach may allow saving resources used by an electronic device.
Stiller US-20020072869-A1 discloses in a similar invention field of endeavor, a consideration for a method of calibrating vehicle sensor systems which“…includes an external camera and … **** radar without the LiDAR.”;
(Stiller [0027] …A simple method of analyzing object data is explained below on the basis of the flow chart illustrated in FIG. 3. The basic sensor system includes only one camera (sensor 2) and radar sensor 3. A detailed explanation of the flow chart according to FIG. 3 will be given after a description of FIG. 4.)
(Stiller [0012] The results of calibration of one sensor of the sensor system can be easily transferred to one or more other sensors on the motor vehicle for calibration of these sensors as well. Thus, with the method according to the present invention, joint calibration of most of the vehicle sensors after installation is possible without requiring any special boundary conditions. In particular, the combination of sensors such as camera, radar, wheel sensor, acceleration sensor is advantageous here, but an application in conjunction with LIDAR sensors or ultrasonic sensors is also possible.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include an external camera and radar without LiDAR with a reasonable expectation for success, as taught by Stiller, for the benefit of providing a basic sensor system for detecting objects within a control system.
2. (Original) Lee Jeong Woo (KR-20210070853-A) discloses
The method for developing an autonomous driving vehicle according to claim 1, wherein the autonomous driving system applies the machine learning model to the recognition process, the planning process, and the vehicle control process [0024, 0077], the machine learning model being configured to output a control amount of the autonomous driving control by taking the detection result of the external sensor as an input [0061] The deep learning model execution module 140 is an electronic control device 13 in the autonomous driving vehicle 10 that requires predictive path information and/or vehicle control information generated from the deep learning model through the internal communication module 110 transmit. Here, the electronic control device 13 includes all control devices related to autonomous driving control, such as an engine control device, a speed control device, and a steering angle control device, and
in the first step, the machine learning model is trained by adapting the recognition process, the planning process, [0008] present invention for solving the above problems utilizes predictive information of a pre-learned deep learning model provided from an infrastructure system for autonomous driving so that an autonomous vehicle can safely find a route, and at the same time, while the autonomous vehicle is driving and the vehicle control process using the first autonomous driving vehicle [0032-39] when the autonomous vehicle A enters the irregular road, the vehicle terminal device of the autonomous vehicle A learns the sensor data obtained from the vehicle sensor that detects the environment of the irregular road, and a vehicle sensor-based deep learning model (M1) is created.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Jeong Woo (KR-20210070853-A), Golomedv (US-20200210715-A1), Hori US-20030160866-A1 and Stiller US-20020072869-A1, as applied to claim 1 above and further in view of Sridhara US-20140187177-A1.
3. (New) Lee Jeong Woo (KR-20210070853-A) discloses The method for developing an autonomous driving vehicle according to claim 1, wherein the machine learning model trained in the first step is applied to the second autonomous driving vehicle ***and applying the*** machine learning model on the second autonomous driving vehicle.
[0042] Subsequently, in S29 , the infrastructure system 400 generates a deep learning model M2 evolved by the autonomous vehicle B among the advanced deep learning models M1 and M2 received from the vehicle terminal device of the autonomous vehicle B. is re-learned based on sensor data obtained from an infrastructure sensor that detects the autonomous vehicle B as an object. Accordingly, the evolved deep learning model M2 further evolves… the infrastructure system 400 receives the deep learning models M1 and M2 stored in the database from the vehicle terminal device of the autonomous vehicle B. The evolved deep learning model M1 and the further evolved Update to deep learning model (M2).
[0044] As such, the infrastructure system 400 in charge of the road uses the deep learning model M1 learned from the vehicle terminal device of the autonomous vehicle A driving on the road and the deep learning model M2 learned through the infrastructure sensor information to the roadside device 300 is transmitted to the vehicle terminal device 100 of the autonomous vehicle B
[0045] Upon receiving this, the vehicle terminal device of the autonomous vehicle B receives the sensor data obtained from the vehicle sensor as an input, and the route or control information predicted through the corresponding models (deep learning models M1 and M2) received from the infrastructure system 400 At the same time
[0045-46] the sensor data collected until the autonomous vehicle B exits the relevant road is input to retrain the corresponding models (deep learning models M1 and M2). … The re-learned and evolved models (deep learning models M1 and M2) are transferred back to the infrastructure system 400 just before the autonomous vehicle B enters (or enters into) the corresponding road.
Sridhara US-20140187177-A1 discloses in a similar invention field of endeavor, a consideration for vehicle to vehicle communications and the use machine learning techniques wherein a model is used “…without re-adapting the machine learning model…”;
(Sridhara [0072] The use of boosted decision stumps allows the observer and/or analyzer modules to generate and apply lean data models without communicating with the cloud or a network to re-train the data, which significantly reduces the mobile device's dependence on the network server and the cloud. This eliminates the feedback communications between the mobile device and the network server, which further improves the performance and power consumption characteristics of the mobile device.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a consideration for machine learning without re-adapting the machine learning model with a reasonable expectation for success, as taught by Sridhara, for the benefit of allowing the observer and/or analyzer modules to generate and apply lean data models without communicating with the cloud or a network to re-train the data, which significantly reduces the mobile device's dependence on the network server and the cloud. This eliminates the feedback communications between the mobile device and the network server, which further improves the performance and power consumption characteristics of the mobile device [0072].
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee Jeong Woo (KR-20210070853-A), Golomedv (US-20200210715-A1), Hori US-20030160866-A1 and Stiller US-20020072869-A1, as applied to claim 1 above and further in view of Hoffberg US-20060200259-A1 and Mosher US-20180121576-A1.
4. (New) Lee Jeong Woo (KR-20210070853-A) discloses The method for developing an autonomous driving vehicle according to claim 1, wherein the machine learning model [0008] includes a ***computation*** applied to the recognition process [0024] and a ***computation*** applied to the planning process (e.g. determination of a route), and the ***computations*** trained in the first step are applied to the second autonomous driving vehicle [0044].
[0008-11] The present invention for solving the above problems utilizes predictive information of a pre-learned deep learning model provided from an infrastructure system for autonomous driving so that an autonomous vehicle can safely find a route… an infrastructure sensor-based deep learning model and a vehicle sensor-based deep learning model learned from another autonomous vehicle
[0024] In one example, the vehicle terminal device 100 generates (constructs) a vehicle sensor-based deep learning model (M1) for autonomous driving by itself, and uses sensor data obtained from the vehicle sensor to generate the deep learning model (M1), the autonomous driving process including recognition of the surrounding environment, determination of the route to be driven, and vehicle control is performed
[0077] the executed deep learning model uses the sensor data processed by the data processing module (120 in FIG. 2) as input to predict path information for autonomous driving and/or vehicle control information of the autonomous driving vehicle, etc. create Such predicted path information and/or vehicle control information is transmitted to the electronic control device 13 that controls autonomous driving of the autonomous driving vehicle 10 through an internal communication module ( 110 in FIG. 2 ) to be utilized for autonomous driving
[0044] As such, the infrastructure system 400 in charge of the road uses the deep learning model M1 learned from the vehicle terminal device of the autonomous vehicle A driving on the road and the deep learning model M2 learned through the infrastructure sensor information to the roadside device 300 is transmitted to the vehicle terminal device 100 of the autonomous vehicle B
Hoffberg US-20060200259-A1 discloses in a similar invention field of endeavor, a consideration for object detection using “…a first recognition model”;
(Hoffberg [0814] The present invention also provides a model-based pattern recognition system, for determining the presence of an object within an image. By providing models of the objects within an image, the recognition process is relatively unaffected by perspective, and the recognition may take place in a higher dimensionality space than the transmitted media. Thus, for example, a motion image may include four degrees of freedom; x, y, chroma/luma, and time. A model of an object may include further dimensions, including z, and axes of movement. Therefore, the model allows recognition of the object in its various configurations and perspectives.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a first recognition model with a reasonable expectation for success, as taught by Hoffberg, for the benefit of providing a system the capability to of recognition of the object in its various configurations and perspectives [0814].
Mosher US-20180121576-A1 discloses in a similar invention field of endeavor, a consideration for machine learning comprising “…a first planning model”;
(Mosher [claim 4.] The system of claim 1, wherein the network planning model generates the layout of the utility network based one or more objectives for the utility network and weights assigned to each feature of a set of features, the features including at least one of (i) characteristics of objects located along physical paths along which the network can be routed, (ii) characteristics of clearance to route wires or transmit wireless signals along the physical paths, (iii) data related to existing agreements between an owner of the network and other utility providers or property owners in the geographic area, or (iv) weather data for the geographic area.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Lee Jeong Woo to include a first planning model with a reasonable expectation for success, as taught by Mosher, for the benefit of generating control features according to one or more objectives or weights associated with operation(s).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW JOHN MOSCOLA whose telephone number is (571)272-6944.
The examiner can normally be reached M-F 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn can be reached on (571) 272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.J.M./Examiner, Art Unit 3663
/ABBY J FLYNN/Supervisory Patent Examiner, Art Unit 3663
Prosecution Insights