DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 18 August 2026 has been entered.
Summary
The Amendment filed on 18 August 2026 has been acknowledged.
Claims 1, 5 – 7, 9 – 10, 13 – 17 and 20 are amended.
Currently, claims 1 – 20 are pending and considered as set forth.
Response to Arguments
Applicant’s arguments with respect to claims 1 – 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shashua et al. (Hereinafter Shashua) (US 2017/0010618 A1) in view of Tu et al. (Hereinafter Tu) (US 2021/0279640 A1).
As per claim 1, Shashua teaches the limitations of:
an autonomous or semi-autonomous machine (See at least paragraph 6; systems and methods for autonomous vehicle navigation) comprising:
a first sensor including a first sensor modality, the first sensor to obtain first sensor data (See at least paragraph 511; navigation system 1700 may include one or more sensors, such as camera 122, GPS unit 1710, road profile sensor 1730, speed sensor 1720, and accelerometer 1725. Vehicle 1205 may include other sensors, such as radar sensors. The sensors included in vehicle 1205 may collect data related to road segment 1200 as vehicle 1205 travels along road segment 1200.);
a second sensor including a second sensor modality that is different than the first sensor modality, the second sensor to obtain second sensor data (See at least paragraph 511; navigation system 1700 may include one or more sensors, such as camera 122, GPS unit 1710, road profile sensor 1730, speed sensor 1720, and accelerometer 1725. Vehicle 1205 may include other sensors, such as radar sensors. The sensors included in vehicle 1205 may collect data related to road segment 1200 as vehicle 1205 travels along road segment 1200.);
one or more controllers (See at least paragraph 115; the control system may include at least one of a steering control, an acceleration control, and a braking control.);
one or more actuation components (See at least paragraph 138; FIG. 2F, vehicle 200 may include throttling system 220, braking system 230, and steering system 240.); and
one or more processors, the one or more processors comprising processing circuitry (See at least abstract and paragraph 266; navigation system for a vehicle may include at least one processor. The at least one processor may be programmed to determine a navigational maneuver for the vehicle based, at least in part, on a comparison of a motion of the vehicle with respect to a predetermined model representative of a road segment. The at least one processor may be further programmed to receive, from a camera, at least one image representative of an environment of the vehicle. The at least one processor may be further programmed to determine, based on analysis of the at least one image, an existence in the environment of the vehicle of a navigational adjustment condition, cause the vehicle to adjust the navigational maneuver based on the existence of the navigational adjustment condition, and store information relating to the navigational adjustment condition. … Both applications processor 180 and image processor 190 may include various types of processing devices. For example, either or both of applications processor 180 and image processor 190 may include a microprocessor, preprocessors (such as an image preprocessor), graphics processors, a central processing unit (CPU), support circuits, digital signal processors, integrated circuits, memory, or any other types of devices suitable for running applications and for image processing and analysis) to:
determine, using the one or more controllers, one or more controls to control the autonomous or semi-autonomous machine according to the one or more trajectory points (See at least paragraph 439 and 443; As vehicles 1205-1225 travel on road segment 1200, navigation information collected (e.g., detected, sensed, or measured) by vehicles 1205-1225 may be transmitted to server 1230. In some embodiments, the navigation information may be associated with the common road segment 1200. The navigation information may include a trajectory associated with each of the vehicles 1205-1225 as each vehicle travels over road segment 1200. In some embodiments, the trajectory may be reconstructed based on data sensed by various sensors and devices provided on vehicle 1205. For example, the trajectory may be reconstructed based on at least one of accelerometer data, speed data, landmarks data, road geometry or profile data, vehicle positioning data, and ego motion data. In some embodiments, the trajectory may be reconstructed based on data from inertial sensors, such as accelerometer, and the velocity of vehicle 1205 sensed by a speed sensor. In addition, in some embodiments, the trajectory may be determined (e.g., by a processor onboard each of vehicles 1205-1225) based on sensed ego motion of the camera, which may indicate three dimensional translation and/or three dimensional rotations (or rotational motions). The ego motion of the camera (and hence the vehicle body) may be determined from analysis of one or more images captured by the camera. …The autonomous vehicle road navigation model may use map data included in sparse map 800 for determining target trajectories along road segment 1200 for guiding autonomous navigation of autonomous vehicles 1205-1225 or other vehicles that later travel along road segment 1200. For example, when the autonomous vehicle road navigation model is executed by a processor included in a navigation system of vehicle 1205, the model may cause the processor to compare the trajectories determined based on the navigation information received from vehicle 1205 with predetermined trajectories included in sparse map 800 to validate and/or correct the current traveling course of vehicle 1205. ); and
send one or more control signals corresponding to the one or more controls to the one or more actuation components to cause the autonomous or semi-autonomous machine to navigate according to the one or more trajectory points (See at least paragraph 39 – 40; he recognized landmark may include at least one of a traffic sign, an arrow marking, a lane marking, a dashed lane marking, a traffic light, a stop line, a directional sign, a reflector, a landmark beacon, a lamppost, a change is spacing of lines on the road, or a sign for a business. The predetermined road model trajectory may include a three-dimensional polynomial representation of a target trajectory along the road segment. Navigation between recognized landmarks may include integration of vehicle velocity to determine a location of the vehicle along the predetermined road model trajectory. The processor may be further programmed to adjust the steering system of the vehicle based on the autonomous steering action to navigate the vehicle. The processor may be further programmed to: determine a distance of the vehicle from the at least one recognized landmark; and determine whether the vehicle is positioned on the predetermined road model trajectory associated with the road segment based on the distance. The processor may be further programmed to adjust the steering system of the vehicle to move the vehicle from a current position of the vehicle to a position on the predetermined road model trajectory when the vehicle is not positioned on the predetermined road model trajectory. … A method of navigating a vehicle may include receiving, from an image capture device associated with the vehicle, at least one image representative of an environment of the vehicle; analyzing, using a processor associated with the vehicle, the at least one image to identify at least one recognized landmark; determining a current position of the vehicle relative to a predetermined road model trajectory associated with the road segment based, at least in part, on a predetermined location of the recognized landmark; determining an autonomous steering action for the vehicle based on a direction of the predetermined road model trajectory at the determined current location of the vehicle relative to the predetermined road model trajectory; and adjusting a steering system of the vehicle based on the autonomous steering action to navigate the vehicle.).
While Shashua teaches End-to-end neural network and applying sensor data to the neural network which is compute and generate one or more trajectory points in 3D world space (See at least paragraph 380, 421 – 423, 485 and 576), Shashua is silent regarding limitations of:
apply the first sensor data to one or more first layers of an end-to-end (E2E) neural network and the second sensor data to one or more second layers of the E2E neural network; and
directly generate, by the E2E neural network and based at least on the E2E neural network processing the first sensor data and the second sensor data, output data defining one or more coordinates of one or more trajectory points within a three-dimensional (3D) world-space coordinate system.
Tu teaches the limitations of:
apply the first sensor data to one or more first layers of an end-to-end (E2E) neural network and the second sensor data to one or more second layers of the E2E neural network (See at least figure 7A and paragraph 155 – 156; For example, FIG. 7A depicts an example surrogate training scenario 700 according to example embodiments of the present disclosure. FIG. 7A includes a first portion 705 of a target machine-learned model configured to generate an intermediate representation 710 based on sensor data 715A. In addition, the scenario 700 includes a first portion 720 and second portion 730 of a surrogate machine-learned model. The first portion 720 of the surrogate machine-learned model can be configured to generate an intermediate representation 725 based on sensor data 715B. The first portion 720 of the surrogate machine-learned model can be trained, by a discriminator model 735, to generate a deviating intermediate representation 725 matching the distribution of the first portion 705 of the target model. The discriminator model 735 can be trained based on a discriminator loss 740. The first 720 and second portion 730 of the surrogate machine-learned model can be trained based, at least in part, on a task loss 745. More particularly, a computing system can access a plurality of samples of intermediate feature maps (e.g., intermediate representation 710) generated by the machine-learned model (e.g., a first portion 705 thereof). For example, the computing system can obtain a plurality of intermediate representations 710 representative of a surrounding environment of the target vehicle computing system (e.g., of the target autonomous vehicle, etc.) at a plurality of times. For instance, the computing system can “spy” on a communication channel between the target vehicle computing system and another transmitting computing system (e.g., another transmitting autonomous vehicle, infrastructure element, etc.). The computing system can utilize adversarial descriptive domain adaptation to align the distribution of the received intermediate representation 710 (e.g., denoted) m and surrogate intermediate representations 725 (e.g., denoted m′) generated by the surrogate machine-learned model (e.g., a first portion 720 thereof) without explicit input-feature pairs.); and
PNG
media_image1.png
518
703
media_image1.png
Greyscale
directly generate, by the E2E neural network and based at least on the E2E neural network processing the first sensor data and the second sensor data, output data defining one or more coordinates of one or more trajectory points within a three-dimensional (3D) world-space coordinate system (See at least figure 7A and paragraph 155 – 156).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include apply the first sensor data to one or more first layers of an end-to-end (E2E) neural network and the second sensor data to one or more second layers of the E2E neural network; and directly generate, by the E2E neural network and based at least on the E2E neural network processing the first sensor data and the second sensor data, output data defining one or more coordinates of one or more trajectory points within a three-dimensional (3D) world-space coordinate system as taught by Tu in the system of Shashua, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 2, Shashua teaches the limitation of:
wherein the one or more processors include at least one of: one or more graphics processing units (GPUs), one or more central processing units (CPUs), or one or more hardware accelerators (See at least paragraph 266).
As per claim 3, Shashua teaches the limitation of:
wherein the autonomous or semi-autonomous machine further comprises one or more systems-on-a-chip (SOCs), and the one or more processors are included in the one or more SOCs (See at least paragraph 266 - 267).
As per claim 4, Shashua teaches the limitation of:
wherein the one or more trajectory points correspond to a turn or a lane change (See at least paragraph 517).
As per claim 5, the combination of Shashua and Tu teaches the limitation of:
wherein map data is further applied to the E2E neural network, and the output data defining the one or more coordinates of the one or more trajectory points in 3D world-space coordinate system are directly generated further based at least on the E2E neural network processing the map data (Shashua, See at least paragraph 12 and 540).
As per claim 6, Shashua teaches the limitation of:
wherein vehicle state data is further applied to the E2E neural network, and the output data representative defining the one or more coordinates of the one or more trajectory points in 3D world-space coordinate system are directly generated further based at least on the E2E neural network processing the vehicle state data (See at least paragraph 422 – 423).
As per claim 7, Shashua teaches the limitation of:
first sensor includes an image sensor; and
the second sensor includes at least one of: a LiDAR sensor; an image sensor; a SONAR sensor; a depth sensor; a microphone sensor; a RADAR sensor; or an ultrasonic sensor (See at least abstract, paragraph 330, 506 and 511).
As per claim 8, Shashua teaches the limitation of:
wherein the autonomous or semi-autonomous machine is a passenger vehicle, a truck, a bus, a robot, a warehouse vehicle, a flying vessel, or a boat (See at least paragraph 6 and 88).
As per claim 10, Shashua teaches the limitation of:
wherein the autonomous or semi-autonomous machine further comprises one or more internal sensors having fields of view or sensory fields internal to the autonomous or semi-autonomous machine, and wherein the computing system or another computing system of the autonomous or semi-autonomous machine perform in-cabin monitoring of one or more passengers using third sensor data obtained using the one or more internal sensors (See at least paragraph 6 and 418).
Regarding claims 9 and 11 – 20:
Claims 9 and 11 – 20 are rejected using the same rationale, mutatis mutandis, applied to claims 1 – 8 above, respectively.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IG T AN whose telephone number is (571)270-5110. The examiner can normally be reached M - F: 10:00AM- 4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached at (571) 270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
IG T AN
Primary Examiner
Art Unit 3662
/IG T AN/Primary Examiner, Art Unit 3662
Prosecution Insights