DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant asserts that Hu et al. (US 20240420344 A1) does not teach the limitations of the independent claims. Examiner agrees, however this is moot in light of the new grounds of rejection.
Applicant asserts that the rejection of claims 6, 8, 16 and 18 under 35 USC 103 is improper. Examiner agrees and the rejection of these claims is withdrawn.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 10-13 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Philbin et al. (US 20210103285 A1).
Regarding claim 1, Philbin teaches: A system, comprising a computer including a processor and a memory, the memory storing instructions executable by the processor (See Philbin [00390]-[0040] for processor, memory and instructions) to:
obtain a portion of an occupancy grid map for an area, wherein the occupancy grid map is generated based on collected data of a host object in the area and collected data of respective target objects in the area; (See Philbin [0011]-[0017] for generation of occupancy maps based on sensor data)
generate a predicted portion of the occupancy grid map based on predicted data of the host object and predicted data of the respective target objects; (See Philbin [0013] for generation of predicted occupancy map associated with future point in time)
determine an action based on inputting the portion and the predicted portion of the occupancy grid map to a deep reinforcement learning neural network; and (See Philbin [0009]-[0010], [0018]-[0020] for determination of trajectory, [0024] for trajectory determined by planning component based on information received from perception component and examples of data included in trajectory, and that perception components, planning components, and collision avoidance components may include one or more machine-learned models. See Figs. 3-4 and [0052]-[0066] for operation of collision avoidance system based on present and predicted occupancy maps. See [0053]-[0056] for sets of occupancy maps representing present and future states, [0015]-[0017] and [0056] for aggregation of sets to determine final estimation. See [0049] for examples of neural networks including deep learning algorithms.)
operate the host object based on the action. (See Philbin [0018] and throughout for execution of trajectory)
Regarding claim 2, Philbin teaches: The system of claim 1, wherein the instructions further include instructions to receive the collected data of the respective target objects from an infrastructure element in the area. (See Philbin [0023] where sensors may be separate from or disposed remotely from the vehicle. See [0036] for communication with other devices including traffic signals.)
Regarding claim 3, Philbin teaches: The system of claim 1, wherein the instructions further include instructions to determine the collected data of the host object based on host object sensor data. (See Philbin Fig. 2 and [0009]-[0012] for sensors 206, data used for trajectory planning and collision avoidance)
Regarding claim 10, Philbin teaches: The system of claim 1, wherein the occupancy grid map is generated based additionally on at least one of signal phase and timing (SPaT) data for traffic signals in the area and map data for the area. (See Philbin Fig. 2 and [0024] for maps and localization data, [0025] for local map data including road characteristics and static objects like buildings. See [0022] where sensor data includes location sensors such as GPS. See [0011] for occupancy maps based on sensor data, which would potentially include GPS data, which inherently must be associated with a map in order to be useful. See [0036] for network interface 210 which may communicate with other nearby devices including traffic signals. While Philbin does not explicitly teach the inclusion of SPaT data in the determination of the occupancy maps, it does have access to that information.)
Regarding claims 11-13 and 20, the claims are directed to a method for operating the system of claims 1-3 and 10 and are rejected under the same rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Philbin et al. (US 20210103285 A1) in view of Ramamoorthy et al. (US 20210370980 A1).
Regarding claim 4, Philbin teaches: The system of claim 1,
Philbin does not explicitly teach:
wherein the deep reinforcement learning neural network is trained based on a reward function, a reward for the reward function being determined based on comparing the action to a virtual scenario.
However, Ramamoorthy teaches a method of autonomous vehicle path planning based on predicted occupancy grid (See [0003], [0023], [0209]), the model being trained based on a reward function (See [0438]), by comparing different virtual scenarios (See [0139] for comparison of reward function scores generated for different simulated paths. Examiner considers the different paths to be comparable to the virtual scenarios.)
It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the application, to modify the system of Philbin to incorporate the multiple scored simulations of Ramamoorthy in order to provide more effective iterative training of the neural network model.
Regarding claim 14, the claim is directed to a method for operating the system of claim 4 and is rejected under the same rationale.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Philbin et al. (US 20210103285 A1) in view of Ramamoorthy et al. (US 20210370980 A1) and Mueller et al. (US 20250368231 A1).
Regarding claim 5, Philbin in view of Ramamoorthy teaches: The system of claim 4, wherein the virtual scenario includes virtual target vehicles operating in a virtual area, …and map data for the virtual area. (Each of the simulations of Ramamoorthy [0139] would include the relevant information)
Hu in view of Ramamoorthy does not explicitly teach: …simulated signal phase and timing (SPaT) data for virtual traffic signals in the virtual area,…
As noted in claim 10, Philbin [0036] teaches communication with traffic signals, which would include signal state data, but does not teach specific use of that data for model training.
However, Mueller teaches a method of determining an occupancy grid map for a vehicle environment (See [0006]) which incorporates signal phase and timing data (See [0046] for information about traffic lights and their state. While the specific SPaT terminology is not used, Examiner considers these to be comparable.)
It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the application, to modify the system of Philbin in view of Ramamoorthy to incorporate traffic signal state and timing information, as taught in Mueller, in order to more accurately predict the upcoming behavior of surrounding vehicles.
Regarding claim 15, the claim is directed to a method for operating the system of claim 5 and is rejected under the same rationale.
Allowable Subject Matter
Claims 6-9 and 16-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACOB KENT BESTEMAN-STREET whose telephone number is (571)272-2501. The examiner can normally be reached M-TH 8:00-5:00. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Nolan can be reached on 571-270-7016. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format.
For additional questions, contact the Electronic Business Center (EBC) at 866 217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACOB KENT BESTEMAN-STREET/
Examiner, Art Unit 3661
Prosecution Insights