DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 9, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20150210312 to Stein in view of WO 2024040171 to Park in view of CN 112990002 to Zhi.
!!! In reference of WO 2024040171 to Park there are two paragraph numbers at beginning of each paragraph. In this rejection the second paragraph number is recited !!
Regarding claim 1, Stein discloses a method, comprising:
identifying, by a computing device (paragraph 80; processing unit 110), an image captured of a driving scene associated with a vehicle (paragraph 244; in 1902 image is captured by capturing device 122 of vehicle area (scene) of vehicle));
detecting, by the computing device, a traffic light associated with the image using an object detector (paragraph 155; detecting traffic light objects in image by processing unit 110 (object detector));
predicting, by the computing device, a relevance attribute of the traffic light using a relevance classifier (paragraph 147, 244; relevancy of traffic light predicted by classification using criteria such as object type (classifier) based on vehicle position by processing unit);
predicting, by the computing device, a state attribute of the traffic light (paragraph 244; status of traffic light is determined (red to green));
generating, by the computing device, an image-level recommendation (paragraph 244, 249; based on determined change in status based on image analysis, a notification/warning (recommendation) can be generated).
However Stein does not disclose creating, by the computing device, a feature map based on the image; creating, by the computing device, an enhanced feature map based on the feature map and an output of the object detector; generating, by the computing device, an image-level recommendation using an image-level classifier, wherein the image-level recommendation is based on the enhanced feature map being provided as an input to the image-level classifier.
Park discloses creating, by the computing device (paragraph 58; processor 304), a feature map based on the image (paragraph 179-180; feature maps generated based on image);
creating, by the computing device, an enhanced feature map based on the feature map and an output of the object detector (paragraph 161, 181-182; enriched feature maps (enhanced) generated from feature maps by “multi-view stage 516 and the ROI stage 518”; paragraph 105-106; object detection via query);
generating, by the computing device, an image-level recommendation using an image-level classifier, wherein the image-level recommendation is based on the enhanced feature map being provided as an input to the image-level classifier (paragraph 127, 150-153, 155; paragraph 69-70; classification of objects into groups; paragraph 176; detection stage 510 uses queries from BEV stage that process enriched feature maps as input to generate bounding boxes for object classified in certain classes in image (image-level classifier used to generate bounding box) ; paragraph 70, 99, 195; planning system 404 generates update to trajectory (recommendation) based on the bounding box from perception system).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Stein as taught by Park to provide enhanced feature maps for generating recommendation.
The motivation to combine the references is to provide enhanced feature maps that can improve object identification capability of the vehicle and provide updated trajectory in response to object identification (paragraph 73, 97-98).
However Stein does not disclose predicting, by the computing device, a state attribute of the traffic light using a state classifier.
Zhi discloses predicting, by the computing device (paragraph n0011; microprocessor), a state attribute of the traffic light using a state classifier (paragraph n0116-n0118, n0120; traffic light status (attribute) is predicted by the AdaBoost classifier (state classifier)).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Stein as taught by Zhi to provide state classifier for estimating state attributes of traffic light.
The motivation to combine the references is to provide accurate traffic light status classifier that can be used especially for downhill roads that requires fast determination of traffic light state thereby resulting in safe driving environment (n0004, n0061-n0062).
Regarding claim 9, Stein discloses a computing device (paragraph 80; system 100), comprising:
one or more processors (paragraph 80; processing unit 110) configured to:
identify an image captured of a driving scene associated with a vehicle (paragraph 244; in 1902 image is captured by capturing device 122 of vehicle area (scene) of vehicle));
detect a traffic light associated with the image using an object detector (paragraph 155; detecting traffic light objects in image by processing unit 110 (object detector));
predict a relevance attribute of the traffic light using a relevance classifier (paragraph 147, 244; relevancy of traffic light predicted by classification based on vehicle position by processing unit (classifier));
predict a state attribute of the traffic light (paragraph 244; status of traffic light is determined (red to green));
generate an image-level recommendation (paragraph 244, 249; based on determined change in status based on image analysis, a notification/warning (recommendation) can be generated).
However Stein does not disclose one or more processors configured to: create a feature map based on the image; create an enhanced feature map based on the feature map and an output of the object detector; and generate an image-level recommendation using image-level classifier; wherein the image-level recommendation is based on the enhanced feature map being provided as an input to the image-level classifier.
Park discloses one or more processors configured to (paragraph 58; processor 304): create a feature map based on the image (paragraph 179-180; feature maps generated based on image); create an enhanced feature map based on the feature map and an output of the object detector (paragraph 161, 181-182; enriched feature maps (enhanced) generated from feature maps; paragraph 105-106; object detection via query); and generate an image-level recommendation using image-level classifier; wherein the image-level recommendation is based on the enhanced feature map being provided as an input to the image-level classifier (paragraph 127, 150-153, 155; paragraph 69-70; classification of objects into groups; paragraph 176; detection stage 510 uses queries from BEV stage that process enriched feature maps as input to generate bounding boxes for object class in image (image-level classifier used to generate bounding box) ; paragraph 70, 99, 195; planning system 404 generates update to trajectory (recommendation) based on the bounding box from perception system).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Stein as taught by Park to provide enhanced feature maps for generating recommendation.
The motivation to combine the references is to provide enhanced feature maps that can improve object identification capability of the vehicle and provide updated trajectory in response to object identification (paragraph 73, 97-98).
However Stein does not disclose processor configured to predict a state attribute of the traffic light using a state classifier.
Zhi discloses processor configured to predict (paragraph n0011; microprocessor), a state attribute of the traffic light using a state classifier (paragraph n0116-n0118, n0120; traffic light status (attribute) is predicted by the AdaBoost classifier (state classifier)).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Stein as taught by Zhi to provide state classifier for estimating state attributes of traffic light.
The motivation to combine the references is to provide accurate traffic light status classifier that can be used especially for downhill roads that requires fast determination of traffic light state thereby resulting in safe driving environment (n0004, n0061-n0062).
Regarding claim 17, see rejection of claim 1. Further Stein discloses a non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising (paragraph 25; storage media storing programs):
one or more instructions that, when executed by one or more processors of a computing device, cause the computing device to (paragraph 25; processing device executes programs).
Claim(s) 3, 2, 6, 10, 11, 14, 18 s/are rejected under 35 U.S.C. 103 as being unpatentable over US 20150210312 to Stein in view of WO 2024040171 to Park in view of CN 112990002 to Zhi further in view of US 20230211801 to Hendler.
Regarding claim 3, Stein does not disclose the method of claim 1, wherein the image-level recommendation indicates that the vehicle is to stop when a relevant traffic light is red.
Hendler discloses wherein the image-level recommendation indicates that the vehicle is to stop when a relevant traffic light is red (paragraph 354-355; image captured includes traffic light; when traffic light is relevant to vehicle and it is red ; second trained system can recommend vehicle to make stop).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Stein as taught by Hendler to provide stop recommendation for red light detected.
The motivation to combine the references is to provide determination of relevant traffic light such that when relevant light is red recommendation of stopping is provided and even when status of traffic light is unknown a holistic processing can generate accurate recommendation for stopping (paragraph 354-355).
Regarding claim 2, Stein discloses the method of claim 1, further comprising:
determining, by the computing device, the relevance attribute and the state attribute in accordance with a local task (paragraph 244, 246; relevancy determined based on analyzing local area or region of image for “relative position of traffic light to vehicle” using lane information (local task); paragraph 247; state determination is based on analyzing only an image area where color of traffic light is present as local area (local task)); and Hendler discloses determining, by the computing device, the image-level recommendation, in conjunction with the relevance attribute and the state attribute, in accordance with a global task (paragraph 354-355; relevancy attribute and status attribute of red light determined; paragraph 355; holistic approach (global task) used to determine recommendation).
Regarding claim 6, Hendler discloses the method of claim 1, further comprising:
using, by the computing device, a region-based convolutional neural network (R-CNN) detector to identify a relevant traffic light in the image and provide the image-level recommendation (paragraph 375; traffic light relevancy information determined using bounding box (region based) is used to output recommendation by network 2940/2950; paragraph 368; network 2940/2950 can be CNN type).
Regarding claim 10, see rejection of claim 2.
Regarding claim 11, see rejection of claim 3.
Regarding claim 14, see rejection of claim 6.
Regarding claim 18, see rejection of claim 2.
Claim(s) 7, 15 s/are rejected under 35 U.S.C. 103 as being unpatentable over US 20150210312 to Stein in view of WO 2024040171 to Park in view of CN 112990002 to Zhi further in view of JP 2024056959 to TAKAHASHI.
Regarding claim 7, Stein does not disclose the method of claim 1, further comprising:
identifying, by the computing device, a risky driving behavior based on the image-level recommendation; and
providing, by the computing device, a notification based on the risky driving behavior, wherein the notification indicates a recommended driving practice in view of the risky driving behavior.
Takahashi discloses identifying, by the computing device (PARAGRAPH 185; CPU 20a), a risky driving behavior based on the image-level recommendation; and providing, by the computing device, a notification based on the risky driving behavior, wherein the notification indicates a recommended driving practice in view of the risky driving behavior (paragraph 430; recommendation of dangerous driving location is output using video data and location (image level) ; based on this recommendation, it can detect dangerous driving incident (risky behavior) and provide notification such as to “warn the driver against engaging in that dangerous driving”).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Stein as taught by Takahashi to provide notification related to risky driving behavior.
The motivation to combine the references is to provide history of dangerous driving based on location in storage such that user can be notified ahead of time of approaching these locations such that user can make adjustments in advance to avoid the danger (paragraph 430).
Regarding claim 15, see rejection of claim 7.
Claim(s) 8, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20150210312 to Stein in view of WO 2024040171 to Park in view of CN 112990002 to Zhi further in view of US 20240029559 to Bond.
Regarding claim 8, Stein does not disclose the method of claim 1, further comprising: identifying, by the computing device, a relevant traffic light in the image without using a positioning system or a high-definition map.
Bond discloses further comprising: identifying, by the computing device (paragraph 39; processor), a relevant traffic light in the image without using a positioning system or a high-definition map (paragraph 54; relevant traffic lights are identified using images captured and using machine learning to identify relevant traffic light (no GPS or map used)).
It would have been obvious to one of ordinary skill in the art at the time of the invention was made to modify the system of Stein as taught by Bond to provide traffic light relevancy without using positioning system.
The motivation to combine the references is to provide relevancy determination of traffic light with high accuracy without using GPS information but instead using image data thereby in situation where GPS is not available relevancy can be determined using images from camera (paragraph 54).
Regarding claim 16, see rejection of claim 8.
Allowable Subject Matter
Claims 4, 5, 12, 13, 19, 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Other Prior Art Cited
14. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230280183 to Hayat discloses machine learning system that determines relevancy of traffic signals.
US 20250336288 to Di Lorenzo discloses determination of danger area for vehicles including analyzing traffic signals.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENIYAM MENBERU whose telephone number is (571) 272-7465. The examiner can normally be reached on Monday-Friday, 10:00am-6:30pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi Sarpong can be reached on (571) 270-3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Any inquiry of a general nature or relating to the status of this application or proceeding should be directed to the customer service office whose telephone number is (571) 272-2600. The group receptionist number for TC 2600 is (571) 272-2600.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
For more information about the PAIR system, see <http://pair-direct.uspto.gov/>. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Patent Examiner
Beniyam Menberu
/BENIYAM MENBERU/Primary Examiner, Art Unit 2681
07/17/2026
Prosecution Insights