DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This action is in response to the amendments and remarks filed on 08/04/2026.
Claims 1-12 are pending.
Corrective Actions by Applicant
Claims 1 and 12 have been amended.
Claim 13 has been canceled.
Response to Arguments
The examiner has fully considered Applicant’s presented arguments.
On pages 5-6 of the remarks, Applicant argues that Shen fails to disclose every element of amended claims 1 and 12. This is persuasive, based on Shen failing to disclose “wherein the filtering module analyzes content of the corresponding cropped image and, in response to determining that the object faces toward the image capturing module, selects the corresponding cropped image for classification by the image classifier.” However, the amendments necessitate new 103 rejections below.
On pages 5-6 of the remarks, Applicant argues that Angerer fails to disclose every element of amended claims 1 and 12. This is persuasive, based on Angerer not detecting and selecting forward-facing objects in particular for further analysis. However, the amendments necessitate new 103 rejections below.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al. (U.S. Publ. US-2020/0175326-A1) in view of Angerer et al. (U.S. Publ. US-2023/0286530-A1) and Heilbron et al. (U.S. Publ. US-2022/0383745-A1).
Regarding claim 1, Shen discloses Shen discloses a real-time vehicle traffic assistance system (see figure 1), comprising:
an image capturing module for capturing an image with one or more objects therein (see figure 2, step 202 and paragraph 0032);
a processing module (see figure 1, image processing network 102 and paragraphs 0020-0021) comprising:
a cropping module implemented by the processing module for cropping the one or more objects out of the image to generate one or more cropped images from the cropped-out portions of the image (see figure 2, step 204, figure 4, step 204, and paragraphs 0024-0026 and 0034, where a priority FOV is identified in the input image see figure 2, step 206, figure 4, step 206 and paragraph 0036, where the input image is cropped according to the priority FOV to obtain a high-resolution cropped image depicting objects such as vehicles);
an image classifier implemented by the processing module for classifying the one or more filtered images based on a neural network approach (see figure 2, step 210, figure 4, step 210, and paragraph 0038, where object detection and classification are performed on the cropped images and the following downsampled input image);
and an image reducer implemented by the processing module for reducing resolution of the image to generate a resized image (see figure 2, step 208; figure 4, step 208; and paragraph 0037, where the original input image is downsampled to obtain a low resolution image);
wherein the processing module detects an orientation of each object within the resized image and assigns the detected orientation to a corresponding cropped image (see paragraph 0039, where labeled boxes for the objects in the low resolution input image can be optionally labeled with object characteristics such as the object class or pose; see paragraphs 0039-0040, where the outputs for both the low resolution input image and cropped images can be combined, thus the pose can be assigned to the cropped image as well),
and a traffic analysis module that is operably connected to the image reducer, wherein the traffic analysis module locates object locations of the one or more objects on the resized image (see figure 2, step 212; figure 4, step 212; and paragraph 0039, where the objects are detected in the low resolution input image),
maps the object locations of the one or more objects to a map (see figure 4 and paragraph 0039, where a set of labeled boxes/windows identify the object locations on the low resolution input image),
and then scales back the map to the image to provide real-time traffic assistance to a vehicle (see figure 2, step 212; figure 4, step 212; and paragraphs 0041-0042, where the detected objects from the low resolution input image can be scaled up based on scaling factors between the high-resolution crop and the original input image; paragraph 0039 specifies that the output can be used to drive and/or otherwise control operation of an autonomous vehicle; paragraph 0016 specifies that the method is performed in real-time).
Shen fails to disclose a filtering module implemented by the processing module for filtering the one or more cropped images based on the orientation of the corresponding object within each cropped image of the one or more cropped images to generate one or more filtered images.
Pertaining to the same field of endeavor, Angerer discloses a filtering module implemented by the processing module for filtering the one or more cropped images based on the orientation of the corresponding object within each cropped image of the one or more cropped images to generate one or more filtered images (see paragraphs 0020-0024, where sets of sensor data, such as cropped images, can be filtered out based on properties such as road sign sizes, orientations, or angles; see in particular paragraph 0021, where a deep neural network can be applied to candidate cropped images to detect characteristics of the objects, such as sizes, orientations, angles, levels of occlusion, etc.).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen because doing so allows for filtering through large amounts of images to obtain those that most likely depict objects of interest (see Angerer paragraph 0020).
Shen in view of Angerer fails to further disclose wherein the filtering module analyzes content of the corresponding cropped image and, in response to determining that the object faces toward the image capturing module, selects the corresponding cropped image for classification by the image classifier.
Pertaining to the same field of endeavor, Heilbron discloses wherein the filtering module analyzes content of the corresponding cropped image and, in response to determining that the object faces toward the image capturing module, selects the corresponding cropped image for classification by the image classifier (see figure 40 and paragraphs 0558-0568, where traffic signs/objects in images are analyzed to determine their relevancy to the vehicle; paragraph 0568 particularly states this can include the signs facing towards the drivable path, or in other words towards the vehicle; see paragraphs 0575-0576, where relevant traffic signs are then classified using a crowd-sourced map; in combination with Shen and Angerer, this would predictably result in only using cropped images for classification where the signs are facing the vehicle).
Shen and Heilbron are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Heilbron into Shen and Angerer by only using images of objects facing the vehicle for classification because doing so allows for autonomous vehicles to determine which traffic signs are relevant to its travel path (see Heilbron paragraph 0559).
Regarding claim 2, Shen in view of Angerer and Heilbron discloses wherein the vehicle is either a manually driven vehicle or a self- driven vehicle (see Shen paragraph 0019).
Regarding claim 3, Shen in view of Angerer and Heilbron discloses wherein the image capturing module is a camera mounted onboard the vehicle (see Shen paragraph 0032).
Regarding claim 4, Shen in view of Angerer and Heilbron discloses wherein the camera captures images of either a road, a driveway, a traffic, or a runway (see Shen paragraph 0013, where images can depict roads or traffic).
Regarding claim 5, Shen fails to disclose the limitations of claim 5.
Pertaining to the same field of endeavor, Angerer discloses wherein the one or more objects are traffic signal lights, traffic road signs, road safety information, runway safety information, driveway regulatory signs, or runway regulatory signs (see paragraphs 0020-0022, where traffic/road signs can be the subjects analyzed in the images).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen and Heilbron because recognizing traffic/road signs improves the operation of autonomous vehicles (see Angerer paragraph 0001).
Regarding claim 6, Shen fails to disclose the limitations of claim 6.
Pertaining to the same field of endeavor, Angerer discloses wherein the image classifier identifies a classification code related to the type of traffic information displayed by the one or more objects in real time (see paragraph 0032, where particular sign types can be identified; paragraphs 0172-0174 specify that neural network inferencing can be conducted in real-time).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen and Heilbron because recognizing traffic/road signs improves the operation of autonomous vehicles (see Angerer paragraph 0001).
Regarding claim 7, Shen in view of Angerer fails to disclose the limitations of claim 7.
Pertaining to the same field of endeavor, Heilbron discloses wherein the traffic information includes green light, speed limits, school proximity, landslide hazards, or turns (see paragraph 0562, where the road signs can include speed limit signs and road curve signs).
Shen and Heilbron are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Heilbron into Shen and Angerer by detecting speed limit and turn signs because doing so improves safety of operation of autonomous vehicles (see Heilbron paragraph 0560).
Regarding claim 8, Shen in view of Angerer and Heilbron discloses wherein the neural network approach can be selected from either of Capsule Neural Network, Traffic Sign Yolo, or Convolutional Neural Network (see Shen paragraph 0020, where convolutional neural networks can be used).
Regarding claim 9, Shen in view of Angerer and Heilbron discloses wherein the traffic analysis module compares the one or more filtered images on the image and locates the one or more filtered images on the resized image before mapping (see Shen paragraph 0038-0039, where the objects are located across both images before mapping via labeled boxes occurs).
Regarding claim 10, Shen in view of Angerer and Heilbron discloses wherein the traffic analysis module uses mapping of the one or more cropped images to detect traffic information on a road in real time (see Shen figure 4, where traffic information such as vehicle locations can be determined as output; paragraph 0016 specifies that the method is performed in real-time).
Regarding claim 11, Shen fails to disclose the limitations of claim 11.
Pertaining to the same field of endeavor, Angerer discloses wherein the one or more cropped images are associated with the classification code (see paragraph 0032, where particular sign types can be identified).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen and Heilbron because recognizing traffic/road signs improves the operation of autonomous vehicles (see Angerer paragraph 0001).
Regarding claim 12, Shen discloses a method for providing real-time vehicle traffic assistance (see figures 2 and 4), comprising:
capturing an image with one or more objects (see figure 2, step 202 and paragraph 0032);
resizing the image by reducing the resolution of the image to generate a resized image (see figure 2, step 208; figure 4, step 208; and paragraph 0037, where the original input image is downsampled to obtain a low resolution image);
locating one or more resized objects within the resized image (see figure 2, step 212; figure 4, step 212; and paragraph 0039, where the objects are detected in the low resolution input image);
mapping the one or more resized objects of the resized image to the one or more objects of the image (see figure 4 and paragraph 0039, where a set of labeled boxes/windows identify the object locations on the low resolution input image);
cropping the one or more objects out of the image to generate one or more cropped images from the cropped-out portions of the image (see figure 2, step 204, figure 4, step 204, and paragraphs 0024-0026 and 0034, where a priority FOV is identified in the input image see figure 2, step 206, figure 4, step 206 and paragraph 0036, where the input image is cropped according to the priority FOV to obtain a high-resolution cropped image depicting objects such as vehicles);
and classifying the one or more filtered images based on a neural network approach to provide real-time traffic assistance to a vehicle (see figure 2, step 210; figure 4, step 210; and paragraph 0038, where object detection and classification are performed on the cropped images and the downsampled input image; paragraph 0039 specifies that the output can be used to drive and/or otherwise control operation of an autonomous vehicle; paragraph 0016 specifies that the method is performed in real-time),
and detecting an orientation of each object within the resized image and assigning the detected orientation to a corresponding cropped image (see paragraph 0039, where labeled boxes for the objects in the low resolution input image can be optionally labeled with object characteristics such as the object class or pose; see paragraphs 0039-0040, where the outputs for both the low resolution input image and cropped images can be combined, thus the pose can be assigned to the cropped image as well),
Shen fails to disclose filtering the one or more cropped images based on the orientation of the corresponding object within each cropped image of the one or more cropped images to generate one or more filtered images.
Pertaining to the same field of endeavor, Angerer discloses filtering the one or more cropped images based on the orientation of the corresponding object within each cropped image of the one or more cropped images to generate one or more filtered images (see paragraphs 0020-0024, where sets of sensor data, such as cropped images, can be filtered out based on properties such as road sign sizes, orientations, or angles; see in particular paragraph 0021, where a deep neural network can be applied to candidate cropped images to detect characteristics of the objects, such as sizes, orientations, angles, levels of occlusion, etc.).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen because doing so allows for filtering through large amounts of images to obtain those that most likely depict objects of interest (see Angerer paragraph 0020).
Shen in view of Angerer fails to further disclose analyzing content of the corresponding cropped image, and, in response to determining that the object faces toward the image capturing module, selecting the corresponding cropped image for classification using the neural network approach.
Pertaining to the same field of endeavor, Heilbron discloses analyzing content of the corresponding cropped image, and, in response to determining that the object faces toward the image capturing module, selecting the corresponding cropped image for classification using the neural network approach (see figure 40 and paragraphs 0558-0568, where traffic signs/objects in images are analyzed to determine their relevancy to the vehicle; paragraph 0568 particularly states this can include the signs facing towards the drivable path, or in other words towards the vehicle; paragraph 0574 specifies that this can be done using a neural network; see paragraphs 0575-0576, where relevant traffic signs are then classified using a crowd-sourced map; in combination with Shen and Angerer, this would predictably result in only using cropped images for classification where the signs are facing the vehicle).
Shen and Heilbron are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Heilbron into Shen and Angerer by only using images of objects facing the vehicle for classification because doing so allows for autonomous vehicles to determine which traffic signs are relevant to its travel path (see Heilbron paragraph 0559).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS JOHN HELCO whose telephone number is (703)756-5539. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella, can be reached at telephone number 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/NICHOLAS JOHN HELCO/Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights