DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This action is in response to the amendments and remarks filed on 12/01/2025.
Claims 1-13 are pending.
Corrective Actions by Applicant
Claims 1, 6, and 11-12 have been amended.
Response to Arguments
The examiner has fully considered Applicant’s presented arguments.
On page 5 of the remarks, Applicant accepts the Examiner’s interpretation using the broadest reasonable interpretation consistent with the specification for claims 5 and 7. Accordingly, the objections of claims 5 and 7 have been withdrawn.
On page 5 of the remarks, Applicant argues that the amendment to claim 6 overcomes the objection of claim 6. This is persuasive. The objection of claim 6 has been withdrawn.
On pages 5-6, Applicant argues that amended claim 1 and the specification provide sufficient corresponding structure and algorithmic support for the limitations of claims 1-11. This is persuasive. The 35 U.S.C. 112(b) rejections of claims 1-11 have been withdrawn in view of Applicant’s statement of corresponding structure and algorithmic support.
On pages 6-7 of the remarks, Applicant argues that none of Shen, Angerer, or Watanabe disclose or reasonably suggest every element of amended independent claims 1 and 12. The examiner respectfully disagrees.
The examiner argues that Shen discloses cropping the one or more objects out of the image to generate one or more cropped images from the cropped-out portions of the image, as recited by amended independent claim 1.
Paragraph 0036 of Shen describes step 206 of figures 2 and 4 in detail. After a “priority FOV” defining a portion of the image is determined in step 204, step 206 requires cropping the image according to the priority FOV to generate a high-resolution crop of the image. The examiner regards this as an instance of cropping one or more objects, such as the vehicles in the priority FOV of figure 4, out of the image to generate a separate cropped image that contains only the cropped-out portion of the original image.
Notably, paragraph 0037 of Shen states that step 208 requires downsampling the rest of the image that was not part of the cropped portion. Paragraph 0037 of Shen further states that, as a result of performing step 208, two separate images are obtained – a high resolution crop/cropped image and a low resolution version of the original image. The claim does not preclude doing so by cropping the entire original image down to obtain the cropped-out portion while maintaining a copy of the original image, as performed by Shen.
The same argument applies to the similarly-amended limitations of independent claim 12.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6 and 8-13 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al. (U.S. Publ. US-2020/0175326-A1) in view of Angerer et al. (U.S. Publ. US-2023/0286530-A1).
Regarding claim 1, Shen discloses a real-time vehicle traffic assistance system (see figure 1), comprising:
an image capturing module for capturing an image with one or more objects therein (see figure 2, step 202 and paragraph 0032);
a processing module (see figure 1, image processing network 102 and paragraphs 0020-0021) comprising:
a cropping module implemented by the processing module for cropping the one or more objects out of the image to generate one or more cropped images from the cropped-out portions of the image (see figure 2, step 206, figure 4, step 206, and paragraph 0036, where the input image is cropped according to a "priority FOV" to obtain a high-resolution cropped image depicting objects such as vehicles);
an image classifier implemented by the processing module for classifying the one or more filtered images based on a neural network approach (see figure 2, step 210, figure 4, step 210, and paragraph 0038, where object detection and classification are performed on the cropped images and the following downsampled input image);
and an image reducer implemented by the processing module for reducing resolution of the image to generate a resized image (see figure 2, step 208; figure 4, step 208; and paragraph 0037, where the original input image is downsampled to obtain a low resolution image);
and a traffic analysis module that is operably connected to the image reducer, wherein the traffic analysis module locates object locations of the one or more objects on the resized image (see figure 2, step 212; figure 4, step 212; and paragraph 0039, where the objects are detected in the low resolution input image),
maps the object locations of the one or more objects to a map (see figure 4 and paragraph 0039, where a set of labeled boxes/windows identify the object locations on the low resolution input image),
and then scales back the map to the image to provide real-time traffic assistance to a vehicle (see figure 2, step 212; figure 4, step 212; and paragraphs 0041-0042, where the detected objects from the low resolution input image can be scaled up based on scaling factors between the high-resolution crop and the original input image; paragraph 0039 specifies that the output can be used to drive and/or otherwise control operation of an autonomous vehicle; paragraph 0016 specifies that the method is performed in real-time).
Shen fails to disclose a filtering module implemented by the processing module for filtering the one or more cropped images based on the orientation of the one or more cropped images to generate one or more filtered images.
Pertaining to the same field of endeavor, Angerer discloses a filtering module implemented by the processing module for filtering the one or more cropped images based on the orientation of the one or more cropped images to generate one or more filtered images (see paragraphs 0020-0024, where sets of sensor data, such as cropped images, can be filtered out based on properties such as road sign sizes, orientations, or angles).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen because doing so allows for filtering through large amounts of images to obtain those that most likely depict objects of interest (see Angerer paragraph 0020).
Regarding claim 2, Shen in view of Angerer discloses wherein the vehicle is either a manually driven vehicle or a self- driven vehicle (see Shen paragraph 0019).
Regarding claim 3, Shen in view of Angerer discloses wherein the image capturing module is a camera mounted onboard the vehicle (see Shen paragraph 0032).
Regarding claim 4, Shen in view of Angerer discloses wherein the camera captures images of either a road, a driveway, a traffic, or a runway (see Shen paragraph 0013, where images can depict roads or traffic).
Regarding claim 5, Shen fails to disclose the limitations of claim 5.
Pertaining to the same field of endeavor, Angerer discloses wherein the one or more objects are traffic signal lights, traffic road signs, road safety information, runway safety information, driveway regulatory signs, or runway regulatory signs (see paragraphs 0020-0022, where traffic/road signs can be the subjects analyzed in the images).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen because recognizing traffic/road signs improves the operation of autonomous vehicles (see Angerer paragraph 0001).
Regarding claim 6, Shen fails to disclose the limitations of claim 6.
Pertaining to the same field of endeavor, Angerer discloses wherein the image classifier identifies a classification code related to the type of traffic information displayed by the one or more objects in real time (see paragraph 0032, where particular sign types can be identified; paragraphs 0172-0174 specify that neural network inferencing can be conducted in real-time).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen because recognizing traffic/road signs improves the operation of autonomous vehicles (see Angerer paragraph 0001).
Regarding claim 8, Shen in view of Angerer discloses wherein the neural network approach can be selected from either of Capsule Neural Network, Traffic Sign Yolo, or Convolutional Neural Network (see Shen paragraph 0020, where convolutional neural networks can be used).
Regarding claim 9, Shen in view of Angerer discloses wherein the traffic analysis module compares the one or more filtered images on the image and locates the one or more filtered images on the resized image before mapping (see Shen paragraph 0038-0039, where the objects are located across both images before mapping via labeled boxes occurs).
Regarding claim 10, Shen in view of Angerer discloses wherein the traffic analysis module uses mapping of the one or more cropped images to detect traffic information on a road in real time (see Shen figure 4, where traffic information such as vehicle locations can be determined as output; paragraph 0016 specifies that the method is performed in real-time).
Regarding claim 11, Shen fails to disclose the limitations of claim 11.
Pertaining to the same field of endeavor, Angerer discloses wherein the one or more cropped images are associated with the classification code (see paragraph 0032, where particular sign types can be identified).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen because recognizing traffic/road signs improves the operation of autonomous vehicles (see Angerer paragraph 0001).
Regarding claim 12, Shen discloses a method for providing real-time vehicle traffic assistance (see figures 2 and 4), comprising:
capturing an image with one or more objects (see figure 2, step 202 and paragraph 0032);
resizing the image by reducing the resolution of the image to generate a resized image (see figure 2, step 208; figure 4, step 208; and paragraph 0037, where the original input image is downsampled to obtain a low resolution image);
locating one or more resized objects within the resized image (see figure 2, step 212; figure 4, step 212; and paragraph 0039, where the objects are detected in the low resolution input image);
mapping the one or more resized objects of the resized image to the one or more objects of the image (see figure 4 and paragraph 0039, where a set of labeled boxes/windows identify the object locations on the low resolution input image);
cropping the one or more objects out of the image to generate one or more cropped images from the cropped-out portions of the image (see figure 2, step 206; figure 4, step 206; and paragraph 0036, where the input image is cropped according to a "priority FOV" to obtain a high-resolution cropped image depicting objects such as vehicles);
and classifying the one or more filtered images based on a neural network approach to provide real-time traffic assistance to a vehicle (see figure 2, step 210; figure 4, step 210; and paragraph 0038, where object detection and classification are performed on the cropped images and the downsampled input image; paragraph 0039 specifies that the output can be used to drive and/or otherwise control operation of an autonomous vehicle; paragraph 0016 specifies that the method is performed in real-time).
Shen fails to disclose filtering the one or more cropped images based on the orientation of the one or more cropped images to generate one or more filtered images.
Pertaining to the same field of endeavor, Angerer discloses filtering the one or more cropped images based on the orientation of the one or more cropped images to generate one or more filtered images (see paragraphs 0020-0024, where sets of sensor data, such as cropped images, can be filtered out based on properties such as road sign sizes, orientations, or angles).
Shen and Angerer are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Angerer into Shen because doing so allows for filtering through large amounts of images to obtain those that most likely depict objects of interest (see Angerer paragraph 0020).
Regarding claim 13, Shen in view of Angerer discloses detecting the orientation of the one or more resized objects (see Shen paragraph 0039, where labeled boxes for the objects in the low resolution input image can include object pose/orientation); and assigning the orientation of the one or more resized objects to the one or more cropped images (see Shen paragraphs 0039-0040, where the outputs for both the low resolution input image and cropped images can be combined).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Shen et al. (U.S. Publ. US-2020/0175326-A1) in view of Angerer et al. (U.S. Publ. US-2023/0286530-A1), and further in view of Watanabe et al. (U.S. Publ. US-2019/0286917-A1).
Regarding claim 7, Shen in view of Angerer fails to disclose the limitations of claim 7.
Pertaining to the same field of endeavor, Watanabe discloses wherein the traffic information includes green light, speed limits, school proximity, landslide hazards, or turns (see paragraph 0020, where the recognition device identifies targets depicted in images; figure 2 and paragraphs 0026-0030 specify that these targets can include speed limit signs).
Shen and Watanabe are considered analogous art, as they are both directed to object detection models for autonomous vehicle assistance. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Watanabe into Shen and Angerer because doing so allows for controlling the vehicle to drive at regulation speeds (see Watanabe paragraph 0097).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS JOHN HELCO whose telephone number is (703)756-5539. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella, can be reached at telephone number 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/NICHOLAS JOHN HELCO/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667