Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on October 29, 2025 has been entered.
Claim Objections
Claim 22 is objected to because of the following informalities:
Claim 22 as recited is ambiguous as it contains a typographical error. For the record, the examiner recommends claim 22 to be rewritten as follow, and interpretation will be as such until clarification is made of record or applicant accepts this proposal and makes changes accordingly.
22. The method of claim 13, further comprising capturing [[the]] location data via an Inertial Navigation System ("INS") sensor coupled to the robot concurrently with captur[[es]]ing the images, wherein tracking the one or more objects relative to the robot is based at least in part on the location data.
.
Response to Arguments
Applicant's arguments filed October 29, 2025 have been fully considered but they are not persuasive.
Regarding claim 1,
(1) applicant states that “Wang does not disclose comparing a time period of blob detection to a threshold.” Examiner disagrees with this statement. Wang teaches “the stereo-optical sensor unit 1302 tracks the blob through the counting zone for a specified duration. The specified duration stipulates a period of time used for a particular counting zone. [0215]”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 7, 9, 11, 13-14, 18-19 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Driessen (Object Tracking in a Computer Vision based Autonomous See-and-Avoid System for Unmanned Aerial Vehicles, Master’s Thesis in Computer Science at the School of Vehicle Engineering, Royal Institute of Technology year 2004), in view of Wang (US Patent Pub. No.: US 2019/0019017 A1), hereinafter Wang, further in view of Nguyen (An Improved Real-time Blob Detection for Visual Surveillance, 2009 2nd Internation Congress on Image and Signal Processing, 2009), hereinafter Nguyen.
Regarding claim 1, Driessen teaches a system, comprising: an imaging sensor (This thesis first looks at the basics of a see-and-avoid system. It then discusses the basics of computer imaging and image analysis and also investigates the performance of the sensor hardware that is available, i.e. commercial video cameras, and tries to answer whether they are suitable for use in a see-and-avoid system. Page 2 5th paragraph) coupled to a robot (This thesis is focused mainly on the tracking aspect of the see-and-avoid problem for unmanned aerial vehicles (UAVs). Abstract); an imaging module configured to capture images of a scene within a field of view of the imaging sensor (Only objects that appear above the horizon from the UAV’s line of sight will be considered. Abstract) throughout a time period (
PNG
media_image1.png
774
658
media_image1.png
Greyscale
); an image-processing module configured to process the captured images to generate a series of binary images based on the captured images and identify one or more objects within the field of view of the imaging sensor in the scene relative to the robot (For detection of moving objects subtraction of two consecutive frames was used to perform a differential motion image. This image was combined with a threshold to receive a binary differential image containing areas with motion. Page 44 1st paragraph); and a tracking module configured to track, based on the captured images (
PNG
media_image2.png
780
678
media_image2.png
Greyscale
), the one or more objects relative to the robot while the one or more objects are in motion (The objects can also be divided into two classes, based on the way they appear to the camera: stationary and moving objects. Page 24 2nd paragraph).
Driessen does not teach the following limitations as further recited, but Wang further teaches to track the one or more objects (The present disclosure generally relates to the field of object detection, tracking, and counting [0002]) by: identifying binary blobs within the series of images (As further described herein, the stereo-optical sensor unit 1302 may identify binary large objects ("blobs") in the height map that is representative of a desired tracking object. [0209]); selecting a set of the identified binary blobs that comprises foreground pixels (if either but not all detection is in the background, ignore the one in the background since it is most likely a static object (the local maximum in the foreground has higher priority over the one in the background). [0098]) in response to detecting the identified binary blobs within the series of images for at least a threshold time interval within the time period (The stereo-optical sensor unit 1302 tracks the blob through the counting zone for a specified duration. The specified duration may be a period of time used for a particular counting zone. The stereo-optical sensor unit 1302 may thereafter determine whether the tracking of the blob is complete. For example, the tracking of the blob may be complete once the tracked object is no longer in the scene. As another example, the tracking of the blob may be complete once the specified duration has elapsed. [0215]); and tracking a center point of each binary blob of the selected set of binary blobs throughout the series of binary images (Further, in block 1514, the stereo-optical sensor unit 1302 calculates properties in each blob relative to the one or more properties. For instance, the stereo-optical sensor unit 1302 determines a total area of the blob ( e.g., a total number of pixels associated with the blob), a low blob area ( e.g., a number of pixels associated with the blob with height values less than the cart height), a minimal blob size, a maximal blob size, a maximal parallelogram for the blob (if present), and the like. [0217]. A person having ordinary skill in the art would recognize a center point of each blob can also be determined as one of the properties of each blob.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Driessen to incorporate the teachings of Wang to track objects by identifying binary blobs that comprise foreground pixels in response to detecting the identified binary blobs within the series of images for a threshold time interval within the time period and tracking a center point of each binary blob in order to provide a tracking system that involves low cost, easy and low maintenance, high-speed processing, and capable of providing time-stamped results that can be further analyzed.
Driessen and Wang does not teach the following limitations as further recited, but Nguyen further teaches identifying binary blobs within the series of binary images (
PNG
media_image3.png
422
748
media_image3.png
Greyscale
Page 4 left column last paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Driessen and Wang to incorporate the teachings of Nguyen to identify binary blobs within the series of binary images in order to provide better processing speed and more precise blob detection.
Claims 2, 7, 9 and 11, unamended and are rejected based on the combination of Driessen, in view of Wang, and further in view of Nguyen. The grounds of rejection established in the last Office Action is fully incorporated herein.
Regarding claim 23, Driessen in the combination teaches the system of claim 11, wherein: the classification module is further configured to classify at least one object of the one or more objects as an above-horizon object (The high altitude actually implies the next restriction, that only obstacles in the air need to be considered, and thus anything below the horizon can be disregarded. Page 4 last paragraph); and the image-processing module is further configured to increase a rate of image capture in response to the classification module classifying the at least one object of the one or more objects as an above-horizon object (It is common knowledge that the rate of image capture can be increased for better object detection and tracking.).
Method claims 13, 14 and 18 are drawn to the method of using the corresponding apparatus claimed in claims 1, 2 and 11. Therefore method claims 13, 14 and 18 correspond to apparatus claims 1, 2 and 11 and are rejected for the same reasons of obviousness as used above.
Claim 19 is drawn to a computer program product comprising a computer readable storage medium having program code embodied therein, the program code that in response to execution by a processor, perform operations for executing the method of using the corresponding apparatus as claimed in claim 1. Therefore, claim 19 corresponds to apparatus claim 1, and is rejected for the same reasons of obviousness as used above.
Claims 3, 5, 15 and 20, unamended and are rejected based on the revised combination of Driessen (Object Tracking in a Computer Vision based Autonomous See-and-Avoid System for Unmanned Aerial Vehicles, Master’s Thesis in Computer Science at the School of Vehicle Engineering, Royal Institute of Technology year 2004), in view of Wang (US Patent Pub. No.: US 2019/0019017 A1), hereinafter Wang, further in view of Nguyen (An Improved Real-time Blob Detection for Visual Surveillance, 2009 2nd Internation Congress on Image and Signal Processing, 2009), hereinafter Nguyen as applied to claim 1 above, and further in view of Chan (US Patent No.: US 7460689 B1), hereinafter Chan. The ground of rejection established in the last Office Action is fully incorporated herein.
Regarding claim 3, Driessen teaches the system of claim 2, wherein the image-processing module is configured to: generate the sequence of binary images based on the one or more generated background-subtracted images to further eliminate the static background information (For detection of moving objects subtraction of two consecutive frames was used to perform a differential motion image. This image was combined with a threshold to receive a binary differential image containing areas with motion. Page 44 first paragraph), wherein generating the sequence of binary images comprises assigning a value of ‘1’ to the identified pixels and a value of ‘0’ to a plurality of remaining pixels in the captured images (Thresholding is a very basic image operation that tries to separate the background from objects based on intensity. In short, all the pixels in the image are compared to a suitable threshold value, and if their intensity value is higher, they are set to 1, otherwise they are set to 0. Page 17 last paragraph).
The combination of Driessen, Wang and Nguyen does not teach the following limitations as further recited, but Chan further teaches compute a running average of previous overlapping frames of the captured images; [[and]] subtract the running average from a current input frame to identify the pixels representing the one or more objects that are in motion in the captured images (A system and method of tracking moving targets in video images comprises retrieving a reference video frames each comprising arrays of digital pixels; computing a first averaged image for each of the reference video frames, wherein each pixel value of the first averaged image comprises an average pixel intensity value associated with all corresponding pixel values in the reference video frames; computing a second averaged image for a second set of the reference video frames, wherein each pixel value of the second averaged image comprises an average pixel intensity value associated with all corresponding pixel values in the reference video frames; viewing an input video frame comprising arrays of digital pixels; subtracting the input video frame from the first and second averaged images. Abstract).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Driessen, Wang and Nguyen to incorporate the teachings of Chan to compute a running average of previous overlapping frames of the captured images and subtract the running average from a current input frame to identify and track moving targets in video images with less false alarms.
Regarding claim 5, Driessen in the combination teaches the system of claim [[4]]3, wherein the image-processing module is configured to distinguish foreground pixels from background pixels of the background-subtracted images based on a threshold pixel value (Thresholding is a very basic image operation that tries to separate the background from objects based on intensity. Page 17 last paragraph. A possible way to deal with this is adaptive thresholding, where an individual threshold level is set for each pixel based on the range of the intensity values in its local neighbourhood. Page 18 1st paragraph).
Method claim 15 is drawn to the method of using the corresponding apparatus claimed in claim 3. Therefore method claim 15 corresponds to apparatus claim 3 and is rejected for the same reasons of obviousness as used above.
Claim 20 is drawn to a computer program product for executing the method of using the corresponding apparatus as claimed in claims 2, 3, 5 and 11. Therefore, claim 20 corresponds to apparatus claims 2, 3, 5 and 11, and is rejected for the same reasons of obviousness as used above.
Claim 8, unamended and is rejected based on the combination of Driessen (Object Tracking in a Computer Vision based Autonomous See-and-Avoid System for Unmanned Aerial Vehicles, Master’s Thesis in Computer Science at the School of Vehicle Engineering, Royal Institute of Technology year 2004), in view of Wang (US Patent Pub. No.: US 2019/0019017 A1), hereinafter Wang, further in view of Nguyen (An Improved Real-time Blob Detection for Visual Surveillance, 2009 2nd Internation Congress on Image and Signal Processing, 2009), hereinafter Nguyen, further in view of Kawamoto (Japan Patent Publication Number: JP 6236600 B1), hereinafter Kawamoto. The grounds of rejection established in the last Office Action is fully incorporated herein.
Claims 21, 22 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Driessen (Object Tracking in a Computer Vision based Autonomous See-and-Avoid System for Unmanned Aerial Vehicles, Master’s Thesis in Computer Science at the School of Vehicle Engineering, Royal Institute of Technology year 2004), in view of Wang (US Patent Pub. No.: US 2019/0019017 A1), hereinafter Wang, further in view of Nguyen (An Improved Real-time Blob Detection for Visual Surveillance, 2009 2nd Internation Congress on Image and Signal Processing, 2009), hereinafter Nguyen, further in view of Kawamoto (Japan Patent Publication Number: JP 6236600 B1), hereinafter Kawamoto, further in view of Russell (US Patent Pub. No.: US 2023/0304801 A1), hereinafter Russell.
Claim 21, unamended and is rejected based on the combination of Driessen, in view of Wang, further in view of Nguyen, further in view of Kawamoto, further in view of Russell. The grounds of rejection established in the last Office Action is fully incorporated herein.
Regarding claim 22, Russell in the combination teaches the method of claim 13, further comprising capturing [[the]] location data via an Inertial Navigation System ("INS") sensor (In a typical operational scenario, the INS outputs raw inertial data, position, velocity and attitude which a coupled device accepts as input. [0013]) coupled to the robot (In particular, an imaging sensor may acquire information from the environment through which a vehicle travels by detecting information reflected by objects within an observed field of view in the environment. [0014]) concurrently with captur[[es]]ing the images (In some implementations, an imaging sensor may emit signals towards objects in the environment through which the navigation system 100 travels and may detect portions of the signals that are reflected by surfaces in the environment. [0014]), wherein tracking the one or more objects relative to the robot is based at least in part on the location data (To provide the navigation parameters, the INS 311 may track the position and orientation of an object relative to a known starting point, orientation, and velocity. [0040]).
Regarding claim 24, Russell in the combination teaches the system of claim 21, wherein: the imaging sensor comprises a first imaging sensor (In some implementations, an imaging sensor may emit signals towards objects in the environment through which the navigation system 100 travels and may detect portions of the signals that are reflected by surfaces in the environment. [0014]); the INS sensor comprises a first INS sensor (In some embodiments, the inertial navigation system includes one or more inertial sensors and an input interface for receiving measurements. Abstract); the location data comprises first location data associated with the captured images (The processors may use the observed environmental information to perform visual odometry, lidar odometry, point cloud registration, simultaneous localization and mapping (SLAM) or other algorithms that acquire navigation information from the information provided by the imaging sensors. [0012]); the system further comprises: a housing attached to an underside of the robot and enclosing the INS sensor and the first imaging sensor (For example, the navigation system 300 may include an image sensor based component 310 and an inertial navigation system (INS) based component 320. [0038]. A person having ordinary skill in the art would recognize that attaching such a navigation system to an underside of the robot in a housing is one of the implementation options.); a second imaging sensor configured to capture additional images (As illustrated, the navigation system 300 may include imaging sensors, such as a camera 303, a lidar 305, or other types of imaging sensors. [0038]); and a second INS sensor configured to capture second location data corresponding to the additional images (In certain embodiments, a device includes an inertial navigation system. In some embodiments, the inertial navigation system includes one or more inertial sensors and an input interface for receiving measurements. Abstract); the image-processing module is further configured to generate the series of images based on the additional images (In some embodiments, an imaging sensor may include multiple imaging sensors. Further, the multiple imaging sensors may be the same type of sensors (i.e. multiple cameras) or implement multiple image sensing technologies. [0015]); and the tracking module is further configured to track the one or more objects relative to the robot based on the additional images and the second location data (The INS 311 may provide the navigation parameters to the processing device 301 for fusing with the odometry measurements provided by the imaging sensors ( camera 303 and lidar 305). [0039]).
Nguyen in the combination further teaches the image-processing module is further configured to generate the series of binary images (In this process, foreground masks means the set of all foreground pixels, and are represented as binary images where white (value 1) pixels are foreground pixels and black (value 0) pixels are background pixels. Page 2 left column 3rd paragraph).
Claim 10, unamended and is rejected based on the revised combination of Driessen (Object Tracking in a Computer Vision based Autonomous See-and-Avoid System for Unmanned Aerial Vehicles, Master’s Thesis in Computer Science at the School of Vehicle Engineering, Royal Institute of Technology year 2004), in view of Wang (US Patent Pub. No.: US 2019/0019017 A1), hereinafter Wang, further in view of Nguyen (An Improved Real-time Blob Detection for Visual Surveillance, 2009 2nd Internation Congress on Image and Signal Processing, 2009), hereinafter Nguyen as applied to claim 1 above, and further in view of Zhang (AOA-Based Three-Dimensional Positioning and Tracking Using the Factor Graph Technique Symmetry 2020, 12, 1400; doi:10.3390/sym12091400, Published: 22 August 2020), hereinafter Zhang, and further in view of Tabassum (Obstacle Avoiding Robot, Global Journal of Researches in Engineering: H Robotics & Nano-Tech, Volume 17 Issue 1 Version 1.0 Year 2017), hereinafter Tabassum. The ground of rejection established in the last Office Action is fully incorporated herein.
Claim 12, unamended and is rejected based on the revised combination of Driessen (Object Tracking in a Computer Vision based Autonomous See-and-Avoid System for Unmanned Aerial Vehicles, Master’s Thesis in Computer Science at the School of Vehicle Engineering, Royal Institute of Technology year 2004), in view of Wang (US Patent Pub. No.: US 2019/0019017 A1), hereinafter Wang, further in view of Nguyen (An Improved Real-time Blob Detection for Visual Surveillance, 2009 2nd Internation Congress on Image and Signal Processing, 2009), hereinafter Nguyen as applied to claim 1 above, and further in view of Cristaldi (Aircraft Classification Using a Microwave Barrier, IMS 2006 - IEEE International Workshop on Measurement Systems for Homeland Security, Contraband Detection and Personal Safety, Alexandria, VA, USA, 18-19 October 2006), hereinafter Cristaldi. The ground of rejection established in the last Office Action is fully incorporated herein.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668