Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bylicka et. al. (United States Patent Application Publication US 2021/0112238 A1) in view of Goyal et. al. (United States Patent Application Publication US 2024/0185613 A1).
Regarding claim 1, Bylicka et. al. discloses a system for tracking objects comprising: one or more image sensors configured to capture one or more images; a detector processor configured to: detect one or more objects from the images, generate one or more first bounding boxes around the objects, wherein each of the first bounding boxes surrounds each of the objects (Bylicka et. al. Figure 2, Figure 7: Sensor component 706; [0033]: the 3D bounding boxes are formed from projection of 2D bounding boxes from 2D images and into a 3D space or graph to form at least two hyper-planes as sides of the 3D bounding box, and this may be repeated for each pair of objects on different 2D image views); and a tracking processor configured to: refine the first bounding boxes by minimizing space between each objects and the corresponding bounding boxes (Bylicka et. al. [0066]: process may include “prune 3D bounding boxes from graph by removing irrelevant edges”. To reduce computation load (or cost) once the 3D bounding boxes as constructed on the graph or 3D space, the system can check to determine whether or not the 3D bounding boxes meet at least some minimal characteristic conditions or criteria to be an associated subject that should be tracked. The conditions factor distinct features of the 3D bounding boxes and can be easily tailored to specific scenarios or environments. [0067] the size of the 3D bounding box can be limited to the general size, volume, height, and so forth related to a person (the context is an object being tracked) This would consider any geometrical tolerances such as camera perspective, distance from the camera, and so forth.); estimate one or more real-world locations of the objects based on the first bounding boxes (Bylicka et. al. [0066]: the conditions may be related to 3D geometry such as an approximate location and volume of the potential subject in the 3D space or graph).
However, Bylicka et. al. fails to disclose project the real-world locations of the objects into one or more second bounding boxes; receive a new detection of the objects; associate the new detection with the second bounding boxes; update the estimation of the real-world locations of the objects; and predict, based on the updated real-world locations, one or more future real-world locations of the objects.
Goyal teaches project the real-world locations of the objects into one or more second bounding boxes; receive a new detection of the objects; associate the new detection with the second bounding boxes; update the estimation of the real-world locations of the objects; and predict, based on the updated real-world locations, one or more future real-world locations of the objects (Goyal et. al. Abstract, Figure 3, 4, [0011]: processor applies an object detection model image frame to predict a location of a first bounding box of an object in the first image frame; and apply a confidence value to the predicted location of first bounding box. In response to the confidence level exceeding a predetermined threshold, the instructions may direct the processor to estimate a location of a second bounding box of the object in a second image frame based on the location of the first bounding box and non-zero movement of the vehicle and update the object detection model based on the estimated location of the second bounding box in the second image frame. The processor may further predict a location of a third bounding box of the object in a third image frame using the updated object detection model and control an operation of the vehicle based on the predicted location of the third bounding box. See also Figure 5; [0077] i.e., geographic coordinates and/or IMU).
This is important to the claimed invention because being able to predict the location of a moving object based on refined bounding boxes allows systems to make decisions based on information of the location of the object more precisely, especially when the object is in motion. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Bylicka et. al. and Goyal et. al. so that these features are shown in the solution of the claimed invention.
Regarding claim 12, which recites a method for tracking objects, corresponding to the system of claim 1, which the rejection analysis is incorporated herein.
Regarding claim 20, which recites a non-transitory computer readable medium containing computer executable instructions that corresponds to the system of claim 1, which the rejection analysis is incorporated herein.
Regarding claim 2, Bylicka et. al. and Goyal et. al. disclose the system of claim 1, and Goyal et. al. further discloses wherein one or more objects comprise at least one or more vehicles (Goyal et. al. Figure 5, [0011]: …second bounding box of the object in a second image frame based on the location of the first bounding box and non-zero movement of the vehicle.)
Regarding claim 3 and claim 19, Bylicka et. al. and Goyal et. al. disclose the system of claim 1 and method of claim 12, and Goyal et. al. further discloses wherein the refinement further comprises segmenting the first bounding boxes (Goyal et. al. Figure 4: Deployed segmentation model, [0041]: processor applies the segmentation model to the pixels in image frame to produce or output a first prediction segmentation map including pixel labels. The pixel labels may identify each individual pixel as being part of an object or environment/surroundings to the object. The pixel labels may identify the boundary, shape or edge of the object. [0051]: Segmentation involves pixel-wise labelling of every pixel of an input image.).
Regarding claim 4 and claim 16, Goyal et. al. further discloses the system of claim 3 and method of claim 12, wherein the refinement further comprises segmenting the first bounding boxes by matching the segmentation to a reference image of the one or more objects (Goyal et. al. claim 13: image segmentation system, [0073]: (1) apply a segmentation model to the image to output a first predicted segmentation map including pixel labels, (5) compute a first quantity objectness score for an object in the first predicted segmentation map. [0023]: In some implementations, the bounding box may represent a portion of the image frame that is to be set apart and segmented to determine the shape and perimeter coordinates of the object or obstacle contained within the bounding box.).
Regarding claim 5, Goyal et. al. further discloses the system of claim 4, wherein the reference image comprises at least one selected from the group of a blueprint, model, or computer-aided design (CAD) (Goyal et. al. [0010]: The medium may direct the processor to apply a segmentation model to the image to output a first predicted segmentation map including pixel labels, fuse the first predicted segmentation map and the point cloud and label pixels in the point cloud.).
Regarding claim 6, Bylicka et. al. and Goyal et. al. disclose the system of claim 1, and Goyal et. al. further discloses wherein the one or more image sensors are monocular imaging devices configured for ground-to-ground applications (Goyal et. al. [0070]: Cameras may comprise a monocular/2D camera or may comprise a stereo/3D camera).
Regarding claim 7, Bylicka et. al. and Goyal et. al. disclose the system of claim 1, and Bylicka et. al. further discloses wherein the estimation of the real-world locations of the objects is further based on at least a height of the bounding boxes, a location of the camera, and a pointing angle of the camera (Bylicka et. al. [0033]: the 3D bounding boxes are formed from projection of 2D bounding boxes from 2D images and into a 3D space or graph to form at least two hyper-planes as sides of the 3D bounding box, and this may be repeated for each pair of objects on different 2D image views. [0066]-[0067]: The conditions may be related to 3D geometry such as an approximate location and volume of the potential subject in the 3D space or graph. The size of the 3D bounding box can be limited to the general size, volume, height, and so forth related to a person (object of interest). Figure 6, [0058]: multi-view association graph.).
Regarding claim 8 and claim 17, Bylicka et. al. and Goyal et. al. disclose the system of claim 1 and method of claim 12, and Bylicka et. al. further discloses wherein the projection the real-world locations of the objects into one or more second bounding boxes further comprises: tracking the objects across multiple locations across a predetermined time period, sampling the locations, and projecting the sampled locations into one or more image coordinates (Bylicka et. al. [0057]: “obtain image data of multiple images of the same scene”. This may involve obtaining multiple images of the same scene and at the same time (same or similar time stamp) as mentioned above. [0033]: the 3D bounding boxes are formed from projection of 2D bounding boxes from 2D images and into a 3D space or graph to form at least two hyper-planes as sides of the 3D bounding box, and this may be repeated for each pair of objects on different 2D image views.).
Regarding claim 9 and claim 18, Bylicka et. al. and Goyal et. al. disclose the system of claim 1 and the method of claim 12, and Goyal et. al. further discloses wherein the association of the new detection with the second bounding boxes further comprises associating the new detection with a largest average bounding box overlap among the second bounding boxes (Goyal et. al. [0035]: Several image-based correlation metrics like mutual information, cross-correlation or normalized cross-correlation can be used to optimally find the location of the bounding box within the image. Here, the largest peak indicates largest overlap among the second bounding boxes, which produces the highest cross-correlation value.).
Regarding claim 10, Bylicka et. al. and Goyal et. al. disclose the system of claim 1, and Goyal et. al. further disclose wherein the one or more images sensors comprise a monocular camera and an inertial measurement unit (IMU) (Goyal et. al. Figure 5, [0070]: Cameras may comprise a monocular/2D camera or may comprise a stereo/3D camera. [0067]: Tractor is configured to be steered in an automated fashion by controller from various cameras or sensors provided on tractor and/or according to a predefined steering routine, route or path based upon signals from GPS and/or inertial measurement units.).
Regarding claim 11, Goyal et. al. further discloses the system of claim 10, wherein the monocular camera and IMU are associated with a standalone camera, a drone, or a moving vehicle (Goyal et. al. Figure 5, [0070]: Cameras may comprise a monocular/2D camera or may comprise a stereo/3D camera. [0067]: Tractor is configured to be steered in an automated fashion by controller from various cameras or sensors provided on tractor and/or according to a predefined steering routine, route or path based upon signals from GPS and/or inertial measurement units.).
Regarding claim 13, Bylicka et. al. and Goyal et. al. disclose the method of claim 12, and Goyal et. al. further discloses wherein the updating of the estimation of the real-world location of the objects is achieved through Kalman filtering (Goyal et. al. [0011]: The estimating of the location of the second bounding box in the second image comprises applying a Kalman filter and correlating the estimated location of the second bounding box within a margin around the location of the first bounding box. [0035]: To alleviate the effects of noisy bounding boxes, high confidence bounding boxes are taken from previous frames and correlated within a margin around the estimated bounding boxes from the Kalman Filter. Several image-based correlation metrics like mutual information, cross-correlation or normalized cross-correlation can be used to optimally find the location of the bounding box within the image.).
Regarding claim 14, Bylicka et. al. and Goyal et. al. disclose the method of claim 12, and Goyal et. al. further discloses wherein the predicting of the future location is achieved through at least one selected from the group of a motion model, a Kalman filter, or a particle filter (Goyal et. al. [0035]: To alleviate the effects of noisy bounding boxes, high confidence bounding boxes are taken from previous frames and correlated within a margin around the estimated bounding boxes from the Kalman Filter. Several image-based correlation metrics like mutual information, cross-correlation or normalized cross-correlation can be used to optimally find the location of the bounding box within the image. This step ensures that the tracking capabilities of the Kalman Filter and also the pixel information content in the images are integrated.).
Regarding claim 15, Bylicka et. al. and Goyal et. al. disclose the method of claim 12, and Goyal et. al. further discloses wherein the estimation of the real-world location of the one or more objects is represented as at least one selected from the group of a global positioning system (GPS), a latitude and longitude unit, or a measurement from a position of the image sensors (Goyal et. al. Figure 5, [0070]: Cameras may comprise a monocular/2D camera or may comprise a stereo/3D camera. [0067]: Tractor is configured to be steered in an automated fashion by controller from various cameras or sensors provided on tractor and/or according to a predefined steering routine, route or path based upon signals from GPS and/or inertial measurement units.).
PNG
media_image1.png
998
1048
media_image1.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA YIFANG LIN whose telephone number is (571)272-6435. The examiner can normally be reached M-F 7:00am-6:15pm, with optional day off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSICA YIFANG LIN/Examiner, Art Unit 2668 June 29, 2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights