DETAILED ACTIONNotice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant Response to Official Action
The response filed on 12/22/2025 has been entered and made of record.
Acknowledgment
Claims 1-4, 6-10, 12, and 15-20, amended on12/22/2025, are acknowledged by the examiner.
Response to Arguments
Applicant’s arguments with respect to claims 1, 16, 19 and their dependent claims have been considered but they are moot in view of the new grounds of rejection necessitated by amendments initiated by the applicant. Examiner addresses the main arguments of the Applicant as below.
Regarding the claim objection, the amendment filed on 12/22/2025 addresses the issue. As a result, the claim objection is withdrawn.
Regarding the drawing objection, the amendment filed on 12/22/2025 addresses the issue. As a result, the drawing objection is withdrawn.
Regarding the 35 U.S.C. 101 rejection, the amendment filed on 12/22/2025 addresses the issue. As a result, the 35 U.S.C. 101 rejection is withdrawn.
Regarding the 35 U.S.C. 112(a) rejection, the amendment filed on 12/22/2025 addresses the issue. As a result, the 35 U.S.C. 112(a) rejection is withdrawn.
Regarding the 35 U.S.C. 112(b) rejections, the amendment filed on 12/22/2025 addresses the issues. As a result, the 35 U.S.C. 112(b) rejections are withdrawn.
Claim Rejection – 35 U.S.C. § 112
The following is a quotation of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention. The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph because of a new matter. The amended independent claims include “estimate a translation of the traffic image sensor using the homography”. It is noted that “homography” was mentioned 15 times in the specification, but none of these occasions indicates that homography is used to estimate the translation for the traffic image sensor. The specification, on other hand, describes homography is used to estimate the translation of images, which are data associated with the traffic image sensor, but it does not use the homography to translate the traffic image sensor itself. Therefore, the claim limitation “estimate a translation of the traffic image sensor using the homography” is a new matter, which is not described in the application as originally filed. The new matter is required to be canceled from the claims (Please see MPEP 608.04).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102(e), (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 1-7, 10-11, 13-14, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Viguier (US Patent Application Publication 2022/0189039 A1), (“Viguier”), in view of Sablak et al. (US Patent 9,210,312 B2), (“Sablak”).
Regarding claim 1, Viguier meets the claim limitations as follow.
A system for traffic image sensor movement detection and handling (system for camera-based detection, classification and tracking of distributed objects, and particularly to detecting, classifying and tracking moving objects along surface terrain through multiple zones) [Viguier: para. 0002], comprising: at least one non-transitory storage medium (The non-transitory computer readable medium could also be distributed among multiple data storage elements) [Viguier: para. 0071] storing instructions (the stored instructions) [Viguier: para. 0071]; and at least one processor that executes the instructions to (The computing device that executes some or all of the stored instructions could be a sensor unit. Alternatively, the computing device that executes some or all of the stored instructions could be another computing device, such as a cloud computer) [Viguier: para. 0071]: mask out vehicles (track and predict traffic patterns of humans, animals, vehicles) [Viguier: para. 0011; Note: Please the vehicle was masked in Figs. 7 and 10] in a first image captured by the traffic image sensor and a second image captured by the traffic image sensor ((FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]; (the method and system disclosed herein may use a single side mounted camera to monitor each zone or intersection, and track objects across multiple discontiguous zones) [Viguier: para. 0007]; (the camera view angle off the perpendicular axis, such as for example, to place it on a lamp post along a road or at a street comer looking across the traffic area rather than down from overhead) [Viguier: para. 0004; Figs. 7, 10]; (The sensor unit 101 is preferably adapted to be mounted to a pole, wall or any similar shaped surface that allows the sensor unit 101 to overlook the intersection and provides an unobstructed view of the terrain to be monitored) [Viguier: para. 0026; Figs. 7-10] ) using a mask segmentation model (A convolutional neural network trained to segment and identify pixels on a road) [Viguier: para. 0045; Figs. 7, 10]; perform background extraction on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor ((FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]; (the method and system disclosed herein may use a single side mounted camera to monitor each zone or intersection, and track objects across multiple discontiguous zones) [Viguier: para. 0007]; (the camera view angle off the perpendicular axis, such as for example, to place it on a lamp post along a road or at a street comer looking across the traffic area rather than down from overhead) [Viguier: para. 0004; Figs. 7, 10]; (The sensor unit 101 is preferably adapted to be mounted to a pole, wall or any similar shaped surface that allows the sensor unit 101 to overlook the intersection and provides an unobstructed view of the terrain to be monitored) [Viguier: para. 0026; Figs. 7-10]; (A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc.) [Viguier: para. 0045; Figs. 7, 10] ; (FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]); perform a homography transform on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (a homography transform is used to transform any pixel coordinate to the real-world coordinate. FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below. FIG. 8 is an example of a homography transform where FIG. 7 is projected onto the ground plane.) [Viguier: para. 0045; Figs 7-8]; estimate a translation of the traffic image sensor using the homography transform ((The detection module 601 uses a homography transform to translate the points where the object touches the ground and the bounding box into real world coordinates) [Viguier: para. 0056]; (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010]; (The GPS position of the sensor unit or an arbitrary point in the sample image is used as the origin to translate the real-world coordinates previously obtained into latitude and longitude coordinates. In an exemplary embodiment, the GPS position and other metadata is stored in the Sensor Database 118 in the cloud computer. An exemplary SFM algorithm is dense multi-view reconstruction. In this example, every pixel in the image sensor's field of view is mapped to the real-world coordinate system. An additional exemplary SFM algorithm is a homography transform illustrated in FIG. 5. In this example, a plane is fit to tie points that are known to be on the ground. A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc. Then a homography transform is used to transform any pixel coordinate to the real-world coordinate) [Viguier: para. 0043-0044; Figs 5-8]); determine whether the translation of the traffic image sensor is over a threshold amount (If more than one object in the second intersection meet the matching criteria a similarity metric D (e.g. mean squared distance) is calculated for each object meeting the matching criteria in the second intersection. A matching object is selected from the plurality of objects in the second intersection, based on the similarity metric exceeding a predetermined threshold to merge with the first object) [Viguier: para. 0065]; and upon determining that the translation of the traffic image sensor is over the threshold amount (a feature point matching algorithm finds matching points between the image from the sensor and the image from the mobile device; the mobile device indicates if enough matching points have been found in excess of a predetermined threshold) [Viguier: claim 12], take an action to handle of the traffic image sensor (If more than one object in the second intersection meet the matching criteria a similarity metric D (e.g. mean squared distance) is calculated for each object meeting the matching criteria in the second intersection. A matching object is selected from the plurality of objects in the second intersection, based on the similarity metric exceeding a predetermined threshold to merge with the first object. The object appearance information may be incorporated into the similarity metric and the predetermined threshold. This improves accuracy when object mergers are attempted at a third, fourth or subsequent intersection. If a plurality of matching objects have a similarity metric above the predetermined threshold, the object with the highest similarity metric is selected to merge with the first object. A high similarity metric is an indication that two objects are likely the same) [Viguier: para. 0065-0067].
Viguier does not explicitly disclose the following claim limitations (Emphasis added).
mask out vehicles in a first image and a second image.
However, in the same field of endeavor Sablak further discloses the claim limitations and the deficient claim limitations as follows:
mask out vehicles in a first image and a second image ((FIG. 16 is a plan view of a motion mask derived from a sequential series of images acquired by the camera) [Sablak: col. 3, line 36-37; Fig. 16]; (The invention comprises, in yet another form thereof a method of operating a surveillance camera system, including acquiring images with a camera. A motion mask is created based upon the acquired images. A source of static motion is located within the acquired images. A virtual mask is defined over the source of static motion within the acquired images. The motion mask is modified by use of the virtual mask. A moving object of interest is tracked in the acquired images based upon the modified motion mask) [Sablak: col. 2, line 43-51]),
perform background extraction on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (A possible approach to masking static motion or "background motion" involves removing or deleting a large preselected area, that may possibly include static motion, from a calculated motion mask. The computer vision system may transform each mask on image frames from cameras, and may process each frame to remove static motion. Such an approach may remove a large portion of useful information in addition to removing static motion.) [Sablak: col. 15, line 7-14];
perform a homography transform on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (The calculation of the Rotational and homography matrices is used to transform the privacy mask to align it with the current image and may require the translation, scaling and
rotation of the mask) [Sablak: col. 10, line 52-55; Fig. 13a]; (After computation of the homography matrix M, the vertices of the current mask visible in the field of view are identified, as indicated at 158, and then the homography matrix is used to determine the transformed image coordinates of the mask vertices) [Sablak: col. 12, line 2-6; Fig. 13a];estimate a translation of the traffic image sensor using the homography transform ((Transformation of the mask for an image acquired at a different focal length than the focal length at which the mask was defined requires scaling and rotation of
the mask as well as translation of the mask to properly position the mask in the current image. Masks produced by such geometric operations are approximations of the original) [Sablak: col. 10, line 55-60];determine whether the translation of the traffic image sensor (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010] is over a threshold amount (A possible approach to masking static motion or "background motion" involves removing or deleting a large preselected area, that may possibly include static motion, from a calculated motion mask. The computer vision system may transform each mask on image frames from cameras, and may process each frame to remove static motion. Such an approach may remove a large portion of useful information in addition to removing static motion.) [Sablak: col. 15, line 7-14]; andtake an action to handle the translation of the traffic image sensor (System 20 includes a camera 22 which is located within a partially spherical enclosure 24. Enclosure 24 is tinted to allow the camera to acquire images of the environment outside of enclosure 24 and simultaneously prevent individuals in the environment who are being observed by camera 22 from determining the orientation of camera 22. Camera 22 includes motors which provide for the panning, tilting and adjustment of the focal length of camera 22. Panning movement of camera 22 is represented by arrow 26, tilting movement of camera 22 is represented by arrow 28 and the changing of the focal length of the lens 23 of camera 22, i.e., zooming, is represented by arrow 30. As shown with reference to coordinate system 21, panning motion corresponds to movement along the x-axis, tilting motion corresponds to movement along the y-axis and focal length adjustment corresponds to movement along the z-axis) [Sablak: col. 1, line 50-65].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier with Sablak to program the system to implement of Sablak’s method.
Therefore, the combination of Viguier with Sablak will enable the system to increase the robustness of an auto-tracker system [Sablak: col. 2, line 52-62].
Regarding claim 2, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
wherein, upon determining that the translation of the traffic image sensor (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010] is not over the threshold amount (if there are not enough matching points from the additional data to meet the predetermined threshold) [Viguier: claim 13], the at least one processor further executes the instructions (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] to omit taking the action to handle the translation of the traffic image sensor (terminating the tracker if the path does not lead to the second cell) [Viguier: claim 6].
Regarding claim 3, Viguier meets the claim limitations as set forth in claim 3. Viguier further meets the claim limitations as follow.
wherein the at least one processor further executes the instructions (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format, or on other non-transitory media or articles of manufacture) [Viguier: para. 0070]; (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071]) to estimate the translation of the traffic image sensor (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010] periodically (In an exemplary embodiment, the sensor unit 101 transmits the path to the cloud computer 103 or other sensor units 101. The path may be transmitted after each iteration, at regular intervals (e.g. after every minute) or once the sensor unit 101 determines that the path is complete. A path is considered complete if the object has not been detected for a predetermined period of time or if the path took the object out of the sensor unit's field of view. The completion determination may be made by the cloud computer instead of the sensor unit.) [Viguier: para. 0053].
Regarding claim 4, Viguier meets the claim limitations as set forth in claim 3. Viguier further meets the claim limitations as follow.
wherein the period is at least one of one minute or changes (In an exemplary embodiment, the sensor unit 101 transmits the path to the cloud computer 103 or other sensor units 101. The path may be transmitted after each iteration, at regular intervals (e.g. after every minute) or once the sensor unit 101 determines that the path is complete. A path is considered complete if the object has not been detected for a predetermined period of time or if the path took the object out of the sensor unit's field of view. The completion determination may be made by the cloud computer instead of the sensor unit.) [Viguier: para. 0053].
Regarding claim 5, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
wherein the at least one processor executes the instructions using cloud services (the computing device that executes some or all of the stored instructions could be another computing device, such as a cloud computer) [Viguier: para. 0071]; (In an exemplary embodiment, the sensor unit 101 transmits the path to the cloud computer 103 or other sensor units 101. The path may be transmitted after each iteration, at regular intervals (e.g. after every minute) or once the sensor unit 101 determines that the path is complete. A path is considered complete if the object has not been detected for a predetermined period of time or if the path took the object out of the sensor unit's field of view. The completion determination may be made by the cloud computer instead of the sensor unit.) [Viguier: para. 0053]).
Regarding claim 6, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
wherein the mask segmentation model (A convolutional neural network trained to segment and identify pixels on a road) [Viguier: para. 0045; Figs. 7, 10] comprises at least one of a semantic image segmentation model (A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc.) [Viguier: para. 0045; Figs. 7, 10], a convolutional neural network model that performs object detection and instance segmentation (A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc.) [Viguier: para. 0045; Figs. 7, 10] or a computer vision object detection convolutional neural network (A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc.) [Viguier: para. 0045; Figs. 7, 10].
Regarding claim 7, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
the translation of the traffic image sensor is at least one of in pixels (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010] or an absolute pixel movement (A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc.) [Viguier: para. 0045; Figs. 7, 10].
Regarding claim 10, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
wherein the action to handle the translation of the traffic image sensor comprises transmitting an instruction to move the traffic image sensor (System 20 includes a camera 22 which is located within a partially spherical enclosure 24. Enclosure 24 is tinted to allow the camera to acquire images of the environment outside of enclosure 24 and simultaneously prevent individuals in the environment who are being observed by camera 22 from determining the orientation of camera 22. Camera 22 includes motors which provide for the panning, tilting and adjustment of the focal length of camera 22. Panning movement of camera 22 is represented by arrow 26, tilting movement of camera 22 is represented by arrow 28 and the changing of the focal length of the lens 23 of camera 22, i.e., zooming, is represented by arrow 30. As shown with reference to coordinate system 21, panning motion corresponds to movement along the x-axis, tilting motion corresponds to movement along the y-axis and focal length adjustment corresponds to movement along the z-axis) [Sablak: col. 1, line 50-65; Figs. 2-3].
Regarding claim 11, Viguier meets the claim limitations as set forth in claim 10. Viguier further meets the claim limitations as follow.
wherein the at least one processor further executes the instructions (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] to transmit the instruction to the traffic image sensor (Upon receiving the positive feedback, in step 207 the calibration application preferably prompts the user to move the phone in a slow sweeping motion, keeping the camera oriented toward the sensor unit field of view (e.g., intersection). The sweeping process is illustrated in FIG. 3. The installer/user with the mobile device takes the first image and the calibration application identifies the tie points 303 that match with the sample image 302. The user then sweeps the mobile device through N mobile device positions.) [Viguier: para. 0002; Figs. 2-3].
Regarding claim 13, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
wherein an area of interest is no longer visible when the translation is over the threshold amount (a similarity metric above the predetermined threshold) [Viguier: para. 0067].
Viguier does not explicitly disclose the following claim limitations (Emphasis added).
wherein an area of interest is no longer visible.
However, in the same field of endeavor Sablak further discloses the deficient claim limitations as follows:
wherein an area of interest is no longer visible ((If the user has finished adding vertices to the mask and indicates that the mask is complete, the program proceeds to box 140 where the user is asked to select the type of obscuring infill to be used with the mask) [Sablak: col. 9, line 46-50; Figs. 4-5]; (obscures a portion of the video image and thereby limits the effectiveness of the) [Sablak: col. 8, line 46-47]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier with Sablak to program the system to implement of Sablak’s method.
Therefore, the combination of Viguier with Sablak will enable the system to increase the robustness of an auto-tracker system [Sablak: col. 2, line 52-62].
Regarding claim 14, Viguier meets the claim limitations as set forth in claim 13. Viguier further meets the claim limitations as follow.
wherein the area of interest comprises an intersection (A track completion module 114 in the cloud computer monitors the paths in the intersection) [Viguier: para. 0061], an approach to the intersection, or an exit from the intersection (monitored intersections that include the predicted location of the object) [Viguier: para. 0061].
Regarding claim 16, Viguier meets the claim limitations as follow.
A method of traffic image sensor movement detection and handling (Method for Camera-Based Distributed Object Detection, Classification and Tracking) [Viguier: para. 0001], comprising:masking out vehicles in a first image captured by the traffic image sensor and a second image captured by the traffic image sensor ((FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045] ; (the method and system disclosed herein may use a single side mounted camera to monitor each zone or intersection, and track objects across multiple discontiguous zones) [Viguier: para. 0007]; (the camera view angle off the perpendicular axis, such as for example, to place it on a lamp post along a road or at a street comer looking across the traffic area rather than down from overhead) [Viguier: para. 0004; Figs. 7, 10]; (The sensor unit 101 is preferably adapted to be mounted to a pole, wall or any similar shaped surface that allows the sensor unit 101 to overlook the intersection and provides an unobstructed view of the terrain to be monitored) [Viguier: para. 0026; Figs. 7-10]; (track and predict traffic patterns of humans, animals, vehicles) [Viguier: para. 0011; Note: Please the vehicle was masked in Figs. 7 and 10]) using a mask segmentation model (A convolutional neural network trained to segment and identify pixels on a road) [Viguier: para. 0045; Figs. 7, 10]; performing background extraction on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor ((FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]; (the method and system disclosed herein may use a single side mounted camera to monitor each zone or intersection, and track objects across multiple discontiguous zones) [Viguier: para. 0007]; (the camera view angle off the perpendicular axis, such as for example, to place it on a lamp post along a road or at a street comer looking across the traffic area rather than down from overhead) [Viguier: para. 0004; Figs. 7, 10]; (The sensor unit 101 is preferably adapted to be mounted to a pole, wall or any similar shaped surface that allows the sensor unit 101 to overlook the intersection and provides an unobstructed view of the terrain to be monitored) [Viguier: para. 0026; Figs. 7-10]; (A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc.) [Viguier: para. 0045; Figs. 7, 10] ; (FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]); performing a homography transform on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (a homography transform is used to transform any pixel coordinate to the real-world coordinate. FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below. FIG. 8 is an example of a homography transform where FIG. 7 is projected onto the ground plane.) [Viguier: para. 0045; Figs 7-8]; estimating a translation of the traffic image sensor using the homography transform ((The detection module 601 uses a homography transform to translate the points where the object touches the ground and the bounding box into real world coordinates) [Viguier: para. 0056]; (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010]); (The GPS position of the sensor unit or an arbitrary point in the sample image is used as the origin to translate the real-world coordinates previously obtained into latitude and longitude coordinates. In an exemplary embodiment, the GPS position and other metadata is stored in the Sensor Database 118 in the cloud computer. An exemplary SFM algorithm is dense multi-view reconstruction. In this example, every pixel in the image sensor's field of view is mapped to the real-world coordinate system. An additional exemplary SFM algorithm is a homography transform illustrated in FIG. 5. In this example, a plane is fit to tie points that are known to be on the ground. A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc. Then a homography transform is used to transform any pixel coordinate to the real-world coordinate) [Viguier: para. 0043-0044; Figs 5-8]); determining whether the translation of the traffic image sensor is over a threshold amount (If more than one object in the second intersection meet the matching criteria a similarity metric D (e.g. mean squared distance) is calculated for each object meeting the matching criteria in the second intersection. A matching object is selected from the plurality of objects in the second intersection, based on the similarity metric exceeding a predetermined threshold to merge with the first object) [Viguier: para. 0065]; and upon determining that the translation of the traffic image sensor is over the threshold amount (a feature point matching algorithm finds matching points between the image from the sensor and the image from the mobile device; the mobile device indicates if enough matching points have been found in excess of a predetermined threshold) [Viguier: claim 12], take an action (If more than one object in the second intersection meet the matching criteria a similarity metric D (e.g. mean squared distance) is calculated for each object meeting the matching criteria in the second intersection. A matching object is selected from the plurality of objects in the second intersection, based on the similarity metric exceeding a predetermined threshold to merge with the first object. The object appearance information may be incorporated into the similarity metric and the predetermined threshold. This improves accuracy when object mergers are attempted at a third, fourth or subsequent intersection. If a plurality of matching objects have a similarity metric above the predetermined threshold, the object with the highest similarity metric is selected to merge with the first object. A high similarity metric is an indication that two objects are likely the same) [Viguier: para. 0065-0067].
Viguier does not explicitly disclose the following claim limitations (Emphasis added).
masking out vehicles in a first image and a second image.
However, in the same field of endeavor Sablak further discloses the deficient claim limitations as follows:
masking out vehicles in a first image and a second image ((FIG. 16 is a plan view of a motion mask derived from a sequential series of images acquired by the camera) [Sablak: col. 3, line 36-37; Fig. 16]; (The invention comprises, in yet another form thereof a method of operating a surveillance camera system, including acquiring images with a camera. A motion mask is created based upon the acquired images. A source of static motion is located within the acquired images. A virtual mask is defined over the source of static motion within the acquired images. The motion mask is modified by use of the virtual mask. A moving object of interest is tracked in the acquired images based upon the modified motion mask) [Sablak: col. 2, line 43-51]);
performing background extraction on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (A possible approach to masking static motion or "background motion" involves removing or deleting a large preselected area, that may possibly include static motion, from a calculated motion mask. The computer vision system may transform each mask on image frames from cameras, and may process each frame to remove static motion. Such an approach may remove a large portion of useful information in addition to removing static motion.) [Sablak: col. 15, line 7-14];
performing a homography transform on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (The calculation of the Rotational and homography matrices is used to transform the privacy mask to align it with the current image and may require the translation, scaling and
rotation of the mask) [Sablak: col. 10, line 52-55; Fig. 13a]; (After computation of the homography matrix M, the vertices of the current mask visible in the field of view are identified, as indicated at 158, and then the homography matrix is used to determine the transformed image coordinates of the mask vertices) [Sablak: col. 12, line 2-6; Fig. 13a];estimating a translation of the traffic image sensor using the homography transform ((Transformation of the mask for an image acquired at a different focal length than the focal length at which the mask was defined requires scaling and rotation of
the mask as well as translation of the mask to properly position the mask in the current image. Masks produced by such geometric operations are approximations of the original) [Sablak: col. 10, line 55-60]; (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010]);determining whether the translation of the traffic image sensor is over a threshold amount (A possible approach to masking static motion or "background motion" involves removing or deleting a large preselected area, that may possibly include static motion, from a calculated motion mask. The computer vision system may transform each mask on image frames from cameras, and may process each frame to remove static motion. Such an approach may remove a large portion of useful information in addition to removing static motion.) [Sablak: col. 15, line 7-14]; andtake an action to handle the translation of the traffic image sensor (System 20 includes a camera 22 which is located within a partially spherical enclosure 24. Enclosure 24 is tinted to allow the camera to acquire images of the environment outside of enclosure 24 and simultaneously prevent individuals in the environment who are being observed by camera 22 from determining the orientation of camera 22. Camera 22 includes motors which provide for the panning, tilting and adjustment of the focal length of camera 22. Panning movement of camera 22 is represented by arrow 26, tilting movement of camera 22 is represented by arrow 28 and the changing of the focal length of the lens 23 of camera 22, i.e., zooming, is represented by arrow 30. As shown with reference to coordinate system 21, panning motion corresponds to movement along the x-axis, tilting motion corresponds to movement along the y-axis and focal length adjustment corresponds to movement along the z-axis) [Sablak: col. 1, line 50-65].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier with Sablak to program the system to implement of Sablak’s method.
Therefore, the combination of Viguier with Sablak will enable the system to increase the robustness of an auto-tracker system [Sablak: col. 2, line 52-62].
Regarding claim 17, Viguier meets the claim limitations as set forth in claim 16. Viguier further meets the claim limitations as follow.
wherein the information is output on the second image captured by the traffic image sensor ((FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]; (the method and system disclosed herein may use a single side mounted camera to monitor each zone or intersection, and track objects across multiple discontiguous zones) [Viguier: para. 0007]; (the camera view angle off the perpendicular axis, such as for example, to place it on a lamp post along a road or at a street comer looking across the traffic area rather than down from overhead) [Viguier: para. 0004; Figs. 7, 10]; (The sensor unit 101 is preferably adapted to be mounted to a pole, wall or any similar shaped surface that allows the sensor unit 101 to overlook the intersection and provides an unobstructed view of the terrain to be monitored) [Viguier: para. 0026; Figs. 7-10]).
In the same field of endeavor Sablak further discloses the claim limitations as follows:
wherein the information is output on the second image ((FIG. 16 is a plan view of a motion mask derived from a sequential series of images acquired by the camera) [Sablak: col. 3, line 36-37; Fig. 16] captured by the traffic image sensor (The invention comprises, in yet another form thereof a method of operating a surveillance camera system, including acquiring images with a camera. A motion mask is created based upon the acquired images. A source of static motion is located within the acquired images. A virtual mask is defined over the source of static motion within the acquired images. The motion mask is modified by use of the virtual mask. A moving object of interest is tracked in the acquired images based upon the modified motion mask) [Sablak: col. 2, line 43-51]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier with Sablak to program the system to implement of Sablak’s method.
Therefore, the combination of Viguier with Sablak will enable the system to increase the robustness of an auto-tracker system [Sablak: col. 2, line 52-62].
Regarding claim 18, Viguier meets the claim limitations as set forth in claim 16. Viguier further meets the claim limitations as follow.
wherein the information comprises an indicator showing an amount of the translation of the traffic image sensor (These modules work together to track the movement of objects through a specific intersection which the sensor unit observes. Each object is assigned a path which moves through the intersection. Each path includes identifying information such as the object's position, class label, current timestamp and a unique path ID) [Viguier: para. 0045; Fig. 6] or a position of the first image captured by the traffic image sensor in the second image captured by the traffic image sensor ((The relative position and orientation measurements of each image are used to align the SFM coordinate frame with an arbitrary real-world reference frame, such as East-North-Up ("ENU"), and rescale distances to a realworld measurement system such as meters or the like) [Viguier: para. 0042; Figs. 2-4]; (FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]; (the method and system disclosed herein may use a single side mounted camera to monitor each zone or intersection, and track objects across multiple discontiguous zones) [Viguier: para. 0007]; (the camera view angle off the perpendicular axis, such as for example, to place it on a lamp post along a road or at a street comer looking across the traffic area rather than down from overhead) [Viguier: para. 0004; Figs. 7, 10]; (The sensor unit 101 is preferably adapted to be mounted to a pole, wall or any similar shaped surface that allows the sensor unit 101 to overlook the intersection and provides an unobstructed view of the terrain to be monitored) [Viguier: para. 0026; Figs. 7-10]).
In the same field of endeavor Sablak further discloses the claim limitations as follows:
wherein the information comprises an indicator showing an amount of the translation of the traffic image sensor ((Transformation of the mask for an image acquired at a different focal length than the focal length at which the mask was defined requires scaling and rotation of the mask as well as translation of the mask to properly position the mask in the current image. Masks produced by such geometric operations are approximations of the original) [Sablak: col. 10, line 55-60]) or a position of the first image captured by the traffic image sensor in the second image captured by the traffic image sensor ((FIG. 16 is a plan view of a motion mask derived from a sequential series of images acquired by the camera) [Sablak: col. 3, line 36-37; Fig. 16]; (The invention comprises, in yet another form thereof a method of operating a surveillance camera system, including acquiring images with a camera. A motion mask is created based upon the acquired images. A source of static motion is located within the acquired images. A virtual mask is defined over the source of static motion within the acquired images. The motion mask is modified by use of the virtual mask. A moving object of interest is tracked in the acquired images based upon the modified motion mask) [Sablak: col. 2, line 43-51]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier with Sablak to program the system to implement of Sablak’s method.
Therefore, the combination of Viguier with Sablak will enable the system to increase the robustness of an auto-tracker system [Sablak: col. 2, line 52-62].
Regarding claim 19, Viguier meets the claim limitations as follow.
At least one non-transitory machine readable medium that stores a computer program product (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format, or on other non-transitory media or articles of manufacture) [Viguier: para. 0070] comprising: first instructions stored in the at least one non-transitory machine readable medium (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format) [Viguier: para. 0070] an executable by at least one processor to (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] mask out vehicles in a first image captured by the traffic image sensor and a second image captured by the traffic image sensor ((track and predict traffic patterns of humans, animals, vehicles) [Viguier: para. 0011; Note: Please the vehicle was masked in Figs. 7 and 10]; (FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]; (the method and system disclosed herein may use a single side mounted camera to monitor each zone or intersection, and track objects across multiple discontiguous zones) [Viguier: para. 0007]; (the camera view angle off the perpendicular axis, such as for example, to place it on a lamp post along a road or at a street comer looking across the traffic area rather than down from overhead) [Viguier: para. 0004; Figs. 7, 10]; (The sensor unit 101 is preferably adapted to be mounted to a pole, wall or any similar shaped surface that allows the sensor unit 101 to overlook the intersection and provides an unobstructed view of the terrain to be monitored) [Viguier: para. 0026; Figs. 7-10]) using a mask segmentation model (A convolutional neural network trained to segment and identify pixels on a road) [Viguier: para. 0045; Figs. 7, 10]; second instructions stored in at least one non-transitory machine readable medium (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format) [Viguier: para. 0070] executable by at least one processor to (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] perform background extraction on the first image captured by a traffic sensor and the second image captured by a traffic sensor ((A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc.) [Viguier: para. 0045; Figs. 7, 10] ; (FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below) [Viguier: para. 0045]); third instructions stored in at least one non-transitory machine readable medium (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format) [Viguier: para. 0070] executable by at least one processor to (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] perform a homography transform on the first image and the second image (a homography transform is used to transform any pixel coordinate to the real-world coordinate. FIG. 7 is an exemplary illustration of an image taken by the sensor unit. In this illustration, the objects already have bounding boxes and two of the objects have a path. The bounding box 701 identifies the location of the object on the ground plane as discussed further below. FIG. 8 is an example of a homography transform where FIG. 7 is projected onto the ground plane.) [Viguier: para. 0045; Figs 7-8]; fourth instructions stored in at least one non-transitory machine readable medium (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format) [Viguier: para. 0070] executable by at least one processor to (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] estimate a translation of the traffic image sensor using the homography transform ((The detection module 601 uses a homography transform to translate the points where the object touches the ground and the bounding box into real world coordinates) [Viguier: para. 0056]; (The GPS position of the sensor unit or an arbitrary point in the sample image is used as the origin to translate the real-world coordinates previously obtained into latitude and longitude coordinates. In an exemplary embodiment, the GPS position and other metadata is stored in the Sensor Database 118 in the cloud computer. An exemplary SFM algorithm is dense multi-view reconstruction. In this example, every pixel in the image sensor's field of view is mapped to the real-world coordinate system. An additional exemplary SFM algorithm is a homography transform illustrated in FIG. 5. In this example, a plane is fit to tie points that are known to be on the ground. A convolutional neural network trained to segment and identify pixels on a road surface is used to distinguish between points that are on the ground and points associated with buildings, objects etc. Then a homography transform is used to transform any pixel coordinate to the real-world coordinate) [Viguier: para. 0043-0044; Figs 5-8]); fifth instructions stored in at least one non-transitory machine readable medium (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format) [Viguier: para. 0070] executable by at least one processor to (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] determine whether the translation of the traffic image sensor is over a threshold amount (If more than one object in the second intersection meet the matching criteria a similarity metric D (e.g. mean squared distance) is calculated for each object meeting the matching criteria in the second intersection. A matching object is selected from the plurality of objects in the second intersection, based on the similarity metric exceeding a predetermined threshold to merge with the first object) [Viguier: para. 0065]; and sixth instructions stored in at least one non-transitory machine readable medium (In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a non-transitory computer-readable storage media in a machine-readable format) [Viguier: para. 0070] executable by at least one processor to (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071], upon determining that the translation of the traffic image sensor is over the threshold amount (a feature point matching algorithm finds matching points between the image from the sensor and the image from the mobile device; the mobile device indicates if enough matching points have been found in excess of a predetermined threshold) [Viguier: claim 12], take an action to handle the translation of the traffic image sensor (If more than one object in the second intersection meet the matching criteria a similarity metric D (e.g. mean squared distance) is calculated for each object meeting the matching criteria in the second intersection. A matching object is selected from the plurality of objects in the second intersection, based on the similarity metric exceeding a predetermined threshold to merge with the first object. The object appearance information may be incorporated into the similarity metric and the predetermined threshold. This improves accuracy when object mergers are attempted at a third, fourth or subsequent intersection. If a plurality of matching objects have a similarity metric above the predetermined threshold, the object with the highest similarity metric is selected to merge with the first object. A high similarity metric is an indication that two objects are likely the same) [Viguier: para. 0065-0067].
Viguier does not explicitly disclose the following claim limitations (Emphasis added).
mask out vehicles in a first image and a second image.
However, in the same field of endeavor Sablak further discloses the deficient claim limitations as follows:
mask out vehicles in a first image and a second image ((FIG. 16 is a plan view of a motion mask derived from a sequential series of images acquired by the camera) [Sablak: col. 3, line 36-37; Fig. 16]; (The invention comprises, in yet another form thereof a method of operating a surveillance camera system, including acquiring images with a camera. A motion mask is created based upon the acquired images. A source of static motion is located within the acquired images. A virtual mask is defined over the source of static motion within the acquired images. The motion mask is modified by use of the virtual mask. A moving object of interest is tracked in the acquired images based upon the modified motion mask) [Sablak: col. 2, line 43-51]),
perform background extraction on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (A possible approach to masking static motion or "background motion" involves removing or deleting a large preselected area, that may possibly include static motion, from a calculated motion mask. The computer vision system may transform each mask on image frames from cameras, and may process each frame to remove static motion. Such an approach may remove a large portion of useful information in addition to removing static motion.) [Sablak: col. 15, line 7-14];
perform a homography transform on the first image captured by the traffic image sensor and the second image captured by the traffic image sensor (The calculation of the Rotational and homography matrices is used to transform the privacy mask to align it with the current image and may require the translation, scaling and
rotation of the mask) [Sablak: col. 10, line 52-55; Fig. 13a]; (After computation of the homography matrix M, the vertices of the current mask visible in the field of view are identified, as indicated at 158, and then the homography matrix is used to determine the transformed image coordinates of the mask vertices) [Sablak: col. 12, line 2-6; Fig. 13a];estimate a translation of the traffic image sensor using the homography transform ((Transformation of the mask for an image acquired at a different focal length than the focal length at which the mask was defined requires scaling and rotation of
the mask as well as translation of the mask to properly position the mask in the current image. Masks produced by such geometric operations are approximations of the original) [Sablak: col. 10, line 55-60]; (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010]);determine whether the translation of the traffic image sensor (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010] is over a threshold amount (A possible approach to masking static motion or "background motion" involves removing or deleting a large preselected area, that may possibly include static motion, from a calculated motion mask. The computer vision system may transform each mask on image frames from cameras, and may process each frame to remove static motion. Such an approach may remove a large portion of useful information in addition to removing static motion.) [Sablak: col. 15, line 7-14]; andupon determining that the translation of the traffic image sensor (The user captures additional data including image, position, orientation and similar data from the mobile device and produces a 3D structure from the additional data. The GPS position of the sensor or an arbitrary point is used as an origin to translate pixel coordinates into a position in real space) [Viguier: para. 0010] is over the threshold amount, take an action to handle the translation of the traffic image sensor (System 20 includes a camera 22 which is located within a partially spherical enclosure 24. Enclosure 24 is tinted to allow the camera to acquire images of the environment outside of enclosure 24 and simultaneously prevent individuals in the environment who are being observed by camera 22 from determining the orientation of camera 22. Camera 22 includes motors which provide for the panning, tilting and adjustment of the focal length of camera 22. Panning movement of camera 22 is represented by arrow 26, tilting movement of camera 22 is represented by arrow 28 and the changing of the focal length of the lens 23 of camera 22, i.e., zooming, is represented by arrow 30. As shown with reference to coordinate system 21, panning motion corresponds to movement along the x-axis, tilting motion corresponds to movement along the y-axis and focal length adjustment corresponds to movement along the z-axis) [Sablak: col. 1, line 50-65].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier with Sablak to program the system to implement of Sablak’s method.
Therefore, the combination of Viguier with Sablak will enable the system to increase the robustness of an auto-tracker system [Sablak: col. 2, line 52-62].
Regarding claim 20, Viguier meets the claim limitations as set forth in claim 19. Viguier further meets the claim limitations as follow.
wherein the action to handle of the traffic image sensor comprises transmitting an instruction to move the traffic image sensor (Upon receiving the positive feedback, in step 207 the calibration application preferably prompts the user to move the phone in a slow sweeping motion, keeping the camera oriented toward the sensor unit field of view (e.g., intersection). The sweeping process is illustrated in FIG. 3. The installer/user with the mobile device takes the first image and the calibration application identifies the tie points 303 that match with the sample image 302. The user then sweeps the mobile device through N mobile device positions.) [Viguier: para. 0002; Figs. 2-3].
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Viguier (US Patent Application Publication 2022/0189039 A1), (“Viguier”), in view of Sablak et al. (US Patent 9,210,312 B2), (“Sablak”), in view of Sharma et al. (US Patent 11,599,744 B2), (“Sharma”).
Regarding claim 8, Viguier meets the claim limitations as set forth in claim 1. Viguier and Sablak do not teach the claim limitations as follow.
comprises flagging data associated with the traffic image sensor as not to be used.
However, in the same field of endeavor Sharma further discloses the deficient claim limitations as follows:
comprises flagging data associated with the traffic image sensor as not to be used (to flag the intermediate data as suspect, so that it is omitted when re-training the network.) [Romanik: col. 35, line 12-13].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier and Sablak with Sharma to program the system to implement of Sharma’s method.
Therefore, the combination of Viguier and Sablak with Sharma will enable the system improving image recognition technologies to mitigate the accuracy failings of the prior art [Sharma: col. 1, line 65-66].
Regarding claim 9, Viguier meets the claim limitations as set forth in claim 8. Viguier and Sablak further disclose the claim limitations as follow.
wherein the at least one processor further executes the instructions (The computing device that executes some or all of the stored instructions) [Viguier: para. 0071] to unflag the data upon estimating that the translation of the traffic image sensor is reversed (In order to enable processing device 50 to track person 202 with little or no regard for the static motion of flag 200, the user may define a virtual mask 204 to "cover" the static motion of flag 200. That is, areas of the acquired image that are within virtual mask 204 include the source of static motion 200. The user may define virtual mask 204 by drawing a visual representation of virtual mask 204 on screen 38. In one embodiment, the user selects vertices A, B, C, D of mask 204 on screen 38 such as by use of joystick 36 or a computer mouse (not shown). After the user has selected vertices A-D, processing device 50 may add to the display visible boundary lines 206 which join adjacent pairs of the vertices) [Sablak: col. 21, line 51-62; Figs. 14-17].
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Viguier (US Patent Application Publication 2022/0189039 A1), (“Viguier”), in view of Sablak et al. (US Patent 9,210,312 B2), (“Sablak”), in view of Riaz et al. (US Patent 11,599,744 B2), (“Riaz”).
Regarding claim 12, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
wherein the action to handle the translation of the traffic image sensor comprises compensating for the translating of the traffic image sensor when using data associated with the traffic image sensor for analysis (the monitoring and analysis of traffic patterns of people, vehicles) [Viguier: para. 0003].
Viguier does not explicitly disclose the following claim limitations (Emphasis added).
wherein the action to handle the translation of the traffic image sensor comprises compensating for the translating.
However, in the same field of endeavor Riaz further discloses the deficient claim limitations as follows:
wherein the action to handle the translation of the traffic image sensor comprises compensating for the translating ((compensating for the global motion of the background in the second image using said item of information representing the global motion of the background in order to obtain an adjusted version of the second image, referred to as the adjusted second image) [Riaz: col. 2, line 38-42; Fig. 4]; (motion compensation means for compensating for the global motion of the background in the second image using said item of information representing the global motion of the background in order to obtain an adjusted version of the second image, referred to as the adjusted second image) [Riaz: col. 4, line 49-54; col. 7, line 62-65]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier and Sablak with Riaz to program the system to implement of Riaz’s method.
Therefore, the combination of Viguier and Sablak with Riaz will enable the system to improving a quality of image restoration [Riaz: col. 1, line 10-11].
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Viguier (US Patent Application Publication 2022/0189039 A1), (“Viguier”), in view of Sablak et al. (US Patent 9,210,312 B2), (“Sablak”), in view of Romanik et al. (US Patent 10,210,427 B2), (“Romanik”).
Regarding claim 15, Viguier meets the claim limitations as set forth in claim 1. Viguier further meets the claim limitations as follow.
wherein the threshold amount (the mobile device indicates if enough matching points have been found in excess of a predetermined threshold) [Viguier: claim 12] is at least one of: a number of feet; or a number of pixels.
Viguier and Sablak do not explicitly disclose the following claim limitations (Emphasis added).
wherein the threshold amount is at least one of: a number of feet; or a number of pixels.
However, in the same field of endeavor Romanik further discloses the deficient claim limitations as follows:
wherein the threshold amount is at least one of: a number of feet; or a number of pixels ((Accordingly, in some aspects hereof, only the best matching
feature points between the template image and a test image are used to establish the coordinate transformation between the two images. An initial homography is determined using a large reprojection error threshold, e.g., using the RANSAC algorithm. If a sufficient number of matching points (e.g., 2:6 points) are found between the template image and test image, additional homography estimates are
computed using a smaller reprojection error threshold. When fewer points are used as input, a correct homography can often be computed that has a smaller reprojection error when computed with more points. One advantage of using a more accurate homography to describe the relationship between a template image and test image is to compute additional statistics on the match, such as a correlation coefficient to describe the similarity between the two matching
regions.) [Romanik: col. 9, line 61 – col. 10, line 10]; (The feature point identifies the same object in both images, and matches the predicted location of the feature points closely. If a test image contains a pristine copy of the template image, the error in location can be expressed as fractions of a pixel. When the test image is less than pristine (e .g., highly skewed or distorted), the location error can be 1-3 pixels) [Romanik: col. 2, line 43-51]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Viguier and Sablak with Romanik to program the system to implement of Romanik’s method.
Therefore, the combination of Viguier and Sablak with Romanik will enable the system to improve the speed and/or accuracy of object recognition/identification/location image processing techniques [Romanik: col. 1, line 33-34].
Reference Notice
Additional prior arts, included in the Notice of Reference Cited, made of record and not relied upon is considered pertinent to applicant's disclosure.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Philip Dang whose telephone number is (408) 918-7529. The examiner can normally be reached on Monday-Thursday between 8:30 am - 5:00 pm (PST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached on 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Philip P. Dang/Primary Examiner, Art Unit 2488