DETAILED ACTION
The present Office action is in response to the amendments filed on 12 FEBRUARY 2026 and the Information Disclosure statements.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The Information Disclosure Statement (IDS) submitted on 04/13/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the Information Disclosure Statement is being considered by the Examiner.
Response to Amendment
Claims 1-11 and 13-14 have been amended. No claims have been added or cancelled. The claim amendments address the claim objections and 35 U.S.C. § 112 rejection. Therefore, the claim objections and 35 U.S.C. § 112 rejection are withdrawn. The title of the invention has been amended to be more descriptive. Therefore, the specification object is withdrawn. Claims 1-15 are pending and herein examined.
Response to Arguments
Applicant's arguments filed 13 FEBRUARY 2026 have been fully considered but they are not persuasive.
With regard to claim 1, rejected under 35 U.S.C. § 103 as being unpatentable over U.S. Publication No. 2017/0177947 A1 (hereinafter “Citerin”) in view of U.S. Publication No. 2018/0098082 A1 (hereinafter “Burns”), Applicant alleges:
“Accordingly, the comparison performed in Citerin is between target paths captured by more than one camera. But, Citerin is silent to using an event driven sensor as a secondary image sensor or anywhere in the system. Accordingly, Citerin fails to disclose detection of a target by an image sensor and subsequent tracking of the detection target by second image sensor (e.g., an event driven vision sensor). Citerin also fails to teach cross-modal comparison, such as comparison between position information derived from a first image sensor and from an event driven vision sensor.” (Remarks, p. 3.)
The Examiner respectfully disagrees, because the broadest reasonable interpretation of “an event driven sensor” is any sensor capable of determining some functionality based on the captured information (i.e., “event driven”). Citerin discloses a plurality of cameras (i.e., sensors) that are capable of keeping track of detected objects (i.e., event driven). See Citerin, ¶¶ [0070-0073]. Furthermore, Citerin discloses the use of position information between the two sensors for continuous tracking, thereby teaching the comparison limitation. See Citerin, FIGS. 3a-3e and ¶ [0121]. The rejection does not rely on Citerin for teaching one of the mono camera trackers is an event driven sensor because of the “trigger” for driving an event being “asynchronously generates an image signal when an intensity change in light incident to each pixel is detected.” However, as per the rejection, Burns is relied upon for disclosing the limitation.
“Burns fails to cure the deficiencies of Citerin noted above. For example, although Burns describes motion estimation using hybrid video imaging based on frame-based capture and event-based capture, Burns fails to teach or suggest using a first image sensor to detect a target and then performing event driven tracking of the target. For example, Burns discloses that "motion vectors are calculated from asynchronously generated pixel motion events provided by the event-based camera" and that "these pixel motion vectors are then mapped to tiled regions of the image frames provided by the frame-based camera to estimate motion of the tiles from frame to frame or between frames." See Burns, para. [0012]. Accordingly, in contrast to detecting a detection target based on an image sensor signal and tracking the detection target based on a second image sensor signal from an event driven vision sensor, Burns describes using event-based data for calculating motion vectors and then mapping the motion vectors to images frames. Burns is further silent to comparing position information for a detected target using two different sensor modalities (image sensor and event driven sensor).” (Remarks, p. 3.)
The Examiner respectfully disagrees, because the limitations in question are rejected in view of Citerin and not Burns. Furthermore, Burns does disclose “comparing position information for a detected target using two different sensor modalities.” See Burns, ¶ [0030], “The mapping is based on calibration process that correlates the locations of pixels in the 2-dimensional sensor planes of the event-based camera and the frame-based camera.” The mapping in Burns occurs in response to a detected event. See Burns, ¶ [0025]. Therefore, the rejection is maintained.
The Remarks highlights the importance of comparing two different types of sensors with some modal differences. As currently recited, the claims do not accurately reflect such modal differences. For instance, the first image sensor can also be understood to be an event driven sensor, where the event is the detection of a target. Under such an interpretation there is no modal difference. For these reasons and those explained above, the rejection is maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3 and 5-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2017/0177947 A1 (hereinafter “Citerin”) in view of U.S. Publication No. 2018/0098082 A1 (hereinafter “Burns”).
Regarding claim 1, Citerin discloses an information processing device (FIG. 2, multi-mono tracker 120 with first and second camera 200-1 and 200-2, respectively) comprising:
a computer ([0184], “computing device”) configured to:
detect a detection target on a basis of a first image signal generated by a first image sensor ([0070], “modules 105-1 and 105-n are tracking modules based on a tracking-by-detection algorithm.” FIG. 3a depicts two targets detected. [0102], “targets 300-1 and 300-2 are identified in the images obtained from camera #1.” FIG. 2, first camera 200-1 with module 105-1));
set a region of interest including at least a part of the detection target ([0071], “Detection modules 110-1 and 110-n may deliver bounding boxes (e.g. x-coordinate, y-coordinate, width, and height) and a corresponding detection score representing a level of confidence associated with each detected object or person, generically referred to as a target in the following, in each image.” FIG. 2, detection module 110-1);
track the detection target in the region of interest on a basis of a second image signal generated by a second image sensor ([0072], “The results obtained by the detection modules are used by mono-camera tracking modules 115-1 and 115-n in order to generate target paths with consistent identity labels by relying on object detection in successive images,” [0073], “In turn, the target paths generated in mono-camera tracking modules 115-1 and 115-n are fed into tracking module 120 to be combined in order to perform a persistent tracking, that is to say to track targets over several cameras.” FIG. 2, tracking 115-1, 115-2, multi-mono tracker 120)
compare position information on the detection target represented by a result of the detection on the basis of the first image signal with position information on the detection target represented by a result of the tracking on the basis of the second image signal associated with the first image signal ([0121], “comparing a portion of a target path as generated by a mono-camera tracker of a first camera with portions of target paths as generated by mono-camera trackers of other cameras.” Note, the target paths represent positional data of the tracked targets, as illustrated by the positional data in FIGS. 3a-3e).
Citerin fails to expressly disclose a second sensor including an event driven vision sensor that asynchronously generates an image signal when an intensity change in light incident to each pixel is detected.
However, Burns teaches a second sensor including an event driven vision sensor (FIG. 1, event-based video camera 102) that asynchronously generates an image signal when an intensity change in light incident to each pixel is detected ([0025], “The event signal processor 302 is configured to receive a sequence of pixel events, generated asynchronously by an event-based video camera 102. The events, which represent illumination change in a pixel, are captured asynchronously with relatively high temporal resolution. Events may include an address of the location of the pixel that generated the event and a timestamp, along with any other suitable data. The amount of illumination change required to trigger an event may be fixed or adjustable over a suitable range of values depending, for example, on the application and processing requirements”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to have used an asynchronous even-based video-camera, as taught by Burns ([0025]), in Citerin’s invention. One would have been motivated to modify Citerin’s invention, by incorporating Burns’ invention, to computer motion estimation of an event with greater temporal resolution and increased efficiency (Burns: [0012]).
Regarding claim 2, Citerin and Burns disclose every limitation of claim 1, as outlined above. Additionally, Citerin discloses wherein the computer is further configured to set the region of interest again when a difference is more than a predetermined threshold value on a basis of a result of the comparison of the position information on the detection target represented by the result of the detection on the basis of the first image signal with the position information on the detection target represented by the result of the tracking on the basis of the second image signal associated with the first image signal ([0071], “Detection modules 110-1 and 110-n may deliver bounding boxes (e.g. x-coordinate, y-coordinate, width, and height) and a corresponding detection score representing a level of confidence associated with each detected object or person, generically referred to as a target in the following, in each image.” [0073], “an error that is generated by a mono-camera tracking module such as a switch of identity between two targets due to an occlusion can be recovered by another tracking module.” Note, the mono-camera tracking module detects occlusions and corrects them in the tracking module of the camera with the occlusion to correct for detection and the detection process provides a confidence score. As per Burns, it is well-known for detection to trigger when pixel values are more than a threshold, see Burns, [0023], which is well-known to use as per the rationale of claim 1).
Regarding claim 3, Citerin and Burns disclose every limitation of claim 1, as outlined above. Additionally, Citerin discloses wherein the computer is further configured to correct the result of the tracking the detection target on a basis of a result of the comparison of the position information on the detection target represented by the result of the detection on the basis of the first image signal with the position information on the detection target represented by the result of the tracking on the basis of the second image signal associated with the first image signal ([0073], “the target paths generated in mono-camera tracking modules 115-1 and 115-n are fed into tracking module 120 to be combined in order to perform a persistent tracking, that is to say to track targets over several cameras and to correct errors of the individual mono-camera tracking modules.” [0073], “an error that is generated by a mono-camera tracking module such as a switch of identity between two targets due to an occlusion can be recovered by another tracking module.” Note, the occlusion happens at specific locations where tracking should be occurring).
Claims 7, 10, and 13 are independent claims with different statutory categories from claim 1. Claims 5, 8, 11, and 14 are independent claims equivalent to 1, 7, 10, and 13 without the comparison limitation, which is present in the dependent claims 6, 9, 12, and 15, respectively.
Regarding claim 5, the limitations are the same as those in claim 1. Therefore, the same rationale of 1 applies equally as well to claim 5.
Regarding claim 6, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 6.
Regarding claim 7, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 7.
Regarding claim 8, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 8.
Regarding claim 9, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 9.
Regarding claim 10, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 10.
Regarding claim 11, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 11.
Regarding claim 12, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 12.
Regarding claim 13, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 13.
Regarding claim 14, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 14.
Regarding claim 15, the limitations are the same as those in claim 1. Therefore, the same rationale of claim 1 applies equally as well to claim 15.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2017/0177947 A1 (hereinafter “Citerin”) in view of U.S. Publication No. 2018/0098082 A1 (hereinafter “Burns”), and further in view of U.S. Publication No. 2020/0074165 A1 (hereinafter “Ghafoor”).
Regarding claim 4, Citerin and Burns disclose every limitation of claim 1, as outlined above. Additionally, Citerin discloses wherein the detection target is a person ([0071], “detected object or person”), and
Wherein the computer is further configured to calculate coordinate information ([0081], “the tracking modules may be converted into a two-dimensional (2D) real world coordinate system”)
.
Citerin and Burns fail to expressly disclose coordinate information on at least one joint of the person, and
to set the region of interest for each joint of the person.
However, Ghafoor teaches coordinate information on at least one joint of the person ([0088], “The joints may be provided as a set of two-dimensional joint locations on the image; i.e. each candidate joint may be provided as an image coordinate. As used herein, a ‘joint’ as it relates to a candidate joint refers to a point of interest on the body. Though a joint may correspond to a point of the body that is anatomically considered a joint (e.g. the knee, elbow, ankle etc.), it need not necessarily do so. For example, a candidate joint could include: foot, ankle, knee, hip, torso, shoulder, elbow, wrist, neck or head”), and
to set the region of interest for each joint of the person ([0111], “the regions of interest may be defined so as to bound the distal ends of the arms of a derived pose estimate (i.e., distal joints of the pose estimate), for example the hands and elbows of each derived pose estimate”).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to have identified joint information of a person, as taught by Ghafoor ([0111]), in Citerin and Burns’ invention. One would have been motivated to modify Citerin and Burns’ invention, by incorporating Ghafoor’s invention, to improve detection and activity recognition of humans (Ghafoor: [0004]) and because it is an obvious use of known techniques for human detection to improve similar devices for human detection in the same way (MPEP § 2143(I)(C)).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STUART D BENNETT whose telephone number is (571)272-0677. The examiner can normally be reached Monday - Friday from 9:00 AM - 5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at 571-272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STUART D BENNETT/Examiner, Art Unit 2481
Prosecution Insights