DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant’s response to the Non-final Office Action dated 07/02/2025, filed with the office on 09/30/2025, has been entered and made of record.
Response to Amendment
5. In light of Applicant’s amendment of the claims, the 35 U.S.C. 112(b) rejections of record with respect to the claims have been withdrawn.
Status of Claims
Claims 1-14 are pending. Claims 1-5 and 8-13 are amended.
Response to Arguments
Applicant’s amendment of independent Claims 1, 13 and 14, which has altered the scope of the claims of the instant application, has necessitated the new ground(s) of rejection presented in this office action with respect to claims of the instant application. Accordingly, in response to Applicant’s arguments that are merely directed to the amended portion of the claims, new analyses have been presented below, which make Applicant’s arguments moot.
Consequently, THIS ACTION IS MADE FINAL.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 3 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Specifically, claim 3 recites “the visual feature matching is distributed across the first camera and at least one other camera”. Amended claim 1 recites “the visual feature matching is performed at the first camera”. It is not clear whether the visual feature matching is performed only at the first camera or the task is distributed across multiple cameras.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-14 are rejected under 35 U.S.C. 103 as being unpatentable over Mirza et al. (US 2021/0124953 A1) in view of Lamb et al. (US 2011/0228098 A1) and in further view of Rantala et al. (US 2018/0007245 A1).
Regarding claim 1, Mirza teaches, An apparatus comprising: at least one processor; at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: (Mirza, ¶0479: “The tracking subsystem 3910 may execute the instructions 4004, 4010, 4012, and 4016 described in FIG. 2 (e.g., using one or more processors as described with respect to the device”) use (Mirza, ¶0024: “each camera needs to be processed independently to identify and track people and objects within the field of view of a particular camera”) use an expected location of the first object in a second field of view of a second camera (Mirza, ¶0209: “tracking system 100 stores pixel location information 908 that comprises expected pixel locations 402 within the second frame 302B of the second sensor”) to identify the detected object in the second field of view as the first object, (Mirza, ¶0222: “the tracking system 100 determines the object is within the overlap region 1110 between the first sensor 108 and the second sensor 108”) wherein the second camera is different to the first camera and the second field of view is different to the first field of view, (Mirza, ¶0302: “fields-of-view of adjacent sensors 108 may overlap between about 10% to 30%. As such, the same object may be detected by two different sensors”). However, Mirza does not explicitly teach visual feature matching for a detected object and wherein the visual feature matching is performed at the first camera, and wherein the expected location of the first object is shared camera-to-camera from the first camera to the second camera.
In an analogous field of endeavor, Lamb teaches, visual feature matching for a detected object (Lamb, ¶0039: “a feature recognition process that recognizes a face… particular visual feature of the tracked object”) wherein the visual feature matching is performed at the first camera, (Lamb, ¶0039: “imager 116 employs a feature recognition process that recognizes a face, for example”) and wherein the expected location of the first object (Lamb, ¶0039: “visual feature of the tracked object 103 that is useful to refine determination of the position of the object”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Mirza using the teachings of Lamb to introduce a visual feature recognition. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of identifying an object within the field of view of the camera. Therefore, it would have been obvious to combine the analogous arts Mirza and Lamb to obtain the above-described limitations of claim 1. However, the combination of Mirza and Lamb does not explicitly teach, (data) is shared camera-to-camera from the first camera to the second camera.
In another analogous field of endeavor, Rantala teaches, (data) is shared camera-to-camera (Rantala, ¶0026: “two or more cameras 10 to be coupled to each other via the nubs 30… camera-to-camera communication”) from the first camera to the second camera. (Rantala, ¶0025: “nubs 30 that operate as physical connection ports, power delivery ports, and/or data transfer ports between cameras 10”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Mirza in view of Lamb using the teachings of Rantala to introduce camera-to-camera communication. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of efficient communication between the cameras without the need of a server. Therefore, it would have been obvious to combine the analogous arts Mirza, Lamb and Rantala to obtain the invention in claim 1.
Regarding claim 2, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the apparatus is further caused to track the identified object across fields of view of different cameras of a multiple camera system (Mirza, ¶0025: “a process for handing off tracking information for an object as the object moves from the field of view of one sensor to the field of view of another sensor”) by using the expected location of the first object in the fields of view of the different cameras. (Mirza, ¶0213: “the tracking system 100 can continue comparing the current pixel locations 402 for the shelf markers 906 for the first sensor 108 and the second sensor 108 with the new expected pixel locations 402”).
Regarding claim 3, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the visual feature matching is distributed across the first camera and at least one other camera of multiple cameras. (Mirza, ¶0171: “the tracking system 100 may use multiple sensors 108 to determine the location of the object”).
Regarding claim 4, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the apparatus is further caused to, at the second camera, use the expected location (Mirza, ¶0206: “the tracking system 100 stores pixel location information 908 that comprises expected pixel locations 402 within the second frame 302B of the second sensor”) of the first object in the second field of view of the second camera to identify the detected object in the second field of view as the first object. (Mirza, ¶0171: “tracking system 100 may then apply a second homography 118 that is associated the second sensor 108 to the second pixel location 402 of the object to determine a second (x,y) coordinate 306 that identifies a second x-value and a second y-value in the global plane 104 where the object is located”).
Regarding claim 5, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the first camera is further caused to; provide to other ones of multiple cameras an indication (Mirza, ¶0148: “the tracking system 100 may comprise a first server 106 that is in signal communication with a plurality of sensors 108”) of a location of the detected object in the first field of view of the first camera (Mirza, ¶0150: “the server 106 periodically compares the current shelf location of a rack 112 to an expected shelf location for the rack 112 using a sensor 108”) and an indication of the identity of the detected object as the first object. (Mirza, ¶0224: “Adding the object identifier 1118 for the first person 1106 to the second tracking list 1112B indicates that the first person 1106 is within the field of view of the second sensor 108 and allows the tracking system 100 to begin tracking the first person 1106 using the second sensor 108”).
Regarding claim 6, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 5, wherein the indication of the location of the detected object in the first field of view of the first camera is provided as an indication of a bounding box location. (Mirza, ¶0342: “the tacking subsystem 2400) may determine a first global position 2438 based on the determined pixel positions 2412c and 2416c (e.g., corresponding to pixel coordinates 2412a, 2416a and bounding boxes 2412b, 2416b”).
Regarding claim 7, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 5, wherein the indication of the location of the detected object in the first field of view of the first camera is provided to a selected sub-set of the multiple cameras (Mirza, ¶0148: “the tracking system 100 may comprise a first server 106 that is in signal communication with a first plurality of sensors 108 in a sensor array”) based on the location of the detected object in the first field of view (Mirza, ¶0142: “determine where a person is physically located in the space 102 based on their pixel location 402 within a frame 302… multiple sensors 108 to monitor the entire space 102, each sensor 108 is uniquely associated with a different homography 118 based on the sensor's 108 physical location within the space”) and a spatial relationship of overlapping fields of view of the other ones of the multiple cameras. (Mirza, ¶0142: “the sensor array is configured such that adjacent sensors 108 have at least partially overlapping fields of view”).
Regarding claim 8, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 7, wherein the apparatus is further caused to select the sub-set of the multiple cameras based on the expected location of the first object (Mirza, ¶0560: “The tracking system 100 uses pixel locations 5710 of a plurality of markers 5708 with the corresponding physical locations of the markers 5708 within the global plane 104 to generate a homography 118 for the sensor 108”) relative to the fields of views of the multiple cameras wherein the expected location of the first object lies within the fields of view of the cameras in the sub-set (Mirza, ¶0551: “position a plurality of sensor assemblies 5602 near a ceiling of the store. The adjustable positioning system 5600 allows the sensor assemblies 5602 to be distributed to such that these sensor assemblies 5602 are able to collectively provide coverage for the store”) and wherein the expected location of the first object lies outside the fields of view of the cameras not in first sub-set. (Mirza, ¶0262: “the tracking system 100 can ignore the second person 1826 because the pixel location 402B of the second person 1826 is outside of the predetermined zone 180”).
Regarding claim 9, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the apparatus is further caused to select a sub-set of a field of view of a camera for object detection based on the expected location of the first object in the field of view. (Mirza, ¶0218: “object identifier 1118 associated with the first person 1106 as well as pixel locations 402 in the sensors 108 where the first person 1106 appears in a tracking list… the tracking system 100 may track the people within the field of view of a first sensor 108 using a first tracking list”).
Regarding claim 10, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the apparatus is further caused to determine the expected location of the first object in the second field of view of the second camera, (Mirza, ¶0209: “tracking system 100 stores pixel location information 908 that comprises expected pixel locations 402 within the second frame 302B of the second sensor”) wherein the second field of view is constrained to be simultaneous with or contemporaneous with the first field of view of the first camera (Mirza, ¶0224: “Adding the object identifier 1118 for the first person 1106 to the second tracking list 1112B indicates that the first person 1106 is within the field of view of the second sensor 108 and allows the tracking system 100 to begin tracking the first person 1106 using the second sensor 108”) or (The strikethrough portion is not examined as alternative limitation is being considered).
Regarding claim 11, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the first field of view partially overlaps the second field of view at a first overlapping field of view, (Mirza, ¶0302: “fields-of-view of adjacent sensors 108 may overlap between about 10% to 30%. As such, the same object may be detected by two different sensors”) and at least a third camera in a multiple camera system has a third field of view that partially overlaps the second field of view at a second overlapping field of view, but does not overlap the first overlapping field of view, (Mirza, ¶0339: “the adjacent fields-of-view 2404a-c may overlap by between about 10% and 30%”; also see Fig. 24A) wherein the apparatus is further caused to identify the detected object in the first field of view, outside the first overlapping field of view, as the first object; (Mirza, Fig, 24A) detect when the detected object, identified as the first object, in the first field of view enters the first overlapping field of view, and consequently identifying a corresponding detected object in the second field of view, inside the overlapping field of view, as the first object; (Mirza, ¶0028: “the tracking system is configured to hand off tracking information for an object (e.g. a person) as it moves between the field of views of adjacent sensors”) detect when the detected object, identified as the first object, in the second field of view enters (Mirza, Fig. 24A) the second overlapping field of view, (Mirza, ¶0339: “the adjacent fields-of-view 2404a-c may overlap by between about 10% and 30%”; also see Fig. 24A) and consequently identifying a corresponding detected object in the third field of view, (Mirza, Fig. 24A: Wherein the subject is 2402, the first, second, and third field of views are 108a, 108b and 108c respectively) inside the second overlapping field of view, as the first object. (Mirza, ¶0339: “the adjacent fields-of-view 2404a-c may overlap by between about 10% and 30%”; also see Fig. 24A).
Regarding claim 12, Mirza in view of Lamb, and in further view of Rantala teaches, An apparatus as claimed in claim 1, wherein the apparatus is further caused to: match visual features for a target object (Lamb, ¶0039: “a feature recognition process that recognizes a face… particular visual feature of the tracked object”) in the fields of view of multiple cameras to identify the target object in one or more field of view of the multiple cameras; (Mirza, ¶0221: “compare the features of a person in the first frame 302A to the features associated with the first person 1106 to determine if the person is the first person”) create a sequence of cameras (Mirza, ¶0142: “the sensor array is configured such that adjacent sensors 108 have at least partially overlapping fields of view”) and performing the visual feature matching for the target object in the fields of view of the cameras in the order of the created sequence to identify the target object in one or more field of view of the multiple cameras in the created sequence; (Mirza, ¶0142: “determine where a person is physically located in the space 102 based on their pixel location 402 within a frame 302… multiple sensors 108 to monitor the entire space 102, each sensor 108 is uniquely associated with a different homography 118 based on the sensor's 108 physical location within the space”) and/or create a sequence of detected objects and (Mirza, ¶0369: “View 2702 at initial time to includes a first tracked object region 2704, a second tracked object region 2708, and a third tracked object region 2712”) performing visual feature matching for the target object for the detected objects (Mirza, ¶0368: “people may be tracked in the space 102 and regions 2704, 2708, 2712 may be determined using any appropriate tracking and identification method”) in the order of the created sequence to identify the target object. (Mirza, ¶0369: “the identities of all of the people are generally known (e.g., because the people have recently entered the space 102”).
The proposed combination as well as the motivation for combining Mirza, Lamb and Rantala references presented in the rejection of claim 1, apply to claim 12 and are incorporated herein by reference. Thus, the apparatus recited in claim 12 is met by Mirza, Lamb and Rantala.
Regarding claim 13, it recites a non-transitory computer readable medium including program instructions corresponding to the elements of the apparatus recited in claim 1. Therefore, the recited instructions of the non-transitory computer readable medium of claim 13 are mapped to the proposed combination in the same manner as the corresponding elements of the apparatus claim 1. Additionally, the rationale and motivation to combine Mirza, Lamb and Rantala presented in rejection of claim 1, apply to this claim. In addition, Mirza teaches, A non-transitory computer readable medium comprising program instructions that, when executed by an apparatus, cause the apparatus to perform at least the following: (Mirza, Claim 15: “A computer program comprising executable instructions stored in a non-transitory computer readable medium that when executed by a processor causes the processor to:”).
Regarding claim 14, it recites a method with steps corresponding to the elements of the apparatus recited in claim 1. Therefore, the recited steps of the method claim 14 are mapped to the proposed combination in the same manner as the corresponding elements of the apparatus claim 1. Additionally, the rationale and motivation to combine Mirza, Lamb and Rantala presented in rejection of claim 1, apply to this claim. In addition, Mirza teaches, A method comprising: (Mirza, ¶0157: “tracking system 100 may employ method 200”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEHRAZUL ISLAM whose telephone number is (571)270-0489. The examiner can normally be reached Monday-Friday: 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Saini Amandeep can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEHRAZUL ISLAM/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662