DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) filed on 9/10/2024 was considered and placed on the file of record by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The following highlighted claim elements are vague and indefinite:
“(d) comparing each tracklet created in step (d) to other tracklets.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (US 2006/0098865) in view of Socek et al. (US 2017/0339409).
Regarding claim 1, Yang teaches a method for isolating an individual's image from an image frame, the method comprising the steps of:
(a) recognizing at least a part of said individual, based on predetermined image pattern data (see para. 0053, Yang discusses face rectangle bounding box);
(b) defining a rectangle around said part; (see para. 0053, Yang discusses face rectangle bounding box). However, Yang does not expressly disclose (c) cropping said image frame to said rectangle. However, Socek teaches (c) cropping said image frame to said rectangle (see figure 12E, figure 13C, figure 17C, para. 0053, Socek discusses cropping a face region from an image based on the rectangle bounding box).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Yang with Socek to derive at the invention of claim 1. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Yang in this manner in order to improve object tracking by extracting features to properly identify and combine corresponding matched trajectory tracks. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Yang, while the teaching of Socek continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of extracting features to identify matching tracks of objects captured in images. The Yang and Socek systems perform object tracking, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 2, Yang teaches wherein said part is at least one of: - a head; - a head and shoulders; and - a torso (see para. 0053, Yang discusses face rectangle bounding box).
The same motivation of claim 1 is applied to claim 2. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Yang with Socek to derive at the invention of claim 2. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Regarding claim 3, Yang teaches further comprising the step of generating a descriptor for each cropped image resulting from step (c) (see para. 0054, Yang discusses detecting color in the bounding box and performing color normalization on the detected skin).
The same motivation of claim 1 is applied to claim 3. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Yang with Socek to derive at the invention of claim 3. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Regarding claim 4, Yang teaches further comprising the step of colour-equalization before said descriptor is generated (see para. 0054, Yang discusses performing color normalization on the detected skin).
The same motivation of claim 1 is applied to claim 4. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Yang with Socek to derive at the invention of claim 4. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 5, 8, 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Shin et al. (US 11,354,683).
Regarding claim 5, Shin teaches a method for tracking an individual between multiple image frames, the method comprising the steps of:
(a) receiving a dataset based on each image frame, said dataset including a descriptor (see figure 3, col. 6 lines 5-30, Shin discusses a shopper dataset containing shopper descriptors);
(b) comparing said dataset with each of a plurality of other datasets (see figure 3, claim 14, claim 19, col. 10 lines 19-31, Shin discusses comparing a shopper dataset with other shopper datasets);
(c) adding said dataset to a tracklet based on the results of comparing in step (b ), wherein each tracklet comprises at least one descriptor (see claim 19, col. 10 lines 27-31, Shin discusses creating a new track trajectory with a dataset based on the comparison result);
(d) comparing each tracklet created in step (d) to other tracklets (see col. 10 lines 27-31, Shin discusses creating a new track trajectory with a dataset based on the comparison result); and
(e) merging each tracklet with a customer set, based on the results of comparing in step (d), wherein each customer set comprises at least one tracklet (see claim 1, Shin discusses merging a track trajectories by matching track trajectories for detected customers).
Regarding claim 8, Shin teaches further comprising the step of determining tracking-related metadata for each customer set (see figure 3, claim 14, claim 19, col. 10 lines 19-31, Shin discusses comparing a shopper dataset with other shopper datasets).
Claim 20 is rejected as applied to claim 1 as pertaining to a corresponding non-transitory computer-readable media.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 6, 9 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 11,354,683) in view of Sharma et al. (US 8,706,544).
Regarding claim 6, Shin does not expressly disclose wherein similarity scores are calculated in step (b). However, Sharma teaches wherein similarity scores are calculated in step (b) (see col. 14 lines 16-50, Sharma discusses similarity scores calculated).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin with Sharma to derive at the invention of claim 6. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Shin in this manner in order to improve object tracking by extracting features, timestamps, location, and calculating similarity scores to properly identify and combine corresponding matched trajectory tracks. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Shin, while the teaching of Sharma continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of extracting features, timestamps, location, and calculating similarity scores to identify matching tracks of objects captured in images. The Shin and Sharma systems perform object tracking, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 9, Sharma teaches wherein step (b) comprises calculating a similarity score between said dataset and each of said plurality of other datasets, producing a plurality of similarity scores (see col. 14 lines 1-50, Sharma discusses similarity scores calculated).
The same motivation of claim 6 is applied to claim 9. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin with Sharma to derive at the invention of claim 9. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Claims 7, 10 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 11,354,683) in view of Stephen et al. (US 2011/0169917).
Regarding claim 7, Shin does not expressly disclose wherein tracklet similarity scores are calculated in step (d). However, Stephen teaches wherein tracklet similarity scores are calculated in step (d) (see para. 0209, Stephen discusses matching track records based on similarity scoring).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin with Stephen to derive at the invention of claim 7. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Shin in this manner in order to improve object tracking by extracting features, timestamps, location, and calculating similarity scores to properly identify and combine corresponding matched trajectory tracks. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Shin, while the teaching of Stephen continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of extracting features, timestamps, location, and calculating similarity scores to identify matching tracks of objects captured in images. The Shin and Stephen systems perform object tracking, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 10, Stephen teaches wherein step (d) comprises calculating a tracklet similarity between a pair of tracklets, said method further comprising a step of merging said pair of tracklets to thereby produce a customer set when said tracklet similarity meets a threshold (see para. 0218, Stephen discusses a similarity threshold for matching track records).
The same motivation of claim 6 is applied to claim 10. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin with Stephen to derive at the invention of claim 10. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Claims 11, 12 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 11,354,683) in view of Sharma et al. (US 8,706,544) in view of Boghossian et al. (US 2013/0208124).
Regarding claim 11, Sharma discusses a geometric match score including position, size and time between captured faces (see col 14 lines 16-18). Boghossian teaches wherein said dataset comprises said at least one descriptor, location information for said image, and time information for said image frame (see claim 3, para. 0073, 0107-0108, Sharma discusses appearance, location, and time of captured objects).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin and Sharma with Boghossian to derive at the invention of claim 11. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Shin and Sharma in this manner in order to improve object tracking by extracting features, timestamps, location, and calculating similarity scores to properly identify and combine corresponding matched trajectory tracks. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Shin and Sharma, while the teaching of Boghossian continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of extracting features, timestamps, location, and calculating similarity scores to identify matching tracks of objects captured in images. The Shin, Sharma, and Boghossian systems perform object tracking, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 12, Boghossian teaches wherein said similarity score is based on at least one of an appearance-similarity score based on said descriptor, and a proximity score based on said location information and said time information (see para. 0073, 0106-0108, Sharma discusses appearance similarity matching scores, temporal, and spatial data).
The same motivation of claim 11 is applied to claim 12. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin and Sharma with Boghossian to derive at the invention of claim 12. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 11,354,683) in view of Kant (US 2016/0275376).
Regarding claim 13, Shin does not expressly disclose further comprising dynamically removing a background from said individual's image. However, Kant teaches further comprising dynamically removing a background from said individual's image (see para. 0050, Kant discusses performing background removal from captured images).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin with Kant to derive at the invention of claim 13. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Shin in this manner in order to improve object tracking by extracting features, timestamps, location, and calculating similarity scores to properly identify and combine corresponding matched trajectory tracks. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Shin, while the teaching of Kant continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of extracting features, timestamps, location, and calculating similarity scores to identify matching tracks of objects captured in images. The Shin and Kant systems perform object tracking, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 11,354,683) in view of Harper et al. (US 2016/0048721).
Regarding claim 14, Shin does not expressly disclose wherein each dataset of said plurality of other datasets has a timestamp that is within a predetermined time window, such that said specific dataset is only compared to datasets having timestamps within said predetermined time window. However, Harper teaches wherein each dataset of said plurality of other datasets has a timestamp that is within a predetermined time window, such that said specific dataset is only compared to datasets having timestamps within said predetermined time window (see para. 0094, Harper discusses generating the target data set, the system will compare the time stamps and, if the difference between the timestamps is no greater than a predefined time period, simply accept the target data set as accurate and correct and, if it falls outside the predefined time period, delete or fail to create the target data set).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin with Harper to derive at the invention of claim 14. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Shin in this manner in order to improve object tracking by extracting features, timestamps, location, and calculating similarity scores to properly identify and combine corresponding matched trajectory tracks. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Shin, while the teaching of Harper continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of extracting features, timestamps, location, and calculating similarity scores to identify matching tracks of objects captured in images. The Shin and Harper systems perform object tracking, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claims 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 11,354,683) in view of Harper et al. (US 2016/0048721) in view of Nukaga et al. (US 2016/0343147).
Regarding claim 15, Shin and Harper do not expressly disclose wherein a specific one of said tracklets has a first timestamp corresponding to an earliest time of appearance in said image frames of a specific individual corresponding to said tracklet. However, Nukaga teaches wherein a specific one of said tracklets has a first timestamp corresponding to an earliest time of appearance in said image frames of a specific individual corresponding to said tracklet (see figure 3, para. 0042-0046, 0055, Nukaga discusses a video database that contains tracking IDs related to specific persons include a start appearance timestamp and last appearance timestamp).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin and Harper with Nukaga to derive at the invention of claim 15. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Shin and Harper in this manner in order to improve object tracking by extracting features, timestamps, location, and calculating similarity scores to properly identify and combine corresponding matched trajectory tracks. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Shin and Harper, while the teaching of Nukaga continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of extracting features, timestamps, location, and calculating similarity scores to identify matching tracks of objects captured in images. The Shin, Harper, and Nukaga systems perform object tracking, therefore a person having ordinary skill in the art would have reasonable expectation of success in the combination yielding predictable results. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 16, Nukaga teaches wherein a specific one of said tracklets has a second timestamp corresponding to a last time of appearance in said image frames of a specific individual corresponding to said tracklet (see figure 3, para. 0042-0046, 0055, Nukaga discusses a video database that contains tracking IDs related to specific persons include start appearance timestamp and last appearance timestamp).
The same motivation of claim 15 is applied to claim 16. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin and Harper with Nukaga to derive at the invention of claim 16. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Regarding claim 17, Harper teaches further comprising a step of comparing said second timestamp to a current time (see para. 0094, Harper discusses generating the target data set, the system will compare the time stamps and, if the difference between the timestamps is no greater than a predefined time period, simply accept the target data set as accurate and correct and, if it falls outside the predefined time period, delete or fail to create the target data set).
The same motivation of claim 15 is applied to claim 17. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin and Harper with Nukaga to derive at the invention of claim 17. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Regarding claim 18, Harper teaches wherein datasets in said specific tracklet are included in said plurality of other datasets for comparison when a difference between said second timestamp and said current time is less than a duration of said predetermined time window (see para. 0094, Harper discusses generating the target data set, the system will compare the time stamps and, if the difference between the timestamps is no greater than a predefined time period, simply accept the target data set as accurate and correct and, if it falls outside the predefined time period, delete or fail to create the target data set).
The same motivation of claim 15 is applied to claim 18. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin and Harper with Nukaga to derive at the invention of claim 18. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Regarding claim 19, Harper teaches wherein datasets in said specific tracklet are omitted from said plurality of other datasets for comparison when a difference between said second timestamp and said current time is greater than a duration of said predetermined time window (see para. 0094, Harper discusses generating the target data set, the system will compare the time stamps and, if the difference between the timestamps is no greater than a predefined time period, simply accept the target data set as accurate and correct and, if it falls outside the predefined time period, delete or fail to create the target data set).
The same motivation of claim 15 is applied to claim 19. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Shin and Harper with Nukaga to derive at the invention of claim 19. The result would have been expected, routine, and predictable in order to perform object tracking based on extracted metadata.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Marcheselli et al. (US 10,410,048) discusses counting and tracking defined objects includes the step of receiving subset data with a data capturing device, wherein the subset data is associated with defined objects and includes a unique identifier, an entry time, an exit time, and location data for each defined object.
Li et al. (US 2016/0098591) discusses generates on the basis of the corresponding motion signature information, the second signature and the first signature corresponding to the same predetermined time threshold.
Partis et al. (US 2013/0148852) discusses a visual signature of the track, such as a color histogram.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY A CESE whose telephone number is (571) 270-1896. The examiner can normally be reached on Monday – Friday, 9am – 4pm.
If attempts to reach the primary examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kenny A Cese/
Primary Examiner, Art Unit 2663
Prosecution Insights