DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/23/2025 has been entered.
Response to Arguments
Claims 1, 9, and 17 have been amended. Claims 1-5, and 7-20, and 22 are currently pending.
Applicant’s arguments with respect to amended claim(s) 1, 9, and 17 have been fully considered. With regards to applicant’s argument that “generating a bounding box around a subset of disjointed pixels disjointed pixels identified from the modified set of pixels” a new ground of rejection that does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. With regards to applicant’s argument that limitation “in accordance with a determination that the pixel intensities of the modified set of pixels changed by approximately a same value or intensity between the representative image and the canonical image based at least in part on the comparison of the changes, modifying the prediction of the identification of the object represented by the subset of modified set of pixels” is not taught by Shepherd, Examiner respectfully disagrees. Shadows are identified in an image when pixels in a difference image have substantially the same difference value (Shepherd Col 3 line 48- Col 4 line 26 Col 16 line 35- Col 17 line 9). Pixels that are identified as shadows were originally classified as part of an animal/foreground and are reclassified (the prediction is modified) to be part of the background. Therefore the limitation is considered to be taught. As such this action is made NON-FINAL.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 06/24/2025 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) has/have been considered by the examiner.
Claim Objections
Claim 17 is objected to because of the following informalities:
The claim limitation “smaller that the set of pixels” should be “smaller than the set of pixels” in line 12. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 9-16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows. Independent claim 9 is to “One or more computer -readable medium having instructions stored thereon,” a computer readable medium does not exclude transitory embodiments. The broadest reasonable interpretation of the claim is therefore directed to signals per se with no special definition or disavow in the written description. Since signals do not fall within the statutory categories of 35 U.S.C. 101 (i.e., process, machine, manufacture, or composition of matter), the claim is ineligible under 35 U.S.C. 101. The Examiner suggests “One or more computer readable medium having instructions stored thereon,” in lines 1-2 of claim 9 to be – One or more non-transitory computer readable medium having instructions stored thereon,-- or equivalent. With similar amendments made to depended claims 10-16. Appropriate correction is required.
Claim Interpretation
Although the term “approximately” in claim 21 is a relative term, the specification provides a standard for ascertaining the requisite degree in paragraph [0064]. As in paragraph [0064], “approximately” is taken to mean “within 5%.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 5, 7-10, 13-17 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rigney (Patent No. US 6985172B1) in view of Savvides (Pub. Number US 20190057588A1), and Shepherd (Patent No. US10796141B1).
Regarding claim 1, Rigney teaches A method, comprising: capturing a representative image from a video (Rigney Fig. 1 element 23 image to be analyzed, an image is chosen from a set of frames.) retrieving a canonical image (Rigney column 6 lines 50-59 and Fig. 3 element 22, a representative image is chosen, referred to as the reference image.) that represents one or more frames of the video before the representative image (Rigney column 7 lines 14-19, the reference image may be created from frames before the representative image.) determining a difference between the canonical image and the representative image, the difference identifying a set of pixels of the representative image that are different from corresponding pixels in the canonical image; (Rigney column 6 lines 60-65, column 7 lines 1-4, and Fig. 4 elements 23 and 22 are used to compute a temporal difference image, element 24.) modifying the set of pixels based at least in part on previous frames of the video before the representative image to form a modified set of pixels, the modified set of pixels being smaller than the set of pixels; (Rigney column 8 lines 6-20 a threshold can be applied to the difference image to remove cyclic motions in the video feed, see also Fig. 6 the result 25 is a low-level detected motion image.) identifying a subset of disjointed pixels from the modified set of pixels; (Rigney column 8 lines 29-39, pixels are grouped spatially and small or narrow groups are removed as noise.) executing a classifier using the modified set of pixels with the pixels corresponding to the set of pixels (Rigney column 8 lines 53-58 object classification can be done, see also Fig. 7 in step B. noise objects are deleted then in step C. objects are classified.) outputting the prediction of the identification of the object. (Rigney Fig. 9 spatio-temporal object features and scene motion model (includes classification see 4) under MODEL MAY INCLUDE) are sent to an external interface.)
Rigney does not disclose generating a bounding box around the subset of disjointed pixels identified from the modified set of pixels, or determining that the bounding box is larger than a threshold percentage of the representative image; removing pixels corresponding to the bounding box from the set of pixels based at least in part on the determination that the bounding box is larger than the threshold percentage of the representative image.
Savvides, however, discloses generating a bounding box around the subset of disjointed pixels identified from the modified set of pixels. (Savvides ¶30-31; bounding boxes are fitted around grouped pixels from a difference image. These pixels can be filtered etc. and are considered a modified set of pixels.) determining that the bounding box is larger than a threshold percentage of the representative image; removing pixels corresponding to the bounding box from the set of pixels based at least in part on the determination that the bounding box is larger than the threshold percentage of the representative image. (Savvides ¶34; bounding boxes larger than a predetermined maximum area like half the image (threshold percentage) are removed as potential objects.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine Rigney with the teachings of Savvides by including fitting bounding boxes around potential object detections and removal of object detections that are too large. One skilled in the art would have been motivated to modify Rigney in this manner in order to insure the full detected object is found within the bounding box for later classification and to prevent wasting processing resources on detections that are unlikely to be objects (Savvides ¶34).
Although the combination of Rigney and Savvides, teaches normalizing image intensities of the difference image (Rigney paragraph 6 lines 60 -67), it does not explicitly teach comparing changes in pixel values or pixel intensities to each other for pixels within the modified set of pixels; in accordance with a determination that the pixel intensities of the modified set of pixels changed by approximately a same value or intensity between the representative image and the canonical image based at least in part on the comparison of the changes, modifying the prediction of the identification of the object represented by the subset of modified set of pixels.
Shepherd, however, discloses comparing changes in pixel values or pixel intensities to each other for pixels within the modified set of pixels;; (Shepherd Col 3 line 48- Col 4 line 26; a shadow can be identified in a difference by pixels that have substantially the same difference value.) in accordance with a determination that the pixel intensities of the modified set of pixels changed by approximately a same value or intensity between the representative image and the canonical image based at least in part on the comparison of the changes, modifying the prediction of the identification of the object represented by the subset of modified set of pixels. (Shepherd Col 3 line 48- Col 4 line 26; a shadow can be identified in a difference by pixels that have substantially the same difference value. Then “shadow” pixels are designated/ reclassified (i.e. the prediction is changed) as background pixels and not part of the region of interest of the animal. The predefined range can be based on the expected difference in brightness levels of the pixel, therefore a person of ordinary skill in the art could have chosen it to be within 5% difference. See also Col 16 line 35- Col 17 line 9.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to further combine Rigney and Savvides with Shepherd by including a threshold to remove sets of pixels that have approximately the same intensity differences between two images. One skilled in the art would have been motivated to modify the combination of Rigney and Savvides in this manner in order to remove pixels due to shadows rather than regions of interest (Shepherd Col 3 line 48 – Col 4 line 26).
Regarding claim 4, the combination of Rigney, Savvides and Shepherd teaches the claim limitations related to claim 1, as described above. They further teach further comprising: averaging amounts of change in subgroups of the set of pixels, (Rigney column 7 lines 20-24 the mean temporal difference is taught.) wherein averaging the amount of change in the subgroups of the set of pixels comprises averaging an amount of change of a first pixel within a subgroup of the set of pixels with amounts of change of other pixels within the subgroup of the set of pixels; (Rigney column 9 lines 34-49, statistical features (from column 7 lines 20-24 includes a mean) are generated as part of the model. Features associated with regions of the scene are also encoded, which include motion. Therefore a mean within a subregion is disclosed.) identifying a portion of the subgroups having averaged amounts of change that are below a threshold amount of change, wherein pixels within the portion of the subgroups comprise low density pixels; and removing the low density pixels from the set of pixels. (Rigney column 8 lines 29-39 and Fig. 7, pixels are grouped spatially and small or narrow groups are removed as noise, using a spatial filter. Thresholding can be considered a subset of filtering and is thus taught.)
Regarding claim 5, the combination of Rigney, Savvides and Shepherd teaches the claim limitations related to claim 1 as described above. They further teach wherein the subset of disjointed pixels is a first subset of disjointed pixels, and wherein the method further comprises: identifying a second subset of disjointed pixels from the modified set of pixels; (Rigney column 8 lines 21-35, pixels are grouped spatially and small or narrow groups are removed as noise. Disclosing removing small or narrow groups indicates that more than one grouping was made) generating a second bounding box around the second subset of disjointed pixels; and adding additional pixels to the modified set of pixels to fill the second bounding box, (Savvides ¶30-31; bounding boxes (plural) are fitted around grouped pixels from a difference image. These pixels can be filtered etc. and are considered a modified set of pixels. Wherein it would have been obvious to generate bounding boxes around disjointed pixels in order to insure the full detected object is found within the bounding box for later classification) wherein the modified set of pixels with the additional pixels is used in the executing of the classifier (Rigney column 8 lines 53-58 object classification can be done, see also Fig. 7.)
Regarding claim 7, the combination Rigney, Savvides and Shepherd teaches the claim limitations with regards to claim 1 as described above. They further teach wherein modifying the set of pixels includes: applying temporal averaging to the set of pixels; (Rigney column 7 lines 20-24 the mean temporal difference is taught.) identifying high frequency pixels from the set of pixels based at least in part on the temporal averaging indicating that the high frequency pixels have been unstable within the previous frames; and (Rigney column 8 lines 6-20 a threshold can be applied to the difference image to remove cyclic motions in the video feed, see also Fig. 6 the result 25 is a low-level detected motion image.) removing the high frequency pixels from the set of pixels based at least in part on the temporal averaging indicating that the high frequency pixels have been unstable within the previous frames. (Rigney column 7 lines 20-37 a threshold can be based off of a median or a mean.)
Regarding claim 8, the combination of Rigney, Savvides and Shepherd teaches the claim limitations related to claim 7 as described above. They further teach, wherein applying the temporal averaging includes averaging amounts of change for each pixel within the set of pixels within the previous frames, and wherein the averaged amounts of change for the high frequency pixels are below a temporal averaging threshold. (Rigney column 7 lines 20-37 a threshold can be based off of a median or a mean.)
Regarding claim 9, it has parallel claim limitations to claim 1 and those limitations are rejected under the combination of Rigney, Savvides and Shepherd for similar reasons as described above. The combination of Rigney, Savvides and Shepherd additionally teach One or more computer-readable media having instructions stored thereon, wherein the instructions, when executed by a system, cause the system to: (Rigney column 4 line 67- column 5 lines 1-2, teaches instructions stored in a computer memory.)
Regarding claim 10, the combination of Rigney, Savvides and Shepherd teaches the claim limitations with regards to claim 9 as described above. They further teach wherein the prediction of the object includes a prediction that the object is a package, and wherein the instructions, when executed by the system, further cause the system to: (Rigney column 2 lines 35-40, “objects…such as a package or suitcase, can be detected.”) determine that notification settings associated with a camera that captures the video indicate that a notification is to be provided when the prediction is that the object is a package; and cause the notification to be provided based at least in part on the prediction that the object is a package. (Rigney column 10 lines 9-18, an alert or notification can be sent based on the detection of an object of interest, column 6 line 13, detection of an object or package left by a person is taught.)
Regarding claim 13, the combination of Rigney, Savvides and Shepherd teaches the claim limitations with regards to claim 9 as described above. Additional claim limitations are parallel to claim 4 and are rejected for similar reasons.
Regarding claim 14, the combination of Rigney, Savvides and Shepherd teaches the limitations related to claim 9, as described above. Additional claim limitations are parallel to claim 5 and are rejected for similar reasons.
Regarding claim 15, the combination of Rigney, Savvides and Shepherd teach the limitations related to claim 14 as described above. They further teach to modify the set of pixels includes to: apply temporal averaging to the set of pixels; identify high frequency pixels from the set of pixels based at least in part on the temporal averaging indicating that the high frequency pixels have been unstable within the previous frames; and (Rigney column 8 lines 6-20; a threshold can be applied to the difference image to remove cyclic motions in the video feed, see also Fig. 6 the result 25 is a low-level detected motion image.) remove the high frequency pixels based at least in part on the temporal averaging indicating that the high frequency pixels have been unstable within the previous frames. (Rigney column 7 lines 20-37; it is taught that a threshold can be based off of a median or a mean.)
Regarding claim 16, the combination of Rigney, Savvides and Shepherd the limitations related to claim 15 as described above. They further teach to apply the temporal averaging includes to average amounts of change for each pixel within the set of pixels within the previous frames, and wherein the averaged amounts of change for the high frequency pixels are below a temporal averaging threshold. (Rigney column 7 lines 20-37 it is taught that a threshold can be based off of a median or a mean.)
Regarding claim 17, it has parallel claim limitations to claim 1 and those limitations are rejected under the combination of Rigney, Savvides and Shepherd for similar reasons as described above. The combination of Rigney, Savvides and Shepherd additionally teach A system, comprising: memory to store images from video received from a camera; and one or more processors coupled to the memory, the one or more processors to: (Rigney column 4 lines 9-22 and Fig. 1 elements 50 (video camera), 32 (memory), and 34 (processor).)
Regarding claim 19, the combination of Rigney, Savvides and Shepherd teaches the claim limitations with regards to claim 17, as described above. Additional claim limitations are parallel to claim 7 and are rejected for similar reasons.
Regarding claim 20, the combination of Rigney, Savvides and Shepherd teaches the claim limitations with regards to claim 19, as described above. Additional claim limitations are parallel to claim 8 and are rejected for similar reasons.
Claim(s) 2-3, 11-12, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rigney (Patent No. US 6985172B1) in view of Savvides (Pub. Number US 20190057588A1), Shepherd (Patent No. US10796141B1), and Jabbar (J. Jabbar and L. R. Ragha, "Delta E colour feature based mutiobject tracking using merge and split approach and Kalman filtering," 2017 Second International Conference on Electrical, Computer and Communication Technologies (ICECCT), Coimbatore, India, 2017, pp. 1-5, doi: 10.1109/ICECCT.2017.8117927.).
Regarding claim 2, the combination of Rigney, Savvides and Shepherd teaches the claim limitations with regards to claim 1 as described above. They further teach for related pixels within the representative image and the canonical image, and wherein the set of pixels that are identified as being different is based at least in part on the determined distances within the color space. (Rigney column 3 lines 28-35, a median pixel characteristic value (can be pixel color) is found and moving objects are identified as ones that do not match the median value.)
The combination of Rigney, Savvides and Shepherd does not specifically disclose using Delta E to determine the difference canonical image and the representative image.
Jabbar however teaches, determining the difference between the canonical image and the representative image includes determining distances within a color space using Delta E (Jabbar page 2 Section III. The Proposed Method paragraph 2 and equation (1), Delta E is used for object tracking.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to further combine Rigney, Savvides and Shepherd with Jabbar. One skilled in the art would have been motivated to modify the combination of Rigney, Savvides and Shepherd in this manner because Delta E basically calculates the Euclidean color distance between two pixels. This gives a single value to compare the pixels with, rather than having to compare 3 color space dimensions separately.
Regarding claim 3, the combination of Rigney, Savvides, Shepherd, and Jabbar teach the claim limitations related to claim 2 as described above. Additionally, they teach the color space is a LAB color space (Jabbar page 2 equation (1), the LAB color space is being used for the difference. Wherein it would have been obvious to use the LAB color space in order to use Delta E because it provides a single value to compare the pixels with.)
Regarding claim 11, the combination of Rigney, Savvides and Shepherd teaches the limitations related to claim 9, as described above. Additional claim limitations are parallel to claim 2 and are rejected for similar reasons.
Regarding claim 12, the combination of Rigney, Savvides, Shepherd and Jabbar teach the limitations related to claim 11 as described above. They further teach the set of pixels that are identified as being different are identified based at least in part on distances within the color space corresponding to the set of pixels being greater than a threshold distance. (Rigney column 3 lines 29-35 a median pixel value can be calculated based on pixel color and moving objects are identified as pixels that do not match the median value. Matching is evaluated using a difference image see column 6 lines 60-65, column 7 lines 1-4, and Fig. 4 elements 23 and 22 are used to compute a temporal difference image, element 24. To which a threshold can be applied Rigney column 8 lines 6-20 a threshold can be applied to the difference image to remove cyclic motions in the video feed, see also Fig. 6 the result 25 is a low-level detected motion image.)
Regarding claim 18, the combination of Rigney, Savvides and Shepherd teaches the limitations related to claim 17, as described above. They further teach to determine the difference between the canonical image and the representative image (Rigney column 6 lines 60-65, column 7 lines 1-4, and Fig. 4 elements 23 and 22 are used to compute a temporal difference image, element 24.) the difference determined based at least in part on the distances. (Rigney column 3 lines 28-35, a median pixel characteristic value (can be pixel color) is found and moving objects are identified as ones that do not match the median value.)
The combination of Rigney, Cohen, Piovano, and Shepherd does not disclose using the LAB color space.
Jabbar, however discloses includes to determine distances between the canonical image and the representative image in a LAB color space, (Jabbar page 2 equation (1), the LAB color space is being used for the difference.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to further combine Rigney, Savvides and Shepherd with Jabbar. One skilled in the art would have been motivated to modify the combination of Rigney, Savvides and Shepherd in this manner because Delta E basically calculates the Euclidean color distance between two pixels. This gives a single value to compare the pixels with, rather than having to compare 3 color space dimensions separately.
Claim(s) 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rigney (Patent No. US 6985172B1) in view of Savvides (Pub. Number US 20190057588A1), Shepherd (Patent No. US10796141B1) and Haverlag (Pub. No. US20180295704A1).
Regarding claim 22, the combination of Rigney, Savvides and Shepherd teaches the limitations related to claim 1, as described above.
The combination of Rigney, Savvides and Shepherd does not explicitly teach determining amounts of time each pixel within the modified set of pixels have been at a corresponding current value; and determining whether to modify the modified set of pixels to remove one or more pixels having corresponding determined amounts of time less than a threshold amount of time.
Haverlag, however, discloses determining amounts of time each pixel within the modified set of pixels have been at a corresponding current value; and (Haverlag ¶82; it is determined if a subset of pixels (considered a modified set) has increased by more than a predetermined threshold (has been at the current value) over a predetermined time (i.e. how long the pixel has been at the current value).) determining whether to modify the modified set of pixels to remove one or more pixels having corresponding determined amounts of time less than a threshold amount of time. (Haverlag ¶82; fast changes of light can be filtered out as a change of light of the environment from a neighboring luminaire.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to further combine Rigney, Savvides and Shepherd with Haverlag by including a threshold to remove sets of pixels that have fast intensity changes. One skilled in the art would have been motivated to modify the combination of Rigney, Savvides and Shepherd in this manner in order to filter out changes in light (Haverlag ¶82).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5. Examiner’s email is Meredith.taylor@uspto.gov.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached on (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671