Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments have been considered. Please see the new combination below.
Karyodisa teaches calibrated low and high resolution streams, conversion of low-resolution bounding box coordinates into the high-resolution stream, and recognition in the corresponding high resolution region.
Zhou teaches generating a new classification request when the current tracker bounding box has increased relative to the bounding box at the last classification.
This rejection is made final.
Claim Rejections - 35 USC 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 18-23,25-26 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Karyodisa (20190213420) in view of Zhou (20190130188).
Regarding claim 18, Karyodisa teaches a first stream of images, at a first resolution, wherein: the first stream of images is received from a first image source; and the first stream of images comprises a first field of view of an environment (pars. 88-89, supplies high-resolution stream and source).
Karyodisa teaches a second stream of images, at a second resolution, wherein: the second resolution is lower than said first resolution; the second stream of images is received from a second image source (pars. 88-89, supplies separate lower-resolution source).
the second stream of images comprises a second field of view that overlaps with over half of the first field of view of said first image source (par. 89, captures the same field of view).
a localizer component configured to localize at least one moving object into one or more locations (pars. 60 and 90-91, detects moving objects into boxes).
a classifier configured to: receive the one or more locations for the at least one moving object (par. 93, recognition receives converted detection results).
identify a corresponding portion of a first image acquired from said first stream within a pre-determined period of time from when the at least one moving object is localized from said second stream, wherein identifying the corresponding portion of the first image is done by mapping coordinates from the second stream to the first stream based on the overlapping fields of view (pars. 89 and 92-93, maps synchronized low-high bounding coordinates).
return at least one classification for the at least one moving object based on the corresponding portion of the first image (pars. 93-94, recognizes object within mapped region).
wherein the localizer component is configured to perform the localizing of the at least one moving object within successive images of said second stream of images (par. 55, tracks objects across successive frames).
wherein the localizer component is further configured to perform the localizing of the at least one moving object independently of classification of said at least one moving object (par. 55, tracks while skipping full recognition).
Karyodisa does not teach a tracker configured to re-classify the at least one moving object when a bounding size of the one or more locations localized by the localizer component for the at least one moving object increases.
Zhou teaches this in pars. 148-149, reclassifies when tracker box enlarges.
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to modify Karyodisa to invoke re-classification when the tracked bounding size increases as taught by Zhou. The reason is to more accurately classify.
Regarding claim 19, Karyodisa teaches wherein said localizer component is further configured to identify temporal movement of an object located in a number of images of said second stream based on generic object spatial features (par. 56, tracks spatiotemporal generic image features).
Regarding claim 20, Zhou teaches wherein said tracker is further configured to provide, to said classifier, a determined location, where the determined location corresponds to at least one of: an object newly-located in a second image of said second stream; or an object identified to have stopped moving within the second field of view of the second image source (par. 141, submits newly detected tracker for classification).
Regarding claim 21, Zhou teaches wherein when, for a second image from said second stream, said classifier fails to classify an object, said tracker is configured to provide, to said classifier, a determined location of said object in a subsequent image of said second stream, so long as said determined location corresponds to at least one of: a level of confidence above a predetermined threshold; or said object with a larger bounding size (par. 149, retries failed small object later).
Regarding claim 22, Zhou teaches wherein said classifier is configured to intermittently vary the classification of the at least one moving object being classified from said corresponding portion of said first image acquired from said first stream (pars. 170 and 173, updates classification in later frames).
Regarding claim 23, Karyodisa teaches wherein the second stream comprises a sub-sampled version of the first stream (par. 87, downscales high-resolution video stream).
Regarding claim 25, Zhou teaches wherein the image processing system is housed in a common housing associated with a static security camera (pars. 64-65, places analytics inside static camera).
Regarding claim 26, Karyodisa teaches wherein said localizer component is adapted to receive inputs from a plurality of previously acquired images of said second stream when localizing previous moving objects in an initial image of said second stream (pars. 55-60, tracks using prior-frame motion information).
Regarding claim 35, see the rejection of claim 18 above.
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Karyodisa (20190213420) in view of Zhou (20190130188) in further view of McMordie (20110063446).
Regarding claim 24, McMordie teaches wherein the second image source comprises at least one of: a thermal infra-red camera; a near infra-red camera; a LIDAR transceiver; an ultrasound transceiver; or an event camera (par. 28, uses nearinfrared or thermographic camera).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to modify Karyodisa and Zhou to include that the second image source uses the mapped infrared modality as taught by McMordie. The reason is to maintain lower resolution object detection and tracking under low visible-light conditions while reserving the higher resolution stream for recognition.
Claim(s) 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Karyodisa (20190213420) in view of Zhou (20190130188) in further view of Tantalo (20020149693).
Regarding claim 27, Karyodisa teaches wherein said localizer component is further adapted to receive inputs from a plurality of later acquired images of said second stream (par. 55, tracks across later video frames).
Tantalo teaches wherein the plurality of later acquired images comprises a specific duration corresponding to an interval (pars. 27 and 42, acquires successive interval-separated image frames).
wherein the interval is determined to be inversely proportional to observed speed of the previous moving objects localized in the initial image (pars. 10, 33 and 35, sets interval inverse to speed).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Karyodisa and Zhou a tracker to set its sampling interval from measured tracked-object speed under a fixed maximum-displacement constraint as taught by Tantalo. The reason is to improve trackability for faster objects.
Regarding claim 28, Karyodisa teaches wherein said localizer comprises a neural network (par. 54, uses neural-network object detection).
Zhou teaches a neural network comprising a plurality of layers (par. 198, CNN includes multiple hidden layers).
Regarding claim 29, Karyodisa teaches wherein said localizer is configured to produce a map of previous moving object locations based on said plurality of previously acquired images (pars. 56 and 91, maps tracked motion into regions).
Claim 34 is rejected under 35 U.S.C. 103 as being unpatentable over Karyodisa (20190213420) in view of Zhou (20190130188) in further view of Bigioi (20190065410).
Regarding claim 34, Karyodisa teaches A computer program product comprising a non-transitory computer-readable medium (par. 42, stores data on non-transitory medium).
Karyodisa teaches when executed, are configured to cause a processor to perform a process for implementing an image processing system (par. 138, processor executes stored image-processing instructions).
The combination of Karyodisa and Zhou does not expressly teach that the stored instructions comprise a neural-network configuration and stored weight information for the localizer.
Bigioi teaches this (par. 29, stores network configuration with instructions and par. 53 teaches stored neural-network configuration and weights).
It would have been obvious prior to the effective filing date of the invention to one of ordinary skill in the art to include in Karyodisa and Zhou a combination to store the neural network configuration and weights with the executable system as taught by Bigioi. The reason is to use a trained neural-network localizer with the parameters needed to perform inference.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Fisher (20200074393) teaches determining point correspondence and a transformation between cameras with overlapping fields of view (pars. 82-84 and 168).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HADI AKHAVANNIK whose telephone number is (571)272-8622. The examiner can normally be reached 9 AM - 5 PM Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HADI AKHAVANNIK/ Primary Examiner, Art Unit 2676
Prosecution Insights