DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 5, 7, 9-11 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 2017/0206669 by Saleemi et al.
Regarding claim 1, a system for monitoring regions of interest (fig. 1), the system comprising:
one or more processors (paragraph 0062 teaches “In some implementations, architecture 800 includes one or more processor(s) 802 (e.g., dual-core Intel® Xeon® Processors)…”);
one or more cameras positioned to view a region of interest (in addition to discussion above, paragraph 0020 teaches “Capturing device(s) 107 can be monocular intensity cameras, stereo cameras, structured light cameras, time-of-flight (TOF) cameras or any other camera, sensor or system that is capable of capturing grayscale or color intensity images or depth images.”);
a memory to store instructions (in addition to discussion above, paragraph 0062 teaches “one or more storage device(s) 804 (e.g., hard disk, optical disk, flash memory) and one or more non-transitory, computer-readable storage medium(s) 808 (e.g., hard disk, optical disk, flash memory, etc.).”);
wherein the one or more processors execute the instructions to perform operations that include (in addition to discussion above, fig. 6):
processing image data captured by the one or more cameras (paragraph 20 teaches cameras), including (i) detecting, from the image data, an object entering a region of interest, (ii) tracking, from the image data, the object traversing the region of interest, and (iii) detecting, from the image data, the object exiting the region of interest (in addition to discussion above, paragraph 0047 teaches “The tracking is continued one batch at a time, and the association of tracks over batch boundaries is explicitly handled by allowing a small temporal overlap between adjacent batches. A resulting track describes the position of the object of interest in a world metric coordinate system as well as image pixel coordinate system over the period of time that the object of interest is observable in the field of view of the image capturing device 202, 203. The object tracks may be used for counting the frequency of entrance and exit events when a bounding region is defined, as well as for estimating the duration of persistence of the object of interest within the bounding region. For example, in a retail store a bounding region can be defined at a particular location in the store, such as the front entrance. Object tracks generated by the foregoing embodiments that enter and exit the boundary region can be counted to determine how many customers entered or exited the retail store. In another example, a bounding region can be defined to be around a particular department in the retail store (e.g., electronics section) and the duration of persistence of objects in the bounding region is indicative of the number of customers that visited the electronics section.”); and
in response to determining the object leaving the region of interest, updating a count of objects the type traversing the region (in addition to discussion above, paragraph 0059 teaches “Object tracks are a foundational element of data acquired using video analytics. For example, foot traffic, e.g., the number of people, shopping carts, and other objects of interest passing by or through a specific area, is generally calculated by counting the number of object tracks that cross an arbitrary line in the scene, or that enter and then exit a bounded shape such as a polygon, or that exit one bounded shape and enter another.”).
Regarding claim 2, the system wherein the operations further comprise determining, from the image data, a type of the object, and wherein updating the count of objects includes updating a count of objects of the type (in addition to discussion above, paragraph 0059 teaches “Object tracks are a foundational element of data acquired using video analytics. For example, foot traffic, e.g., the number of people, shopping carts, and other objects of interest passing by or through a specific area, is generally calculated by counting the number of object tracks that cross an arbitrary line in the scene, or that enter and then exit a bounded shape such as a polygon, or that exit one bounded shape and enter another.”).
Regarding claim 3, the system wherein the operations further comprise: determining a dwelling time while the object is in the region of interest (in addition to discussion above, paragraph 0059-0060 teaches “Similarly, waiting or dwell time is another important metric used by retailers to measure shopper activity and staff performance, and is generally calculated by measuring the duration that a track spends within a bounded shape, such as a polygon, that is drawn on the scene”).
Regarding claim 5, the system wherein the cameras are mounted to be elevated and askew of the region of interest, and wherein processing image data includes mapping the image data to a set of real-world coordinates (in addition to discussion above, paragraph 0019-0020 teaches “Image capturing device(s) 107 can be mounted on walls and/or ceilings at various locations throughout the retail store and directed toward transaction devices(s) 108, ingress and egress points and shopping aisles or any other desired location in the retail store. Capturing device(s) 107 can be monocular intensity cameras, stereo cameras, structured light cameras, time-of-flight (TOF) cameras or any other camera, sensor or system that is capable of capturing grayscale or color intensity images or depth images. As used herein, a depth image is an image that contains information relating to the distance of the surfaces of scene objects from a viewpoint.”, paragraph 0029-0030 teaches “Using geometry and algebra, the points that appear in the 2D stereo images can be mapped as coordinates in a 3D world coordinate system. Object detection module 208 generates a dictionary of potential 2D projections of a 3D human model mimicking an average human. These projections correspond to an exhaustive set of potential locations in the world coordinate system that a human can occupy, and are realized as a discretized grid with adjacent locations at a pre-specified metric distance.”).
Regarding claim 7, the system wherein the operations further comprise: defining multiple trigger zones, each trigger zone being configured in shape and dimension specifically for the region of interest; making a determination as to which of the multiple trigger zones the object traverses and in which sequence; and based on the determination, determining a direction of travel for the object of interest (in addition to discussion above, paragraph 0020 teaches “Image capturing device(s) 107 can be mounted on walls and/or ceilings at various locations throughout the retail store and directed toward transaction devices(s) 108, ingress and egress points and shopping aisles or any other desired location in the retail store. Capturing device(s) 107 can be monocular intensity cameras, stereo cameras, structured light cameras, time-of-flight (TOF) cameras or any other camera, sensor or system that is capable of capturing grayscale or color intensity images or depth images. As used herein, a depth image is an image that contains information relating to the distance of the surfaces of scene objects from a viewpoint.”, paragraph 0059-0061 teaches “Other applications include heat maps of several object tracks acquired over a period of time that provides a graphical view of aggregate shopper movement within that period, directional statistics that indicate the probability of a shopper moving in a certain direction in the store, etc.”).
Regarding claim 9, the system wherein the operations further comprise: determining, based on the count, a heat map for the region of interest (in addition to discussion above, paragraph 0061 taches “Other applications include heat maps of several object tracks acquired over a period of time that provides a graphical view of aggregate shopper movement within that period, directional statistics that indicate the probability of a shopper moving in a certain direction in the store, etc.”).
Claim 10 is rejected for the reason as discussed in the corresponding claim 1 above.
Claim 11 is rejected for the reason as discussed in the corresponding claim 2 above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4, 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2017/0206669 by Saleemi et al. in view of US 2018/0005033 by Ding et al.
Regarding claim 4, Saleemi et al. discloses the object traversing the region of interest (as discussed above), but fails to disclose the system wherein the operations further comprise: determining a speed of the object traversing the region of interest.
Ding et al. discloses the system wherein the operations further comprise: determining a speed of the object traversing the region of interest (paragraph 0045 teaches “One or more regions of interest (ROIs) may be arranged in each image frame. The ROIs shown in FIG. 3A include a left ROI 311, a center ROI 312, and a right ROI 313. Although three ROIs are shown to be arranged in image frame 301, the disclosed systems and methods may use only one ROI, two ROIs, or more than three ROIs, such as four ROIs, five ROIs. When one ROI is used, the system may detect a moving object entering or exiting the ROI. The one-ROI system, however, may not detect a moving direction of the ROI. When two ROIs are used, the system may detect the moving direction and speed of the moving object. The two-ROI system, however, may be sensitive to environmental changes. When three ROIs are used, as shown in FIG. 3A, the system may achieve robust, fast, and accurate detection of a moving object.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include determining a speed of the object traversing the region of interest, as taught by Ding et al. into the system of Saleemi et al., because such incorporation would allow for the benefit of calculating speed of the object for monitoring the object in the entryway/exit way, thus increase user flexibility of the system.
Regarding claim 6, Saleemi et al. discloses the object traversing the region of interest (as discussed above), but fails to disclose the system wherein processing the image data includes determining a speed of an object traversing the region of interest, by mapping display coordinates reflecting the object's position as displayed to a corresponding set of real-world coordinates.
Ding et al. discloses the system wherein processing the image data includes determining a speed of an object traversing the region of interest, by mapping display coordinates reflecting the object's position as displayed to a corresponding set of real-world coordinates. (paragraph 0045 teaches “One or more regions of interest (ROIs) may be arranged in each image frame. The ROIs shown in FIG. 3A include a left ROI 311, a center ROI 312, and a right ROI 313. Although three ROIs are shown to be arranged in image frame 301, the disclosed systems and methods may use only one ROI, two ROIs, or more than three ROIs, such as four ROIs, five ROIs. When one ROI is used, the system may detect a moving object entering or exiting the ROI. The one-ROI system, however, may not detect a moving direction of the ROI. When two ROIs are used, the system may detect the moving direction and speed of the moving object. The two-ROI system, however, may be sensitive to environmental changes. When three ROIs are used, as shown in FIG. 3A, the system may achieve robust, fast, and accurate detection of a moving object.”, paragraph 0054 teaches “As shown in FIG. 3C, when processor 210 identifies moving object 305 and determines to track moving object 305, a tracking indicator 310 may be superimposed in the image frame displayed on a display (e.g., a screen, a monitor, etc.). Tracking indicator 310 may follow moving object 305 as the moving object 305 continues to move to the left. Although a circle is used as an example of tracking indicator 310, tracking indicator 310 may be in other forms or shapes, such as a rectangle surrounding moving object 305. In some embodiments, the tracking indicator 310 may not be displayed. When processor 210 identifies moving object 305 and determines to track moving object 305, processor 210 may send a control signal to motor 120 (FIG. 1) to start driving motor 120 to adjust an angle of camera 105, such that the field of view of camera 105 follows the movement of moving object 305. In other words, during tracking, motor 120 may adjust the angle of camera 105 such that moving object 305 is kept within the field of view.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include the system wherein processing the image data includes determining a speed of an object traversing the region of interest, by mapping display coordinates reflecting the object's position as displayed to a corresponding set of real-world coordinates, as taught by Ding et al. into the system of Saleemi et al., because such incorporation would allow for the benefit of calculating speed of the object for monitoring the object in the entryway/exit way, thus increase user flexibility of the system.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2017/0206669 by Saleemi et al. in view of US 20230267667 by Brudy et al.
Regarding claim 8, Saleemi et al. discloses the system wherein the operations further comprise: transmitting information about the monitored region to an external computing resource (in addition to discussion above, paragraph 0024 teaches “A system administrator can use console 109 to analyze and display data, run search queries and generally facilitate user interaction with analytics engine 102 through a number of graphical user interfaces (GUIs) and input devices. Console 109 can be physically located at the point-of-sale (POS) and/or located remotely and coupled to analytics engine 102 through a network-based connection (e.g., in Internet or Intranet connection). Console 109 can be any device capable of providing a human interface to analytics engine 102, including but not limited to a desktop computer or mobile device (e.g., a tablet computer, smart phone).”); based on the transmitted information, rendering a visualization of the monitored region, the visualization identifying objects by type, and each detected objects track when traversing the region of interest (in addition to discussion above, paragraph 0059 teaches “Object tracks are a foundational element of data acquired using video analytics. For example, foot traffic, e.g., the number of people, shopping carts, and other objects of interest passing by or through a specific area, is generally calculated by counting the number of object tracks that cross an arbitrary line in the scene, or that enter and then exit a bounded shape such as a polygon, or that exit one bounded shape and enter another. Foot traffic is an important metric used by retailers to measure store performance, and calculating accurate foot traffic data is only possible if objects of interest in the scene are detected and tracked with the highest possible accuracy.”).
Saleemi et al. fails to disclose rendering a ghost visualization.
Brudy et al. discloses rendering a ghost visualization (paragraph 0064-0065 teaches “In various embodiments, the one or more data visualizations 114(1) include, for a virtual avatar displayed within an AR scene 124, a ghost visualization corresponding to the virtual avatar…. In some embodiments, motion analysis application 112(1) receives a request to generate a ghost visualization corresponding to a person at a specific location..”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the ability to include rendering a ghost visualization, as taught by Brudy et al. into the system of Saleemi et al., because such incorporation would allow a user to check ghost image to have an accurate count of the object, thus increase user flexibility of the system.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIGAR CHOWDHURY whose telephone number is (571)272-8890. The examiner can normally be reached Monday-Friday 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Tran can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NIGAR CHOWDHURY/Primary Examiner, Art Unit 2484