DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of US Provisional Applications No. 63/532,277 and No. 63/532,282, both filed August 11, 2023. Claims 1-20 have been afforded the benefit of this filing date.
Information Disclosure Statement
The IDSs dated 8/26/2024 has been considered and placed in the application file.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 3, 4, 8, 9, 10, 11, 13, 14, 15, 17, 18, 19, and 20 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by US Patent Publication 2021 0407131 A1, (Kallakuri et al.)
Claim 1
Regarding claim 1, Kallakuri et al. teach a method for self-service installation of monitoring operations in an area of real space, the method including: scanning the area of real space, using a sensor, to generate a three-dimensional (3D) representation of the area of real space; (the cameras 114 are installed in a shopping store (such as a supermarket) such that sets of cameras 114 (two or more) with overlapping fields of view are positioned over each aisle to capture images of real space in the store," par. 53) placing a camera at an initial location and orientation for monitoring a zone within the area of real space; ("Cameras 114 are placed and oriented such that areas of the floor 220 and shelves can be seen by at least two cameras," par. 72) configuring a computing device to be (i) connected to a cloud network hosting an image processing service ("The storage subsystem 2330 is an example of a computer readable memory comprising a non-transitory data storage medium, having computer instructions stored in the memory executable by a computer to perform all or any combinations of the data processing and image processing functions," par. 299) and (ii) couplable to the camera via a local connection, wherein the configuration of the computing device enables the computing device to mediate communications between the image processing service and the camera; (“Network nodes can be implemented in a cloud-based server system. More than one virtual device configured as a network node can be implemented using a single physical device," par. 51) coupling the camera to the computing device using the local connection and a unique identifier associated with the camera; ("The data structure 1200 stores the subject-related data as a key-value dictionary. The key is a frame_number and the value is another key-value dictionary where the key is the camera_id," par. 188) and finetuning a placement of the camera to a calibrated location and orientation for monitoring the zone within the area of real space, wherein the finetuning is assisted by information received from a cloud-based application associated with the image processing service ("point correspondences are identified for each 3D scene using the results of the image recognition engines 112a-112n for the purposes of the external calibration," par. 77).
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of system claim 13 and non-transitory computer readable medium claim 17 while noting that the rejection above cites to both device and method disclosures. Claims 13 and 17 are mapped below for clarity of the record and to specify any new limitations not included in claim 1.
Claim 2
Regarding claim 2, Kallakuri et al. teach the method of claim 1, further including identifying a region of interest within the monitored zone based on an output received from an object classification model, ("The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254) wherein: the region of interest is a 3D space at which a particular object is expected to be located, ("As described above, the semantic diffing events (or diff events) classify items put on or taken from shelves," par. 255) and the output of the object classification model is determined by processing, as input, the 3D
PNG
media_image1.png
406
676
media_image1.png
Greyscale
representation of the area of real space and a flattened 2D image ("FIG. 3 presents creation of two-dimensional (2D) and three-dimensional (3D) maps," par. 86) produced from RGB data ("Multiple types of sensors can be used, including for example [AltContent: textbox (Figure 3 shows the 3D and 2D images of the regions of interest.)]ultrasound or RF sensors in addition to the cameras 114 that generate RGB color output," par. 54) corresponding to respective points in the 3D representation of the area of real space ("The map defines an area for inventory locations or shelves where inventory items are positioned. In FIG. 3, a 2D location of the shelf unit shows an area formed by four coordinate positions (x1, y1), (x1, y2), (x2, y2), and (x2, y1). These coordinate positions define a 2D region on the floor 220 where the shelf is located," par. 86).
Kallakuri et al. are cited as per claim 1.
Claim 3
Regarding claim 3, Kallakuri et al. teach the method of claim 2, further including identifying updated regions of interest based on one or more updated images of the area of real space ("The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254).
Kallakuri et al. are cited as per claim 1.
Claim 4
Regarding claim 4, Kallakuri et al. teach the method of claim 2, wherein the monitored zone comprises a location within shelving at which a corresponding inventory item is expected to be found, and wherein the method includes: ("As described above, the semantic diffing events (or diff events) classify items put on or taken from shelves," par. 255) receiving the 3D representation of the area of real space and the flattened 2D image of the area of real space; ("creation of two-dimensional (2D) and three-dimensional (3D) maps," par. 86) processing the received the 3D representation of the area of real space and the flattened 2D image of the area of real space using the object classification model to identify a region of interest within the monitored zone, wherein the region of interest corresponds to a location for an expected inventory item; ("defining a 3D region in which inventory items are positioned on the shelf 1. Similar 3D regions are defined for inventory locations in all shelf units," par. 87) generating a label for the region of interest comprising one or more of (i) a set of boundaries indicating a location of the region of interest and (ii) an identifier of the expected inventory item for the region of interest; ("The second image processors 1606 use bounding boxes of hand images of subjects in the area of real space and perform time series analysis of the classification of hand images," par. 200) and storing the label for the region of interest at the location of the region of interest within a field of view of the camera ("The coordination logic component 1636 selects bounding boxes representing the inventory items having the same SKU and the same inventory event type (take or put) from multiple cameras with overlapping fields of view," par. 250).
Kallakuri et al. are cited as per claim 1.
Claim 8
Regarding claim 8, Kallakuri et al. teach the method of claim 1, further including: provisioning and initializing an instance of the cloud network, configuring network and IP management processes for the cloud network; ("Network nodes can be implemented in a cloud-based server system," par. 51) installing a node cluster configured to schedule and run a plurality of cloud containers; ("The system 100 includes cameras 114, network nodes hosting image recognition engines 112a, 112b, and 112n, a subject tracking engine 110 deployed in a network node 102 (or nodes) on the network, a subject database 140, a calibration database 150, a proximity (or location) events database 160, a feature descriptors and keypoints database 170, a proximity event detection and classification engine 180 deployed in a network node 104 (or nodes), a camera calibration engine 190 deployed in a network node (or nodes) 106 and a communication network or networks 181," par. 50) and connecting the computing device to the cloud network using an ephemeral wireless connection ("The communication is generally over a network such as a … wireless network," par. 68).
Kallakuri et al. are cited as per claim 1.
Claim 9
Regarding claim 9, Kallakuri et al. teach the method of claim 1, further including, for a plurality of cameras with overlapping corresponding fields of view connected to the computing device: ("Cameras 114 are connected to the tracking engine 110 through network nodes hosting image recognition engines 112a, 112b, and 112n. In one embodiment, the cameras 114 are installed in a shopping store (such as a supermarket) such that sets of cameras 114 (two or more) with overlapping fields," par. 53) processing respective sequences of frames of the overlapping corresponding fields of view ("classify background changes by processing the sequence of factored images," par. 244) to detect an occlusion in at least one field of view corresponding to a camera of the plurality of cameras, ("the system can store masks and unmodified images, and conditioned on an elsewhere computed region & time of interest, process the masks to determine the latest time before and earliest time after the time of interest in which the region is not occluded by a person. The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254) and generating a factored image, ("the mask logic component 1626 combines, such as by averaging or summing by pixel, sets of N masked images in the sequences of images to generate sequences of factored images for each camera," par. 244) wherein the generation of the factored images includes factoring that results in the detected occlusion being patched based on at least one other field of view corresponding to another camera of the plurality of cameras ("The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254).
Kallakuri et al. are cited as per claim 1.
Claim 10
Regarding claim 10, Kallakuri et al. teach the method of claim 1, further including storing, in an image storage, images captured by the camera ("The method can include storing the transformation information and images used to calibrate the cameras in a database," par. 14).
Kallakuri et al. are cited as per claim 1.
Claim 11
Regarding claim 11, Kallakuri et al. teach the method of claim 1, further including storing, in a region of interest database, at least one identified region of interest within the monitored zone ("Similar 3D regions are defined for inventory locations in all shelf units in the shopping store and stored as a 3D map of the real space (shopping store) in the maps database," par. 87).
Kallakuri et al. are cited as per claim 1.
Claim 13
Regarding claim 13, Kallakuri et al. teach a system including one or more processors and memory accessible by the processors, the memory loaded with computer instructions self-service installation of monitoring operations in an area of real space, which computer instructions, when executed on the processors, implement actions comprising: (The logic can be implemented using processors configured as described above programmed using computer programs stored in memory accessible and executable by the processors," par. 190) scanning the area of real space, using a sensor, to generate a three-dimensional (3D) representation of the area of real space; (the cameras 114 are installed in a shopping store (such as a supermarket) such that sets of cameras 114 (two or more) with overlapping fields of view are positioned over each aisle to capture images of real space in the store," par. 53) placing a camera at an initial location and orientation for monitoring a zone within the area of real space; ("Cameras 114 are placed and oriented such that areas of the floor 220 and shelves can be seen by at least two cameras," par. 72) configuring a computing device to be (i) connected to a cloud network hosting an image processing service ("The storage subsystem 2330 is an example of a computer readable memory comprising a non-transitory data storage medium, having computer instructions stored in the memory executable by a computer to perform all or any combinations of the data processing and image processing functions," par. 299) and (ii) couplable to the camera via a local connection, wherein the configuration of the computing device enables the computing device to mediate communications between the image processing service and the camera; (Network nodes can be implemented in a cloud-based server system. More than one virtual device configured as a network node can be implemented using a single physical device," par. 51) coupling the camera to the computing device using the local connection and a unique identifier associated with the camera; ("The data structure 1200 stores the subject-related data as a key-value dictionary. The key is a frame_number and the value is another key-value dictionary where the key is the camera_id," par. 188) and finetuning a placement of the camera to a calibrated location and orientation for monitoring the zone within the area of real space, wherein the finetuning is assisted by information received from a cloud-based application associated with the image processing service ("point correspondences are identified for each 3D scene using the results of the image recognition engines 112a-112n for the purposes of the external calibration," par. 77).
Kallakuri et al. are cited as per claim 1.
Claim 14
Regarding claim 14, Kallakuri et al. teach the system of claim 13, further including identifying a region of interest within the monitored zone based on an output received from an object classification model, ("The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254) wherein: the region of interest is a 3D space at which a particular object is expected to be located, ("As described above, the semantic diffing events (or diff events) classify items put on or taken from shelves," par. 255) and the output of the object classification model is determined by processing, as input, the 3D representation of the area of real space and a flattened 2D image ("FIG. 3 presents creation of two-dimensional (2D) and three-dimensional (3D) maps," par. 86) produced from RGB data ("Multiple types of sensors can be used, including for example ultrasound or RF sensors in addition to the cameras 114 that generate RGB color output," par. 54) corresponding to respective points in the 3D representation of the area of real space ("The map defines an area for inventory locations or shelves where inventory items are positioned. In FIG. 3, a 2D location of the shelf unit shows an area formed by four coordinate positions (x1, y1), (x1, y2), (x2, y2), and (x2, y1). These coordinate positions define a 2D region on the floor 220 where the shelf is located," par. 86).
Kallakuri et al. are cited as per claim 1.
Claim 15
Regarding claim 15, Kallakuri et al. teach the system of claim 14, wherein the monitored zone comprises a location within shelving at which a corresponding inventory item is expected to be found, and further including: ("As described above, the semantic diffing events (or diff events) classify items put on or taken from shelves," par. 255) receiving the 3D representation of the area of real space and the flattened 2D image of the area of real space; ("creation of two-dimensional (2D) and three-dimensional (3D) maps," par. 86) processing the 3D representation of the area of real space and the flattened 2D image using the object classification model to identify a region of interest within the monitored zone, wherein the region of interest corresponds to a location for an expected inventory item; ("defining a 3D region in which inventory items are positioned on the shelf 1. Similar 3D regions are defined for inventory locations in all shelf units," par. 87) generating a label for the region of interest comprising one or more of (i) a set of boundaries indicating a location of the region of interest and (ii) an identifier of the expected inventory item for the region of interest; ("The second image processors 1606 use bounding boxes of hand images of subjects in the area of real space and perform time series analysis of the classification of hand images," par. 200) and storing the label for the region of interest at the location of the region of interest within a field of view of the camera ("The coordination logic component 1636 selects bounding boxes representing the inventory items having the same SKU and the same inventory event type (take or put) from multiple cameras with overlapping fields of view," par. 250).
Kallakuri et al. are cited as per claim 1.
Claim 17
Regarding claim 17, Kallakuri et al. teach a non-transitory computer readable storage medium impressed with computer program instructions for self-service installation of monitoring operations in an area of real space, which computer program instructions when executed implement a method comprising: ("a computer readable memory comprising a non-transitory data storage medium, having computer instructions stored in the memory executable by a computer to perform all or any combinations of the data processing and image processing functions," par. 299) scanning the area of real space, using a sensor, to generate a three-dimensional (3D) representation of the area of real space; (the cameras 114 are installed in a shopping store (such as a supermarket) such that sets of cameras 114 (two or more) with overlapping fields of view are positioned over each aisle to capture images of real space in the store," par. 53) placing a camera at an initial location and orientation for monitoring a zone within the area of real space; ("Cameras 114 are placed and oriented such that areas of the floor 220 and shelves can be seen by at least two cameras," par. 72) configuring a computing device to be (i) connected to a cloud network hosting an image processing service ("The storage subsystem 2330 is an example of a computer readable memory comprising a non-transitory data storage medium, having computer instructions stored in the memory executable by a computer to perform all or any combinations of the data processing and image processing functions," par. 299) and (ii) couplable to the camera via a local connection, wherein the configuration of the computing device enables the computing device to mediate communications between the image processing service and the camera; (Network nodes can be implemented in a cloud-based server system. More than one virtual device configured as a network node can be implemented using a single physical device," par. 51) coupling the camera to the computing device using the local connection and a unique identifier associated with the camera; ("The data structure 1200 stores the subject-related data as a key-value dictionary. The key is a frame_number and the value is another key-value dictionary where the key is the camera_id," par. 188) and finetuning a placement of the camera to a calibrated location and orientation for monitoring the zone within the area of real space, wherein the finetuning is assisted by information received from a cloud-based application associated with the image processing service ("point correspondences are identified for each 3D scene using the results of the image recognition engines 112a-112n for the purposes of the external calibration," par. 77).
Kallakuri et al. are cited as per claim 1.
Claim 18
Regarding claim 18, Kallakuri et al. teach the non-transitory computer readable medium of claim 17, further including identifying a region of interest within the monitored zone based on an output received from an object classification model, ("The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254) wherein: the region of interest is a 3D space at which a particular object is expected to be located, ("As described above, the semantic diffing events (or diff events) classify items put on or taken from shelves," par. 255) and the output of the object classification model is determined by processing, as input, the 3D representation of the area of real space and a flattened 2D image ("FIG. 3 presents creation of two-dimensional (2D) and three-dimensional (3D) maps," par. 86) produced from RGB data ("Multiple types of sensors can be used, including for example ultrasound or RF sensors in addition to the cameras 114 that generate RGB color output," par. 54) corresponding to respective points in the 3D representation of the area of real space ("The map defines an area for inventory locations or shelves where inventory items are positioned. In FIG. 3, a 2D location of the shelf unit shows an area formed by four coordinate positions (x1, y1), (x1, y2), (x2, y2), and (x2, y1). These coordinate positions define a 2D region on the floor 220 where the shelf is located," par. 86).
Kallakuri et al. are cited as per claim 1.
Claim 19
Regarding claim 19, Kallakuri et al. teach the non-transitory computer readable medium of claim 17, further including: provisioning and initializing an instance of the cloud network, configuring network and IP management processes for the cloud network; ("Network nodes can be implemented in a cloud-based server system," par. 51) installing a node cluster configured to schedule and run a plurality of cloud containers; ("The system 100 includes cameras 114, network nodes hosting image recognition engines 112a, 112b, and 112n, a subject tracking engine 110 deployed in a network node 102 (or nodes) on the network, a subject database 140, a calibration database 150, a proximity (or location) events database 160, a feature descriptors and keypoints database 170, a proximity event detection and classification engine 180 deployed in a network node 104 (or nodes), a camera calibration engine 190 deployed in a network node (or nodes) 106 and a communication network or networks 181," par. 50) and connecting the computing device to the cloud network using an ephemeral wireless connection ("The communication is generally over a network such as a … wireless network," par. 68).
Kallakuri et al. are cited as per claim 1.
Claim 20
Regarding claim 20, Kallakuri et al. teach the non-transitory computer readable medium of claim 17, further including, for a plurality of cameras with overlapping corresponding fields of view connected to the computing device: ("Cameras 114 are connected to the tracking engine 110 through network nodes hosting image recognition engines 112a, 112b, and 112n. In one embodiment, the cameras 114 are installed in a shopping store (such as a supermarket) such that sets of cameras 114 (two or more) with overlapping fields," par. 53) processing respective sequences of frames of the overlapping corresponding fields of view ("classify background changes by processing the sequence of factored images," par. 244) to detect an occlusion in at least one field of view corresponding to a camera of the plurality of cameras, ("the system can store masks and unmodified images, and conditioned on an elsewhere computed region & time of interest, process the masks to determine the latest time before and earliest time after the time of interest in which the region is not occluded by a person. The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254) and generating a factored image, ("the mask logic component 1626 combines, such as by averaging or summing by pixel, sets of N masked images in the sequences of images to generate sequences of factored images for each camera," par. 244) wherein the generation of the factored images includes factoring that results in the detected occlusion being patched based on at least one other field of view corresponding to another camera of the plurality of cameras ("The system can then take the images from those two times, crop to the region of interest, and classify the background changes between those two crops," par. 254).
Kallakuri et al. are cited as per claim 1.
1st Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 5, 6, 7, and 16 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0407131 A1, (Kallakuri et al.) in view of US Patent Publication 2022 0383546 A1, (Bronicki et al.)
Claim 5
Regarding claim 5, Kallakuri et al. teach the method of claim 4 as noted above.
Kallakuri et al. do not explicitly teach all of processing captured images, labelled with the label for the region of interest, to detect a presence of an inventory item at the region of interest; in response to a detection of no inventory item being present at the region of interest, determining, that the region of interest is empty facing; and generating a report for the empty facing region of interest.
However, Bronicki et al. teach processing captured images, ("may acquire and transmit an up-to-date image of the area-of-interest," par. 140) labelled with the label for the region of interest, ("The real-time video stream may be augmented with markings identifying to the store associate an area-of-interest," par. 139) to detect a presence of an inventory item at the region of interest; ("sensors to determine empty spaces on the store shelves," par. 175) in response to a detection of no inventory item being present at the region of interest, determining, that the region of interest is empty facing; ("Based on detected products and/or empty spaces, determined using the first signals and second signals, the one or more processors may determine one or more aspects of planogram compliance," par. 185) and generating a report for the empty facing region of interest ("image processing unit 130 may generate may include identification of products, indicators of product quantity, indicators of planogram compliance, indicators of service-improvement events (e.g., a cleaning event, a restocking event, a rearrangement event, etc.), and various reports indicative of the performances of retail stores," par. 113).
Therefore, taking the teachings of Kallakuri et al. and Bronicki et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify region of interest monitoring and camera calibration techniques as taught by Kallakuri et al. to use inventory item detection methods as taught by Bronicki et al. The suggestion/motivation for doing so would have been that, “the monitoring of the online digital activities of customers 2422, 2424, and 2426 while in the retail store, may facilitate the detection of condition 2402 by processing unit 2404. Consequently, condition 2402 may be addressed conveniently and efficiently by automatically initiating a remedial action for condition” as noted by the Bronicki et al. disclosure in paragraph [0407], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that doing so would provide a comprehensive, real-time understanding of both customer intent and store inventory, significantly improving the precision of inventory tracking and condition detection; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 6
Regarding claim 6, Kallakuri et al. teach processing the subsequent image, that is labeled for a plurality of regions of interest ("The first image processors 1604 use locations of subjects and locations of inventory display structures to detect “proximity events” which are further processed to detect put and take events. The second image processors 1606 use bounding boxes of hand images of subjects in the area of real space and perform time series analysis of the classification of hand images," par. 200).
Kallakuri et al. do not explicitly teach all of corresponding to a plurality of locations for a particular inventory item; in response to determining that each of the plurality of regions of interest is empty facing, further determining that the particular inventory item is out of stock; and generating a report for the out of stock inventory item.
However, Bronicki et al. teach corresponding to a plurality of locations for a particular inventory item; ("image processing unit 130 may utilize suitably trained machine learning algorithms and models to perform the product identification," par. 113) in response to determining that each of the plurality of regions of interest is empty facing, further determining that the particular inventory item is out of stock; ("identify a target product as being out of stock, such as through an image analysis identifying empty store shelf space," par. 252) and generating a report for the out of stock inventory item ("Store managers and regional managers, as well as other stakeholders, may access custom dashboards and online reports to see how in-store conditions," par. 222).
Kallakuri et al. and Bronicki et al. are combined as per claim 5.
Claim 7
Regarding claim 7, Kallakuri et al. teach processing the subsequent images, labelled for the region of interest ("The first image processors 1604 use locations of subjects and locations of inventory display structures to detect “proximity events” which are further processed to detect put and take events. The second image processors 1606 use bounding boxes of hand images of subjects in the area of real space and perform time series analysis of the classification of hand images," par. 200).
Kallakuri et al. do not explicitly teach all of to detect a presence of an inventory item at the region of interest; in response to a detection of the inventory item being present at the region of interest, determining whether the detected inventory item matches the expected inventory item, wherein (i) in response to the detected inventory item matching the expected inventory item, further determining that the expected inventory item is correctly stocked, and (ii) in response to the detected inventory item not matching the expected inventory item, further determining that the detected inventory item is incorrectly stocked; and generating a report for one or more of: (i) a correctly inventory item and (ii) an incorrectly stocked inventory item.
However, Bronicki et al. teach to detect a presence of an inventory item at the region of interest; ("the one or more processors may identify products and their locations on the shelves, determine quantities of products within particular areas," par. 185) in response to a detection of the inventory item being present at the region of interest, determining whether the detected inventory item matches the expected inventory item, "Based on detected products and/or empty spaces, determined using the first signals and second signals, the one or more processors may determine one or more aspects of planogram compliance," par. 185) wherein (i) in response to the detected inventory item matching the expected inventory item, further determining that the expected inventory item is correctly stocked, and (ii) in response to the detected inventory item not matching the expected inventory item, further determining that the detected inventory item is incorrectly stocked; ("image processing unit 130 may generate may include identification of products, indicators of product quantity, indicators of planogram compliance, indicators of service-improvement events (e.g., a cleaning event, a restocking event, a rearrangement event, etc.), and various reports indicative of the performances of retail stores," par. 113) and generating a report for one or more of: (i) a correctly inventory item and (ii) an incorrectly stocked inventory item ("Store managers and regional managers, as well as other stakeholders, may access custom dashboards and online reports to see how in-store conditions," par. 222).
Kallakuri et al. and Bronicki et al. are combined as per claim 5.
Claim 16
Regarding claim 16, Kallakuri et al. teach the system of claim 15 as noted above.
Kallakuri et al. do not explicitly teach all of processing captured images, labelled with the label for the region of interest, to detect a presence of an inventory item at the region of interest; in response to a detection of no inventory item being present at the region of interest, determining, that the region of interest is empty facing; and generating a report for the empty facing region of interest.
However, Bronicki et al. teach processing captured images, ("may acquire and transmit an up-to-date image of the area-of-interest," par. 140) labelled with the label for the region of interest, ("The real-time video stream may be augmented with markings identifying to the store associate an area-of-interest," par. 139) to detect a presence of an inventory item at the region of interest; ("sensors to determine empty spaces on the store shelves," par. 175) in response to a detection of no inventory item being present at the region of interest, determining, that the region of interest is empty facing; ("Based on detected products and/or empty spaces, determined using the first signals and second signals, the one or more processors may determine one or more aspects of planogram compliance," par. 185) and generating a report for the empty facing region of interest ("image processing unit 130 may generate may include identification of products, indicators of product quantity, indicators of planogram compliance, indicators of service-improvement events (e.g., a cleaning event, a restocking event, a rearrangement event, etc.), and various reports indicative of the performances of retail stores," par. 113).
Kallakuri et al. and Bronicki et al. are combined as per claim 5.
2nd Claim Rejections - 35 USC § 103
Claims 12 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2021 0407131 A1, (Kallakuri et al.) in view of US Patent Publication 2018 0332235 A1, (Glaser)
Claim 12
Regarding claim 12, Kallakuri et al. teach the method of claim 1 as noted above.
Kallakuri et al. do not explicitly teach all of wherein the computing device is a power over ethernet (POE) sitebox.
However, Glaser teach wherein the computing device is a power over ethernet (POE) sitebox ("this may involve a single integrated cable for data and/or power connections such as using a power-over-ethernet (PoE) cable," par. 48).
Therefore, taking the teachings of Kallakuri et al. and Glaser as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify region of interest monitoring and camera calibration techniques as taught by Kallakuri et al. to use power and internet connectivity method as taught by Glaser. The suggestion/motivation for doing so would have been that because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, region of interest monitoring and camera calibration techniques as modified by power and internet connectivity method can yield a predictable result of remote continuous camera operation and centralized data transmission since using a PoE sitebox allows a single Ethernet cable to simultaneously provide both power and network connectivity, thereby eliminating the need for separate, bulky power supplies and redundant wiring near the camera. Thus, a person of ordinary skill would have appreciated including in region of interest monitoring and camera calibration techniques the ability to use power and internet connectivity method since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent Publication 2021 0409648 A1 to Kallakuri et al. discloses machine-learning method for optimizing camera placement to accurately track subjects handling items in a 3D space.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KARSTEN F LANTZ whose telephone number is (571) 272-4564. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Karsten F. Lantz/Examiner, Art Unit 2664
Date: 6/18/2026
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights