DETAILED ACTION
Remarks
This non-final office action is in response to the RCE filled on 02/06/2026. Claims 1, 2, 9, 10, 17 and 18 are amended. Claims 1-24 are pending and examined below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/06/2026 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 9, 10, 17 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2022/0383384 (“Bronicki”), and further in view of US 2024/0037752 (“Gottin”).
Regarding claim 1 (and similarly claim 9 and 17), Bronicki discloses a method comprising: tracking, by at least one imaging assembly, a location of an individual associated with a container (see at least [0154], where “As shown in FIG. 6A, the at least one processor contained in a second housing 504 may control a plurality of image capture devices 506… Controlling image capturing device 506 may include instructing image capturing device 506 to capture an image and/or transmit captured images to a remote server”; see also fig 5B, where 506 is camera. see also fig 6A, where location of a person/individual, 608 with cart inside a retail store is identified by using camera.);
detecting at least one of the container or at least one object within the container present in first image data captured by the at least one imaging assembly (see at least [0127], where “server 135 may execute an image processing algorithm to identify in received images one or more products and/or obstacles, such as shopping carts, people, and more.”; shopping cart is interpreted as container);
identifying at least one of the at least one object or a region of interest associated with the container present in the first image data captured by the at least one (see at least [0141], where “the robotic capturing devices may use input from sensors (e.g., image sensors, depth sensors, proximity sensors, etc.), to avoid collision with objects or people, and to complete the scan of the desired area of retail store 105.”);
determining, based on the identification, at least one of a value of at least one attribute of the at least one object or a first sub-area and a second sub-area of the region of interest (see at least [0354], where “the property associated with the first product may include one or more characteristics of the product, such as one or more visually discernable features attributed to the product or one or more features that may be detected by one or more of the plurality of sensors (e.g., size of the product, shape of the product, information on a label associated with the product, weight of the product, etc.).”; see also [0162]);
determining whether at least one of the value of the at least one attribute is greater than a first threshold or a ratio of the first sub-area and the second sub-area is less than a second threshold (see at least [0105], where “the characteristic of the product may be associated with the ornamental design of the product, the size of the product, the shape of the product, the colors of the product, the brand of the product, a logo or text associated with the product (e.g., on a product label), and more. In addition, embodiments of the present disclosure further include determining a confidence level associated with the determined type of the product…the confidence level may have a value between 1 and 10, alternatively, the confidence level may be expressed as a percentage.”; confidence level of product is determined based on characteristics of the product. Characteristics of the product is interpreted as attribute of the product. see also [0106], where “the system may compare the confidence level to a threshold. The term “threshold” as used herein denotes a reference value, a level, a point, or a range of values, for which, when the confidence level is above it (or below it depending on a particular use case), the system may follow a first course of action and, when the confidence level is below it (or above it depending on a particular use case), the system may follow a second course of action. The value of the threshold may be predetermined for each type of product or may be dynamically selected based on different considerations.); and
generating and transmitting a notification to a device when at least one of the value of the at least one attribute is greater than the first threshold or the ratio of the first sub-area and the second sub-area is less than the second threshold (see at least fig 15C, fig 27A-B, fig 18A-B, where product information is displayed to customer. See also fig 29, where the product is displayed to customer after determining product availability. Product availability (type, amount of product) is determined based on images, see citation above.), the notification being indicative of the location of the individual (see at least [0274], where “physical information about a shopper's in-store location, shopper's view direction 145 (i.e., the direction the shopper is looking), the shopper's view direction time 146 (i.e., how much time the shopper spends looking in a particular direction), and the products the shopper is picking up and/or otherwise inspecting are examples of physical information the disclosed system may collect and analyze to make a first determination about the product category actions of an in-store shopper…to making the first determination of shopper product category actions from information about shopper physical actions and the shopper location or environment”; see also [0279]) and instructions to the device to navigate to the individual based on the location (see at least [0329] and [0416]).
Bronicki does not disclose fixed overhead camera. However, Gottin discloses a method wherein object inside a warehouse is detected using image data received from cameras installed inside the warehouse, see at least [0037], fig 1 and fig 6.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Bronicki to incorporate the teachings of Gottin by including the above feature for reducing interference during movement for computing images by installing overhead cameras.
Regarding claim 2 (and similarly claim 10 and 18), Bronicki further discloses a method wherein the at least one imaging assembly is disposed within a venue and is configured to capture the first image data over at least a portion of a zone within the venue (see at least fig 6A, where camera installed inside the retail store is capturing image of a zone inside the store/venue),
the container is at least one of a cart, a basket, a bin, a platform truck, a hand truck, or a dolly (see at least fig 6A, where a shopping cart is shown).
Bronicki does not disclose the following limitation:
fixed overhead imaging assembly and the device is at least one of an autonomous mobile robot (AMR) transporting a container or an AMR integrated with a container.
However, Gottin further discloses a method wherein fixed overhead imaging assembly (see at least fig 1) and the device is at least one of an autonomous mobile robot (AMR) transporting a container or an AMR integrated with a container (see at least [0018] and fig 6).
Claim(s) 3, 11 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2022/0383384 (“Bronicki”), in view of US 2024/0037752 (“Gottin”), as applied to claim 1, 9 and 17 above, and further in view of US 2015/0187080 (“Kundu”).
Regarding claim 3 (and similarly claim 11 and 19), Bronicki further discloses a method wherein the at least one attribute of the at least one object is a weight of the at least one object (see at least [0181]).
Bronicki in view of Gottin does not disclose the following limitation:
the first sub-area is indicative of non-occupied space within the container and the second sub-area is indicative of occupied space within the container.
However, Kundu discloses a method wherein the first sub-area is indicative of non-occupied space within the container and the second sub-area is indicative of occupied space within the container (see at least [0059], where “During video analysis 150-1, the Cart Inspector 150-1 obtains a target image. The target image can be a video image 170-1 that shows the shopping cart 210 was empty at 2 o'clock when it was in the transaction area 200. In addition, the Cart Inspector 150-1 obtains a reference representation, which can be a predefined image of an empty cart 150-3.”; see also fig 3).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Bronicki in view of Gottin to incorporate the teachings of Kundu by including the above feature for reducing number of transaction and faster task completion by evaluation amount of space remain on the cart.
Claim(s) 4, 12 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2022/0383384 (“Bronicki”), and in view of US 2024/0037752 (“Gottin”), as applied to claim 1, 9 and 17 above, and further in view of US 2025/0029406 (“Kim”).
Regarding claim 4 (and similarly claim 12 and 20), Bronicki further discloses a method comprising localizing at least one of the detected container or the detected at least one object (see at least [0108], where “a trained machine learning algorithm may include an object detector, the input may include an image, and the inferred output may include one or more detected objects in the image and/or one or more locations of objects within the image.”; see also fig 6A, where location of cart is shown).
Bronicki in view of Gottin does not disclose the following limitation:
localizing at least one of the detected container or the detected at least one object by removing background noise from the first image data.
However, Kim discloses a method wherein localizing at least one of the detected container or the detected at least one object by removing background noise from the first image data (see at least [0150], where “the noise removal part 325 may identify a preset object in the image through segmentation based on artificial intelligence (AI) that has been previously machine-trained.”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Bronicki in view of Gottin to incorporate the teachings of Kim by including the above feature for increasing accuracy of object detection by removing noise.
Claim(s) 5-8, 13-16 and 21-24 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 2022/0383384 (“Bronicki”), and in view of US 2024/0037752 (“Gottin”), as applied to claim 1, 9 and 17 above, and further in view of US 2022/0292815 (“Donnelly”).
Regarding claim 5 (and similarly claim 13 and 21), Bronicki further discloses a method wherein identifying the at least one object comprises:
generating, by applying a feature extractor model to the first image data, at least one object descriptor indicative of one or more features of the detected at least one object (see at least [0117], [0231] and [0354]);
executing, by a visual search engine, a (see at least [0231] and [0117]); and
selecting, by the visual search engine, a known object corresponding to the detected at least one object from a ranked list of known objects, (see at least [0183] and [0513]).
Bronicki in view of Gottin does not disclose the following limitation:
a nearest neighbor search within a database storing one or more known object descriptors corresponding to respective image data of one or more known objects to determine a respective metric distance between the at least one object descriptor and the one or more known object descriptors; and
a known object corresponding to the detected at least one object from a ranked list of known objects, the ranked list of known objects being prioritized based on a respective metric distance between the at least one object descriptor and the one or more known object descriptors.
However, Donnelly discloses a method wherein executing a nearest neighbor search within a database storing one or more known object descriptors corresponding to respective image data of one or more known objects to determine a respective metric distance between the at least one object descriptor and the one or more known object descriptors (see at least [0088], where “a CNN.sub.1 classifies the target item by using the descriptor F of the target item to retrieve a most similar shape in a data set, rather than by supplying the descriptor F to a second stage CNN.sub.2. For example, all of the objects in the training set may be supplied to the first stage CNN.sub.1 to generate a set of known descriptors {F.sub.ds(m)}, where the index m indicates a particular labeled shape in the training data. A similarity metric is defined to measure the distance between any two given descriptors (vectors) F and F.sub.ds(m). Some simple examples of similarity metrics are a Euclidean vector distance and a Mahalanobis vector distance.”); and
selecting a known object corresponding to the detected at least one object from a ranked list of known objects, the ranked list of known objects being prioritized based on a respective metric distance between the at least one object descriptor and the one or more known object descriptors (see at least [0084], where “In max-pooling, the n feature vectors are combined to generate a single combined feature vector or descriptor F, where the j-th entry of the descriptor F is equal to the maximum among the j-th entries among the n feature vectors f The resulting descriptor F has the same length (or rank) as the n feature vectors f and therefore descriptor F can also be supplied as input to the second stage CNN.sub.2 to compute a classification of the object.”; see also [0072]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to have modified Bronicki in view of Gottin to incorporate the teachings of Donnelly by including the above feature for reducing shopping error by identifying the object using database models and similarity search.
Regarding claim 6 (and similarly claim 14 and 22), Bronicki further discloses a method wherein the feature extractor model is a machine learning model comprising a convolutional neural network classifier or visual transformer classifier trained on one or more of supervised learning tasks or unsupervised learning tasks (see at least [0110], where “analyzing image data (for example by the methods, steps and modules described herein) may comprise analyzing the image data and/or the preprocessed image data using one or more rules, functions, procedures, artificial neural networks, object detection algorithms”).
Regarding claim 7 (and similarly claim 15 and 23), Bronicki further discloses a method wherein for each known object represented in the database, the database stores at least one attribute of the known object including one or more of (i) a known object location, (ii) a known object weight, or (iii) a known object volume (see at least [0117], where “Server 135 may access database 140 to detect and/or identify products. The detection may occur through analysis of features in the image using an algorithm and stored data. The identification may occur through analysis of product features in the image according to stored product models…the product model may include a description of visual and contextual properties of the particular product (e.g., the shape, the size, the colors, the texture, the brand name, the price, the logo, text appearing on the particular product, the shelf associated with the particular product, adjacent products in a planogram, the location within the retail store, etc.)”).
Regarding claim 8 (and similarly claim 16 and 24), Bronicki further discloses a method wherein the at least one object descriptor and the one or more known object descriptors are indicative of one or more features comprising one or more of a shape, a color, a height, a width, or a length (see at least [0117]).
Bronicki in view of Gottin does not disclose the following limitation:
at least one object descriptor and the one or more known object descriptors correspond to vectors and the respective metric distance between the at least one object descriptor and the one or more known object descriptors corresponds to differences between respective vectors of the at least one object descriptor and the one or more known object descriptors.
However, Donnelly further discloses a method wherein the at least one object descriptor and the one or more known object descriptors correspond to vectors and the respective metric distance between the at least one object descriptor and the one or more known object descriptors corresponds to differences between respective vectors of the at least one object descriptor and the one or more known object descriptors (see at least [0015], where “The identification model includes feature vectors attributable to visual patterns of a tray or surgical tool, implant, fastener, or other object identified in each training dataset. The feature vectors can be combined into matrices to provide a 2-dimensional array of feature vectors.”; see also [0036], [0040], [0082-84]). Same motivation of claim 5 applies.
Response to Arguments
Applicant’s arguments with respect to claim 1-20 have been considered but are moot because the arguments do not apply to the new combination used in the current rejection that is due to the newly added claim amendments.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOHANA TANJU KHAYER whose telephone number is (408)918-7597. The examiner can normally be reached on Monday - Thursday, 7 am-5.30 pm, PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on 5712703976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SOHANA TANJU KHAYER/Primary Examiner, Art Unit 3657