DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed inventions absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4-6, 11, 12 and 14 under 35 U.S.C. 103 as being unpatentable over U.S. Pat. No. 10296814 to Kumar et al. (hereinafter “Kumar”) in view of SKU-Patch: Towards Efficient Instant Segmentation for Unseen Objects in Auto-Store to Yang et al. (hereinafter “Yang”) and in further view of U.S. Pat. Appl. Pub. No. 20250131357 (filed 21 October 2023) to Ranatunga et al. (hereinafter “Ranatunga”).
Regarding claim 1, Kumar teaches an apparatus for using computer vision for product identification in association with a self-service retail market environment, the apparatus comprising:
a processing system (Kumar, FIG. 7, “Server System 203”) that includes one or more memories (Kumar, FIG. 7, “Memory 712”) and one or more processors (Kumar, FIG. 7, “Processor 700”) coupled with the one or more memories (See Kumar at FIG. 7), the processing system configured to cause the apparatus to:
obtain (Kumar, FIG. 4, steps 402, 404, 600, 406, 408 and 410), via an imaging device (Kumar, col. 3, ll. 50-55, “cameras”) associated with a receiving area for inventory order fulfillment (Kumar, col. 3, ll. 14-17, “store them until a user ordered or retrieves one or more of the items”; col. 3, ll. 50-55, “For example, the receiving area 120 may include cameras that capture images of the items as they are unpacked and/or otherwise prepared for storage. These images may be provided to and processed by the inventory management system 150 to identify features of the items and to determine an arrangement of those features.”; FIG. 2 shows a person holding item 207.), first image data corresponding to a first product item having a product type (Kumar, col. 3, ll. 28-34, “the inventory management system 150 (which, as described below, may include one or more software applications executing on a computer system) may be updated to reflect the type, quantity, condition, cost, location, images or any other suitable parameters with respect to newly received items 135.”); and
determine, in association with the receiving area, the product type (Kumar, col. 3, ll. 26-34, “Upon being received from a supplier at receiving area 120, items 135 may be prepared for storage. For example ... the inventory management system 150 ... may be updated to reflect the type ... with respect to newly received items 135.”), but does not teach that which is explicitly taught by Yang.
Yang teaches generate, using at least the first image data, a computer vision component associated with at least the product type (Yang, section III.A, “we tokenize the input image and an SKU patch for feature extraction, and use learnable object queries to predict the instance-level categories, positions, and masks”; section IV.A, “For AutoStore dataset [9] with 1,000 images, the training takes about 17h and the inference runs at around 11.0 Fps on a single GeForce GTX 1080 Ti GPU. We use Swin-T (Tiny) as our base architecture of Swin Transformer and initialize the length of the object queries K as 200, i.e., the maximum predicted instance number is 200.”), wherein the processing system, to cause the apparatus to generate the computer vision component, is configured to cause the apparatus to train a computer vision model using at least the first image data and a product type identifier corresponding to the product type (Yang, section I, “Importantly, SKU patches can be collected with very little effort without requiring any image annotation: it takes only a few seconds to capture 10 patches for an SKU with an industrial collection system [10], [11]. So, the challenges lie in how to extract vision contexts from the SKU patches and use the patch-level information to guide the image-level instance segmentation.”); and
provide the computer vision component for use with a merchandiser device (The phrase “for use” is an intended use of the “computer vision component” and the “merchandiser device” is equivalent to a generic computer.), wherein the computer vision component is configured to facilitate, based on second image data, identification of a second product item having the product type (Yang, section IV.G, “We deploy our method in a storehouse setting using the Nachi MZ07 robot arm. Our method is responsible for generating precise segmentation results for a given scenario with unseen SKUs, then the system selects one of the results by heuristic height and point cloud smoothness analysis.”).
Kumar discloses an image-based inventory management system that updates a database as images of products are captured and verified using a visual code. See Kumar at col. 12, ll. 46-63. Thus, Kumar shows that it was known in the art before the effective filing date of the claimed invention to use computer vision to maintain an inventory of merchandise, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, reducing recognition error of product images. Yang discloses SKU-Patch, an instance segmentation approach that adapts to unseen SKU images while maintaining the ability to categorize different types of products. Thus, Yang shows that it was known in the art before the effective filing date of the claimed invention to train product classification models to detect and categorize seen and unseen SKUs, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention reducing recognition error of product images.
A person of ordinary skill in the art would have been motivated to combine Yang’s SKU-Patch with Kumar’s inventory management system and replace the visual code with a SKU, to thereby associate the database of tracked inventory with object instances containing corresponding SKU images and associated data for classification and/or segmentation so that the managed inventory can be updated from any location with a computer, such as a local grocery store. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of more comprehensive tracking of relevant information about each type of product.
Kumar in view of Yang does not teach that which is explicitly taught by Ranatunga.
Ranatunga teaches a picking station (Ranatunga, par. 38, “The system captures data through multiple cameras at designated points, such as the point where containers arrive at a picking station, human-operated picking stations, and robotic picking stations.”; par. 43, “The second camera monitors human picking stations 106. At these stations, the camera observes the methods human operators use to pick items from containers. Specific metrics captured include the type of grasp used by human operators, which is classified according to a standardized grasp taxonomy, and the smoothness of the pick path.”; Figure 5 shows a person holding/grasping a product imaged by a camera).
Kumar and Yang are analogous to the claimed invention for the reasons provided above. Ranatunga discloses a system that uses computer vision to capture manual picking at picking stations and robotic picking to optimize for successful and accurate picking in a warehouse. Thus, Ranatunga shows that it was known in the art before the effective filing date of the claimed invention to apply computer vision in a warehouse setting to a scene depicting a picking station, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, reducing recognition error of product images.
A person of ordinary skill in the art would have been motivated to combine Ranatunga’s camera layout and bin picking monitoring processing with the inventory management system of Kumar in view of Yang, to thereby manage inventory in either manual or robotic environments of bin picking including instances of pickers holding products as they are imaged by cameras oriented towards picking stations. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of extending the applicability of the system to additional types and configurations of warehouses or other handling facilities.
Regarding claim 4, Kumar in view of Yang and in further view of Ranatunga teaches the apparatus of claim 1, wherein the product type identifier comprises a stock keeping unit (SKU) (Yang, section III.A, “we tokenize the input image and an SKU patch for feature extraction, and use learnable object queries to predict the instance-level categories, positions, and masks”).
The rationale for obviousness is the same as provided for claim 1.
Regarding claim 5, Kumar in view of Yang and in further view of Ranatunga teaches the apparatus of claim 1, wherein, to cause the apparatus to train the computer vision model, the processing system is further configured to:
cause the apparatus to obtain, via the imaging device, third image data corresponding to a third product item having an additional product type (Multiple types of product instances are identified. See Yang at Figure 3. The model is trained using 40 categories of products. See Yang at section IV.B); and
train the computer vision model based at least in part on the third image data (Yang, section III.A, “the training takes about 17h and the inference runs at around 11.0 Fps on a single GeForce GTX 1080 Ti GPU.”).
The rationale for obviousness is the same as provided for claim 1.
Regarding claim 6, Kumar in view of Yang and in further view of Ranatunga teaches the apparatus of claim 1, wherein the imaging device comprises one or more cameras (Kumar, col. 3, ll. 50-55, “the receiving area 120 may include cameras that capture images of the items as they are unpacked and/or otherwise prepared for storage.”), wherein each of the one or more cameras has a respective orientation relative to the picking station (Kumar, col. 3, ll. 50-55, “capture images of the items as they are unpacked and/or otherwise prepared for storage.”).
Claim 11 substantially corresponds to claim 1 by reciting a method corresponding to the functions of the apparatus of claim 1.
The rationale for obviousness is the same as provided for claim 1.
Regarding claim 12, Kumar in view of Yang and in further view of Ranatunga teaches the method of claim 11, wherein training the computer vision component comprises:
receiving, from a computing device associated with the picking station, labeling data indicative of the product type identifier (The training dataset is labeled. See Yang at section IV.B, “40 training categories”); and
associating the product type identifier with the first image data (Yang, section III.A, “we tokenize the input image and an SKU patch for feature extraction, and use learnable object queries to predict the instance-level categories, positions, and masks”).
The rationale for obviousness is the same as provided for claim 1.
Regarding claim 14, Kumar in view of Yang and in further view of Ranatunga teaches the method of claim 12, wherein the labeling data comprises the product type identifier (Yang, section III.A, “we tokenize the input image and an SKU patch for feature extraction, and use learnable object queries to predict the instance-level categories, positions, and masks”).
The rationale for obviousness is the same as provided for claim 1.
Claims 2, 3, 13 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Yang, in view of Ranatunga, and further in view of U.S. Pat. Appl. Pub. No. 20190333039 to Glaser et al. (hereinafter “Glaser”).
Regarding claim 2, Kumar in view of Yang and in further view of Ranatunga teaches the apparatus of claim 1, wherein the first image data is associated with a bin-packing operation in which the first product item is removed from a storage component of the picking station and placed in a stocking bin (Figure 2 depicts a stocking bin, i.e., shelves that contain like products within a space assigned to a specific location. See Kumar at col. 4, ll. 1-6, “After arriving through receiving area 120, items 135 may be stored within storage area 130. In some implementations like items 135 may be stored or displayed together in bins, on shelves or via other suitable storage mechanisms, such that all items 135 of a given kind are stored in one inventory location within the storage area 130.”), the first image data corresponding to one or more images of a packer (‘picker’ and ‘packer’ are considered interchangeable terms, as they both correspond to a human in an image processed by a computer vision system. Furthermore, since both are people, they correspond to one another.) holding the first product item (See Kumar at Figure 2), but does not teach that which is explicitly taught by Glaser.
Glaser teaches a stocking bin associated with the self-service retail market environment (Glaser, par. 91, “CV processing of image data of stocking activity and/or signage in the environment may be used to facilitate detection and classification of items. For example, detecting item identity can include detecting a stocking event through detecting of a work and stocking activity gestures which may include detecting items from a transient storage item (e.g., a stocking bin, box, or cart) to a permanent or substantially stationary (e.g., remaining in place in the environment for longer than 24 hours)”; par. 49, “The virtual cart system 140 may enable forms of automated checkout such as automatic self-checkout (e.g., functionality enabling a user to select items and walk out) or accelerated checkout (e.g., selected items can be automatically prepopulated in a POS system for faster checkout). Product transactions could even be reduced to per-item transactions (e.g., purchases or returns based on the selection or de-selection of an item for purchase). A virtual cart or other suitable accounting for customer related actions is preferably maintained for the duration of a customer's shopping session.”).
Kumar, Yang and Ranatunga are analogous to the claimed invention for the reasons provided above. Glaser discloses an automated self-checkout system that uses computer vision to monitor a user as they shop and acquire various products, and also to detect a stocking condition when products are removed from a stocking bin. Thus, Glaser shows that it was known in the art before the effective filing date of the claimed invention to apply computer vision to monitor a stocking bin as part of an inventory management system that also manages self-checkout kiosks, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, reducing recognition error of product images.
A person of ordinary skill in the art would have been motivated to combine Glaser’s stocking event detection and self-checkout system with the inventory management system of Kumar in view of Yang and in further view of Ranatunga, to thereby manage inventory in either manual or robotic environments of bin picking as products are received and then moved to a stocking bin before they are ultimately used to replace products purchased at a self-checkout kiosk. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of more accurately tracking inventory throughout the entire process of a product being received at a warehouse to being purchased at a self-checkout kiosk.
Regarding claim 3, Kumar in view of Yang, in view of Ranatunga and in further view of Glaser teaches the apparatus of claim 2, wherein the processing system, to cause the apparatus to determine the product type, is configured to cause the apparatus to:
obtain a storage component identifier corresponding to the storage component (Kumar, col. 4, ll. 1-20, “In general, the inventory management system 150 may maintain a mapping or location information identifying where within the materials handling facility each item is stored. Each inventory item may be associated with the corresponding inventory location in which it is stored and the association may be maintained in an inventory data store 715 (FIG. 7) accessible by the inventory management system”); and
access a database (Kumar, col. 4, ll. 1-20, “accessible by the inventory management system”) comprising a stored indication of an association between the storage component identifier and the product type identifier (Kumar, col. 15, ll. 38-46, “As used herein, the term "data store" refers to any device or combination of devices capable of storing, accessing and retrieving data, which may include any combination and 40 number of data servers, databases, data storage devices and data storage media, in any standard, distributed or clustered environment.”).
Claim 13 substantially corresponds to claim 3 by reciting a method corresponding to the functions of the apparatus of claim 3.
The rationale for obviousness is the same as provided for claim 2.
Claim 16 substantially corresponds to claim 2 by reciting a method corresponding to the functions of the apparatus of claim 2.
The rationale for obviousness is the same as provided for claim 2.
Regarding claim 17, Kumar in view of Yang, in view of Ranatunga and in further view of Glaser teaches the method of claim 16, further comprising:
determining, based on the first image data corresponding to the first product item, that the first product item has been placed in the stocking bin (Kumar, FIG. 2, “You Just Picked Product A”); and
providing, to an inventory management system associated with the merchandiser device, an inventory indication corresponding to the first product item (Kumar, col. 3, ll. 9-13, “an inventory management system 150 configured to interact with each of receiving area 120, storage area 130, transition area 140 and/or users within”; col. 4, ll. 16-20, “Each inventory item may be associated with the corresponding inventory location in which it is stored and the association may be maintained in an inventory data store 715 (FIG. 7) accessible by the inventory management system 150.”).
Claim 18 substantially corresponds to claim 1 by reciting a merchandiser device that corresponds to the apparatus of claim 1, and primarily differing by also reciting:
obtain a computer vision component (i.e., the resulting trained model. See Yang as cited above.), wherein the computer vision component is based on a computer vision model (i.e., the initial state of the model before training. See Yang as cited above.).
The rationale for obviousness is the same as provided for claim 1.
Kumar in view of Yang and in further view of Ranatunga does not teach that which is explicitly taught by Glaser.
Glaser teaches facilitate a product transaction operation based on determining that the second product item has the product type (Glaser, par. 31, “the CV monitoring system 110 in one exemplary implementation is used to track a virtual cart of subjects for offering automated checkout.”).
The rationale for obviousness is the same as provided for claim 2.
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Yang, in view of Ranatunga, and in further view of U.S. Pat. Appl. Pub. No. 20230098811 to Shevtsov et al. (hereinafter “Shevtsov”).
Regarding claim 7, Kumar in view of Yang and in further view of Ranatunga teaches the apparatus of claim 1, but does not teach that which is explicitly taught by Shevtsov.
Shevtsov teaches wherein the computer vision model comprises a localized model associated with the merchandiser device (Shevtsov, par. 15, “described embodiments of the computer vision model also provide improvements to other supporting systems as well as ongoing improvements to the computer vision model itself to adapt a generalized model to a local environment and the imaging conditions therein.”; par. 30, “the computer vision model 210 may be locally hosted on each kiosk 110 of a deployment (e.g., as a separate instance for every individual kiosk 110 in a given deployment), hosted in a central server connected to each kiosk 110 in a deployment (e.g., as a central model for use by several terminals in a given store), or in a server used across deployments (e.g., as a general or baseline model for use in different stores). Local variation (e.g., in lighting conditions, in capabilities of the imaging system 140, product availability, etc.) may be localized to given kiosks 110 based on permutation files or more locally hosted instances of the computer vision model 210. The permutations allow for localized adjustments to account for regional or store-by-store variations in items (e.g., avocados in Texas may be larger than in Alaska, store A may sell tomatoes with a different hue than store B), or localized imaging conditions (register A may be under fluorescent lights, while register B is under incandescent lights) to thereby adjust to different characteristics in the image 220 collected by different imaging systems 140.”).
Kumar, Yang and Ranatunga are analogous to the claimed invention for the reasons provided above. Shevtsov discloses using computer vision to adapt a generalized modal to a local retail environment. Thus, Shevtsov shows that it was known in the art before the effective filing date of the claimed invention to adapt computer vision models to local conditions, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, reducing recognition error of product images.
A person of ordinary skill in the art would have been motivated to combine Shevtsov’s model adaptation to the model of Kumar in view of Yang and in further view of Ranatunga, to thereby adapt the model to local lighting and viewing conditions in a store. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of increasing the accuracy of the model when deployed locally.
Regarding claim 8, Kumar in view of Yang, in view of Ranatunga and in further view of Shevtsov teaches the apparatus of claim 7, wherein the imaging device comprises one or more cameras (Kumar, col. 3, ll. 50-55, “the receiving area 120 may include cameras that capture images of the items as they are unpacked and/or otherwise prepared for storage.”), and wherein at least one of the one or more cameras has an orientation that corresponds to an orientation of a merchandiser camera associated with the merchandiser device (Shevtsov, par. 24, “computer vision model 210 is trained using various supervised learning data sets according to a machine learning scheme to receive various images 220a-e (generally or collectively, image 220) from the imaging system of various kiosks 110 to provide a consistent output.”; The training uses the same camera orientation as the kiosk cameras.).
The rationale for obviousness is the same as provided for claim 7.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Yang, in further view of Ranatunga, in view of Shevtsov and in further view of Context-Aware Hybrid Classification System for Fine-Grained Retail Product Recognition to Baz et al. (hereinafter “Baz”).
Regarding claim 9, Kumar in view of Yang, in view of Ranatunga and in further view of Shevtsov teaches the apparatus of claim 7, wherein the imaging device comprises a first camera (Kumar, col. 3, ll. 50-55, “the receiving area 120 may include cameras that capture images of the items as they are unpacked and/or otherwise prepared for storage.”), wherein a first illumination device associated with the camera has an orientation (Kumar, col. 2, ll. 48-58, “When the item is later imaged for identification under the same or similar conditions, the now stored item image information can be used to verify the identity of the item. By automatically updating the item images data store with additional images of items and/or additional item image information, the item identification process improves and accounts for changes in the environment. For example, if the lighting changes, once images captured under the new lighting are obtained and associated with the item, the identification process will accurately identify the item.”), and wherein the second camera is associated with the merchandiser device (Shevtsov, par. 23, “The imaging system 140 can also include one or more external cameras 130, which may be particular to a given kiosk 110 or shared among several kiosks 110 to optionally provide different viewing angles and image capturing capabilities to help identify the unpackaged items 120. In various embodiments, the imaging system 140 includes various motors to reposition the various cameras to focus in different areas, light sources (including flashes) to provide additional illumination for the unpackaged items 120, and range finders to aid in focusing or determining a distance to an unpackaged item 120.”), but does not teach that which is explicitly taught by Baz.
Baz teaches a first illumination device has an orientation that corresponds to an orientation of a second illumination device (Baz, section I, “Fine–grained classification is one of the challenging problems in computer vision [1–3]. In retail stores, there are a large number of fine-grained product classes and many products have similar appearance in terms of shape, color, texture and metric size. Besides, the product images are captured under real world conditions. So, the captured images are very likely to suffer from many problems such as different viewing angles, blurriness, occlusions, unexpected background parts, and very different lighting conditions. Such complications in the images make the retail product recognition problem more challenging. Accordingly, an effective product classification system needs further information in addition to knowledge obtained from the product image.”).
Kumar, Yang, Ranatunga and Shevtsov are analogous to the claimed invention for the reasons provided above. Baz discloses a classification system for retail product recognition and states that when computer vision models are deployed, they can suffer due to the local conditions deferring from the conditioned used to train the model, including viewing angle and lighting conditions. Baz implies that in order to not have this negative effect, one would do the opposite: ensure that the model is trained using the same or similar conditions as the deployed environment wherever possible. Thus, Baz shows that it was known in the art before the effective filing date of the claimed invention to train computer vision models using the same lighting conditions and camera orientations as the deployed environment to avoid poor performance, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, reducing recognition error of product images.
A person of ordinary skill in the art would have been motivated to combine Baz’s training strategy with the computer vision model of Kumar in view of Yang, in view of Ranatunga and in further view of Shevtsov to thereby train the model under the same lighting and camera orientation conditions as the point-of-sale system deployed in a local retail environment. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of increased accuracy.
Claims 10 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Yang, in view of Ranatunga, and in further view of U.S. Pat. Appl. Pub. No. 20220223019 to Shakedd et al. (hereinafter “Shakedd”).
Regarding claim 10, Kumar in view of Yang and in further view of Ranatunga teaches the apparatus of claim 1, but does not teach that which is explicitly taught by Shakedd.
Shakedd teaches wherein the processing system is further configured to cause the apparatus to generate, based at least in part on the computer vision model, a first planogram associated with the merchandiser device (Shakedd, par. 379, “In some embodiments, the at least one processor may retrieve one or more visual, textual, iconographic, schematic, contextual or other representations of one or more wirelessly tagged products from a data storage (e.g., data structure 11006 of FIG. 11) based on product classification information to generate a planogram.”; par. 367, “In some embodiments, the classification information includes a product model code and wherein the planogram indicates on the map a location of at least one group of products sharing the product model code. As used herein, product model code may include a number, code, or other forms of data uniquely associated with a type of product, object, material, or any other item, as part of a stock management system, inventory keeping system, or other such data management platform. Examples of product model codes include SKU codes, EPC codes, barcodes, ISBN codes, product numbers, part numbers, catalog numbers, or any other identifying indicia. A product model code may include an identifier, e.g., a textual, graphical, numerical, alphanumeric, or digital representation that describes a model, brand name, manufacturer, or other classification information associated with a product.”).
Kumar, Yang and Ranatunga are analogous to the claimed invention for the reasons provided above. Shakedd discloses a theft prevention system deployed in a retail environment that generates planograms from a computer vision model that recognizes product identifiers including SKUs. Thus, Shakedd shows that it was known in the art before the effective filing date of the claimed invention to generate planograms from computer vision models that track retail products, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, reducing recognition error of product images.
A person of ordinary skill in the art would have been motivated to combine Shakedd’s planogram generation with the inventory management system of Kumar in view of Yang and in further view of Ranatunga, to thereby use the computer vision model to generate planograms from the stored product data including images and associated metadata (e.g., location, SKU). Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of providing the initial training of the model at a warehouse with a more accurate representation of how the products will appear in a local retail environment, which better informs the training to produce a more robust classifier.
Claim 15 substantially corresponds to claim 10 by reciting a method corresponding to the functions of the apparatus of claim 10.
The rationale for obviousness is the same as provided for claim 10.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Yang, in view of Ranatunga, in view of Glaser, and in further view of Shevtsov.
Claim 19 substantially corresponds to claim 8 by reciting a merchandiser device that corresponds to the apparatus of claim 8, primarily differing by specifying a warehouse camera (See Kumar at col. 2, l. 62) which corresponds to the “one or more cameras” of claim 8.
Kumar, Yang, Ranatunga, Glaser, and Shevtsov are analogous to the claimed invention for the same reasons provided above. The rationale for obviousness is the same as provided for claim 7.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Kumar in view of Yang, in view of Ranatunga, in view of Glaser, in view of Shevtsov, and in further view of Baz.
Claim 20 substantially corresponds to claim 9 by reciting a merchandiser device that corresponds to the apparatus of claim 9, primarily differing by specifying a warehouse camera (See Kumar at col. 2, l. 62) which corresponds to the “first camera” of claim 9.
Kumar, Yang, Ranatunga, Glaser, Shevtsov and Baz are analogous to the claimed invention for the same reasons provided above. The rationale for obviousness is the same as provided for claim 9.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN P POTTS whose telephone number is (571)272-6351. The examiner can normally be reached M-F, 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN P POTTS/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights