DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action responds to the amendment and argument filed by applicant on April 27, 2026, in response to the Office Action mailed on January 29, 2026.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1–10 and 12-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (an abstract idea) without reciting significantly more than the judicial exception.
Under the 35 U.S.C. §101 subject matter eligibility two-part analysis, Step 1 addresses whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter. See MPEP §2106.03. If the claim does fall within one of the statutory categories, it must then be determined in Step 2A [prong 1] whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea). See MPEP §2106.04. If the claim is directed toward a judicial exception, it must then be determined in Step 2A [prong 2] whether the judicial exception is integrated into a practical application. See MPEP §2106.04(d). Finally, if the judicial exception is not integrated into a practical application, it must additionally be determined in Step 2B whether the claim recites "significantly more" than the abstract idea. See MPEP §2106.05.
Examiner note: The Office's 2019 Revised Patent Subject Matter Eligibility Guidance (2019 PEG) is currently found in the Ninth Edition, Revision 10.2019 (revised June 2020) of the Manual of Patent Examination Procedure (MPEP), specifically incorporated in MPEP §2106.03 through MPEP §2106.07(c).
Step 1
Claims 1–10, 12-21 are directed to one of the four statutory categories, namely a machine (claims 1–10 and 14–21), a process (claim 12), and a manufacture in the form of a non-transitory computer-readable storage medium (claim 13). Accordingly, the claims satisfy Step 1 of the eligibility analysis.
Step 2A, Prong One
Claims 1–10, 12-21 recite an abstract idea.
Representative independent claim 1 recites limitations including: acquiring a plurality of images of a product shelf; specifying a stockout region; comparing stockout regions; selecting one image based on the comparison; identifying products associated with the stockout region;
estimating a stockout product; generating planogram data.
These limitations describe collecting information, analyzing information, evaluating information, comparing information, recognizing patterns, and generating information for inventory management.
The claimed operations constitute mental processes because they describe observations, evaluations, comparisons, judgments, and conclusions that can practically be performed in the human mind or with the aid of pen and paper.
Additionally, the claims are directed to managing retail inventory, identifying missing merchandise, and generating merchandising information used for product placement and stocking decisions.
Managing inventory, merchandising, planogram generation, and stock replenishment are commercial interactions and constitute certain methods of organizing human activity, including commercial or business relations.
Accordingly, claims 1–10, 12-21 recite the following abstract ideas: mental processes; and certain methods of organizing human activity. Therefore, the claims recite a judicial exception.
Step 2A, Prong Two
The claims do not integrate the judicial exception into a practical application.
Although claim 1 additionally recites: a memory; at least one processor; image acquisition; image storage; generation of planogram data; these additional elements merely use generic computer technology as tools to perform the abstract idea.
The processor performs generic data processing functions including: acquiring data; comparing data; selecting data; estimating products; generating output.
The memory merely stores information.
The claimed images merely provide data inputs.
The planogram data merely represents information generated from the abstract analysis.
The claims do not improve: computer technology; image-processing technology; machine vision algorithms; image acquisition hardware; computer architecture; memory management;
processor operation; network communications; or another technology.
Instead, the computer components merely automate what retail personnel traditionally perform when reviewing shelves, determining empty locations, identifying likely missing products, and preparing planograms.
The additional limitations therefore merely implement the abstract idea using generic computing components.
Accordingly, the judicial exception is not integrated into a practical application.
Step 2B
The claims do not include additional elements that amount to significantly more than the judicial exception.
The additional elements include: processor; memory; image data; product shelf images;
product information; planogram data; recognized shelf labels; marketed product information.
These components are described at a high level of generality and perform only well-understood, routine, and conventional computer functions including: storing information;
retrieving information; receiving images; processing images; comparing information; selecting information; generating output.
The specification does not disclose any specialized processor architecture, unconventional image acquisition hardware, improved neural network architecture, improved image recognition algorithm, or other technological improvement.
Instead, the claims merely automate an existing merchandising workflow using generic computing components.
Limiting the abstract idea to planogram generation or stockout estimation within a retail environment merely limits the abstract idea to a particular field of use and does not amount to significantly more.
Likewise, reciting image recognition, shelf labels, adjacent products, product weights, product sizes, numbers of pieces sold, or stockout measurements merely represents additional data considered during the abstract analysis.
These limitations do not improve computer functionality and merely improve the quality of the business decision itself.
Accordingly, the additional elements, individually and in ordered combination, do not amount to significantly more than the abstract idea.
Dependent Claims
Claims 2–10 and 14–21 recite additional limitations including: selecting images below a stockout threshold; selecting images having the least stockout region; identifying stockout candidates; using product information including sales, weight, size, and pricing; positional relationships of stockout regions; recognizing shelf labels; counting product types; calculating display widths; selecting recent images; selecting adjacent products; restricting product information to displayed products.
These limitations merely specify additional information to collect, analyze, compare, or evaluate during implementation of the abstract idea.
The additional limitations represent insignificant extra-solution activity and refinement of the abstract analysis rather than a technological improvement.
Accordingly, these dependent claims likewise fail to integrate the judicial exception into a practical application and do not recite significantly more than the judicial exception.
Independent claims 12 and 13 recite substantially the same abstract idea as claim 1 in method and computer-readable medium formats and therefore are rejected for the same reasons.
Accordingly, claims 1–10, 12-21 are directed to judicial exceptions, namely mental processes and certain methods of organizing human activity, implemented using generic computer components. The claims do not integrate the judicial exception into a practical application and do not include additional elements amounting to significantly more than the judicial exception. Therefore, claims 1–10, 12-21 are not directed to patent-eligible subject matter under 35 U.S.C. §101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1–10, 12, and 13 are rejected under 35 U.S.C. § 103(a) as being unpatentable over Chaubard (US 2020/0005225 A1) in view of Shaw et al. (US 2021/0042588 A1, hereinafter Shaw).
With respect to claims 1, 12 and 13 Chaubard discloses a planogram data generation device comprising a memory and at least one processor coupled to the memory, wherein the processor executes modules including an image collection module, product detection module, out-of-stock detection module, user interface module, and data store for processing shelf images and generating inventory information (Chaubard ¶¶ 14–15, 27–29; Fig. 9).
acquiring a plurality of first images including a product shelf for displaying products; (Chaubard teaches cameras mounted on store shelves capturing images of product display areas and periodically obtaining shelf images during operation, ¶¶ 2–4, 14–16, 24–28; Figs. 1–4.)
specifying a stockout region of the product shelf included in the plurality of first images; (Chaubard identifies product display locations using product labels, generates bounding boxes corresponding to product display regions, analyzes those regions for voids, and identifies regions where products are absent or less than full, thereby specifying stockout regions, ¶¶ 16–28; Figs. 2–4.)
comparing an amount of the stockout region of the plurality of first images; (Chaubard periodically captures additional shelf images and compares the occupancy status of corresponding product display regions to determine whether products are missing or shelves are less than full, thereby comparing stockout conditions across images, ¶¶ 3, 24–29.)
determining a second image from among the plurality of first images based on the comparison of the amount of the stockout region, the second image having an amount of the stockout region that is less than an amount of the stockout region of at least one other first image among the plurality of the first images; (Chaubard teaches obtaining onboarding images and subsequent deployment images, evaluating corresponding shelf regions over time, and utilizing images reflecting differing stock conditions to determine product availability, ¶¶ 2–4, 15–29.)
generating planogram data on the product shelf based on the second image.
Chaubard teaches identifying products associated with shelf regions, maintaining positional information for product display areas, and using the identified product locations for inventory management and shelf monitoring (¶¶ 18–22, 27–29).
However, Chaubard does not expressly teach selecting, from multiple images, a second image having a lesser stockout region specifically for generating planogram data and generating planogram data from the selected image.
Shaw teaches this limitation by disclosing capturing multiple shelf images, identifying products from the shelf images, generating region proposals corresponding to displayed products, analyzing the shelf images for planogram compliance, and utilizing recognized shelf images to estimate and generate planogram information based upon images representing product placement on retail shelves (Shaw ¶¶ 7–10, 22–32; Figs. 2–5).
generating and utilizing planogram information by recognizing products on retail shelves, determining product locations through region proposal techniques, classifying detected products, estimating planogram compliance, and recommending products based upon the analyzed shelf image for planogram generation (Shaw ¶¶ 3, 7–10, 22–32; Figs. 1–5).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify the shelf monitoring and out-of-stock detection system of Chaubard with the planogram generation and compliance techniques of Shaw. Chaubard is directed toward automated detection of out-of-stock products using periodically captured shelf images, while Shaw teaches using recognized shelf images to generate planogram information and evaluate shelf compliance. Combining Shaw's planogram generation techniques with Chaubard's image-based stock monitoring would have predictably enabled the system not only to detect stockout conditions but also to automatically generate updated planogram information from the most representative shelf image. Such a combination merely applies a known image-analysis technique to another known retail shelf monitoring system to improve inventory management accuracy and merchandising consistency, yielding the predictable result of enhanced automated shelf analysis.
The combination therefore would have been obvious under KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398 (2007) because it represents the predictable use of prior art elements according to their established functions to improve retail shelf monitoring, planogram generation, and stockout analysis.
With respect to claim 2 Chaubard discloses the feature of second image includes the stockout region less than a threshold (¶¶ [0058], [0060] discloses identifying degrees of shelf emptiness and comparing detected voids to thresholds to determine stock status).
With respect to claim 3 Chaubard discloses the feature of second image includes the least stockout region (¶¶ [0062], [0065] discloses comparing multiple shelf images to identify images with fewer empty regions).
With respect to claim 4 Shaw discloses the feature of specifying stockout product candidates by comparing displayed products with marketed products (¶¶ [0036], [0041]; Fig. 4 discloses comparing detected shelf products with expected planogram / product databases to identify missing or mismatched products).
With respect to claim 5 Shaw discloses the feature of acquiring product information including sales number, size, weight, or price (¶¶ [0038], [0042] discloses using product metadata associated with detected products, including size and sales-related attributes, to analyze shelf status).
With respect to claim 6 using sales volume as a criterion for determining missing products is an obvious optimization of Chaubard’s stockout detection when combined with Shaw’s product data usage (¶¶ [0042], [0045]).
With respect to claim 7 Shaw discloses the feature of estimating stockout product based on weight or shelf position (¶¶ [0039], [0043]; Fig. 5 discloses associating physical attributes and shelf position with detected products to infer product identity).
With respect to claim 8 Chaubard discloses the feature of estimating stockout product based on extent of stockout region (¶¶ [0059], [0061] teaches quantifying the size of empty shelf regions and using region size in stockout analysis).
With respect to claim 9 Chaubard discloses the feature of estimating stockout product further based on extent of stockout region (¶ [0061] teaches further evaluation using region size metrics).
With respect to claim 10 Shaw discloses the feature of estimating stockout product based on size equal to or less than extent of stockout region (¶¶ [0040], [0044] discloses correlating product dimensions with shelf space to identify products suitable for a given region).
With respect to Claim 14, claim 14 depends from claim 1 and further recites:
wherein the second image is selected to improve accuracy of planogram data generation by reducing influence of stockout regions on product recognition.
Chaubard discloses selecting and processing shelf images to determine product presence or absence by analyzing product display regions corresponding to previously generated bounding boxes. Chaubard teaches comparing shelf images over time and analyzing only the relevant product display regions to improve the accuracy of detecting products that remain on the shelf while identifying stockout locations (Chaubard ¶¶ 3, 15–29; Figs. 2–4).
However, Chaubard does not expressly disclose selecting an image having a reduced stockout region specifically to improve planogram generation accuracy by reducing the influence of stockout regions on product recognition.
Shaw discloses identifying products from shelf images for determining planogram compliance, wherein region proposals are generated from recognized products, classification is performed using CNN-based recognition, ambiguous detections are removed, and recommendation processing improves recognition accuracy for planogram estimation (Shaw ¶¶ 7–10, 22–32; Figs. 2–5). Shaw therefore teaches improving planogram generation accuracy by utilizing image processing that minimizes erroneous recognition caused by missing or ambiguous product regions.
Accordingly, it would have been obvious to utilize Shaw's improved product-recognition techniques within Chaubard's image-selection process to improve planogram generation accuracy.
With respect to Claim 15 that recites, wherein the at least one processor further performs operations to specify the stockout region by recognizing a background of the product shelf and identifying a region where the background is visible as the stockout region.
Chaubard discloses identifying voids within predefined bounding boxes corresponding to product display locations and determining that those voids represent out-of-stock conditions. The classifier analyzes whether products are present within each bounded shelf region and identifies empty shelf regions corresponding to missing merchandise (Chaubard ¶¶ 24–29; Fig. 4).
However, Chaubard does not expressly disclose recognizing the visible background of the shelf to identify the stockout region.
Shaw teaches processing shelf images to identify product regions and distinguish products from surrounding shelf areas using region proposal generation, image segmentation, geometric analysis, and classification processing, thereby distinguishing shelf background from displayed products for accurate product localization (Shaw ¶¶ 7–10, 30–32; Figs. 2–5).
It would therefore have been obvious to employ Shaw's image segmentation and region proposal techniques in Chaubard to identify exposed shelf background corresponding to stockout regions, thereby improving stockout detection accuracy.
With respect to Claim 16, that recites wherein the at least one processor further performs operations to specify, based on a shelf label recognized in the stockout region, how many types of products are to be displayed in the stockout region.
Chaubard discloses identifying product labels (price tags) associated with shelf positions, generating bounding boxes based upon those labels, identifying products associated with each label, and storing the identified products for subsequent stockout analysis (Chaubard ¶¶ 16–22; Figs. 2–4).
However, Chaubard does not expressly teach determining the number of product types to be displayed in the stockout region using the recognized shelf labels.
Shaw discloses planogram compliance analysis that identifies products expected to occupy shelf regions, recognizes products associated with those regions, and determines product placement based upon planogram information and recognized shelf image content (Shaw ¶¶ 22–32).
It would have been obvious to utilize Shaw's planogram compliance information together with Chaubard's recognized shelf labels to determine the number of product types expected within the identified stockout region because such information predictably improves automated shelf reconstruction and merchandising analysis.
With respect to Claim 17 that recites:
wherein the at least one processor further performs operations to specify, based on a display width of a product displayed adjacent to the stockout region, how many types of products are to be displayed in the stockout region by dividing a width of the stockout region by the display width of the adjacent product.
Chaubard discloses identifying bounding boxes corresponding to product display areas, determining product locations adjacent to empty shelf regions, and analyzing product display areas for inventory monitoring (Chaubard ¶¶ 16–29; Figs. 2–5).
However, Chaubard does not expressly disclose calculating the number of products based upon the display width of adjacent products and the width of the stockout region.
Shaw teaches estimating product dimensions, identifying region proposals corresponding to products, determining geometric transformations and scale relationships between products and shelf regions, and utilizing physical dimensions during planogram estimation (Shaw ¶¶ 22–32; Figs. 3–5).
One of ordinary skill in the art would have recognized that Shaw's dimensional analysis of adjacent products could readily be incorporated into Chaubard's stockout detection system to estimate the number of products capable of occupying an identified empty shelf region based upon the relative dimensions of adjacent displayed products.
With respect to Claim 18, that recites wherein the amount of the stockout region comprises at least one selected from the group consisting of: an area of the stockout region, a number of pixels of the stockout region, a number of stockout regions, and a number of types of products to be displayed in the stockout region.
Chaubard discloses determining whether a product display region is less than full by analyzing predefined bounding boxes corresponding to product display locations, identifying one or more voids within each bounding box, determining the extent of empty shelf regions, and classifying products as out-of-stock based upon the detected voids (Chaubard ¶¶ 3, 16–29; Figs. 2–5).
Accordingly, Chaubard teaches evaluating the size or extent of stockout regions represented by the empty portions of the bounding boxes.
However, Chaubard does not expressly disclose determining the amount of the stockout region using one or more of area, pixel count, number of stockout regions, and number of product types expected within the stockout region.
Shaw teaches determining region proposals corresponding to product display areas, estimating geometric dimensions of detected regions, determining scale transformations, measuring product regions within shelf images, and utilizing those measurements to determine planogram compliance (Shaw ¶¶ 22–32; Figs. 3–5).
One of ordinary skill in the art would have recognized that Shaw's geometric region measurements could readily be employed within Chaubard's stockout detection system to quantify the amount of the stockout region using region size, pixel area, and expected product occupancy.
With respect to Claim 19, that recites, wherein the at least one processor further performs operations to, when a plurality of first images each have a stockout region less than a predetermined threshold, select, as the second image, an image having a latest capturing date and time from among the plurality of first images.
Chaubard discloses periodically capturing images of retail shelves during deployment, maintaining image data over time, and utilizing subsequently captured shelf images to monitor changing inventory conditions (Chaubard ¶¶ 3, 14–29; Fig. 4).
However, Chaubard does not expressly disclose selecting the most recently captured image when multiple images satisfy a stockout threshold.
Shaw teaches processing multiple shelf images captured during planogram analysis and selecting image information for subsequent product recognition and planogram compliance analysis using captured shelf image data (Shaw ¶¶ 22–32).
It would have been obvious to one of ordinary skill in the art to utilize the latest available shelf image in Chaubard's monitoring system because the most recently captured image would predictably provide the most current shelf condition for planogram generation and stockout determination.
With respect to Claim 20 that recites:
wherein the at least one processor further performs operations to, when no stockout product candidate is specified by comparing the displayed product with the marketed products, estimate, as the stockout product, a product displayed adjacent to the stockout region.
Chaubard discloses identifying products associated with shelf positions through product labels and bounding boxes and determining products corresponding to detected stockout regions (Chaubard ¶¶ 16–29).
However, Chaubard does not expressly disclose estimating the stockout product using an adjacent displayed product when no candidate product is identified.
Shaw teaches recognizing neighboring products within shelf images, determining product locations through region proposals, analyzing product arrangement relative to adjacent products, and utilizing contextual shelf relationships during planogram compliance estimation (Shaw ¶¶ 22–32; Figs. 2–5).
It would have been obvious to estimate the identity of a missing product using adjacent products because retail planograms arrange neighboring products in known spatial relationships, thereby improving automated identification when direct recognition is unavailable.
With respect to Claim 21, that recites wherein the at least one processor further performs operations to acquire the product information only for marketed products whose display area coincides with an area included in the second image.
Chaubard discloses extracting product information from product labels, barcodes, OCR data, planograms, and identified products associated with corresponding bounding boxes within the captured shelf image (Chaubard ¶¶ 18–22, 27–29).
However, Chaubard does not expressly disclose restricting acquisition of product information only to marketed products whose display areas coincide with the selected second image.
Shaw teaches identifying products located within analyzed shelf image regions, generating region proposals only for products appearing within the processed image, classifying those detected regions, and recommending products based only upon the analyzed image content for planogram compliance (Shaw ¶¶ 22–32).
Accordingly, Shaw teaches limiting product analysis to products appearing within the image region currently being processed.
It would have been obvious to incorporate Shaw's selective image-region processing into Chaubard's system to reduce unnecessary processing, improve computational efficiency, and focus planogram generation on products actually visible within the selected shelf image.
Response to Arguments
Applicant’s arguments with respect to claim(s) have been considered but are moot in view of new ground of rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROKIB MASUD whose telephone number is (571)270-5390. The examiner can normally be reached Mon-Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fahd Obeid can be reached at 571-270-3324. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROKIB MASUD/Primary Examiner, Art Unit 3627
Prosecution Insights