DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Summary
This Final Office Action in response to the communication received on December 19, 2025.
Claims 42, 45, 46, 50, 53, 54, and 61 have been amended.
Claims 42-61 are pending.
The effective filing date of the claimed invention is May 3, 2024, and claims priority dating back to March 6, 2019.
Response to Amendment
Amendments to Claims 42, 45, 46, 50, 53, 54, and 61 are acknowledged.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 42-61 are rejected under 35 U.S.C. 101 because the claimed invention is directed a judicial exception (i.e., an abstract idea) without significantly more.
Step 1 – Statutory Categories
As indicated in the preamble of the claim, the examiner finds the claim is directed to a process, machine, manufacture, or composition of matter.(Claim 61 are processes and Claims 42-60 are machines). Accordingly, step 1 is satisfied.
Step 2A – Prong 1: was there a Judicial Exception Recited
Claim 42 (and similarly Claims 46 and 61) recites the following abstract concepts that are found to include “abstract idea.” Any additional elements will be analyzed under Step 2A-Prong 2 and Step 2B:
at least one imaging sensor;
a data interface;
an image processing unit comprising at least one processor in electronic communication with the at least one imaging sensor via the data interface and configured to:
access a plurality of planograms stored in a database, wherein each planogram describes a desired placement of products on shelves of a retail store during a time period, wherein the plurality of planograms includes at least a first planogram and a second planogram (See MPEP 2106.04(a)(2)(III) mental processes, a claim to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016));
from the at least one imaging sensor, via the data interface, electronically receive a first set of images captured at a first time and depicting a first plurality of products displayed on at least one of the shelves of the retail store (See MPEP 2106.04(a)(2)(III) mental processes, a claim to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016));
with a machine learning classifier model, automatically analyze the first set of images to determine an actual placement of the first plurality of products displayed on the at least one of the shelves of the retail store at the first time (See MPEP 2106.04(a)(2)(III) mental processes, a claim to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016) and PEG Example 42, Claim 2, wherein under the broadest reasonable interpretation of the training and application of an artificial neural network encompasses mental processes practically performed in the human mind by observation, evaluation, judgement, and opinion.);
with a relationship determination software component, automatically identify a deviation of the actual placement of at least some of the first plurality of products from the desired placement of products associated with the first planogram (See MPEP 2106.04(a)(2)(III) mental processes, a claim to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016));
with a display screen, issue a first user-notification associated with the deviation of the actual placement of the at least some of the first plurality of products from the desired placement of products associated with the first planogram (See MPEP 2106.04(a)(2)(III) mental processes, a claim to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016));
after issuing the first user-notification, receive from the at least one imaging sensor a second set of images captured at a second time and depicting a second plurality of products displayed on the at least one of the shelves of the retail store (See MPEP 2106.04(a)(2)(III) mental processes, a claim to collecting and comparing known information (claim 1), which are steps that can be practically performed in the human mind, Classen Immunotherapies, Inc. v. Biogen IDEC, 659 F.3d 1057, 1067, 100 USPQ2d 1492, 1500 (Fed. Cir. 2011));
with the machine learning classifier model, automatically analyze the second set of images to determine an actual placement of the second plurality of products displayed on the shelves of the retail store at the second time (See MPEP 2106.04(a)(2)(III) mental processes, a claim to collecting and comparing known information (claim 1), which are steps that can be practically performed in the human mind, Classen Immunotherapies, Inc. v. Biogen IDEC, 659 F.3d 1057, 1067, 100 USPQ2d 1492, 1500 (Fed. Cir. 2011) and PEG Example 42, Claim 2, wherein under the broadest reasonable interpretation of the training and application of an artificial neural network encompasses mental processes practically performed in the human mind by observation, evaluation, judgement, and opinion.);
with the relationship determination software component, automatically:
identify a deviation of the actual placement of at least some of the second plurality of products from the desired placement of products associated with the first planogram (See MPEP 2106.04(a)(2)(III) mental processes, a claim to collecting and comparing known information (claim 1), which are steps that can be practically performed in the human mind, Classen Immunotherapies, Inc. v. Biogen IDEC, 659 F.3d 1057, 1067, 100 USPQ2d 1492, 1500 (Fed. Cir. 2011));
use the second planogram to determine whether an arrangement associated with the second plurality of products conforms to the second planogram rather than to the first planogram (See MPEP 2106.04(a)(2)(III) mental processes, a claim to collecting and comparing known information (claim 1), which are steps that can be practically performed in the human mind, Classen Immunotherapies, Inc. v. Biogen IDEC, 659 F.3d 1057, 1067, 100 USPQ2d 1492, 1500 (Fed. Cir. 2011)); and
when the arrangement associated with the second plurality of products conforms to the second planogram, avoid issuance of a second user- notification indicating a deviation relative to the first planogram (See MPEP 2106.04(a)(2)(III) mental processes, a claim to collecting and comparing known information (claim 1), which are steps that can be practically performed in the human mind, Classen Immunotherapies, Inc. v. Biogen IDEC, 659 F.3d 1057, 1067, 100 USPQ2d 1492, 1500 (Fed. Cir. 2011)).
Claim 42 (and similarly Claims 46 and 61) is directed to a series of steps for managing planogram data, which are mental processes. The mere nominal recitation of at least one imaging sensor, a data interface, and imaging processing unit, a processor, a relationship classifier software component, a display screen, a database, and a non-transitory computer-readable medium (Claim 46) does not take the claim out of the method of organizing human interactions. Thus, Claim 42 (and similarly Claims 46 and 61) recites an abstract idea.
Step 2A – Prong 2: Can the Judicial Exception Recited be integrated into a practical application
Limitations that are indicative of integration into a practical application:
Improvements to the functioning of a computer, or to any other technology or technical field - see MPEP 2106.05(a)
Applying or using a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition – see Vanda Memo
Applying the judicial exception with, or by use of, a particular machine - see MPEP 2106.05(b)
Effecting a transformation or reduction of a particular article to a different state or thing - see MPEP 2106.05(c)
Applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception - see MPEP 2106.05(e) and Vanda Memo
Limitations that are not indicative of integration into a practical application:
Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f)
Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)
Generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h)
The identified abstract idea of exemplary Claim 42 (and similarly Claims 46 and 61) is not integrated into a practical application. The additional elements are: at least one imaging sensor, a data interface, and imaging processing unit, a processor, a relationship classifier software component, a display screen, a database, and a non-transitory computer-readable medium (Claim 46) that implements the underlying abstract idea. These additional elements are broadly recited computer elements that do not add a meaningful limitation to the abstract idea because they amount to merely using a computer as a tool to perform an abstract idea - see MPEP 2106.05(f).
Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application. Claim 42 (and similarly Claims 46 and 61) is directed to an abstract idea.
Step 2B – Significantly More Analysis
Claim 42 (and similarly Claims 46 and 61) does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered separately and in combination, steps a) access a plurality of planograms, b) receive a first set of images, c) analyze the first set of images, d) identify a deviation of the actual placement of at least some of the first plurality of products, e)issue a first user-notification, f) receive a second set of images, g) analyze the second set of images, h) use the second planogram to determine whether an arrangement associated with the second plurality of products conforms to the second planogram, and h) when the arrangement associated with the second plurality of products conforms to the second planogram, avoid issuance of a second user-notification, do not add significantly more to the exception because they amount to merely using a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Claim 42 (and similarly Claims 46 and 61) is ineligible.
Claim 43 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 44 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 45 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 47 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 48 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 49 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 50 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 51 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 52 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 53 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 54 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 55 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 56 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 57 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 58 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 59 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim 60 recites the abstract idea of mental processes. See MPEP 2106.04(a)(2)(III).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 42-61 is/are rejected under 35 U.S.C. 103 as being unpatentable over US Pat Pub 2018/0150788 “Vepakomma”, in view of US Pat Pub 2017/0178227 “Graham”, in view of US Pat Pub 2019/0236526 “Sosna”.
As per Claims 42, 46, and 61, Vepakomma discloses a system, computer program product, and method for processing images captured in a retail store and automatically identifying changes in planograms, comprising:
at least one imaging sensor (Vepakomma: [0024] The environment 100 may comprise an establishment 105 having image capturing unit(s) 104, sensor(s) 106 and a user device 108 connected therewith. The environment 100 may also comprise the inventory control system 102 which receives data such as planogram data associated with the establishment 105, image data captured by the image capturing unit(s) 104, and sensor data associated with plurality of products in the establishment 105 captured by the sensor(s) 106);
a data interface (Vepakomma: [0024] The environment 100 may comprise an establishment 105 having image capturing unit(s) 104, sensor(s) 106 and a user device 108 connected therewith. The environment 100 may also comprise the inventory control system 102 which receives data such as planogram data associated with the establishment 105, image data captured by the image capturing unit(s) 104, and sensor data associated with plurality of products in the establishment 105 captured by the sensor(s) 106);
an image processing unit comprising at least one processor in electronic communication with the at least one imaging sensor via the data interface and configured to (Vepakomma: [0025] The data (i.e., planogram data, image data and sensor data) is processed by the inventory control system 102 to determine the planogram compliance metric 220 and the product stock-out condition 218.):
access a plurality of planograms stored in a database, wherein each planogram describes a desired placement of products on shelves of a retail store during a time period, wherein the plurality of planograms includes at least a first planogram and a second planogram (Vepakomma: [0019]-[0021], [0035], [0036], describes the multiple images and planograms and the places of products on the shelf. [0006], [0057] discloses processor and database);
from the at least one imaging sensor, via the data interface, electronically receive a first set of images captured at a first time and depicting a first plurality of products displayed on at least one of the shelves of the retail store (Vepakomma: [0019]-[0021], [0035], [0036], describes the receiving of multiple images and planograms and the places of products on the shelf, which includes multiple times (i.e. first time));
automatically analyze the first set of images to determine an actual placement of the first plurality of products displayed on the at least one of the shelves of the retail store at the first time (Vepakomma: [0042], describes the receiving of multiple images of which includes the actual location of products on display);
with a relationship determination software component, automatically identify a deviation of the actual placement of at least some of the first plurality of products from the desired placement of products associated with the first planogram (Vepakomma: [0004], [0018]-[0019], [0021], describes identifying the misplacement of products for the desire location);
with a display screen, issue a first user-notification associated with the deviation of the actual placement of the at least some of the first plurality of products from the desired placement of products associated with the first planogram (Vepakomma: [0004], [0019], [0021], describes identifying the misplacement of products for the desire location. [0040] teaches generating alerts within positions are not in compliance, and [0053] system can comprise a variety of different types of visual displays);
after issuing the first user-notification, receive from the at least one imaging sensor a second set of images captured at a second time and depicting a second plurality of products displayed on the at least one of the shelves of the retail store (Vepakomma: [0004], [0019], [0021], describes identifying the misplacement of products for the desire location. [0040], [0043], teaches generating alerts within positions are not in compliance and taking the corrective actions);
automatically analyze the second set of images to determine an actual placement of the second plurality of products displayed on the shelves of the retail store at the second time (Vepakomma: [0037], describes the collecting and sorting of multiple set of images and multiple times of products displayed);
with the relationship determination software component, automatically:
identify a deviation of the actual placement of at least some of the second plurality of products from the desired placement of products associated with the first planogram (Vepakomma: [0004[-[0006], [0019], [0021], [0039], describes identifying the misplacement of products for the desire location. [0037], describes the collecting and sorting of multiple set of images and multiple times of products displayed);
use the second planogram to determine whether an arrangement associated with the second plurality of products conforms to the second planogram rather than to the first planogram (Vepakomma: [0004]-[0006], [0019], [0021], [0039], describes the comparing positions of products in multiple images and rather products conform to the compared images.); and
Vepakomma fails to disclose a system, computer program product, and method for processing images captured in a retail store and automatically identifying changes in planograms, comprising:
with a machine learning classifier model, analyze images to determine placement of products,
when the arrangement associated with the second plurality of products conforms to the second planogram, avoid issuance of a second user- notification indicating a deviation relative to the first planogram.
Graham teaches a system, computer program product, and method for processing images captured in a retail store and automatically identifying changes in planograms, comprising:
when the arrangement associated with the second plurality of products conforms to the second planogram, avoid issuance of a second user- notification indicating a deviation relative to the first planogram (Graham: [0142], [0146]-[0151] teaches customizing of notifications which can include only sending a notification based on placements and planogram, therefore withholding notifications).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
Vepakomma and Graham fail to disclose a system, computer program product, and method for processing images captured in a retail store and automatically identifying changes in planograms, comprising:
with a machine learning classifier model, analyze images to determine placement of products.
Sosna teaches a system, computer program product, and method for processing images captured in a retail store and automatically identifying changes in planograms, comprising:
with a machine learning classifier model, analyze images to determine placement of products (Sosna: [0032] A variety of machine learning model types may be integrated into the client application 121 and supported by the local visual product placement controller 1214. The machine learning model types may support extensive deep learning, and standard models such as tree ensembles, SVMs, and generalized linear modes. [0046] The first or second machine learning model may be used to detect the products. Images of the packages of a number of products may be used to build the machine learning model in advance).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma and Graham to include a machine learning classifier model as taught by Sosna, with the system for identifying changes in planograms as taught by Vepakomma and Graham with the motivation to identify if the quantity and locations of the first target product do not match the contractual planogram, the adjustment needs to be made to correct the deviation may be determined by the local visual product placement controller (Sosna: [0051]).
As per Claim 43, Vepakomma discloses a system, wherein the first planogram is associated with a first time period and the second planogram is associated with a second time period subsequent to the first time period, and wherein the second time is included in the first time period (Vepakomma: [0037]).
As per Claim 44, Vepakomma discloses a system, wherein the plurality of planograms are indicative of the desired placement of products on shelves of the retail store at different times of day (Vepakomma: [0033] and [0037]).
As per Claim 45, Vepakomma discloses a system, wherein the at least one imaging sensor is fixedly mounted on at least of the shelves of the retail store (Vepakomma: [0024], [0031]-[0032], [0034]-[0036], and Fig. 2).
As per Claim 47, Vepakomma discloses a computer program product, wherein the determination whether the arrangement associated with the second plurality of products conforms to the second planogram rather than to the first planogram is based on the second time (Vepakomma: [0004]-[0006], [0019], [0021], and [0039]).
As per Claim 48, Vepakomma discloses a computer program product, wherein the determination whether the arrangement associated with the second plurality of products conforms to the second planogram rather than to the first planogram (Vepakomma: [0004]-[0006], [0019], [0021], and [0039]).
Vepakomma fails to disclose but Graham teaches a computer program product that is based on time elapsed since the first plurality of products were last arranged according to the first planogram (Graham: [0128], [0131], and [0175]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 49, Vepakomma discloses a computer program product, wherein the determination whether the arrangement associated with the second plurality of products conforms to the second planogram rather than to the first planogram (Vepakomma: [0004]-[0006], [0019], [0021], and [0039]).
Vepakomma fails to disclose but Graham teaches a computer program product that is based on data from another retail store indicative of a current planogram being used in the another retail store (Graham: [0087], [0101], and [0176]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 50, Vepakomma discloses a computer program product, wherein the plurality of planograms includes at least three planograms, and wherein the method further includes:
with the display screen, in response to a determination that the second plurality of products were arranged according to the second planogram and that there is a deviation of the actual placement of the second plurality of products from the desired placement of products associated with the second planogram, issuing a user-notification associated with the second planogram (Vepakomma: [0004]-[0006], [0019], [0021], and [0039]); and
with the display screen, in response to a determination that the second plurality of products were arranged according to another planogram and that there is a deviation of the actual placement of the second plurality of products from the desired placement of products associated with the another planogram, issuing a user-notification associated with the another planogram (Vepakomma: [0004]-[0006], [0019], [0021], and [0039]).
Vepakomma fails to disclose a computer program product, wherein the plurality of planograms includes at least three planograms, and wherein the method further includes:
with the machine learning classifier model and the relationship determination software component, analyzing the second set of images and the at least three planograms to determine which planogram was used for arranging the second plurality of products at a time between the first time and the second time.
Graham teaches a computer program product, wherein the plurality of planograms includes at least three planograms, and wherein the method further includes:
with the relationship determination software component, analyzing the second set of images and the at least three planograms to determine which planogram was used for arranging the second plurality of products at a time between the first time and the second time (Graham: [0080], [0098], [0101], [0117], [0135], and [0222]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
Vepakomma and Graham fail to disclose a computer program product, wherein the plurality of planograms includes at least three planograms, and wherein the method further includes:
with the machine learning classifier model.
Sosna teaches a computer program product, wherein the plurality of planograms includes at least three planograms, and wherein the method further includes:
with the machine learning classifier model (Sosna: [0032] and [0046]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma and Graham to include a machine learning classifier model as taught by Sosna, with the system for identifying changes in planograms as taught by Vepakomma and Graham with the motivation to identify if the quantity and locations of the first target product do not match the contractual planogram, the adjustment needs to be made to correct the deviation may be determined by the local visual product placement controller (Sosna: [0051]).
As per Claim 51, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the method further includes: selecting the second planogram out of a group of possible planograms based on the second time (Graham: [0080], [0098], [0101], [0117], [0135], and [0222]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 52, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the method further includes:
determining that the second time is during a transition period between the first planogram and a different planogram (Graham: [0098], [0101], and [0128]); and
in response to the determination that the second time is during the transition period, avoiding issuance of the second user-notification associated with the deviation of the actual placement of the at least some of the second plurality of products from the desired placement of products associated with the first planogram (Graham: [0142], and [0146]-[0151]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 53, Vepakomma discloses a computer program product, wherein the method further includes:
with the relationship determination classifier software component, automatically determining a deviation of an actual placement of at least one product from the desired placement of products associated with the second planogram (Vepakomma: [0004], [0019], [0021], and [0040]); and
with the display screen, causing an issuance of a user-notification indicating that the at least one product is located out of place (Vepakomma: [0004], [0019], [0021], and [0040]).
Vepakomma fails to disclose a computer program product, wherein the method further includes:
from the at least one imaging sensor via the data interface, electronically receiving a third set of images captured at a third time after the determination of whether the arrangement associated with the second plurality of products conforms to the second planogram;
with the machine learning classifier model, automatically analyzing the third set of images to determine an actual placement of products on the shelves of the retail store at the third time.
Graham teaches a computer program product, wherein the method further includes:
from the at least one imaging sensor via the data interface, electronically receiving a third set of images captured at a third time after the determination of whether the arrangement associated with the second plurality of products conforms to the second planogram (Graham: [0080], [0098], [0101], [0117], [0135], and [0222]);
automatically analyzing the third set of images to determine an actual placement of products on the shelves of the retail store at the third time (Graham: [0080], [0098], [0101], [0117], [0135], and [0222]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
Vepakomma and Graham fail to disclose a computer program product, wherein the method further includes:
with the machine learning classifier model.
Sosna teaches a computer program product, wherein the method further includes:
with the machine learning classifier model (Sosna: [0032] and [0046]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma and Graham to include a machine learning classifier model as taught by Sosna, with the system for identifying changes in planograms as taught by Vepakomma and Graham with the motivation to identify if the quantity and locations of the first target product do not match the contractual planogram, the adjustment needs to be made to correct the deviation may be determined by the local visual product placement controller (Sosna: [0051]).
As per Claim 54, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the method further includes:
from the at least one imaging sensor via the data interface, electronically receiving a third set of images captured at a third time after the determination whether the arrangement associated with the second plurality of products conforms to the second planogram, wherein the third set of images depicts a third plurality of products displayed on the shelves of the retail store;
with the machine learning classifier model, automatically analyzing the third set of images to determine an actual placement of products on the shelves of the retail store at the third time;
with the relationship determination software component, automatically:
determining a deviation of an actual placement of at least one product from the desired placement of products associated with the second planogram; comparing the actual placement of the third plurality of products as identified in the third set of images to a desired placement of products associates with the first planogram;
based on the comparison, determining whether an arrangement associated with the third plurality of products conforms to the first planogram; and
when the arrangement associated with the third plurality of products conforms to the first planogram, avoiding issuance of a user-notification associated with the deviation of the actual placement of the at least one product from the desired placement of products associated with the second planogram.
Graham teaches a computer program product, wherein the method further includes:
from the at least one imaging sensor via the data interface, electronically receiving a third set of images captured at a third time after the determination whether the arrangement associated with the second plurality of products conforms to the second planogram, wherein the third set of images depicts a third plurality of products displayed on the shelves of the retail store (Graham: [0080], [0098], [0101], [0117], [0135], and [0222]);
automatically analyzing the third set of images to determine an actual placement of products on the shelves of the retail store at the third time (Graham: [0098], [0101], [0130]-[0132], and [0135]);
with the relationship determination software component, automatically:
determining a deviation of an actual placement of at least one product from the desired placement of products associated with the second planogram; comparing the actual placement of the third plurality of products as identified in the third set of images to a desired placement of products associates with the first planogram (Graham: [0130]-[0132] and [0197]);
based on the comparison, determining whether an arrangement associated with the third plurality of products conforms to the first planogram (Graham: [0063] and [0080]); and
when the arrangement associated with the third plurality of products conforms to the first planogram, avoiding issuance of a user-notification associated with the deviation of the actual placement of the at least one product from the desired placement of products associated with the second planogram (Graham: [0142], [0146]-[0151]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
Vepakomma and Graham fail to disclose a computer program product, wherein the method further includes:
with the machine learning classifier model.
Sosna teaches a computer program product, wherein the method further includes:
with the machine learning classifier model (Sosna: [0032] and [0046]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma and Graham to include a machine learning classifier model as taught by Sosna, with the system for identifying changes in planograms as taught by Vepakomma and Graham with the motivation to identify if the quantity and locations of the first target product do not match the contractual planogram, the adjustment needs to be made to correct the deviation may be determined by the local visual product placement controller (Sosna: [0051]).
As per Claim 55, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the method further includes: retrieving the second planogram from another retail store (Graham: [0087], [0101], [0176]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 56, Vepakomma discloses a computer program product, wherein the first planogram indicates that a group of first products are supposed to be placed on a shelving unit; and identifying the deviation of the actual placement of at least some of the second plurality of products from the desired placement of products associated with the first planogram includes determining that a group of second products are placed on the shelving unit where a group of first products were supposed to be placed (Vepakomma: [0002]).
As per Claim 57, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the second planogram indicates that the group of second products are supposed to be placed on the shelving unit (Graham: [0094]-[0096]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 58, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the group of second products includes more than five products from a same type (Graham: [0101], [0111], and [0123]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 59, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the group of second products includes products from at least two types of products (Graham: [0101], [0111], and [0123]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
As per Claim 60, Vepakomma fails to disclose but Graham teaches a computer program product, wherein the arrangement associated with the second plurality of products conforms to the second planogram when at least 90% of the group of second products placed on the at least one of the shelves of the retail store (Graham: [0103], [0181], and [0196]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vepakomma to include determine issuance of a user-notification as taught by Graham, with the system for identifying changes in planograms as taught by Vepakomma with the motivation to capture overall data about the state of the real-world shelf, including time-changing information and provide real-time feedback (Graham: [0005]).
Response to Arguments
35 USC 101
Applicant's arguments filed December 19, 2025 have been fully considered but they are not persuasive.
First, Applicant argues that the claims are not directed to an abstract idea because the claims include interrelationships between machine components both external to and internal to the image processing unit. However, the claims are not directed to specific machines or machine components that perform mechanical image processing. Instead they are directed to the processing of the content of the collected images, and as such, are found to recite the abstract idea of mental processes.
Second, Applicant argues that the judicial exception is integrated into a practical application because the recited elements reflect an improvement to the technical field of improving store execution of planogram compliance, store procedures and reoccurring tasks, efficiency of restocking, and reordering. These cited improvements are not improvements to a technical field. At best it can be argued that improvements are being made to a retail environment by applying some technical aspects, such as performing them using generic computer elements. MPEP 2106.05(a)(2)(III)(C) recites “In evaluating whether a claim that requires a computer recites a mental process, examiners should carefully consider the broadest reasonable interpretation of the claim in light of the specification. For instance, examiners should review the specification to determine if the claimed invention is described as a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept. In these situations, the claim is considered to recite a mental process.”
Third, Applicant argues that the claims recite a combination of features that are not well-understood, routine, conventional activity in the field, because the combination of the steps (e.g., issuing a notification based on deviation from a first planogram, but not issuing a second notification based on compliance with a second planogram) operates in a non-conventional and non-generic way to capture time-dependent shelf stocking and arrangement goals. Applicant is applying arguments that the abstract idea is being applied in a non-conventional and non-generic way. The sending of a notification is being completed using well-understood, routine, conventional activity of computing devices. The use of the computing devices and notification sending is not applied in any special or unique application. The applicant is confusing the argument that the abstract idea of evaluating shelf data for compliance with a variety of planograms and issuing a user notification when certain conditions are met, with the argument that a specific devices are used in a non-conventional and non-generic way. There is nothing special about the devices or the combination of devices to indicate that they are anything other than they are well-understood, routine, and conventional. Issuing a notification based on deviation from a first planogram, but not issuing a second notification based on compliance with a second planogram could be performed by a human using their voice or pen and paper to convey the information, but is merely being automated by a generic computer system.
35 USC 103
Applicant’s arguments, see Applicant Arguments/Remarks Made in an Amendment, filed December 19, 2025, with respect to the rejection(s) of claim(s) 42-61 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of US Pat Pub 2018/0150788 “Vepakomma”, in view of US Pat Pub 2017/0178227 “Graham”, in view of US Pat Pub 2019/0236526 “Sosna”.
In response to applicant’s arguments that the combination of Vepakomma and Graham do not disclose or suggest “a second planogram”, and in particular do not disclose or suggest “use the second planogram to determine whether an arrangement associated with the second plurality of products conforms to the second planogram rather than to the first planogram” and “when the arrangement associated with the second plurality of products conforms to the second planogram, avoid issuance of a second user-notification indicating a deviation relative to the first planogram.”, the examiner finds that the combination of Vepakomma and Graham do disclose these features. Vepakomma clearly discloses identifying if products comply with a plurality of planograms and sending a notification if they do not. The combination of Graham further supports and further clarifies that a plurality of planograms are used (a second planogram). In [0109] Graham teaches using a realogram, or the placement of items on a shelf based on real images or sensor data, to retrieve a planogram from a database of a plurality of planograms that corresponds to the actual placement of the items on the products. Thus, Graham is not limited to one selected planogram, but instead identifies any of the plurality of planograms that matches the real arrangement of products.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REVA R MOORE whose telephone number is (571)270-7942. The examiner can normally be reached M-Th: 9:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fahd Obeid can be reached at 571-270-3324. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REVA R MOORE/Examiner, Art Unit 3627
/FAHD A OBEID/Supervisory Patent Examiner, Art Unit 3627