Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Applicant’s submission filed 7/17/25 has been entered. Claims 1-12 are presented for examination.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Note: The following analysis is based on the Revised Guidance titled “2019 Revised Patent Subject Matter Eligibility Guidance (Vol. 84, No. 4).
STEP 1
Is the claim(s) directed to a process, machine, manufacture or composition of matter?
Claims 1-12 are all directed to a statutory category (e.g., a process, machine, manufacture, or composition of matter). The answer is YES.
STEP 2A. Prong 1
The claims disclose the abstract idea of analyzing images of products in a retail store .
Exemplary claim 1 recites the following abstract concepts that are found to include “abstract idea”:
“A method comprising:
receiving, a plurality of images of a retail store that includes a first image and a second image;
identifying a product indicated in the first image, wherein the product is associated with a key point; identifying the key point associated with the product in the second image, wherein the key point in the second image indicates an overlapping region between the first image and the second image;
-combining the first image and the second image based on the key point associated with the product, to produce a combined image; and
-performing an image analysis on the combined image.”
The remaining limitations are no more than computer elements (i.e., a cloud computing system) to be used as a tool to perform this abstract idea.
The recited limitations cover a process that, under its broadest reasonable interpretation, covers subject matter viewed as a certain method of organizing human activity with the additional recitation of generic computer components. For example, but for the “at a cloud computing system” language, “receiving, identifying, identifying, combining, performing”, in the context of this claim encompasses the user manually receiving the images, identifying similarity between products, combining the first and second images, and analyze the images.
The practice of receiving, identifying, identifying, combining data as well as analyzing data is a commercial or legal interaction long prevalent in our system of commerce. The claims recite the idea of performing various conceptual steps generically resulting in generating suggestions based on the information received. As determined earlier, none of these steps recites specific technological implementation details, but instead get to this result by receiving, combining and analyzing data. Thus, the claims are directed to a certain method of organizing human activity
STEP 2A, Prong 2
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
The claim recites the following additional elements: 1) a cloud computing system, a client device (claims 11, 12) and a machine learning model (claim 3) are used to perform the steps.
The computing system, client device, machine learning model, in the steps are recited at a high level of generality, i.e., as a generic processor performing a generic computer function of processing data (receiving, at a computer system, data). This generic processor limitation is no more than mere instructions to apply the exception using a generic computer component. 2) Furthermore, applying a machine learning model to identify products in images is also mere instructions to apply the exception using a generic computer component.
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claim is directed to an abstract idea.
STEP 2B
The next issue is whether the claims provide an inventive concept because the additional elements recited in the claims provide significantly more than the recited judicial exception. Taking the claim elements separately, the function performed by the computing system or the client device at each step of the process is purely conventional. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a computer/processor to perform the steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Considered as an ordered combination, the computer components of Applicants' claims add nothing that is not already present when the steps are considered separately. The claimed invention does not focus on an improvement in computers, but rather certain independently abstract ideas that use computers as tools. {Elec. Power, 830 F.3d at 1354). (Step 2B: NO).
There is no indication that the computer system or processor is anything other than a generic, off-the-shelf computer component, and the Symantec, TLI, and OIP Techs. Court decisions cited in MPEP 2106.05(d)(II) indicate that mere collection or receipt of data over a network is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is here).
The dependent claims when analyzed as a whole are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. The claims provide minimal technical structure or components for further consideration either individually or as ordered combinations with the independent claims. As such, additional recited limitations in the dependent claims only refine the identified abstract idea further. Further refinement of an abstract idea does not convert an abstract idea into something concrete.
Accordingly, a conclusion that the collecting step is well-understood, routine, conventional activity is supported under Berkheimer Option 2.
See MPEP 2106.05(d)(II) The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. A VAuto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350,1355,112 USPQ2d 1093,1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hoteis.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial,Vne claims at issue here specify how interactions with the Internet are manipulated to yield a desired result-a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
iv. Storing and retrieving information in memory, VersataDev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306,1334,115 USPQ2d 1681,1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363,115 USPQ2d at 1092-93.
The claims are ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6, 11, 12 are rejected under 35 U.S.C. 103 as being unpatentable over CHAKI et al. (US 20170068840 A1).
Re-claim 1, CHAKI et al. teach -- A method, comprising:
-receiving, at a cloud computing system, a plurality of images of a retail store that includes a first image and a second image;
(see e.g. [0036] . Image processing device 210 may obtain the set of images from one or more image capture devices 220. For example, a particular image capture device 220 may capture the set of images, and may provide the set of images to image processing device 210. As another example, a set of image capture devices 220 may each capture one or more images, of the set of images, and may collectively provide the set of images to image processing device 210.
[0062] For example, image processing device 210 may obtain two or more images from image capture device 220. Two images, of the two or more images, may share an overlapped area. For example, a portion of a first image may depict the same object as a corresponding portion of a second image.
[0022] Image processing device 210 may include a device capable of receiving, storing, generating, processing, and/or providing information. For example, image processing device 210 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, etc.), a server, a device in a cloud computing network, or a similar device.)
-identifying a product indicated in the first image, wherein the product is associated with a key point; --identifying the key point associated with the product in the second image, wherein the key point in the second image indicates an overlapping region between the first image and the second image;
(see e.g. [0038] For example, image processing device 210 may detect common features in an overlapped area between a first image and a second image)
-combining the first image and the second image based on the key point associated with the product, to produce a combined image; and
(see e.g. [0038] In some implementations, image processing device 210 may combine the set of images based on common features in overlapped areas. For example, image processing device 210 may detect common features in an overlapped area between a first image and a second image, and may combine the first image and the second image by aligning the common features. In some implementations, image processing device 210 may blend features of a first image and a second image.)
-performing an image analysis on the combined image.
(see e.g. [0015] In some cases, the image processing device may analyze the stitched image to identify objects in the stitched image. For example, the image processing device may identify inventory products in a stitched image of a store shelf).
Re-claim 3, CHAKI et al. teach ---The method of claim 1, wherein identifying the product comprises identifying the product using a machine learning model.
(see e.g. abstract -- A device may receive a set of images for an object recognition operation to identify one or more objects in the set of images).
Re-claim 6, CHAKI et al. teach ---The method of claim 1, further comprising: determining an ordering associated with the first image and the second image based on an overlap in information between the first image and the second image, and wherein combining the first image and the second image comprises combining the first image and the second image based on the ordering associated with the first image and the second image.
(see e.g. [0036] As shown in FIG. 4, process 400 may include obtaining a set of images, including objects to be recognized, with one or more overlaps between images of the set of images (block 410). For example, image processing device 210 may obtain a set of images that include one or more objects to be recognized. The set of images may be arranged in a grid, a row, or the like, with areas of overlap between adjacent images in the grid, row, or the like. Image processing device 210 may obtain the set of images from one or more image capture devices 220. For example, a particular image capture device 220 may capture the set of images, and may provide the set of images to image processing device 210. As another example, a set of image capture devices 220 may each capture one or more images, of the set of images, and may collectively provide the set of images to image processing device 210.).
Re-claim 11, CHAKI et al. teach --The method of claim 1, wherein receiving the plurality images comprises receiving the plurality of images from a client device.
(see e.g. [0062] As shown in FIG. 6, process 600 may include obtaining two images that share an overlapped area (block 610). For example, image processing device 210 may obtain two or more images from image capture device 220. Two images, of the two or more images, may share an overlapped area. For example, a portion of a first image may depict the same object as a corresponding portion of a second image.
[0023] Image capture device 220 may include a device capable of capturing an image (e.g., a photograph, a digital picture, a video, etc.). For example, image capture device 220 may include a camera (e.g., a digital camera, a web camera, etc.), a video camera (e.g., a camcorder, a movie camera), a smart camera, a mobile telephone (e.g., a smartphone, a cellular telephone, etc.) a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, etc.), or a similar device.
[0035] In some implementations, one or more process blocks of FIG. 4 may be performed by image processing device 210. In some implementations, one or more process blocks of FIG. 4 may be performed by another device or a group of devices separate from or including image processing device 210, such as image capture device 220 and client device 230.)
Re-claim 12, CHAKI et al. teach--The method of claim 1, wherein receiving the plurality images comprises receiving the plurality of images from a client device via an edge computing system associated with the retail store.
(see e.g. [0037] For example, image capture device 220 may capture the set of images, may combine the set of images into a single, stitched image, and may provide the stitched image to image processing device 210.
[0035] In some implementations, one or more process blocks of FIG. 4 may be performed by image processing device 210. In some implementations, one or more process blocks of FIG. 4 may be performed by another device or a group of devices separate from or including image processing device 210, such as image capture device 220 and client device 230.
The Examiner notes that the image capture device and client device 230 perform edge computing functions such as processing, analyzing data locally).
Claims 2, 4, 5 are rejected under 35 U.S.C. 103 as being unpatentable over CHAKI et al. (US 20170068840 A1), in view of Hicks (US 20090192921 A1).
Re-claims 2, 4, 5, CHAKI et al. do not teach the limitations as claimed.
However, Hicks et al. teach – 2.The method of claim 1, wherein identifying the product comprises identifying one or more of: a stock keeping unit associated with the product, a brand associated with the product, or a Universal Product Code description associated with the product.
(see e.g. The method also involves capturing a first image of a first area and a second image of a second area. A stitched image is generated based on the first and second images. The stitched image is associated with product codes based on products appearing in the stitched image.
[0046] In some example implementations, product codes in the product code selection control 1110 can be selected automatically using a character recognition and/or an image recognition process used to recognize products (e.g., types of products, product names, product brands, etc.) in images.)
4.The method of claim 1, wherein the first image of the product is associated with a first retail shelf level and the second image is associated with a second retail shelf level, and combining the first image and the second image comprises aligning the first retail shelf level and the second retail shelf level based on the product indicated in the first shelf level of the first image and the product indicated in the second shelf level of the second image.
[0038] In the example implementations described herein, numerous photographic images of a product display unit can be merged to form a panoramic image of that product display unit such as, for example, a panoramic image 1800 of FIG. 18. In the illustrated example of FIG. 18, the panoramic image 1800 is formed by merging the photographic images 802, 852, 1802, and 1804 as shown. The photographic images 1802 and 802 are merged at merge area 1806, the photographic images 802 and 852 are merged at merge area 1808, and the photographic images 852 and 1804 are merged at merge area 1810. Although four photographs are shown as being merged to form the panoramic image 1800 in FIG. 18, any number of photographs may be merged to form a panoramic image of products on display in a retail establishment.)
5. The method of claim 1, wherein the first image of the product is associated with a first retail shelf level and the second image is associated with a second retail shelf level, and combining the first image and the second image comprises forming a combined retail shelf level based on the first image and the second image.
(see e.g. [0036] Turning to FIGS. 6A and 6B, when the cart 200 captures photographic images of the arrangement of products 502, it does so by capturing two successive photographic images, one of which is shown in FIG. 6A and designated as image A 602 and the other of which is shown in FIG. 6B and designated as image B 652. Image A 602 corresponds to a first section 506 (FIG. 5) of the arrangement of products 502, and image B 652 corresponds to a second section 508 (FIG. 5) of the arrangement of products 502. A merging or stitching process is used to join image A 602 and image B 652 along an area that is common to both of the images 602 and 652.)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify CHAKI et al., and include the steps cited above, as taught by Hicks, in order to conserve processor and memory resources and improve accuracy of object recognition, which saves time and money for the organization implementing the object recognition operation. (see e.g. CHAKI et al. [0087]).
Claims 7, 8, 9, 10 are rejected under 35 U.S.C. 103 as being unpatentable over CHAKI et al. (US 20170068840 A1), in view of Adato et al. (US 20190215424 A1).
Re-claims 7, 8, 10, CHAKI et al. do not teach the limitations as claimed.
However, Adato et al. teach 7. The method of claim 1, further comprising: providing a recommendation based on the image analysis. 8. The method of claim 7, wherein the recommendation is associated with a task to be performed with respect to products in the retail store.
(see e.g. [0574] In some cases, the at least one processor may be configured to receive a plurality of images. Each of the plurality of images may contain a representation of an area of a shelf that is distinct from the area represented in any other of the plurality of images or each of the plurality of images may contain a representation of an area that is at least partially depicted in another of the plurality of images. For example, system 100 may receive image 3001, image 3002, and image 3003, all of which depict a portion of the shelving unit represented in image 3010. In this example, system 100 may process the images 3001, 3002, 3003, to determine that they contain the same subject matter and combine them in a manner that generates image 3010 or, system 100 may process images 3001, 3002, 3003 separately without first generating image 3010.
[0619] The at least one processor may be configured to identify a plurality of possible rearrangements and then choose one possible rearrangement to implement or configured to report the plurality of possible rearrangements to a user. For example, system 100 may determine several possible rearrangements for a shelf or shelving unit any may transmit each of the possible rearrangements to, for example, client device 145.
[0630] Consistent with the present disclosure, the information provided to the user may be part of a product-related task assigned to a store employee and may include details of the recommended rearrangement. For example, when product assortment rules associated with a plurality of products are used to determine a recommended rearrangement, as discussed in relation to process 32A, the information provided to a user may include a product-related task assigning a store employee the task of rearranging the plurality of products. The information, in this example, may also include the assortment rule or rules that were used to develop the recommended rearrangement. The product-related task may comprise instructions on how to rearrange products, how to rearrange shelves, how to rearrange shelving units, or a combination thereof. tity 145, may then use the transmitted possible rearrangements to determine which rearrangement to implement.
[0629] In accordance with the present disclosure, the at least one processor may be configured to generate a visualization of a recommended rearrangement. The visualization may include the image received in, for example, step 3102 of method 3100. The visualization may contain a visual representation of how a shelf should be arranged. For example, system 100 may generate a visual representation of a desired arrangement of a shelf or a shelving unit, the visualization may be substantially the same as image 3030 or image 3040, for example.)
9. The method of claim 7, wherein the recommendation maximizes a shelf impact score that is based on: a revenue impact as a result of acting on the recommendation and a first corresponding weight, a non-monetary impact as a result of acting on the recommendation and a second corresponding weight, and a determination as to whether the recommendation is actionable and a third corresponding weight.
[0783] Based on the determined or estimated impact, the at least one processor may further rank an importance to product sales of each of the plurality of characteristics of planogram compliance for the at least one product type. For example, the at least one processor may analyze and rank any of the various characteristics of planogram compliance described above (e.g., product facing, product placement, planogram compatibility, price correlation, promotion execution, product homogeneity, restocking rate, planogram compliance of adjacent products, etc.) relative to checkout data. For example, the system may determine that product placement has a greater impact on sales than product facing, or that promotion execution has a greater impact that product homogeneity, and the like. In some embodiments, the provided information may include data for prioritizing actions associated with planogram compliance based on the ranking of the plurality of characteristics of planogram compliance for the at least one product type. For example, the characteristics of planogram compliance described herein may be ranked in a list in order of importance, and actions may be determined based on the ranking. For example, if the system determines that product placement has the greatest impact on sales for a given retail store, the system may determine actions to enhance product placement in the retail store. In such embodiments, the actions associated with planogram compliance may include at least two of: rearranging products to change product facing, rearranging products to change product placement, rearranging products to change product homogeneity, rearranging products to change adjacent products compliance, changing restocking rate, changing promotion execution, and changing price labels.
[0536] Consistent with the present disclosure, server 135 may generate within minutes actionable tasks to improve store execution (for example, in less than a second, less than 10 seconds, less than a minute, less than 10 minutes, more than 10 minutes, and so forth). These tasks may help employees of retail store 105 to quickly address situations that can negatively impact revenue and customer experience in the retail store 105. Such tasks may include resolving the occlusion event.
****Note that Adato et al. teach ‘determining and tracking effects of planogram compliance or non-compliance on product sales… estimating the impact of planogram compliance’ (see e.g. [0760]
Furthermore, Adato et al. consider impacts on revenue and customer experience and actionable tasks (see e.g. [0536]).
10. The method of claim 7, wherein the recommendation is based on one of more of: a characteristic of a retail shelf holding products in the retail store, supply chain data associated with the products in the retail store, spatio-temporal trend data associated with the products in the retail store, or a remediation time associated with the products in the retail store.
(see e.g. [0715] Based on this information, along with characteristics associated with shelving unit 3703 and shelving unit 3705, the processor may generate a report or other indicator of which shelving type and/or what shelving characteristics may be most effective in encouraging sales of a particular product. Such information may also be useful in generating recommendations for increasing product sales, for example by recognizing in captured images one or more shelving characteristics (e.g., stocking density, shelf configuration, shelf placement, shelf highest, shelf size, facings, color scheme, or any other shelving/product characteristic) and making a recommendation for changing at least one shelving/product characteristic to more closely match other configurations known to be associated with higher sales volumes.)
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify CHAKI et al., and include the steps cited above, as taught by Adato et al., because the provided information may enable market research entity 110 to give supplier 115 informed shelving recommendations and fine-tune promotional strategies according to in-store marketing trends, to provide store managers with a comparison of store performances in comparison to a group of retail stores 105 or industry wide performances, and so forth. (see e.g. Adato et al. [0227]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LUNA CHAMPAGNE whose telephone number is (571)272-7177. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Florian Zeender can be reached at 571 272-6790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LUNA CHAMPAGNE/Primary Examiner, Art Unit 3627 July 24, 2026
Prosecution Insights