DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are cancelled and claims 21, 31, and 40 are amended. Claims 21-40 filed December 24, 2024 are pending and are hereby examined.
Double Patenting
3. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
4. A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
5. The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
6. The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
7. Claims 21-40 are rejected on the ground of provisional nonstatutory double patenting as being unpatentable over claims 1-9 and 11-19 of U.S. Patent 12,190,287 and claims 1-7, 9-16, and 18-20 of U.S. Patent 12,136,061. In fact, the ‘061 and ‘287 patents are more detailed and more specific and encompasses almost all of the elements of the broader, current ‘106 application as seen below.
8. This is an obviousness nonstatutory double patenting rejection because although the conflicting claims are not identical, they are not patentably distinct from each other because the claimed limitations from the present application and U.S. Patent 12,141,783 above are significantly similar and the claimed features seem to be identical with various obvious alternate methods. The omission of an element with a corresponding loss of function is an obvious expedient. See In re Karlson, 136 USPQ 184 and Ex parte Rainu, 168 USPQ 375. For these reasons, claims 21-40 of the instant application are not identical to claims 1-9 and 11-19 of U.S. Patent 12,190,287 and claims 1-7, 9-16, and 18-20 of U.S. Patent 12,136,061, but they are not patentably distinct.
Claim Rejections - 35 USC § 101
9. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
10. Claims 21-40 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
11. Step 1 Statutory Category: Claims 21-30 and 40 are directed to a method, and claims 31-39 are directed to a system, all of which are statutory. Claims 21-40 are statutory classes of invention.
12. Step 2A – Prong 1: Judicial Exception Recited: Nevertheless, independent claim 21, 31, and 40 recite an abstract idea of inventory tracking on a retail shelf.
The independent claim 21, 31, and 40 recite the following limitations which fall under commercial or legal interactions:
…;
…;
…;
…;
receiving… one or more initial shelf images from one or more… wherein the one or more initial shelf images are images of a shelf that includes the first set of products and an empty space indicating the missing product;
processing the one or more initial shelf images to remove image skewing and image warping;
stitching the one or more initial shelf images to create the shelf image;
receiving… a planogram image of the shelf, wherein the planogram image includes a second set of products;
computing a visual similarity matrix between the first set of products and the second set of products;
generating a transformation matrix that maps the second set of products from the planogram image to the first set of products in the shelf image, wherein values of the transformation matrix are derived from values of the visual similarity matrix;
iteratively refining the transformation matrix for a predetermined number of iterations;
mapping the planogram image to the shelf image to create a transformed planogram image by applying the refined transformation matrix to the planogram image;
determining a shelf location of the empty space within the shelf image;
based on the transformed planogram image, identifying a planogram product within the planogram image that is located in a planogram location that corresponds to the shelf location;
and determining that the missing product is the planogram product.
13. According to the MPEP, "Commercial interactions" or "legal interactions" include agreements in the form of contracts, legal obligations, advertising, marketing or sales activities or behaviors, and business relations. Clearly, inventory tracking on a retail shelf falls under sales activities, therefore commercial or legal interactions. If the claim limitations, under the broadest reasonable interpretation, covers performance of the limitations as a commercial or legal interaction, then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
14. Step 2A – Prong 2: Practical Application: This judicial exception is not integrated into a practical application because the claim as a whole merely recites inventory tracking on a retail shelf with generally recited computer elements such as imaging devices, computing system, processor, and memory, which in these steps are recited at a high-level of generality such that it amounts to more than mere instructions to apply the exception using a generic computer component, and are merely invoked as tools for inventory tracking on a retail shelf. Accordingly, these elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Simply implementing the abstract idea on a generic computing environment is not a practical application of the abstract idea, and does not take the claim out of the Commercial or Legal Interactions subgrouping of Certain Methods of Organizing Human Activity grouping. The claims are directed to an abstract idea.
15. Step 2B – Inventive Concept: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered individually and as an ordered combination, they do not add significantly more (also known as “inventive concept”) to the exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of imaging devices, computing system, processor, and memory to perform these steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, these additional elements, do not change the outcome of the analysis, when considered individually and as an ordered combination as there is no inventive concept sufficient to transform the claimed subject matter into a patent-eligible application. The claims are not patent eligible.
16. Regarding dependent claims 22-23, 25-26, 28-29, 33-36, and 38-39, these claims merely narrow the abstract idea of inventory tracking on a retail shelf, and these claims neither integrate into a practical application nor contain additional elements which amount to significantly more than the abstract idea.
17. Regarding dependent claims 24, 27, and 37, although these claims recite a generally recited deep neural network, these claims merely narrow the abstract idea of inventory tracking on a retail shelf, and these claims neither integrate into a practical application nor contain additional elements which amount to significantly more than the abstract idea.
18. Regarding dependent claim 30, although this claim recites a generally recited additional imaging devices, this claim merely narrows the abstract idea of inventory tracking on a retail shelf, and this claim neither integrates into a practical application nor contains additional elements which amount to significantly more than the abstract idea.
19. Regarding dependent claim 32, although this claim recites a generally recited user computing device, this claim merely narrows the abstract idea of inventory tracking on a retail shelf, and this claim neither integrates into a practical application nor contains additional elements which amount to significantly more than the abstract idea.
20. Therefore, the limitations of the claims, when viewed individually and in ordered combination, are directed to ineligible subject matter.
Examiner Notes
21. Claims 21-40 are novel and unobvious over the prior art, however, there remains a pending 35 U.S.C. 101 rejection and double patenting rejection. The Examiner suggests filing a terminal disclaimer to obviate the double patenting rejection. The Examiner suggests mirroring all of the independent claims (make sure elements of claim 40 are in claims 21 and 31). Then, the Examiner suggests incorporating the elements of dependent claims 24 (dependent on 23), 27, and 30 together into the independent claims. The Examiner suggests incorporating the imaging device, computing system, processor, and memory from independent claim 31 into other independent claims. The Examiner suggests incorporating how the deep learning network is trained or re-trained uniquely. Finally, the Examiner suggests incorporating more hardware from the Specification and any unique arrangements of hardware, unique hardware, or unique ways the hardware is communicating. The aforementioned claim suggestions, in combination together, is suggested to help advance prosecution forward, although further search, examination, and consideration is required.
22. After further search and consideration, the most pertinent U.S. prior art was found to be Higa (US 2021/0166037), Nemati et al (US 2018/0322448), Taylor et al (US 2019/0180150), Bogolea et al (US 2020/0215698), Schwartz (US 2016/0171707), Schwartz 2 (US 2017/0178060), and Chaubard (US 2020/0005225). Schwartz (US 2016/0171707) is directed to a realogram scene analysis of images. Schwartz 2 (US 2017/0178060) is directed to planogram matching. Chaubard (US 2020/0005225) is directed to on-shelf image based out-of-stock detection. Higa (US 2021/0166037) is directed to detecting a change of an object through image recognition. Bogolea et al (US 2020/0215698) is directed to automatically generating planograms of shelving structures within a store. Taylor et al (US 2019/0180150) is directed to a color haar classifier for retail shelf label detection. Nemati et al (US 2018/0322448) is directed to automatically restocking items on shelves. Santra Bikash et al (An end-to-end annotation-free machine vision system for detection of products on the rack, NPL) was found to be the most pertinent NPL prior art, and is directed to detecting products on a rack.
23. However, both the most pertinent U.S. prior art and NPL fail to disclose all of the limitations particularly:
computing a visual similarity matrix between the first set of products and the second set of products; generating a transformation matrix that maps the second set of products from the planogram image to the first set of products in the shelf image, wherein values of the transformation matrix are derived from values of the visual similarity matrix; iteratively refining the transformation matrix for a predetermined number of iterations; mapping the planogram image to the shelf image to create a transformed planogram image by applying the refined transformation matrix to the planogram image; determining a shelf location of the empty space within the shelf image; based on the transformed planogram image, identifying a planogram product within the
planogram image that is located in a planogram location that corresponds to the shelf location; and determining that the missing product is the planogram product.
24. No prior art cited here or in any previous Office Action neither fully anticipates nor supports a conclusion of obviousness with respect to the subject matter present in the independent claims, either alone or in combination. The limitations lacking in the prior art, in combination with the other limitations clearly claimed in the application, are novel and unobvious.
Response to Arguments
25. Applicant's arguments filed 9/8/26 have been fully considered and are not found to be convincing, therefore the 35 U.S.C. 101 rejection and double patenting rejection are hereby maintained.
a) Argument #1: Applicant argues that the abstract idea is integrated into a practical application under Step 2A, Prong 2 (improvement to technology)
26. The Examiner respectfully disagrees. In regards to improving the functioning of the computer/technology/technical field, the claims recite the additional elements of imaging devices, computing system, processor, and memory, and they are recited at a high level of generality, and therefore are merely using computer processing components for inventory tracking on a retail shelf. After further review of the Specification, there is no disclosure of technical enhancements to any of the computing components, as in multiple instances of the Specification it discloses generally recited elements. Interpreting the claims in view of the Specification, the claims recite the judicial exception are mere instructions to apply the exception of inventory tracking on a retail shelf (see MPEP 2106.05(f)). The elements recited above do not recite and are not directed to any elements or functions that improve underlying technology.
27. According to MPEP 2106.05(a), it states: “It is important to note that in order for a method claim to improve computer functionality, the broadest reasonable interpretation of the claim must be limited to computer implementation. That is, a claim whose entire scope can be performed mentally, cannot be said to improve computer technology. Synopsys, Inc. v. Mentor Graphics Corp., 839 F.3d 1138, 120 USPQ2d 1473 (Fed. Cir. 2016) (a method of translating a logic circuit into a hardware component description of a logic circuit was found to be ineligible because the method did not employ a computer and a skilled artisan could perform all the steps mentally). Similarly, a claimed process covering embodiments that can be performed on a computer, as well as embodiments that can be practiced verbally or with a telephone, cannot improve computer technology. See RecogniCorp, LLC v. Nintendo Co., 855 F.3d 1322, 1328, 122 USPQ2d 1377, 1381 (Fed. Cir. 2017) (process for encoding/decoding facial data using image codes assigned to particular facial features held ineligible because the process did not require a computer).”
28. Furthermore, in determining whether a claim integrates a judicial exception into a practical application, a determination is made of whether the claimed invention pertains to an improvement in the functioning of the computer itself or any other technology or technical field (i.e., a technological solution to a technological problem). Here, the claims recite generic computer components, the additional elements of imaging devices, computing system, processor, and memory, i.e., that are recited at a high level of generality and are recited as performing generic computer functions customarily used in computer applications. The pending claims do not describe a technical solution to a technical problem. The pending claims are directed to solving the problem of inventory tracking on a retail shelf. The claims of the instant application describe an improvement to a business process i.e., inventory tracking on a retail shelf, not improvement in the functioning of the computer itself or an improvement to any other technology or technological field. Therefore, the claims do not integrate into a practical application either by improvement to a computer/technology/technical field.
29. The claims are not directed to any improvement in computer technology. The claims are directed to an abstract idea of inventory tracking on a retail shelf. The applicant must take into consideration that in order to view the claims as supplying an inventive concept the technological improvement must be present within the claims themselves (Accenture Global Servs., GmbH v. Guidewire Software, inc., 108 USPQ2d 1173 (Fed. Cir. 2013)), (Synopsys, inc. v. Mentor Graphics Corp... 120 USPQ2d 1473 (Fed. Cir. 2016). Additionally, the Specification does not provide any evidence that how the claims provide an improvement to functioning of computing systems or technology. The applicant failed to provide persuasive arguments supported by any necessary evidence to demonstrate that one of ordinary skill in the art would understand that the disclosed invention improves technology.
b) Argument #2: Applicant argues that the claim elements are significantly more than the abstract idea under Step 2B
30. Having determined under step one of the Mayo/Alice framework that claims 21, 31, and 40 are directed to an abstract idea, we next consider under Step 2B of the Guidance, the second step of the Mayo/Alice framework, whether the claims include additional elements or a combination of elements that provides an “inventive concept,” i.e., whether an additional element or combination of elements adds specific limitations beyond the judicial exception that are not “well-understood, routine, conventional activity” in the field (which is indicative that an inventive concept is present) or simply appends well-understood, routine, conventional activities previously known to the industry to the judicial exception. 2019 Revised Guidance, 84 Fed. Reg. at 56.
31. Under step two of the Mayo/Alice framework, the elements of each claim are considered both individually and “as an ordered combination” to determine whether the additional elements, i.e., the elements other than the abstract idea itself, “transform the nature of the claim” into a patent-eligible application. Alice Corp., 573 U.S. at 217 (citation omitted); see Mayo, 566 U.S. at 72-73 (requiring that “a process that focuses upon the use of a natural law also contain other elements or a combination of elements, sometimes referred to as an ‘inventive concept,’ sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the natural law itself’ (emphasis added) (citation omitted)).
32. Here the only additional elements recited in claims 21, 31, and 40 beyond the abstract idea are: the additional elements of imaging devices, computing system, processor, and memory, i.e., generic computer components. See Alice, 573 U.S. at 223 (“[T]he mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.”). Applicant has not identified any additional elements recited in the claim that, individually or in combination, provides significantly more than the abstract idea.
33. Accordingly, there are no meaningful limitations in the claims that transform the judicial exception into a patent eligible application such that the claims amount to significantly more than the judicial exception itself. The dependent claims do not resolve the deficiency of the independent claims and accordingly stand rejected under 35 U.S.C. 101 based on the same rationale.
Conclusion
34. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Santra Bikash et al (An end-to-end annotation-free machine vision system for detection of products on the rack, NPL) is found to be the most pertinent NPL prior art.
35. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
36. A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
37. Any inquiry concerning this communication or earlier communications from the examiner should be directed to FAWAAD HAIDER whose telephone number is (571)272-7178. The examiner can normally be reached Mon-Fri 8 AM to 5 PM.
38. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
39. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Florian Zeender can be reached on 571-272-6790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
40. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FAWAAD HAIDER/Primary Examiner, Art Unit 3627
