Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/20/2026 has been entered.
Response to Amendment
Applicant’s “Response to Amendment and Reconsideration” filed on 1/20/2026 has been considered.
Claims 1-14, 20-25 are pending in this application and an action on the merits follows.
Claim Objections
Claims 1-5, 8-14, 20 are objected to because the claims do not follow the proper format such as having bullet points that appear throughout the claims. Appropriated format correction is needed such as removing all bullet points.
Double Patenting
The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-14, 20-25 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 10,625,426. Although the claims at issue are not identical, they are not patentably distinct from each other. For example:
Claim 1 of the present application recites:
A method for automatically generating a planogram assigning products to inventory structures within a facility, the method comprising: accessing a set of images captured by a set of cameras during a first scan cycle, the set of images depicting a set of inventory structures within the facility; and" based on confirmation of a preferred stocking condition in the facility during the first scan cycle: initializing a new planogram of the facility; detecting a set of products depicted in the set of images; and for each product in the set of products detected in the set of images:- identifying the product; calculating a first shelf position of the product based on a first vertical position of the product detected in a first image, in the set of images, depicting a first inventory structure in the set of inventory structures; calculating a first lateral position of the first product based on a first horizontal position of the product detected in the first image;- calculating a product location of the product within the facility based on a location of the set of cameras in the facility during capture of the first image
Claim 1 of U.S. Patent No. 10,625,426 recites:
A method for automatically generating a new planogram assigning products to shelving structures within a store, the method comprising: dispatching a robotic system to autonomously collect map data of a floor space within the store during a first mapping routine; initializing the new planogram of the store, the new planogram representing locations of a set of shelving structures within the store based on map data recorded by the robotic system; - in response to receipt of confirmation of a preferred stocking condition in the store, dispatching the robotic system to record optical data at a first waypoint proximal a first shelving structure, in the set of shelving structures, during a first imaging routine; -accessing a first image comprising optical data recorded by the robotic system while occupying the first waypoint; detecting a first shelf at a first vertical position in the first image; detecting a first object in a first lateral position over the first shelf in the first image; identifying the first object as a unit of a first product based on features extracted from a first region of the first image representing the first object; - projecting the first vertical position of the first shelf and the first lateral position of the first object onto a representation of the first shelving structure in the new planogram to define a first slot in the new planogram; and - based on confirmation of the preferred stocking condition in the store during the first imaging routine and in response to identifying the first object as the unit of the first product, writing an assignment for the first product to the first slot in the new planogram, the assignment defining a target stock condition of the first slot.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6-7, 10, 14, 20-25 are rejected under 35 U.S.C. 103 as being unpatentable over Opalach (WO Patent Publication No. 2009027839), in view of Zimmerman et. al. (U.S. Patent Publication No. 20080077511).
Regarding claim 1, Opalach discloses a method for automatically generating a planogram assigning products to inventory structures within a facility, the method comprising: (The object recognition analysis provides object recognition data, which may include one or more recognized product instances within the received image(s), based on stored product images. In turn, a variety of functionalities may be enabled based on the object recognition data. For example, in one embodiment of the present invention, a planogram may be extracted from the one or more images analyzed in this manner, [6-7] see images );
accessing a set of images captured by a set of cameras during the first scan cycle, the set of images depicting a set of inventory structures within the facility; (the image capture devices may comprise relatively mobile (potentially wireless) still image or video cameras mounted on any of a number of movable objects including, but not limited to, fixed tracks that guide movement of the image capture devices, shopping or stocking carts, or even persons moving about the inventory environment 102, [19], In particular, FIG. 3 illustrates an exemplary image 300 of an inventory environment 301 in which a plurality of product support devices (in this case, shelves) 302-306 are used to display a variety of products 308-314, [28]);
and based on confirmation of a preferred stocking condition in the facility during the first scan cycle, (Referring once again to FIG. 2, processing optionally continues at block 212 where the extracted planogram is compared to a target planogram, (preferred condition), [30-36];
initializing a new planogram of the facility; detecting a set of products depicted in the set of images; for each product in the set of products detected in the set of images:
identifying the product; (the output of the object recognition analysis, i.e. the object recognition data, includes identification of any recognized products in the image as well as appropriate indications (e.g., probabilities) that corresponding products are, in fact, in the image, occasionally referred to hereinafter as recognized product instance(s). Additionally, the object recognition data preferably includes location information (relative to the image) for the identified product(s), if any, within the image, i.e., x, y coordinates in the image., [26-27]);
writing the product assignment, linking a product identifier of the product to the location of the product, to the new planogram, (techniques for mapping actual locations within an inventory environment to corresponding locations within a planogram representation are well known in the art, [32]).
Opalach does not explicitly teach calculating a first shelf position of the product based on a first vertical position of the product detected in a first image, in the set of images, depicting a first inventory structure in the set of inventory structures;
calculating a first lateral position of the first product based on a first horizontal position of the product detected in the first image; calculating a product location of the product within the facility;
a location of the set of cameras in the facility during capture of the first image, in the set of images, depicting the product;
and a position of the product within the first inventory structure depicted in the first image, the position defined by the first shelf position and the first lateral position;
generating a product assignment linking a product identifier of the product to the location of the product;
Opalach teaches the location of a given image capture device 104 may be useful, or even necessary, to the various techniques described herein. In the case of fixed image capture devices 104, such location information may be known implicitly, [19], based on one or more training images of objects to be recognized, such algorithms provide an identification of the objects recognized in the image, locations for the recognized objects within the image, [22);
However, Zimmerman teaches additional imaging devices provide a vertical perspective of a shelf, allowing capture of a shelf image from above, [57]; the imaging device is mounted on a flexible section capable of moving the imaging device vertically, in order to image the products on the shelves at different heights, claim 10.
he operator can move the mobile inventory robot 20 from side to side (horizontal displacement) to get different views, [59], navigating the mobile inventory robot to a shelf through the store; capturing an image of the shelf, claim 1.
segmenting the captured shelf image to detect an image of an item on the shelves, abstract – detecting product in image).
the product database 225 comprises..locations of items in the store used for comparison and analysis, [58]; generating a product database that contains a listing of: the products, the location of the products, and the identifiers of the products, claim 6.
The imaging device 30 captures a shelf image (step 610). The product barcode decoder 210 decodes a product barcode from the captured image (step 615). The product image segmentation module 205 segments the captured image to identify a shelf image of the product (step 630). One or more product visual descriptors are extracted from the segmented image of the product (step 1020). The product visual descriptors are placed in the product database 225 (step 1030), [83].
It would have been obvious to one with ordinary skill in the art before the effective filing date of the invention, to modify planogram extraction system of Opalach to incorporate the system of Zimmerman in order to make the process autonomous and more efficient, [9-14].
Regarding claim 2, Opalach discloses detecting the set of products depicted in the set of images comprises detecting a first product, in the set of products, depicted in a first image in the set of images; and " wherein calculating a location of a product in the facility for each product in the set of products comprises, for the first product in the set of products, (With knowledge of the identities of products corresponding to the one or more recognized product instances (shown in bold in FIG. 3) as well as their actual locations within the inventory environment, the extracted planogram 400 may include a corresponding image 408-412 of the recognized products at locations within the extracted planogram 400 corresponding to their actual locations within the inventory environment 301, 102. Once again, techniques for mapping actual locations within an inventory environment to corresponding locations within a planogram representation are well known in the art, [31-32]). However, Zimmerman teaches accessing a spatial map of the facility store; calculating a first field of view of a camera, in the robotic system set of cameras, that captured the first image projecting the first field of view of the camera onto the spatial map of the facility based on a location and an orientation of the robotic system camera in the facility during capture of the first image by the camera; projecting a first position of the first product, extracted from the first image into the first field of view of the camera projected onto the spatial map of the facility store; and calculating a first location of the first product based on the first position of the product projected into the first field of view of the camera projected onto the spatial map of the facility, (see Additional imaging devices provide a vertical perspective of a shelf, allowing capture of a shelf image from above.. The operator can move the mobile inventory robot 20 from side to side to get different views, [57-59], the tracking tags are secured to a fixed structure of the store to assist the mobile inventory robot in determining a location of the mobile inventory robot within the store, claim 3… the product image segmentation module 205 segments the captured image to identify a shelf image of the product (step 630). One or more product visual descriptors are extracted from the segmented image of the product (step 1020), [83].
Regarding claims 6-7, Opalach does not teach, however, Zimmerman teaches dispatching the robotic system to image the set of shelving inventory structures within the facility during the first scan cycle comprises dispatching the robotic system to execute the first scan cycle within the facility following initial delivery of the robotic system to the facility and in response to absence of a preexisting planogram of the facility; dispatching the robotic system to image the set of shelving inventory structures within the facility during the first scan cycle comprises dispatching the robotic system to autonomously navigate toward and to image the set of shelving inventory structures within the facility during the first scan cycle in response to receipt of confirmation of the preferred stocking condition in the facility, (The mobile inventory robot 20A navigates through an integration of the navigation imaging devices 325, the tracking image device 330, tracking tags installed on the ceiling of the store, detectors 365 imbedded in the mobile inventory robot 20, and dead reckoning, [63-66]).
Regarding claim 10, Opalach does not teach, however, Zimmerman teaches wherein detecting the set of products depicted in the set of images comprises detecting a first product, in the set of products, depicted in a first region of a first image in the set of images; and wherein calculating a location of a product in the facility for each product in the set of products comprises, for the first product in the set of products detecting a set of shelf tags in the first image; associating the first product with a first shelf tag, in the set of shelf tags, depicted proximal the first region of the first image; and defining a first location of the first product in the facility relative to the first shelf tag, (The mobile inventory robot 20A navigates through an integration of the navigation imaging devices 325, the tracking image device 330, tracking tags installed on the ceiling of the store, detectors 365 imbedded in the mobile inventory robot 20, and dead reckoning, [63-66]).
Regarding claims 14 and 23, Opalach does not teach, however, Zimmerman teaches prior to the first scan cycle, dispatching the robotic system to autonomously navigate throughout the facility during a mapping cycle; accessing a spatial map of the facility generated from spatial data captured by the robotic system during the mapping cycle; defining a coordinate system within the spatial map; identifying a first shelving inventory structure represented in the spatial map; and defining a first set of waypoints, relative to the coordinate system, along the first shelving inventory structure, the first set of waypoints specifying orientations facing the first shelving inventory structure; and wherein dispatching the robotic system to image the set of shelving inventory structures within the facility store during the first scan cycle comprises dispatching the robotic system to navigate to each waypoint in the first set of waypoints and to record an image of the first shelving inventory structure while occupying each waypoint in the first set of waypoints, ([45-52], the installer places waypoints near the floor of the store for use by the tracking system 55 to navigate through the store (step 525). In one embodiment, the waypoints comprise infrared transmitters. The waypoints provide navigation vectors; detecting two provide the inventory robot an absolute position fix of its location in the store, [66]).
Claim 20 has similar limitations as Claim 1 (see claim 1 rejection above). In addition:
Opalach does not explicitly disclose dispatching a robotic system to image a set of shelving inventory structures within the facility during a first scan cycle and accessing a set of images captured by the robotic system during the first scan cycle; Opalach teaches image capture devices mounted on mobile objects that move through the environment to capture images, [19].
However, Zimmerman teaches the mobile inventory robot 20 traverses the inventory map, images of shelves, barcodes on shelves, items on shelves and bar codes on items are captured in a predetermined order; [45-52].
It would have been obvious to one with ordinary skill in the art before the effective filing date of the invention, to modify planogram extraction system of Opalach to incorporate the robotic system of Zimmerman in order to make the process autonomous and more efficient, [9-14].
Regarding claim 21, Opalach does not teach, however, Zimmerman teaches dispatching the robotic system comprises dispatching the robotic system comprising: a base; a drive system arranged in the base; a mast extending vertically from the base; a set of cameras arranged on the mast; and a wireless communication module configured to: download waypoints and a master map of a facility from a remote computer system; and upload images captured by the set of cameras to the remote computer system, ([45-52], The mobile inventory robot 20A navigates through an integration of the navigation imaging devices 325, the tracking image device 330, tracking tags installed on the ceiling of the store, detectors 365 imbedded in the mobile inventory robot 20, and dead reckoning, [63]).
Regarding claim 22, Opalach teaches detecting the set of products depicted in the set of images comprises detecting a first product, in the set of products, depicted in a first region of a first image in the set of images; and wherein calculating a location of a product in the facility for each product in the set of products comprises, for the first product in the set of products. Opalach does not teach, however, Zimmerman teaches detecting a set of tags in the first image;associating the first product with a first tag, in the set of tags, depicted proximal the first region of the first image; and defining a first location of the first product in the facility relative to the first tag, (The tracking system 55 uses the tracking tags to determine location, [65]).
Regarding claims 24-25, Opalach teaches capturing images by fixed or mobile cameras. Opalach does not teach, however, Zimmerman teaches dispatching a robotic system to image the set of inventory structures within the facility during the first scan cycle; and wherein accessing the set of images captured by the set of cameras comprises accessing the set of images captured by the set of cameras integrated into the robotic system; accessing the set of images captured by the set of cameras comprises accessing the set of images captured by the set of cameras integrated into a mobile computing device accessed by an associate affiliated with the facility, ([45-52], The mobile inventory robot 20A navigates through an integration of the navigation imaging devices 325, the tracking image device 330, tracking tags installed on the ceiling of the store, detectors 365 imbedded in the mobile inventory robot 20, and dead reckoning, [63]).
Allowable Subject Matter
Claims 3-5, 8-9, 11-13 are dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim, any intervening claims, and the applicant overcomes the double patenting rejection.
Response to Arguments
Applicant’s arguments have been considered. Zimmerman reference teaches determining location of the imaging device within the store and associating detected products with location in the store. It would have been obvious to determine product location within facility based on the known location off the imaging device and the position of the product within the captured image. See the new rejection above.
Examiner notes that Applicant’s amendments had some, but not all, of the limitations from claim 3 incorporated into the independent claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MILENA RACIC whose telephone number is (571)270-5933. The examiner can normally be reached M-F 7:30am-4pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Florian (Ryan) Zeender can be reached at (571)272-6790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MILENA RACIC/Patent Examiner, Art Unit 3627
/FLORIAN M ZEENDER/Supervisory Patent Examiner, Art Unit 3627
Prosecution Insights