DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 1 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 7 of U.S. Patent No. 12177387.
Instant application Claim 1 (complete)
12177387 Claim 1+7 (mapped portions)
1. A method comprising: " accessing a first image captured by an optical sensor, arranged within the store, at a first time; "
1. A method comprising: accessing an image captured by an optical sensor, arranged within a store, at a first time;
detecting a first segment of an inventory structure in the store in a first region of the image;
detecting a segment of an inventory structure in the store in a first region of the image;
accessing a first product model representing a first set of visual characteristics of a first product type;
accessing a first product model representing a first set of visual characteristics of a first product type assigned to a first slot, in the set of slots, by a graphical representation of the store;
detecting a first set of features in a subregion of the first image;
detecting a first set of features in a subregion of the image;
detecting presence of a first product unit of the first product type occupying the first segment of the inventory structure in response to the first set of features approximating the first set of visual characteristics;
detecting presence of a first product unit of the first product type occupying the segment of the inventory structure at the first time in response to the first set of features approximating the first set of visual characteristics;
representing presence of the first product unit, occupying the first segment of the inventory structure at the first time, in a first graphical representation of the store corresponding to a current stock condition of the store.
aggregating presence of the first product unit occupying the first segment of the inventory structure, presence of the second product unit occupying the second segment of the inventory structure, and presence of the third product unit occupying the third segment of the inventory structure into a second graphical representation corresponding to a current stock state of the inventory structure.
As can be seen above patented claim 7 discloses all the subject matter of pending claim 1.
Claim 2 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 7 of U.S. Patent No. 12177387 in view of Katircioglu (US 20150262116 A1).
Instant application Claim 2 (complete)
12177387 Claim 1+7 (mapped portions)
1. A method comprising: " accessing a first image captured by an optical sensor, arranged within the store, at a first time; "
1. A method comprising: accessing an image captured by an optical sensor, arranged within a store, at a first time;
detecting a first segment of an inventory structure in the store in a first region of the image;
detecting a segment of an inventory structure in the store in a first region of the image;
accessing a first product model representing a first set of visual characteristics of a first product type;
accessing a first product model representing a first set of visual characteristics of a first product type assigned to a first slot, in the set of slots, by a graphical representation of the store;
detecting a first set of features in a subregion of the first image;
detecting a first set of features in a subregion of the image;
detecting presence of a first product unit of the first product type occupying the first segment of the inventory structure in response to the first set of features approximating the first set of visual characteristics;
detecting presence of a first product unit of the first product type occupying the segment of the inventory structure at the first time in response to the first set of features approximating the first set of visual characteristics;
representing presence of the first product unit, occupying the first segment of the inventory structure at the first time, in a first graphical representation of the store corresponding to a current stock condition of the store.
aggregating presence of the first product unit occupying the first segment of the inventory structure, presence of the second product unit occupying the second segment of the inventory structure, and presence of the third product unit occupying the third segment of the inventory structure into a second graphical representation corresponding to a current stock state of the inventory structure.
2. The method of Claim 1: " further comprising identifying a first set of slots within the first segment of the inventory structure;
identifying a set of slots within the segment of the inventory structure;
wherein accessing the first product model representing the first set of visual characteristics comprises accessing the first product model representing the first setof visual characteristics of the first product type assigned to a first slot, in the set of slots, by a second graphical representation of the store;
accessing a first product model representing a first set of visual characteristics of a first product type assigned to a first slot, in the set of slots, by a graphical representation of the store;
and " further comprising:detecting a first shelf tag in the first image;
detecting a first shelf tag in the image;
locating a first slot boundary around the first slot, depicted in the first image,according to a first position of the first shelf tag in the first image;
extracting a second position of the first product unit relative to the first slot boundary from the first image;
and in response to the second position of the first product unit falling outside of the first slot boundary, flagging the first slot for reorganization of a set of product units comprising the first product unit.
Regarding claim 2, patented claim 7 discloses all the subject matter of claim 2 save the bolded steps above. However Katircioglu discloses:
locating a first slot boundary around the first slot, depicted in the first image,according to a first position of the first shelf tag in the first image (paragraph 24 Based on detected edges and localized identifier tags, the shelf image 202 is segmented into a plurality of inventory images 204, each corresponding to a particular product type.);extracting a second position of the first product unit relative to the first slot boundary from the first image (paragraph 39); andin response to the second position of the first product unit falling outside of the first slot boundary, flagging the first slot for reorganization of a set of product units comprising the first product unit (paragraph 39 flag as displaced).
It would have been obvious to one of ordinary skill in the art to combine this teaching with those of the patented claim by using identifier tags to recognize item slots and flagging when item locations are displaced. The motivation for the combination is improved inventory management (paragraph 3).
Claim 1 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. 11593755.
Instant application Claim 1 (complete)
11593755 Claim 1 (mapped portions)
1. A method comprising: " accessing a first image captured by an optical sensor, arranged within the store, at a first time; "
1. A method for stock keeping in a store comprising: accessing a photographic image captured by a fixed camera, arranged within the store, at a first time;
detecting a first segment of an inventory structure in the store in a first region of the image;
estimating a shelving segment of an inventory structure in the store depicted in the photographic image based on a projection of the geometry of the field of view onto a planogram of the store
accessing a first product model representing a first set of visual characteristics of a first product type;
retrieving a first product model representing a first set of visual characteristics of a first product type assigned to a first slot, in the set of slots, by the planogram;
detecting a first set of features in a subregion of the first image;
extracting a first constellation of features from the photographic image; detecting presence of a first product unit of the first product type occupying the shelving segment of the inventory structure at the first time in response to the first constellation of features approximating the first set of visual characteristics represented in the first product model;
detecting presence of a first product unit of the first product type occupying the first segment of the inventory structure in response to the first set of features approximating the first set of visual characteristics;
extracting a first constellation of features from the photographic image; detecting presence of a first product unit of the first product type occupying the shelving segment of the inventory structure at the first time in response to the first constellation of features approximating the first set of visual characteristics represented in the first product model;
representing presence of the first product unit, occupying the first segment of the inventory structure at the first time, in a first graphical representation of the store corresponding to a current stock condition of the store.
and representing presence of the first product unit, occupying the shelving segment of the inventory structure at the first time, in a realogram.
As can be seen above patented claim 1 discloses all the subject matter of pending claim 1.
Claim 2 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 13 of U.S. Patent No. 11593755 in view of Katircioglu (US 20150262116 A1).
Instant application Claim 2 (complete)
11593755 Claim 1+13 (mapped portions)
1. A method comprising: " accessing a first image captured by an optical sensor, arranged within the store, at a first time; "
1. A method for stock keeping in a store comprising: accessing a photographic image captured by a fixed camera, arranged within the store, at a first time;
detecting a first segment of an inventory structure in the store in a first region of the image;
estimating a shelving segment of an inventory structure in the store depicted in the photographic image based on a projection of the geometry of the field of view onto a planogram of the store
accessing a first product model representing a first set of visual characteristics of a first product type;
retrieving a first product model representing a first set of visual characteristics of a first product type assigned to a first slot, in the set of slots, by the planogram;
detecting a first set of features in a subregion of the first image;
extracting a first constellation of features from the photographic image; detecting presence of a first product unit of the first product type occupying the shelving segment of the inventory structure at the first time in response to the first constellation of features approximating the first set of visual characteristics represented in the first product model;
detecting presence of a first product unit of the first product type occupying the first segment of the inventory structure in response to the first set of features approximating the first set of visual characteristics;
extracting a first constellation of features from the photographic image; detecting presence of a first product unit of the first product type occupying the shelving segment of the inventory structure at the first time in response to the first constellation of features approximating the first set of visual characteristics represented in the first product model;
representing presence of the first product unit, occupying the first segment of the inventory structure at the first time, in a first graphical representation of the store corresponding to a current stock condition of the store.
and representing presence of the first product unit, occupying the shelving segment of the inventory structure at the first time, in a realogram.
2. The method of Claim 1: " further comprising identifying a first set of slots within the first segment of the inventory structure;
identifying a set of slots within the shelving segment of the inventory structure;
wherein accessing the first product model representing the first set of visual characteristics comprises accessing the first product model representing the first setof visual characteristics of the first product type assigned to a first slot, in the set of slots, by a second graphical representation of the store;
retrieving a first product model representing a first set of visual characteristics of a first product type assigned to a first slot, in the set of slots, by the planogram;
and " further comprising:detecting a first shelf tag in the first image;
detecting a first shelf tag in the photographic image
locating a first slot boundary around the first slot, depicted in the first image,according to a first position of the first shelf tag in the first image;
locating a first slot boundary around the first slot, depicted in the photographic image, according to a position of the first shelf tag in the photographic image;
extracting a second position of the first product unit relative to the first slot boundary from the first image;
and in response to the second position of the first product unit falling outside of the first slot boundary, flagging the first slot for reorganization of a set of product units comprising the first product unit.
Regarding claim 2, patented claim 13 discloses all the subject matter of claim 2 save the bolded steps above. However Katircioglu discloses:
locating a first slot boundary around the first slot, depicted in the first image,according to a first position of the first shelf tag in the first image (paragraph 24 Based on detected edges and localized identifier tags, the shelf image 202 is segmented into a plurality of inventory images 204, each corresponding to a particular product type.);extracting a second position of the first product unit relative to the first slot boundary from the first image (paragraph 39); andin response to the second position of the first product unit falling outside of the first slot boundary, flagging the first slot for reorganization of a set of product units comprising the first product unit (paragraph 39 flag as displaced).
It would have been obvious to one of ordinary skill in the art to combine this teaching with those of the patented claim by using identifier tags to recognize item slots and flagging when item locations are displaced. The motivation for the combination is improved inventory management (paragraph 3).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Groenovelt (US 20090063307 A1).
Regarding claim 1, Groenovelt discloses:
1. A method comprising:
accessing a first image captured by an optical sensor, arranged within the store, at a first time (fig. 2 204);
detecting a first segment of an inventory structure in the store in a first region of the image (paragraph 31 “Thereafter, using known geometrical transformation techniques, the location of the shelves 302-306 within the image may be translated into an actual location of the shelf within the inventory environment, which may be reflected by the position of the shelves 402-406 illustrated in the extracted planogram 400. In order to determine the actual location of the shelves, position information of the image capture device within the inventory environment is used.);
accessing a first product model representing a first set of visual characteristics of a first product type (paragraph 23 the database 130 stores a plurality of stored product images and/or features and characteristics of the images and the products that may be used by the object recognition software);
detecting a first set of features in a subregion of the first image (paragraph 206); " detecting presence of a first product unit of the first product type occupying the first segment of the inventory structure in response to the first set of features approximating the first set of visual characteristics (paragraph 27 object recognition analysis to provide “recognized product instances” of a particular product); and
representing presence of the first product unit, occupying the first segment of the inventory structure at the first time, in a first graphical representation of the store corresponding to a current stock condition of the store ([0032] With knowledge of the identities of products corresponding to the one or more recognized product instances (shown in bold in FIG. 3) as well as their actual locations within the inventory environment, the extracted planogram 400 may include a corresponding image 408-412 of the recognized products at locations within the extracted planogram 400 corresponding to their actual locations within the inventory environment 301, 102.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Groenovelt (US 20090063307 A1) in view of Katircioglu (US 20150262116 A1).
Regarding claim 2, Groenovelt further discloses:
2. The method of Claim 1: " further comprising identifying a first set of slots within the first segment of the inventory structure (fig. 4, paragraph 32); " wherein accessing the first product model representing the first set of visual characteristics comprises accessing the first product model representing the first set of visual characteristics of the first product type assigned to a first slot (paragraph 23 stored instructions implementing software routines capable of generating a planogram based on the results provided), in the set of slots, by a second graphical representation of the store (paragraph 56 target planogram); and " fthe first slot boundary (fig. 2 212, paragraph 28), flagging the first slot for reorganization of a set of product units comprising the first product unit (fig. 2 216, paragraph 36-37).
Groenovelt fails to disclose and Katircioglu discloses:
further comprising: detecting a first shelf tag in the first image (paragraph 24 localized identifier tag); locating a first slot boundary around the first slot, depicted in the first image, according to a first position of the first shelf tag in the first image (paragraph 24 In some embodiments, identifier tags associated with the items are located on the shelves and are captured in the shelf image 202. Based on detected edges and localized identifier tags, the shelf image 202 is segmented into a plurality of inventory images 204, each corresponding to a particular product type.).
It would have been obvious to one of ordinary skill in the art to combine this teaching with those of Groenovelt by using identifier tags to recognize item slots. The motivation for the combination is improved inventory management (paragraph 3).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Opalach (US 20090059270 A1) discloses planogram extraction based on image processing. Birch (US 20140201040 A1) discloses planogram display based on image processing. Rizzolo (US 20170032311 A1) discloses inventory capture based on image processing.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NATHAN A MITCHELL whose telephone number is (571)270-3117. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Zeender can be reached at 571-272-6790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATHAN A MITCHELL/Primary Examiner, Art Unit 3627