DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/23/2025 has been entered.
Response to Arguments
Applicant has amended claims 1, 9, 10.
Claims 2-4 were previously been cancelled.
Claims 1 and 5-10 are currently pending.
Applicant’s arguments, in pages 7-8 filed 11/25/2025, with respect to the 35 USC 103 rejection(s) of the amended claim(s) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of newly found prior art Fujii. Further discussion can be found in the prior art rejection below. Based on these facts, this action is made NON-FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5-6, 9, 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tsuchimochi (WO 2015140853 A1, the attached English language translation is used hereinafter as the Official English language translation of this document, as applied in the previous Office Action) in view of Nomura (WO 2013089004 A1, the attached English language translation is used hereinafter as the Official English language translation of this document, as applied in the previous Office Action), Yamashita (US 20180002109 A1), and Fujii (US 20080068456 A1).
Tsuchimochi teaches a processing apparatus comprising: at least one memory configured to store one or more instructions (Tsuchimochi, pg. 91/97, para 1); and at least one processor configured to execute the one or more instructions to (Tsuchimochi, pg. 64/97, para 3): acquire a product pickup image indicating a scene of picking up a product from a first product shelf by a customer (Tsuchimochi, flow line imaging device acquires image of customer interacting with product and shelf, including picking up a product, pg. 65/97, para 3- pg. 66/97, para 1); determine a product group displayed on the first product shelf, based on shelf-based display information indicating a product displayed on each product shelf (Tsuchimochi, “product shelf information includes a product shelf identifier and a product identifier of a product displayed on the product shelf corresponding to the product shelf identifier” e.g. product shelf A contains product group A, itself containing products A1, A2, A3, is identified as the product group that the product belongs to, pg. 68/97, para 3-4); and recognize a product included in the product pickup image by recognition processing in which the determined product is collated among pieces of product feature value information indicating a feature value of an external appearance of a plurality of products (Tsuchimochi, “the product recognition processing unit 230 collates the extracted image feature with the product feature data corresponding to each product stored in the reference product information storage unit 234 for each product,” to recognize the product, pg. 69/97, para 3).
However, Tsuchimochi fails to teach where Nomura teaches to recognize a product included in the product pickup image by recognition processing in which the determined product group is set as a collation target among pieces of product feature value information indicating a feature value of an external appearance of a plurality of products (Nomura, when “the shelf [which is the identified product group to which the product belongs based on the shelf information] is a collation target…the article condition (number of feature points) displayed on the determined shelf is set,” and the final product is recognized, pg. 134/173, para 2).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tsuchimochi using the teachings of Nomura to include Nomura’s collation target to Tsuchimochi’s collation processing for product recognition. Doing so would improve the collation processing by providing a collation target, which would be used to more efficiently recognize the product by specifying the exact goal that the product should match.
However, the combination of Tsuchimochi and Nomura fails to teach where Yamashita teaches wherein the at least one processor is further configured to execute the one or more instructions to: acquire a product replenishment image indicating a scene of replenishing with a product on the first product shelf by a salesperson (Yamashita, Fig. 8, acquires image of clerk/salesperson re-stocking/displaying products on shelf, [0070]), recognize a product included in the product replenishment image by recognition processing in which all products indicated by the product feature value information are set as a collation target (Yamashita, Fig. 8, apparatus moves on to product allocation inspection, which includes product recognition [0052], which “compares multiple features in images of various products, which are registered in the product database…with multiple features in captured images of products so as to specify a product having the largest number of matched features as a product reflected in an image captured by the image acquisition part,” thereby setting the products indicated by the feature value information as a collation target, [0038]); and register, in the shelf-based display information, a product recognized to be included in the product replenishment image, as a product displayed on the first product shelf (Yamashita, database is updated with the product, its number, and positional information [shelf information], thereby registering the product in the shelf information via the database [0038]), wherein a same camera generates the product pickup image and the product replenishment image (Yamashita, Fig. 8, capture monitoring image S41 [0069-0070]); and classify an image generated by the camera into the product pickup image and the product replenishment image (Yamashita, Fig. 8, the image is classified as either being of the customer interacting with the product/shelf [product pickup image] or of the clerk [product replenishment image], [0069-0070]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura, using the teachings of Yamashita to include Yamashita’s capturing and classification of images to distinguish between clerk/salesperson’s and customer’s actions to Tsuchimochi's, as modified by Nomura, imaging and processing of shelf/product-person interaction. Doing so would improve processing of product interaction imaging by providing a way to distinguish between potential restocking actions, such as those by the clerk/salesperson, and potential purchasing/thieving actions, such as those by a customer, which would be used to more efficiently identify images for different purposes (e.g. a customer’s cart tracking vs. a clerk’s inventory tracking).
However, the combination of Tsuchimochi, Nomura, and Yamashita fails to teach where Fujii teaches wherein the classification is performed based on a mode set at a photographing time of the image (Fujii, classify image based on mode set at a photographing time of the image, [0591-0594]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura and Yamashita, using the teachings of Fujii to include Fujii’s classification of images based on a mode set at the photographing time of the image to Tsuchimochi’s, as modified by Nomura and Yamashita, classification of images into pick-up or restocking images. Doing so would improve classification by providing consideration of the mode at photographing time, which would be used to pick out when/how a product was picked up or restocked.
Regarding claim 5, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1, wherein the at least one processor is further configured to execute the one or more instructions to: acquire a product shelf image including a product displayed on the first product shelf (Tsuchimochi, “flow line image obtained by imaging the front of the product shelf,” including the product, pg. 76/97, para 3), recognize a product included in the product shelf image by recognition processing in which all products indicated by the product feature value information are set as a collation target (Nomura, when “the shelf [which is the identified product group to which the product belongs based on the shelf information] is a collation target…the article condition (number of feature points) displayed on the determined shelf is set,” and the final product is recognized, pg. 134/173, para 2); and register, in the shelf-based display information, a product recognized to be included in the product shelf image, as a product displayed on the first product shelf (Nomura, database stores/registers feature values associated with recognition object [product] and shelf information, pg. 133/173, para 5).
Regarding claim 6, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1, wherein the at least one processor is further configured to execute the one or more instructions to: determine, based on the product pickup image, a stage on which a product picked up from the first product shelf is displayed (Tsuchimochi, “detection unit may detect which position (for example, the upper stage / lower stage) of the product shelf” that the product is displayed on, pg. 87/97, para 2), and determine, based on the shelf-based display information indicating a product displayed on each stage of each of the plurality of product shelves, a product group displayed on a the determined stage of the first product shelf (Tsuchimochi, “the product shelf information may indicate which product is displayed at which position [stage] of the product shelf. By configuring in this way, it is possible to further narrow down the products to be searched,” with the product group to be searched being based on the stage information, pg. 87/97, para 2), and recognize a product included in the product pickup image by recognition processing in which the product group determined to be displayed on the determined stage is set as a collation target among pieces of the product feature value information (Tsuchimochi, the product group is searched using collation processing to recognize the product pg. 69/97, para 3, wherein Nomura discloses utilizing a collation target, pg. 134/173, para 2).
Regarding claims 9, 10, the rationale provided in the rejection of claim 1 is incorporated herein. In addition, the processing apparatus of claim 1 corresponds to the method of claim 9 as well as the non-transitory storage medium (Tsuchimochi, pg. 91/97, para 1) of claim 10, and performs the steps disclosed herein.
Claim(s) 7-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tsuchimochi in view of Nomura, Yamashita, and Fujii as applied to claim 1 above, further in view of Bao (US 20180293635 A1).
Regarding claim 7, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1.
However, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii fails to teach where Bao teaches wherein the processor is further configured to execute the one or more instructions to count, based on a recognition result of the recognizing a product included in the product pickup image, a number of products picked up from the first product shelf for each product (Bao, Fig. 6, products are recognized and quantity of products sold [corresponding to picked up and bought] from shelf is calculated, [0012], [0098-0103]), and delete, from a product displayed on the first product shelf indicated by the shelf-based display information, a product in which the number has reached a reference value (Bao, if a product’s, having been displayed on a shelf, quantity/number reaches zero [reference value], it is deleted from the checking database, [0071-0072]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura, Yamashita, and Fujii, using the teachings of Bao to include Bao’s product quantity and deletion to Tsuchimochi's, as modified by Nomura, Yamashita, and Fujii, product tracking. Doing so would improve inventory management by providing a way to delete a product that is no longer present, which would be used to speed up processing by not considering an absent product for recognition.
Regarding claim 8, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1.
However, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii fails to teach where Bao teaches wherein the processor is further configured to execute the one or more instructions to delete, from a product displayed on the first product shelf indicated by the shelf-based display information, a product in which a number of sales indicated by sales information being managed by a point of sales (POS) system has reached a reference value (Bao, when a product, having been displayed on a shelf, is sold and the quantity/number reaches 0 [reference value], it is deleted from the checking database, [0072]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura, Yamashita, and Fujii, using the teachings of Bao to include Bao’s product deletion based on sales information to Tsuchimochi's, as modified by Nomura, Yamashita, and Fujii, product tracking. Doing so would improve inventory management by providing a way to delete a product that is no longer present, which would be used to speed up processing by not considering an absent product for recognition.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEELY G YEARGIN whose telephone number is (571)272-5126. The examiner can normally be reached M-Th 8am-6pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571) 272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEELY GWYNNE YEARGIN/ Examiner, Art Unit 2671
/VINCENT RUDOLPH/ Supervisory Patent Examiner, Art Unit 2671