Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendments filed 4/27/2026 has been entered and made of record. Claims 1 and 5-10 remain pending. Please see updated interpretation and rejection as fully disclosed below.
Response to Arguments
Applicant's arguments filed 4/27/2026 have been fully considered but they are not persuasive given the amendments to independent claims 1 and 9-10 raise new 112(a) and 112(d) issues as fully disclosed below. As such, the arguments are directed solely to the amended portions that raise these 112 issues, and will not be considered, and the prior art rejections are maintained.
Based on these facts, this action is made FINAL.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 and 5-10 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding independent claims 1 and 9-10, each disclose having the processing apparatus “switch a photographing mode of the camera between a normal photographing mode and a product replenishment photographing mode based on a user input.” The closest support for this is in Par. [0091], where in switching of the modes are present in the camera and only achieved by an input by a salesperson. Thus, there is no support for a processing apparatus to perform the switching between modes of the camera when only the camera can switch the modes.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 1 and 5-10 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Regarding independent claims 1 and 9-10, the amended claim reiterates, and does not further limit, the classifying of the image. Both disclose classifying an image into the product pickup image and product replenishment image based on the mode, and nothing else. As such, the newly added limitation does not further limit the claim.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5-6, 9, 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tsuchimochi (WO 2015140853 A1, the attached English language translation is used hereinafter as the Official English language translation of this document, as applied in the previous Office Action) in view of Nomura (WO 2013089004 A1, the attached English language translation is used hereinafter as the Official English language translation of this document, as applied in the previous Office Action), Yamashita (US 20180002109 A1), and Fujii (US 20080068456 A1).
Tsuchimochi teaches a processing apparatus comprising: at least one memory configured to store one or more instructions (Tsuchimochi, pg. 91/97, para 1); and at least one processor configured to execute the one or more instructions to (Tsuchimochi, pg. 64/97, para 3): acquire a product pickup image indicating a scene of picking up a product from a first product shelf by a customer (Tsuchimochi, flow line imaging device acquires image of customer interacting with product and shelf, including picking up a product, pg. 65/97, para 3- pg. 66/97, para 1); determine a product group displayed on the first product shelf, based on shelf-based display information indicating a product displayed on each product shelf (Tsuchimochi, “product shelf information includes a product shelf identifier and a product identifier of a product displayed on the product shelf corresponding to the product shelf identifier” e.g. product shelf A contains product group A, itself containing products A1, A2, A3, is identified as the product group that the product belongs to, pg. 68/97, para 3-4); and recognize a product included in the product pickup image by recognition processing in which the determined product is collated among pieces of product feature value information indicating a feature value of an external appearance of a plurality of products (Tsuchimochi, “the product recognition processing unit 230 collates the extracted image feature with the product feature data corresponding to each product stored in the reference product information storage unit 234 for each product,” to recognize the product, pg. 69/97, para 3).
However, Tsuchimochi fails to teach where Nomura teaches to recognize a product included in the product pickup image by recognition processing in which the determined product group is set as a collation target among pieces of product feature value information indicating a feature value of an external appearance of a plurality of products (Nomura, when “the shelf [which is the identified product group to which the product belongs based on the shelf information] is a collation target…the article condition (number of feature points) displayed on the determined shelf is set,” and the final product is recognized, pg. 134/173, para 2).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tsuchimochi using the teachings of Nomura to include Nomura’s collation target to Tsuchimochi’s collation processing for product recognition. Doing so would improve the collation processing by providing a collation target, which would be used to more efficiently recognize the product by specifying the exact goal that the product should match.
However, the combination of Tsuchimochi and Nomura fails to teach where Yamashita teaches acquire a product replenishment image indicating a scene of replenishing with a product on the first product shelf by a salesperson (Yamashita, Fig. 8, acquires image of clerk/salesperson re-stocking/displaying products on shelf, [0070]), recognize a product included in the product replenishment image by recognition processing in which all products indicated by the product feature value information are set as a collation target (Yamashita, Fig. 8, apparatus moves on to product allocation inspection, which includes product recognition [0052], which “compares multiple features in images of various products, which are registered in the product database…with multiple features in captured images of products so as to specify a product having the largest number of matched features as a product reflected in an image captured by the image acquisition part,” thereby setting the products indicated by the feature value information as a collation target, [0038]); and register, in the shelf-based display information, a product recognized to be included in the product replenishment image, as a product displayed on the first product shelf (Yamashita, database is updated with the product, its number, and positional information [shelf information], thereby registering the product in the shelf information via the database [0038]), wherein a same camera generates the product pickup image and the product replenishment image (Yamashita, Fig. 8, capture monitoring image S41 [0069-0070]); and classify an image generated by the camera into the product pickup image and the product replenishment image (Yamashita, Fig. 8, the image is classified as either being of the customer interacting with the product/shelf [product pickup image] or of the clerk [product replenishment image], [0069-0070]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura, using the teachings of Yamashita to include Yamashita’s capturing and classification of images to distinguish between clerk/salesperson’s and customer’s actions to Tsuchimochi's, as modified by Nomura, imaging and processing of shelf/product-person interaction. Doing so would improve processing of product interaction imaging by providing a way to distinguish between potential restocking actions, such as those by the clerk/salesperson, and potential purchasing/thieving actions, such as those by a customer, which would be used to more efficiently identify images for different purposes (e.g. a customer’s cart tracking vs. a clerk’s inventory tracking).
However, the combination of Tsuchimochi, Nomura, and Yamashita fails to teach where Fujii teaches wherein the classification is performed based on a mode set at a photographing time of the image (Fujii, classify image based on mode set at a photographing time of the image, [0591-0594]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura and Yamashita, using the teachings of Fujii to include Fujii’s classification of images based on a mode set at the photographing time of the image to Tsuchimochi’s, as modified by Nomura and Yamashita, classification of images into pick-up or restocking images. Doing so would improve classification by providing consideration of the mode at photographing time, which would be used to pick out when/how a product was picked up or restocked.
Regarding claim 5, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1, wherein the at least one processor is further configured to execute the one or more instructions to: acquire a product shelf image including a product displayed on the first product shelf (Tsuchimochi, “flow line image obtained by imaging the front of the product shelf,” including the product, pg. 76/97, para 3), recognize a product included in the product shelf image by recognition processing in which all products indicated by the product feature value information are set as a collation target (Nomura, when “the shelf [which is the identified product group to which the product belongs based on the shelf information] is a collation target…the article condition (number of feature points) displayed on the determined shelf is set,” and the final product is recognized, pg. 134/173, para 2); and register, in the shelf-based display information, a product recognized to be included in the product shelf image, as a product displayed on the first product shelf (Nomura, database stores/registers feature values associated with recognition object [product] and shelf information, pg. 133/173, para 5).
Regarding claim 6, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1, wherein the at least one processor is further configured to execute the one or more instructions to: determine, based on the product pickup image, a stage on which a product picked up from the first product shelf is displayed (Tsuchimochi, “detection unit may detect which position (for example, the upper stage / lower stage) of the product shelf” that the product is displayed on, pg. 87/97, para 2), and determine, based on the shelf-based display information indicating a product displayed on each stage of each of the plurality of product shelves, a product group displayed on a the determined stage of the first product shelf (Tsuchimochi, “the product shelf information may indicate which product is displayed at which position [stage] of the product shelf. By configuring in this way, it is possible to further narrow down the products to be searched,” with the product group to be searched being based on the stage information, pg. 87/97, para 2), and recognize a product included in the product pickup image by recognition processing in which the product group determined to be displayed on the determined stage is set as a collation target among pieces of the product feature value information (Tsuchimochi, the product group is searched using collation processing to recognize the product pg. 69/97, para 3, wherein Nomura discloses utilizing a collation target, pg. 134/173, para 2).
Regarding claims 9, 10, the rationale provided in the rejection of claim 1 is incorporated herein. In addition, the processing apparatus of claim 1 corresponds to the method of claim 9 as well as the non-transitory storage medium (Tsuchimochi, pg. 91/97, para 1) of claim 10, and performs the steps disclosed herein.
Claim(s) 7-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tsuchimochi in view of Nomura, Yamashita, and Fujii as applied to claim 1 above, further in view of Bao (US 20180293635 A1).
Regarding claim 7, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1.
However, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii fails to teach where Bao teaches wherein the processor is further configured to execute the one or more instructions to count, based on a recognition result of the recognizing a product included in the product pickup image, a number of products picked up from the first product shelf for each product (Bao, Fig. 6, products are recognized and quantity of products sold [corresponding to picked up and bought] from shelf is calculated, [0012], [0098-0103]), and delete, from a product displayed on the first product shelf indicated by the shelf-based display information, a product in which the number has reached a reference value (Bao, if a product’s, having been displayed on a shelf, quantity/number reaches zero [reference value], it is deleted from the checking database, [0071-0072]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura, Yamashita, and Fujii, using the teachings of Bao to include Bao’s product quantity and deletion to Tsuchimochi's, as modified by Nomura, Yamashita, and Fujii, product tracking. Doing so would improve inventory management by providing a way to delete a product that is no longer present, which would be used to speed up processing by not considering an absent product for recognition.
Regarding claim 8, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii teaches the processing apparatus according to claim 1.
However, the combination of Tsuchimochi, Nomura, Yamashita, and Fujii fails to teach where Bao teaches wherein the processor is further configured to execute the one or more instructions to delete, from a product displayed on the first product shelf indicated by the shelf-based display information, a product in which a number of sales indicated by sales information being managed by a point of sales (POS) system has reached a reference value (Bao, when a product, having been displayed on a shelf, is sold and the quantity/number reaches 0 [reference value], it is deleted from the checking database, [0072]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Tsuchimochi, as modified by Nomura, Yamashita, and Fujii, using the teachings of Bao to include Bao’s product deletion based on sales information to Tsuchimochi's, as modified by Nomura, Yamashita, and Fujii, product tracking. Doing so would improve inventory management by providing a way to delete a product that is no longer present, which would be used to speed up processing by not considering an absent product for recognition.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vincent Rudolph whose telephone number is (571)272-8243. The examiner can normally be reached M-F 7:30 AM - 3:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VINCENT RUDOLPH/ Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights