DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending for examination.
Applicant claims benefit to PRO 61/891,902 Filing Date 10/17/2013. Examiner could not find sufficient support for the claims to the 61/891,902.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/08/2025; 02/14/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites the limitation "the model" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 10-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Goncalves (US 20100217678 A1) in view of Sugasawa (US 20140064569 A1).
Regarding claim 1, Goncalves teaches a method, comprising:
a) receiving a set of items within an examination space of a kiosk ([0024]-[0027] merchandise received on a self-service checkout and passed through a housing containing visual and depth sensors);
b) capturing a plurality of images, comprising depth information, for the set of items using a set of sensors ([0032] In step 304, the checkout system acquires one or more images of the item being purchased.);
c) extracting a set of features for each of the set of items from the plurality of images ([0006] In the exemplary embodiment, the merchandise checkout system of claim 1 employs geometric point features that are scale-invariant, preferably scale-invariant feature transform (SIFT) features.),
d) identifying each of the set of items based on the respective set of features ([0034] In step 308, the item is recognized based on the one or more acquired images which are compared to the current database of visual models of items using one of various pattern recognition algorithms);
e) repeating a)-d) for subsequent sets of items (Fig. 13In step 1310, the process ends if all items have been transacted, or returns to step 1302 to process the next item in the transaction.); and
f) presenting details for identified items, responsive to receiving an indication to complete a session (Fig. 15: In step 1504, the image that led to the item recognition, as well as the recognized model information (an image of the model, the UPC, and or the item description) is displayed.).
Goncalves does not explicitly teach the following limitations, however, in an analogous art, Sugasawa teaches b) capturing a plurality of images, comprising color information ([0042] The feature amount extraction module 51 extracts the feature amount of appearance (appearance feature amount) such as the shape, surface color, pattern, and concave-convex situation and the like of the commodity included in the image from the image captured by the image capturing section 14.)
c) extracting a set of features, comprising item shape and item appearance, for each of the set of items from the plurality of images ([0042] The feature amount extraction module 51 extracts the feature amount of appearance (appearance feature amount) such as the shape, surface color, pattern, and concave-convex situation and the like of the commodity included in the image from the image captured by the image capturing section 14.).
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Sugasawa and apply them to Goncalves. One would be motivated as such to improve recognition among visually similar items.
Regarding claim 2, Goncalves in view of Sugasawa teaches the method of Claim 1. Sugasawa teaches wherein each of the set of items is identified based on the respective item shape and item appearance ([0042] The feature amount extraction module 51 extracts the feature amount of appearance (appearance feature amount) such as the shape, surface color, pattern, and concave-convex situation and the like of the commodity included in the image from the image captured by the image capturing section 14… The candidate recognition module 52 recognizes the commodity with similarity higher than a reference as candidate of the commodity included in the image.). The same motivation used to combine Goncalves in view of Sugasawa in claim 1 is applicable.
Regarding claim 3, Goncalves in view of Sugasawa teaches the method of Claim 1. Goncalves teaches wherein at least a subset of the plurality of images is captured using a downward-facing depth sensor (Fig. 1: one or more depth sensors 140 [0026] A housing 114 may be placed over the conveyor belt 110 and cover a portion of the conveyor belt 110. As the housing 114 functions as a structure where a motion trigger subsystem, one or more visual sensors or cameras 120, one or more depth sensors 140, and a lighting subsystem 118 are mounted, it may have any dimension or shape that accommodates the flow of items therethrough).
Regarding claim 4, Goncalves in view of Sugasawa teaches the method of Claim 3. Goncalves teaches wherein the kiosk further comprises a set of RGB cameras monitoring the examination space from at least four different perspectives (Fig. 1: one or more visual sensors or cameras 120).
Regarding claim 5, Goncalves in view of Sugasawa teaches the method of Claim 1. Goncalves teaches predicting an item identifier using a classifier, given the set of features (In step 512, a correlation score or confidence score is computed for the top visual models with good geometric alignment. In one embodiment, the score is computed by first transforming the acquired image with the computed geometric alignment, and then computing a normalized sum of squared differences between the pixels of the aligned image and the image corresponding to the top matching model.).
Regarding claim 6, Goncalves in view of Sugasawa teaches the method of Claim 1. Goncalves teaches wherein the model comprises a neural network trained using supervised learning ([0009] FIG. 2 is a functional block diagram of the checkout system with automatic learning;).
Regarding claim 10, Goncalves in view of Sugasawa teaches the method of Claim 1. Goncalves teaches re-introducing an item of the set of items within the predetermined examination space of the kiosk to return the item (Fig. 1: 110 Area).
Regarding claim 11, Goncalves in view of Sugasawa teaches the method of Claim 1. Sugasawa teaches wherein an item of the set of items lacks a printed semantic identifier (object recognition technology is proposed to be applied in a recognition apparatus for recognizing commodity especially the commodity without a barcode such as vegetables, fruits and the like bought by a customer in a checkout system of a retail store.). The same motivation used to combine Goncalves in view of Sugasawa is applicable.
Regarding claim 12, Goncalves teaches a kiosk, comprising:
a set of sensors monitoring a measurement volume, the set of sensors comprising depth sensors ([0024]-[0027] merchandise received on a self-service checkout and passed through a housing containing visual and depth sensors);; and
a processing system, configured to:
a) receive a set of items within an examination space of a kiosk; ([0024]-[0027] merchandise received on a self-service checkout and passed through a housing containing visual and depth sensors);
b) capture a plurality of measurements, comprising depth measurements and appearance measurements, for the set of items using a set of sensors ([0032] In step 304, the checkout system acquires one or more images of the item being purchased.);
c) extracting a set of features from the plurality of measurements for each of the set of items; ([0006] In the exemplary embodiment, the merchandise checkout system of claim 1 employs geometric point features that are scale-invariant, preferably scale-invariant feature transform (SIFT) features.),
d) identifying each of the set of items based on the respective set of features; ([0034] In step 308, the item is recognized based on the one or more acquired images which are compared to the current database of visual models of items using one of various pattern recognition algorithms);
e) repeating a)-d) for subsequent sets of items (Fig. 13In step 1310, the process ends if all items have been transacted, or returns to step 1302 to process the next item in the transaction.); and
f) presenting details for identified items, responsive to receiving an indication to complete a session (Fig. 15: In step 1504, the image that led to the item recognition, as well as the recognized model information (an image of the model, the UPC, and or the item description) is displayed.).
Goncalves does not explicitly teach the following limitations, however, in an analogous art, Sugasawa teaches a set of sensors monitoring a measurement volume, the set of sensors comprising color cameras ([0042] The feature amount extraction module 51 extracts the feature amount of appearance (appearance feature amount) such as the shape, surface color, pattern, and concave-convex situation and the like of the commodity included in the image from the image captured by the image capturing section 14.)
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Sugasawa and apply them to Goncalves. One would be motivated as such to improve recognition among visually similar items.
Regarding claim 13, Goncalves in view of Sugasawa teaches the kiosk of Claim 12. Sugasawa teaches wherein the set of features for each of the set of items is extracted from the appearance measurements for the respective item ([0042] The feature amount extraction module 51 extracts the feature amount of appearance (appearance feature amount) such as the shape, surface color, pattern, and concave-convex situation and the like of the commodity included in the image from the image captured by the image capturing section 14… The candidate recognition module 52 recognizes the commodity with similarity higher than a reference as candidate of the commodity included in the image.). The same motivation used to combine Goncalves in view of Sugasawa in claim 1 is applicable.
Regarding claim 14, Goncalves in view of Sugasawa teaches the kiosk of Claim 13. Sugasawa teaches wherein the set of features comprises an observable attribute of the respective item ([0042] The feature amount extraction module 51 extracts the feature amount of appearance (appearance feature amount) such as the shape, surface color, pattern, and concave-convex situation and the like of the commodity included in the image from the image captured by the image capturing section 14… The candidate recognition module 52 recognizes the commodity with similarity higher than a reference as candidate of the commodity included in the image.). The same motivation used to combine Goncalves in view of Sugasawa in claim 1 is applicable.
Regarding claim 15, Goncalves in view of Sugasawa teaches the kiosk of Claim 13. Goncalves teaches wherein each of the set of items is identified using a classifier, given the set of features extracted from the appearance measurements. (In step 512, a correlation score or confidence score is computed for the top visual models with good geometric alignment. In one embodiment, the score is computed by first transforming the acquired image with the computed geometric alignment, and then computing a normalized sum of squared differences between the pixels of the aligned image and the image corresponding to the top matching model.).
Regarding claim 16, Goncalves in view of Sugasawa teaches the kiosk of Claim 13. Goncalves teaches wherein the set of features is extracted using the depth measurements for the set of items. ([0024]-[0027] merchandise received on a self-service checkout and passed through a housing containing visual and depth sensors).
Regarding claim 20, Goncalves in view of Sugasawa teaches the kiosk of Claim 12. Goncalves teaches wherein the user input comprises an indication to complete a purchase.. ([0061] Fig. 15: In step 1504, the image that led to the item recognition, as well as the recognized model information (an image of the model, the UPC, and or the item description) is displayed.).
Claims 7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Goncalves in view of Sugasawa further in view of Chen (US 20190371134 A1).
Regarding claim 7, Goncalves in view of Sugasawa teaches the method of Claim 1. Goncalves does not explicitly teach the following limitations, however, in an analogous art, Chen teaches tracking movement of a hand within the examination space and delaying processing until the hand is no longer present (If the self-checkout system 100 detects that there are still products in hands of the customer without being placed on the platform, the self-checkout system 100 will remind the customer to place the products. [0038] Examiner note: the reminder effectively delays processing).
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Chen and apply them to Goncalves in view of Sugasawa. One would be motivated to reduce checkout errors.
Regarding claim 19, Goncalves in view of Sugasawa teaches the kiosk of claim 12. Goncalves does not explicitly teach the following limitations, however, in an analogous art, Chen teaches wherein the kiosk comprises a single vertical support, wherein the set of sensors is mounted to the single vertical support (Fig. 2).
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Chen and apply them to Goncalves in view of Sugasawa. One would be motivated to provide a more compact machine.
Claims 8-9 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Goncalves in view of Sugasawa further in view of Gao (US 20140034731 A1).
Regarding claim 8, Goncalves in view of Sugasawa teaches the method of Claim 1. Goncalves does not explicitly teach the following limitations, however, in an analogous art, Gao teaches wherein the examination space comprises a base comprising a calibration pattern ([0051] In one embodiment, extrinsic parameters are calibrated using a planar calibration target coupled to the system 12, i.e., in a fixed position directly on a surface of the system 12. For example, FIG. 3 shows two examples of such planar calibration targets: first, patterns 202 and 204 of optical codes printed on respective read-zone confronting surfaces 206 and 208 of the arches 60 and 62; and second, a freestanding calibration template 210 placed onto the conveyor surface 156.).
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Gao and apply them to Goncalves in view of Sugasawa. One would be motivated to increase detection accuracy.
Regarding claim 9, Goncalves in view of Sugasawa and Gao teaches the method of Claim 8. Gao teaches periodically recalibrating the set of sensors based on an appearance of the calibration pattern in the plurality of images captured by the set of sensors (These targets are also suitable both for self-check and for calibration purposes, and may be used to concurrently calibrate multiple imaging systems oriented in several different planes around the read zone 26.).
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Gao and apply them to Goncalves in view of Sugasawa. One would be motivated to increase detection accuracy.
Regarding claim 17, Goncalves in view of Sugasawa teaches the kiosk of claim 12. Goncalves does not explicitly teach the following limitations, however, in an analogous art, Gao teaches wherein the measurement volume comprises a base comprising a calibration pattern. ([0051] In one embodiment, extrinsic parameters are calibrated using a planar calibration target coupled to the system 12, i.e., in a fixed position directly on a surface of the system 12. For example, FIG. 3 shows two examples of such planar calibration targets: first, patterns 202 and 204 of optical codes printed on respective read-zone confronting surfaces 206 and 208 of the arches 60 and 62; and second, a freestanding calibration template 210 placed onto the conveyor surface 156.).
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Gao and apply them to Goncalves in view of Sugasawa. One would be motivated to increase detection accuracy.
Regarding claim 18, Goncalves in view of Sugasawa and Gao teaches the kiosk of claim 17. Gao teaches wherein the processing system is further configured for online calibration using the calibration pattern ([0051] In one embodiment, extrinsic parameters are calibrated using a planar calibration target coupled to the system 12, i.e., in a fixed position directly on a surface of the system 12. For example, FIG. 3 shows two examples of such planar calibration targets: first, patterns 202 and 204 of optical codes printed on respective read-zone confronting surfaces 206 and 208 of the arches 60 and 62; and second, a freestanding calibration template 210 placed onto the conveyor surface 156. [0087] For example, distribution of software may be via CD-ROM or via Internet download.).
It would have been obvious for a person of ordinary skill in the art, before the effective filling date of the claimed invention, to take the teachings of Gao and apply them to Goncalves in view of Sugasawa. One would be motivated to increase detection accuracy.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HESHAM K ABOUZAHRA whose telephone number is (571)270-0425. The examiner can normally be reached M-F 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jamie Atala can be reached at 57127227384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HESHAM K ABOUZAHRA/Primary Examiner, Art Unit 2486
Prosecution Insights