DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action responds to the amendment and argument filed by applicant on June 19, 2026, in response to the Office Action mailed on December 30, 2025.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1–3, 6-8 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Srivastava et al. (US 2024/0095709 A1, hereinafter “Srivastava”), in view of Grabau et al. (US 2019/0235617 A1, hereinafter “Grabau”).
With respect to claim 1, Srivastava discloses a kiosk system comprising a static base, (Srivastava, FIG. 4; system description discussing housing 120, measurement volume 140, base 160, and sensors 180 namely sampling system 100 having housing 120, base 160, and sensors 180, wherein the base is preferably static relative to the arms and sensors);
a set of supports mounted offset from regions of the base, wherein the supports cooperatively define a measurement volume with the base(Srivastava, FIG. 4; housing/measurement-volume disclosure in that housing 120 includes base 160 and one or more arms and measurement volume 140 is defined between the base and the arms. Srivastava explains that the arms can extend from the base or another arm and that the measurement volume can be defined by the interior surfaces of the arms and base);
that the housing may define item insertion regions along the sides or top of the measurement volume, thereby teaching an accessible/open measurement region rather than requiring a physically closed chamber.
a set of optical sensors mounted to the supports, (Srivastava, FIG. 4 and imaging-system disclosure wherein sensors 180 monitor measurement volume 140 and the optical sensors/cameras are retained in predetermined configurations about the measurement volume. Srivastava expressly teaches multiple cameras, including multiple stereo camera pairs and structured-light cameras, sampling the measurement volume);
Srivastava further discloses fixed/static sensor arrangements. In particular, the arms are preferably static, the base is preferably static relative to the arms and sensors, and the calibration pattern can appear within the fields of view of the optical sensors. Thus, the cameras collectively observe the item-receiving region/base from their predetermined positions.
a processing system configured to automatically identify a set of items based on measurements from the optical sensors ( FIGS. 4 and 10 and associated item-identification processing wherein processing system 200 receives image/depth measurements from sampling system 100, generates geometric representations including point clouds and height maps, segments the representations into item-specific regions, and determines identities of detected items);
Srivastava, however, does not expressly disclose the limitation requiring:
a set of guard sensors, wherein the system is operable between a sleep and awake mode based on measurements sampled by the set of guard sensors, wherein the system transitions from the sleep mode to the awake mode by activating a set of non-guard sensors in response to a trigger event detected by the set of guard sensors.
Grabau teaches this limitation. Grabau discloses an activity sensor configured to monitor a zone, a camera/image sensor, a processor placed in a sleep state and wakeable to an awake state, and a microcontroller receiving an interrupt when the activity sensor detects activity. Responsive to the interruption, the microcontroller wakes the processor, and the processor signals the camera to capture images (Grabau, Abstract, Summary, claims 1 and 11).
Grabau further expressly teaches placing an image sensor in an inactive state, receiving an activity-sensor-initiated interrupt, and responsive thereto operating the image sensor to capture images. Thus, Grabau's activity sensor corresponds to the claimed guard sensor, while its camera/image sensor corresponds to a claimed non-guard sensor.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Srivastava's kiosk by incorporating Grabau's activity-triggered power-management arrangement such that a low-power activity/presence sensor remains active while the processing system and/or principal optical sensors are inactive and, upon detection of an item or user, wakes the processing system and activates the optical cameras.
The motivation would have been to reduce unnecessary electrical consumption and image processing while preserving prompt image acquisition when an item or user enters the monitored region. Grabau expressly recognizes computational, and battery/power costs associated with continuously operating image-capture systems and teaches activity-triggered wake-up as a solution. Such modification would merely apply Grabau's known power-management technique to Srivastava's known camera-based kiosk according to its established function, yielding the predictable result of an energy-efficient kiosk that activates its principal sensing system when activity is detected.
Accordingly, claim 1 would have been obvious over Srivastava in view of Grabau.
With respect to claim 2, Srivastava expressly discloses that the plurality of cameras preferably includes multiple stereo camera pairs and a structured-light camera (Srivastava, imaging-system disclosure).
Accordingly, claim 2 is unpatentable over Srivastava in view of Grabau for the reasons discussed with respect to claim 1.
With respect to claim 3, Srivastava expressly discloses determining a height map from the point cloud. The height map represents height values relative to the base and is subsequently used for item segmentation (Srivastava, FIG. 10 and height-map/segmentation disclosure).
Accordingly, claim 3 would have been obvious over Srivastava in view of Grabau.
With respect to claim 6, Srivastava teaches that housing 120 may include one or more arms extending from base 160 and surrounding measurement volume 140. Srivastava therefore contemplates differing numbers of arms/support structures.
Providing two such supports represents selection of a finite number of support configurations for positioning cameras about the measurement region. Two supports would predictably provide opposing structural mounting locations while retaining open access to the measurement volume.
Accordingly, absent a showing of criticality or unexpected results associated with exactly two supports, it would have been obvious to implement Srivastava's plural-arm embodiment with two supports.
With respect to claim 7, Srivastava teaches arms that can extend from the base and/or from another arm and teaches optical sensors positioned on the housing/arms in predetermined configurations around the measurement volume.
The placement of subsets of cameras at spaced locations along such arms, including locations near the connection between upright and lateral portions and locations toward the arm ends—would have constituted routine positioning of cameras to obtain complementary viewpoints of the measurement volume and calibration surface.
Accordingly, claim 7 would have been obvious over Srivastava in view of Grabau.
With respect to claim 8, Srivastava teaches item insertion regions along one or more sides and/or the top of the measurement volume and teaches a housing formed by base 160 and arms rather than requiring surrounding walls.
It would have been obvious to omit unnecessary side walls from such a camera-support framework to provide unobstructed customer access from the front, rear, left and right while maintaining the arms necessary to support the optical sensors.
Accordingly, claim 8 would have been obvious over Srivastava in view of Grabau.
With respect to claim 10, Srivastava teaches positioning multiple optical sensors around measurement volume 140 such that different cameras monitor the measurement volume from complementary viewpoints. Srivastava further teaches that the calibration pattern, or portions thereof, can appear in each optical sensor's field of view.
The collective overlapping fields of view therefore establish an optically monitored region encompassing the item measurement volume even though that region remains physically accessible.
Accordingly, arranging Srivastava's cameras so that their collective fields encompass the measurement volume constitutes the claimed optically enclosed space and would have been obvious to obtain complete optical coverage and reduce blind spots.
Claims 4-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Srivastava, in view of Grabau and, further in view of Yu et al. (US 2022/0245835 A1, hereinafter “Yu”).
With respect to claim 4, Srivastava discloses automatically identifying items by segmenting measurements into item segments. Srivastava teaches segmenting a geometric representation such as a height map generated from a point cloud and generating a mask for each item. Srivastava further teaches Mask R-CNN segmentation and identifying inter-item boundaries to form item-specific representations (Srivastava, FIG. 10 and segmentation disclosure).
Srivastava therefore teaches: “segmenting the set of measurements into a set of item segments.”
Srivastava also teaches processing the resulting item-specific image/geometric information to determine item identities.
Srivastava does not, however, expressly characterize the intermediate learned representations as “item embeddings.”
Yu teaches generating an embedding for an object from sensor-derived representations and using the embedding to determine the identity/association of the object. Yu specifically teaches a geo-motion embedding neural network and an appearance embedding neural network, generation of respective embeddings, combining the embeddings, and processing the resulting representation to determine whether a detected object corresponds to an identified object (Yu, FIGS. 1, 3–4).
It would have been obvious to use Yu's learned embedding representation in Srivastava's item-classification pipeline because embedding-based representations were a known technique for transforming geometric and appearance features into machine-comparable feature vectors for object identification. Doing so would predictably improve or facilitate comparison/classification of Srivastava's segmented item measurements.
Thus, the combination teaches determining item embeddings based on item segments and determining an item identifier using the embeddings. Claim 4 is therefore unpatentable over Srivastava and Grabau further in view of Yu.
With respect to claim 5, Yu expressly teaches both an appearance embedding and a geometry-related geo-motion embedding.
Yu teaches that the geo-motion embedding encodes information concerning the geometry of the object, while the appearance embedding encodes information concerning object appearance derived from camera images. Yu further teaches combining these embeddings for determining whether measurements correspond to a particular object (Yu, FIGS. 1, 3 and 4).
Accordingly, it would have been obvious to use respective appearance and geometric embeddings for each segmented item in Srivastava's identification system. Claim 5 is therefore unpatentable over Srivastava and Grabau further in view of Yu.
Claim 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Srivastava), in view of Grabau and further in view of US 2017/0249491 A1 (“the object-identification reference”).
With respect to claim 9, Srivastava does not expressly disclose the claimed base comprising a display that dynamically displays an item indicator immediately adjacent to a recognized physical item.
The object-identification reference teaches a checkout system wherein the sensed position of an item and determined information such as price are supplied to a display-panel driver, which presents the information on a display panel adjacent to the corresponding physical item. It further teaches visual indicators responsive to camera-system identification, including indicators that follow an item and change when the item becomes identified (US 2017/0249491 A1, checkout display embodiments, including FIGS. 38A–38B).
It would have been obvious to incorporate this known dynamic checkout display into Srivastava's base because Srivastava already determines both item identity and spatial location. Displaying the corresponding identification/price adjacent to each item would provide immediate visual confirmation to the customer regarding which physical item had been recognized and charged. The combination therefore teaches the limitation of claim 9.
Claims 11–15 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Srivastava, in view of Grabau.
With respect to claim 11, Srivastava discloses:
a base (base 160);
a set of supports mounted to the base (housing 120 having one or more arms extending from base 160);
each support comprising optical sensors (sensors/cameras arranged about measurement volume 140 and supported by the housing/arms);
a measurement volume geometrically defined by the base and supports (measurement volume 140 defined between base 160 and the arms); and
a processing system configured to identify objects based on measurements of the measurement volume sampled by the optical sensors (processing system 200 performing the image/depth-based item identification discussed above FIGS. 4 and 10);
Srivastava additionally teaches insertion/access regions along the sides and top of the measurement volume. Providing the open-arm structure without surrounding walls would predictably provide an unobstructed top, front, back, left and right side while retaining optical coverage.
Srivastava does not expressly teach the claimed guard-sensor sleep/wake arrangement.
As discussed with respect to claim 1, Grabau teaches an activity sensor monitoring while a processor/image system is inactive, detection of activity causing an interruption, waking the processor, and signaling the camera/image sensor to capture images.
It would therefore have been obvious to incorporate Grabau's power-management architecture into Srivastava for the energy-efficiency reasons set forth with respect to claim 1.
Accordingly, claim 11 would have been obvious over Srivastava in view of Grabau.
With respect to claim 12, Srivastava teaches multiple cameras distributed around measurement volume 140 and collectively observing the item-receiving region. The calibration pattern can appear in each sensor's field of view.
It would have been obvious to arrange the overlapping fields of view so that measurement volume 140 is fully encompassed by the collectively monitored optical space, thereby reducing blind spots and improving item identification.
With respect to claim 13, Srivastava expressly teaches that housing 120 can include “one or more arms.”
Thus, Srivastava expressly contemplates a single-arm embodiment. Configuring the set of supports to consist of a single support therefore would have been obvious from Srivastava's disclosed alternatives.
With respect to claim 14, Srivastava teaches plural arms positioned around the base and does not require a particular side arrangement.
Locating two supports on a common side of the base would have been an obvious design choice from the finite available mounting locations where the cameras retain adequate optical coverage, particularly were keeping other sides unobstructed facilitates item insertion/removal.
Accordingly, claim 14 would have been obvious.
With respect to claim 15, Srivastava teaches a plurality of cameras positioned statically about the measurement volume. Srivastava permits differing numbers and arrangements of optical sensors.
Selecting four static cameras represents selection of a finite number of known cameras sufficient to obtain multiple viewpoints of the measurement volume. Nothing in the claim establishes a critical or unexpected result uniquely arising from exactly four cameras.
Accordingly, claim 15 would have been obvious over Srivastava in view of Grabau.
With respect to claim 21, Srivastava teaches supports/arms mounted relative to base 160 and optical sensors positioned about measurement volume 140 to obtain different viewing directions.
Positioning respective supports at opposing corner regions of the base would have been an obvious arrangement of Srivastava's supports because opposite-corner placement provides spatially separated camera viewpoints while leaving substantial portions of the perimeter open for insertion and removal of merchandise.
Such placement amounts to routine optimization of the disclosed camera-support locations to obtain desired angular coverage of the measurement volume, and the claimed placement does not recite any particular unexpected result attributable to the precise corner locations.
Claims 16 is rejected under 35 U.S.C. 103 as being unpatentable over Srivastava, in view of Grabau, further in view of US 2017/0249491 A1 (“the object-identification reference”), and Wilson et al. (US 2009/0002344 A1, hereinafter “Wilson”).
With respect to claim 16 Srivastava expressly teaches a calibration pattern 162 on base 160, including checkerboard, ArUco, ChArUco, grid, shape-array, image or other calibration patterns, and teaches using the pattern to determine camera calibration matrices.
Srivastava does not expressly disclose dynamically generating the calibration pattern on an interactive display.
The object-identification reference teaches a checkout surface comprising a display and dynamically presenting item information/indicators adjacent to physical items based upon camera-determined item position.
Wilson teaches an interactive display system in which images are projected onto/displayed at the interactive display surface and calibration operations are performed using displayed/projected regions and image sensing.
It would have been obvious to replace Srivastava's static printed calibration surface with a known electronically controllable interactive display surface so that the same base could selectively present calibration imagery during calibration and item indicators during checkout. Such combination reduces the need for separate physical calibration indicia and permits the base to serve multiple known display functions.
Accordingly, the combined teachings render claim 16 obvious.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Srivastava, in view of Grabau, further in view of Sheikh (US 2022/0050504 A1), and Nakashima (US 2017/0060516 A1).
With respect to claim 17, Srivastava teaches the underlying system having multiple arms/supports about the measurement volume but does not expressly disclose separate outward-facing user interfaces mounted on different supports.
Sheikh teaches a POS device having first and second touch-screen display panels for different users, with the first display usable by a merchant and the second by a customer. Sheikh expressly teaches arranging the two display panels to face outwardly from the base in the operational configuration (Sheikh, US 2022/0050504 A1, FIG. 1 and claim 6).
Nakashima further teaches a POS terminal wherein a display unit is physically supported at the top end of a support column, and the display can be oriented toward an exterior customer side or store-clerk side (Nakashima, US 2017/0060516 A1, FIGS. 1–3 and 7).
It would have been obvious to mount Sheikh's respective outward-facing POS user interfaces on respective structural supports of Srivastava in the manner taught by Nakashima because Srivastava already provides such supports around the item-measurement area.
A person of ordinary skill would have recognized that mounting transaction interfaces to the existing supports avoids occupying the measurement base with additional hardware, leaves the item-measurement region accessible, and presents each interface toward a user standing outside the measurement volume. This constitutes the predictable use of known POS display mounting structures according to their established functions.
Accordingly, claim 17 would have been obvious over Srivastava and Grabau further in view of Sheikh and Nakashima.
Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Srivastava, in view of Grabau and further in view of Yu.
With respect to claim 18, Srivastava teaches:
segmenting the measurements into item segments, including segmenting a height map/geometric representation into masks corresponding to respective items;
processing item-specific geometric and image measurements; and
identifying each respective item from those measurements.
Yu further teaches extracting geometry-related and appearance embeddings from object sensor measurements and processing the embeddings to identify/associate the object.
For the reasons stated with respect to claim 4, it would have been obvious to implement Srivastava's item-identification representations as learned embeddings because Yu demonstrates that such embeddings provide machine-comparable representations suitable for object identity determination.
Accordingly, claim 18 would have been obvious over Srivastava and Grabau further in view of Yu.
With respect to claim 19, Srivastava teaches segmenting a geometric measurement into geometric item segments using heuristics.
Specifically, Srivastava teaches that the geometric representation may be a height map generated from a point cloud and can be segmented by identifying an inter-item boundary based on the height map, images associated with the height-map region, heuristics, and related criteria. Srivastava further teaches Mask R-CNN segmentation resulting in a mask corresponding to each item (Srivastava, FIG. 10).
Srivastava further teaches generating a surface reconstruction for each segmented item and registering/projecting 3D points in a common coordinate system to corresponding 2D points in original camera coordinate systems, thereby determining image regions corresponding to respective geometric item regions.
Yu teaches processing geometric representations with a geo-motion embedding neural network to generate a geometry-containing embedding and processing perspective camera images using an appearance embedding neural network to generate an appearance embedding.
Yu then combines the embeddings and uses the combined representation for object identity determination (Yu, FIGS. 3–4).
It would have been obvious to use Yu's geometric and appearance embedding representations for Srivastava's corresponding geometric and image item segments because the representations correspond to complementary sensor modalities already available in Srivastava and Yu teaches that combining the two improves/completes object association.
Accordingly, claim 19 would have been obvious.
Response to Arguments
Applicant’s arguments with respect to the pending claim(s) have been considered but are moot in view of new ground of rejection.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROKIB MASUD whose telephone number is (571)270-5390. The examiner can normally be reached Mon-Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fahd Obeid can be reached at 571-270-3324. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROKIB MASUD/Primary Examiner, Art Unit 3627
