DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 3-5, 8 and 10-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1) and further in view of Ebrahimi Afrouzi et al (US 2022/0066456 A1).
Regarding Claim 8, Cuan teaches a computing device comprising:
a processor, i.e., processor (102) of server (101) as illustrated in figure 1 and as mentioned at col. 4, lines 5-37, client computing device (109) as mentioned at col. 4, lines 38-59, transacting device (195) with processor (170) as mentioned at col. 4, line 66-col. 5, line 20, for example; and
non-transitory memory storing instructions, i.e., stored in memory (104, 180) as mentioned at col. 5, lines 21-39, the mention of a memory in mobile device circuitry (162) as mentioned at col. 5, lines 43-67, that, when executed by the processor (102, 109, 195, 170), cause the computing device to:
capture, in real time, lidar information captured by at least one lidar device, i.e, the camera device, as mentioned at col. 7, line 57-col. 8, line 5 and col. 10, lines 29-47, for example;
monitor, in real time based on a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device, i.e., noting the pulling of camera footage in col. 11, line 52-col. 12, line 9;
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach cause/causing the computing device to:
capture, in real time, lidar information captured by at least one lidar device;
monitor, in real time based on a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device;
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach, but Sumpter teaches cause/causing the computing device, i.e., processor (113) of transaction terminal (110) or processor (121) of server (120) or processor (131) of review terminal (130) as illustrated in figure 1, to:
capture, in real time, lidar/camera information, i.e., noting step (220) of the flowchart in figure 2 which states “ATM activates camera and begins recording activity”, captured by at least one lidar/camera device, i.e., camera (111);
monitor, in real time based on a lidar/camera stream received from the at least one lidar/camera device, i.e., camera (111), activities within a field of view of the at least one lidar/camera device, i.e., camera (111);
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar/camera device, i.e., camera (111), an indication of a malicious activity, i.e., noting steps (231, 232, 233, 234, 250, 251, 260 and 262, for example;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar/camera device, i.e., camera (111), noting paragraph 49, which states as follows.
[0049] When fraud is potential, at 233, fraud manager 125 determines from the image/video, at 240, based on comparison of a fraud score calculated and compared to the fraud threshold, to notify fraud review interface 133 on review terminal 130. A fraud analyst operates fraud review interface 133 to inspect elements in the images/video, at 241, to decide as to whether fraud is present or not. Fraud review interface 133 based on options selected by the fraud analyst sends a notification to fraud agent 116. When the notification indicates no fraud, at 260, the ATM 110 completes the transaction on behalf of the user and analysis is complete at 262. When the notification indicates fraud, at 260, fraud agent 116 disables the transaction and/or the ATM 110 and alerts are sent to bank personnel and/or bank personnel-operated devices, at 261.
Emphasis provided.
Regarding Claim 8, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided cause/causing the computing device to:
capture, in real time, lidar information captured by at least one lidar device;
monitor, in real time based on a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device;
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device, as taught by Sumpter, in Cuan’s computing device and system for the purpose of enabling the lidar/camera data to be interpreted and identified probable fraud to be communicated to users.
Regarding Claim 8, Cuan does not expressly teach at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach, but Goetz teaches at least one lidar device, i.e., sensors (220) as mentioned at col. 11, line 14-col. 12, line 9 and in particular, col. 11, lines 46-58, which states as follows.
The camera 224 may include a photographic camera, a video camera, a thermal camera, etc., which may be used to prevent collisions between the unmanned vehicle 104 and objects (e.g., trees, buildings, people, etc.) encountered during the delivery path (e.g., the flight path 105). The camera 224 may also be used to scan or capture images of CACs on the customer user devices 106A-106D and to confirm biometrics of the customers 107A-107D retrieving cash from the cash container device 140. The sensors 220 may include proximity sensors, GPS sensors, a Light Imaging Detection and Ranging (LIDAR) device, force sensors for sensing pressure on the display screen 228, ambient sensors that detect surrounding conditions (e.g., sound sensors, light detectors, the microphone 226, other sound detectors, and so on) and biometric sensors (e.g., fingerprint readers, a heart monitor that detects cardiovascular signals, an iris scanner, a face scanner, and so forth).
Emphasis provided.
Regarding Claim 8, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided at least one lidar device, as taught by Goetz, in Cuan’s computing device and system for the purpose of providing a common proximity type device, as is common in the art.
Regarding Claim 8, Cuan does not expressly teach capture in real time and using at least one lidar feed generated from a laser of at least one lidar device, lidar feed information captured by the at least one lidar device, wherein the lidar feed information provides three dimensional details of objects within a field of view;
monitor, in real time based on features extracted from a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device;
identify, in real time and based on the features extracted from the lidar stream and a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity; and
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device, wherein the identified indication of the malicious activity comprises a real-time indication of malicious activity captured in three dimensions.
Regarding Claim 8, Cuan does not expressly teach, but Darrer teaches capture, i.e., via processor (not shown), as mentioned in paragraph 170, fourth sentence, in real time and using at least one lidar feed generated from a laser, i.e., light source (109a) with laser driver (110), as mentioned at paragraph 72, noting also the mention of five laser sources (109a) as mentioned at paragraph 73 and as illustrated at figure 1, of at least one lidar device (100), as mentioned at paragraphs 32 and 170, for example, lidar feed information captured by the at least one lidar device (100), wherein the lidar feed information provides three dimensional details of objects within a field of view, as mentioned in the last two sentences of paragraph 170, for example;
monitor, in real time based on features extracted from a lidar stream received from the at least one lidar device (100, 500), which includes control equipment (101), for example, as mentioned in paragraph 35, stating “[t]he control equipment 101 may obtain (e.g., sequentially, in real time) feedback motion measurements 152 associated to the mirror 122, to adaptively synchronize the motion mirror control signal 172 with a reference signal (not shown) on the basis of feedback motion measurements 152 associated to the motion of the motion of the mirror 122”, activities within a field of view of the at least one lidar device (500), as mentioned in paragraph 170, second, fifth and sixth sentences, i.e., “LIDAR is a remote sensing technique that uses light in the form of a pulsed laser (e.g., 112, 114, 312, 314) to measure ranges (variable distances) to one or more objects in a field of view (e.g., 522)” and “[a]rrays of photodetectors 515 receive reflections from objects illuminated by the light (114, 314) irradiated by the photodiodes 109a of the illumination unit 109, and the time necessary for the reflections to arrive at various sensors in the photodetector array 515 is determined” and “[t]hus, the time-of-flight computations can create distance and depth maps, which may be used to generate images”;
identify, in real time and based on the features extracted from the lidar stream (100, 500) and a historical record of activities within the field of view of the at least one lidar device (100, 500), an indication of a malicious activity, as taught by Cuan; and
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view, as taught by Cuan, of the at least one lidar device (100, 500), wherein the identified indication of the malicious activity comprises a real-time indication of malicious activity captured in three dimensions, as taught by Cuan as well as at Darrer, paragraph 170, last sentence, i.e., “[b]y emitting successive light pulses in different directions (scanning directions) established by the mirror (122, 322, an area can be scanned, a three-dimensional image can be generated, and objects within the area can be detected” for example. See Darrer’s paragraphs 32-34, 72, 73 and 170, which state as follows.
[0032] The laser scanning control system 100 may be used, for example in a light detection and ranging (LIDAR) system for synchronizing light pulses (e.g., laser pulses) 112 with a moving mirror 122, e.g., so that the generated light pulses 112 impinge the mirror 122 at predetermined positions 123 (e.g., angles θ). The laser scanning control system 100 may include control equipment 101 (e.g., including circuitry). The control equipment 101 may perform a coordinated scanning control to control:
[0033] through a motion mirror control signal 172, a mirror driver 121, which may sequentially drive the mirror 122 along preselected mirror positions 123 (e.g., angles θ.sub.1, θ.sub.2, θ.sub.3 . . . ); and/or
[0034] through a pulse trigger control signal 132, a laser driver 110, which may cause the generation light pulses 112 so that the light pulses 112 are impinged onto the mirror 122 at the preselected mirror positions 123.
[0072] A laser driver 110 may be part of an illumination unit 109 for generating the light pulses 112. The illumination unit 109 (a structural layout of which is schematized in FIG. 5) may include at least one light source 109a (e.g., a plurality of light sources 109b. The at least one light source 109a may include laser diode(s) or light emitting diode(s). A plurality of light sources 109a may be configured as an array (or matrix) of linearly aligned light sources 109a. The light pulses 112 emitted by the light source(s) 109a may be infrared light, although light with other wavelength might also be used in some examples. As can be seen from FIG. 5, the shape of the light emitted by the light sources 109a may be spread in a direction perpendicular to a transmission direction to form a light beam with an oblong shape perpendicular to a transmission. The illumination light transmitted from the light source(s) 109a may be directed towards the mirror 122.
[0073] Between the light source(s) 109a and the mirror 122, a transmitter optics 511 may be interposed. The transmitter optics 511 may be configured to focus each the laser light 112 onto the mirror 122. The transmitter optics 511 may be, for example, a lens or a prism. When reflected by the mirror 12, the rays of the light emitted from the light sources 109a may be aligned vertically to form a one-dimensional vertical scanning line of infrared light or a vertical bar of infrared light. Each light source(s) 109a (e.g., photodiode) of the illumination unit 109 may contribute to a different vertical region of a vertical scanning line 523a. While five laser sources 109a are shown in FIG. 5, it will be appreciated that the number of laser sources 109a are not limited thereto. For example, the vertical scanning line 523a may be generated by a single laser source 109a, two laser sources 109a, and so on. It will also be appreciated that the light sources 109a may be arranged in a matrix formation.
[0170] With reference to FIG. 5, examples above may be used, for example in a light detection and ranging (LIDAR) system 500. LIDAR is a remote sensing technique that uses light in the form of a pulsed laser (e.g., 112, 114, 312, 314) to measure ranges (variable distances) to one or more objects in a field of view (e.g., 522). In particular, the mirror 122 or 322 (e.g., a MEMS mirror) may be used to scan light across the field of view 522. A reception unit 514 may be used to receive a response light beam. A processor (not shown) may be used to perform a ranging on the basis of the response light beam. Arrays of photodetectors 515 receive reflections from objects illuminated by the light (114, 314) irradiated by the photodiodes 109a of the illumination unit 109, and the time necessary for the reflections to arrive at various sensors in the photodetector array 515 is determined. This is also referred to as measuring time-of-flight (TOF). The LIDAR system 500 forms depth measurements and makes distance measurements by mapping the distance to objects based on the time-of-flight computations. Thus, the time-of-flight computations can create distance and depth maps, which may be used to generate images. (The operations discuss above for generating the light pulses 112 and for reflecting them towards scanning areas of the field of view 522 are here not repeated.) The photodetector array 515 can be any of a number of photodetector types; including avalanche photodiodes (APD), photocells, and/or other photodiode devices. Imaging sensors such as charge-coupled devices (CCDs) can be the photodetectors. In the examples provided herein, the photodetector array 515 is a two-dimensional (2D) APD array that comprises an array of APD pixels. In other embodiments, the photodetector array 515 may be a 1D array that includes a single column of photodiodes. The activation of the photodiodes 515 may be synchronized with light pulses emitted by the illumination unit 109. In examples, for each distance sampling, a microcontroller triggers a laser pulse from each of the light sources 109a of the illumination unit 509 and also starts a timer in a Time-to-Digital Converter (TDC) Integrated Circuit (IC). The laser pulse (112, 114, 312, 314) is propagated through the transmission optics, reflected by the target field, and captured by an APD of the APD array 515. The APD array 515 may emit a short electrical pulse which may then be amplified by an electrical signal amplifier. A comparator IC may recognize the pulse and send a digital signal to the TDC to stop the timer. The TDC uses a clock frequency to calibrate each measurement. The TDC may send the serial data of the differential time between the start and stop digital signals to the microcontroller, which filters out any error reads, averages multiple time measurements, and calculates the distance to the target at that particular field position. By emitting successive light pulses in different directions (scanning directions) established by the mirror (122, 322, an area can be scanned, a three-dimensional image can be generated, and objects within the area can be detected.
Emphasis provided.
Regarding Claim 8, Cuan does not expressly teach capture/capturing, by a camera, an image;
generate/generating, based on a combination of the image captured by the camera and the at least one lidar feed, an enhanced lidar stream;
monitor in real time based on the features extracted from the enhanced lidar stream, activities within a field of view of the at last one lidar device;
identify, in real time and based on the features extracted from the enhanced lidar stream and a historical record of activities within the field of view of the at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach, but Pandya teaches capture/capturing, by a camera/vision based system (110), as illustrated in figure 1 and as mentioned at col. 10, line 48-col. 11, line 32, for example, an image;
generate/generating, based on a combination of the image captured by the camera (110) and the at least one lidar feed, i.e,. LIDAR system (130), as illustrated in figure 1 and as mentioned at col. 10, lines 22-47 and col. 11, lines 4-18, for example, an enhanced lidar stream, i.e., via an intelligent fusion framework, as mentioned at col. 1, lines 45-54, col. 2, lines 1-25, col. 3, lines 29-43 and using real-time locating system (RTLS), as mentioned at col. 3, lines 6-17, col. 8, lines 1-13, col. 9, line 25-col. 10, line 21, and noting computer vision system (201), LIDAR system (204) and RTLS system (205), as mentioned at col. 18, lines 12-36, and ;
monitor in real time based on the features extracted from the enhanced lidar stream, activities within a field of view of the at last one lidar device (130, 204);
identify, in real time and based on the features extracted from the enhanced lidar stream and a historical record of activities, i.e., “each individual’s or worksite’s historical data, training datasets, data about a predictive model (e.g., parameters, hyper-parameters, model architecture, performance metrics, threshold, rules, etc.), data generated by a predictive model (e.g., intermediary results, output of a model, latent features, input and output of a component of the model system, etc.), incident report, record and user provided information (e.g., user information such as name, credential, etc.), safety law related data, algorithms, and the like“ as mentioned at col. 23, lines 8-17, within the field of view of the at least one lidar device (130, 204).
Regarding Claim 8, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided capture/capturing, by a camera, an image;
generate/generating, based on a combination of the image captured by the camera and the at least one lidar feed, an enhanced lidar stream;
monitor in real time based on the features extracted from the enhanced lidar stream, activities within a field of view of the at last one lidar device;
identify, in real time and based on the features extracted from the enhanced lidar stream and a historical record of activities within the field of view of the at least one lidar device, as taught by Pandya, in Cuan’s computing device and system for the purpose of providing a more accurate and complete model of the three dimensional area around the ATM, based upon combined sensor data from both computer vision/imaging/camera data along with LIDAR sensor three dimensional data along with RTIS location data.
Regarding Claim 8, Cuan does not expressly teach
wherein identifying the indication of the malicious activity comprises calculating a match score based on a comparison between a three dimensional image of the enhanced lidar stream and a historical image, determining, based on the match score, a probability score for a sequence of lidar images, and determining that the probability score satisfies a threshold.
Regarding Claim 8, Cuan does not expressly teach, but Ebrahimi Afrouzi teaches
wherein identifying the indication of the malicious activity comprises calculating a match score based on a comparison between a three dimensional image of the enhanced lidar stream and a historical image, determining, based on the match score, a probability score for a sequence of lidar images, and determining that the probability score satisfies a threshold, as illustrated in figures 154A, 154B and 155A-155E) and as mentioned at paragraphs 142, 596, 1094, 1125, 1225 and 1242, which state as follows.
[0142] FIGS. 154A, 154B and 155A-155E Illustrate examples of structured light patterns that may be used to infer distance and create three-dimensional images, according to some embodiments.
[0596] In some embodiments, structured light, such as a laser light, may be used to infer the distance to objects within the environment using at least some of the methods described in U.S. Non-Provisional patent application Ser. Nos. 15/243,783, 15/954,335, 17/316,006, 15/954,410, 16/832,221, 15/221,112, 15/674,310, 17/071,424, 15/447,122, 16/393,921, 16/932,495, 17/242,020, 15/683,255, 16/880,644, 15/257,798, 16/525,137 each of which is hereby incorporated by reference. An example of a structured light pattern emitted by laser diode may include three rows of three light points. Different examples of different light patterns including light points and lines may be used. In some embodiments, time division multiplexing may be used for point generation. In some embodiments, a light pattern may be emitted onto objects surfaces within the environment. In some embodiments, an image sensor may capture images of the light pattern projected onto the object surfaces. In some embodiments, the processor of the robot may infer distances to the objects on which the light pattern is projected based on the distortion, sharpness, and size of light points in the light pattern and the distances between the light points in the light pattern in the captured images. In some embodiments, the processor may infer a distance for each pixel in the captured images. In some embodiments, the processor may label and distinguish items in the images (e.g., two dimensional images). In some embodiments, the processor may create a three dimensional image based on the inferred distances to objects in the captured images. In one example, a captured image of the environment may include a light pattern projected onto surfaces of objects within the environment. Some light points in the light pattern may appear larger and less concentrated while other light points may appear smaller and sharper. Based on the size, sharpness, and distortion of the light points and the distances between the light points in the light pattern, the processor of the robot may infer the distance to the surfaces on which the light points are projected. The processor may infer a distance for each pixel within the captured image and create a three dimensional image. In some embodiments, the images captured may be infrared images. Such images may capture live objects, such as humans and animals. In some embodiments, a spectrometer may be used to determine texture and material of objects.
[1094] In some embodiments, structured light, such as a laser light, may be used to infer the distance to objects within the environment. FIG. 154A illustrates an example of a structured light pattern 1500 emitted by laser diode 1501. The light pattern 1500 includes three rows of three light points. FIG. 154B illustrates examples of different light patterns including light points and lines (shown in white). In some embodiments, time division multiplexing may be used for point generation. In some embodiments, a light pattern may be emitted onto objects surfaces within the environment. In some embodiments, an image sensor may capture images of the light pattern projected onto the object surfaces. In some embodiments, the processor of the robot may infer distances to the objects on which the light pattern is projected based on the distortion, sharpness, and size of light points in the light pattern and the distances between the light points in the light pattern in the captured images. In some embodiments, the processor may infer a distance for each pixel in the captured images. In some embodiments, the processor may label and distinguish items in the images (e.g., two dimensional images). In some embodiments, the processor may create a three dimensional image based on the inferred distances to objects in the captured images. FIG. 155A illustrates an environment 1600. FIG. 155B illustrates a robot 1601 with a laser diode emitting a light pattern 1602 onto surfaces of objects within the environment 1600. FIG. 155C illustrates a captured two dimensional image of the environment 1600. FIG. 155D illustrates a captured image of the environment 1600 including the light pattern 1602 projected onto surfaces of objects within the environment 1600. Some light points in the light pattern, such as light point 1603, appear larger and less concentrated, while other light points, such as light points 1604, appear smaller and sharper. Based on the size, sharpness, and distortion of the light points and the distances between the light points in the light pattern 1602, the processor of the robot 1601 may infer the distance to the surfaces on which the light points are projected. The processor may infer a distance for each pixel within the captured image and create a three dimensional image, such as that illustrated in FIG. 155E. In some embodiments, the images captured may be infrared images. Such images may capture live objects, such as humans and animals. In some embodiments, a spectrometer may be used to determine texture and material of objects.
[1125] In some embodiments, the refining sample matcher algorithm may pass the few possible matching locations as a seed set to a full scan matcher algorithm. In some embodiments, the full scan matcher algorithm may choose a first seed as a match if the match score or probability of matching is above a predetermined threshold. In some embodiments, the full scan matcher determines a match between two maps using a gauss-newton method on a point cloud. In an example, the refining scan matcher algorithm may identify a wall in a first map (e.g., a map of a current location of the robot), then may match this wall with every wall in a second map (e.g., a stored global map), and compute a translation/angular offset for each of those matches. The algorithm may collect each of those offsets, called a seed, in a seed set. The algorithm may then iterate and reduce the seed set by identifying better matches and discarding worse matches among those seeds at increasingly higher resolutions. The algorithm may pass the reduced seed set to a full scan matcher algorithm that finds the best match among the seed set using gauss-newton method.
[1225] In some embodiments, the processor of the robot marks inferred environmental characteristics of different locations of the environment within a map of the environment based on observations from all or a portion of current and/or historical sensory data. In some embodiments, the processor modifies the environmental characteristics of different locations within the map of the environment as new sensory data is collected and aggregated with sensory data previously collected or based on actions of the robot (e.g., operation history). For example, in some embodiments, the processor of a street sweeping robot determines the probability of a location having different levels of debris accumulation (e.g., the probability of a particular location having low, medium and high debris accumulation) based on the sensory data. If the location has a high probability of having a high level of debris accumulation and was just cleaned, the processor reduces the probability of the location having a high level of debris accumulation and increases the probability of having a low level of debris accumulation. Based on sensed data, some embodiments may classify or score different areas of a working environment according to various dimensions, e.g., classifying by driving surface type in a hierarchical driving surface type ontology or according to a dirt-accumulation score by debris density or rate of accumulation.
[1242] In some embodiments, the processor may represent and distinguish environmental characteristics using ordinal, cardinal, or nominal values, like numerical scores in various dimensions or descriptive categories that serve as nominal values. For example, the processor may denote different driving surface types, such as carpet, grass, rubber, hardwood, cement, and tile by numerical categories, such as 1, 2, 3, 4, 5 and 6, respectively. In some embodiments, numerical or descriptive categories may be a range of values. For example, the processor may denote different levels of debris accumulation by categorical ranges such as 1-2, 2-3, and 3-4, wherein 1-2 denotes no debris accumulation to a low level of debris accumulation, 2-3 denotes a low to medium level of debris accumulation, and 3-4 denotes a medium to high level of debris accumulation. In some embodiments, the processor may combine the numerical values with a map of the environment forming a multidimensional map describing environmental characteristics of different locations within the environment, e.g., in a multi-channel bitmap. In some embodiments, the processor may update the map with new sensor data collected and/or information inferred from the new sensor data in real-time or after a work session. In some embodiments, the processor may generates an aggregate map of all or a portion of the maps generated during each work session wherein the processor uses the environmental characteristics of the same location predicted in each map to determine probabilities of each environmental characteristic existing at the particular location.
Emphasis provided.
Regarding Claim 8, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein identifying the indication of the malicious activity comprises calculating a match score based on a comparison between a three dimensional image of the enhanced lidar stream and a historical image, determining, based on the match score, a probability score for a sequence of lidar images, and determining that the probability score satisfies a threshold, as taught by Ebrahimi Afrouzi, in Cuan’s computing device and system for the purpose of providing a more accurate and complete model of the three dimensional area around the ATM, based upon combined sensor data from both computer vision/imaging/camera data along with LIDAR sensor three dimensional data along with RTIS location data, as previously mentioned, combined by a matching algorithm to arrive at a particular score calibrated by one of ordinary skill to represent the desired characteristics concerned about, such as the images representing malicious activity.
Regarding Claim 10, Cuan does not expressly teach wherein the at least one lidar feed comprises a signal generated by a lidar device co-located with an object of interest.
Regarding Claim 10, Cuan does not expressly teach, but Goetz teaches wherein the at least one lidar feed comprises a signal generated by a lidar device, as taught by Goetz, i.e., the feed from the sensor (220) in the form of a lidar device, co-located with an object of interest, i.e., the camera device which is aimed at an atm, as mentioned at col. 7, line 57-col. 8, line 5 and col. 10, lines 29-47, for example.
Regarding Claim 11, Cuan does not expressly teach wherein the at least one lidar feed comprises a signal generated by a lidar device, as taught by Goetz, i.e., the feed from the sensor (220) in the form of a lidar device, remote from an object of interest, noting that Goetz’ cameras are located on unmanned vehicle, (104) as illustrated at figure 2, for example, wherein the object of interest lies within a field of view of the lidar device (220) .
Regarding Claim 12, Cuan does not expressly teach wherein the instructions further cause the computing device to:
receive, from a video capture device, a video feed of an object of interest within the field of view of the lidar device; and
generate, by the lidar processing server based on the lidar feed and the video feed, a real-color image of the object of interest.
Regarding Claim 12, Cuan does not expressly teach, but Sumpter teaches wherein the instructions further cause the computing device (100, 200) to:
receive, from a video capture device, i.e., camera (111), a video feed of an object of interest within the field of view of the lidar device, as taught by Goetz; and
generate, by the lidar processing server (120) based on the lidar feed and the video feed, a real-color image of the object of interest, as mentioned at paragraphs 28 and 30, for example.
Regarding Claim 1, see the rejection of Claim 8.
Regarding Claim 3, see the rejection of Claim 10.
Regarding Claim 4, see the rejection of Claim 11.
Regarding Claim 5, see the rejection of Claim 12.
Claim(s) 2 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Ebrahimi Afrouzi et al (US 2022/0066456 A1) and further in view of Krebs et al (US 11,631,068 B1).
Regarding Claims 2 and 9, Cuan teaches the system as described above.
Regarding Claim 9, Cuan teaches, wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), i.e., transacting device/terminal (195, 802a, 802b, 802n), as illustrated in figures 1 and 5.
Regarding Claim 9, Cuan does not expressly teach wherein the identified indication of a malicious activity comprises an installation of an unauthorized card reading device at the ATM.
Regarding Claim 9, Cuan does not expressly teach, but Krebs teaches, wherein the identified indication of a malicious activity comprises an installation of an unauthorized card reading device at the ATM, i.e., kiosk (100), as mentioned at col. 2, line 62-col. 3, line 17, col. 3, line 51-col. 4, line 6, col. 6, lines 31-63 and col. 8, lines 36-51, which states as follows.
(13) The kiosk 100 also includes a camera 102 that is positioned to record a plurality of images of a person located in an area in front of the kiosk 100. As shown in FIG. 1, the camera 102 may be positioned on top of the kiosk and pointing down to record images (e.g., still or moving images) of one or more hands of a person that perform an action in front of the kiosk 100. Some examples of an action performed in front of the kiosk 100 include a person entering a passcode using the keypad 108, inserting an electronic card using the card reader 106, withdrawing cash, depositing a cash or check, or installing an unauthorized device (e.g., shimmer, skimmer, or additional camera(s)) on the kiosk 100. The camera 102 is positioned on the kiosk in a way that allows the camera 102 to record images as a person performs action(s) with respect to the card reader 106, the keypad 108, or the chassis of the kiosk 100. Thus, in some embodiments, the camera 102 is positioned above the card reader 106 or the keypad 108 so that the field of view of the camera 102 captures the images needed for the image processing techniques described in this patent document. In some embodiments, the camera 102 can be placed next to the display 104 and pointing at an angle to record images as a person in front of the kiosk 100.
(16) FIGS. 3A and 3B shows two example scenarios where a person is extending his or her hand(s) towards the kiosk to perform an action. In FIG. 3A, the person is entering his or her passcode using the keypad 308, and in FIG. 3B, the person is trying to install an unauthorized device 314 (e.g., shimmer or skimmer device) on a card reader 318. Using the example techniques described in this patent document, the image processing server 202 can receive an image of the persons in FIG. 3A or 3B from a camera, and the image processing server 202 can determine one or more areas (302, 304 in FIGS. 3A and 310, 312 in FIG. 3B) occupied by a person's hand(s) in the received image. In some embodiments, the image processing server 202 may determine that a received image includes a single area where a person has extended one of his or her hands towards the kiosk. In some embodiments, the image processing server 202 may determine that one or more hands of a person are extended towards the kiosk by determining a shortest distance from the one or more areas occupied by the person's hands and a location on the image associated with the kiosk (e.g., an edge of the front plate of the kiosk, shown as 306 in FIGS. 3A and 316 in FIG. 3B) and by determining that the shortest distance calculated is less than a pre-determined value.
(26) FIG. 3E shows an enlarged image of a keypad area where an example image processing technique is used to determine whether an unauthorized device has been installed on a kiosk. A kiosk camera can obtain an image that includes the image of the keypad area as shown in FIG. 3E. Using the image received from the kiosk camera, an image processing server may determine a location of a card reader and may use an array of points (or pixels) around the card reader area for reference (shown in dashed line in FIG. 3E). In some embodiments, an area associated with the location of a card reader may be previously stored on the image processing server by a person (e.g., a technician or maintenance worker). Using the array or set of points, the image processing server may obtain a shape of a card reader and can thus determine a presence of an unauthorized device on top of or in front of the card reader. For example, on the left-side of FIG. 3E shows an unaltered enlarged image of the keypad area where an image processing server can determine a first array of points associated with an outline of the card reader. The right-side of FIG. 3E shows an enlarged image of a card reader on a kiosk that has been tampered with where an unauthorized device (e.g., skimmer device or shimmer device) has been attached to the card reader. The image processing server can determine that a kiosk has been tampered with by comparing a second array of points associated with an outline of the card reader and the unauthorized device. By determining that the two arrays of points are different in location or are related to or comprise different pixels or describe different areas, the image processing server can determine that an unauthorized device has been installed and can send a message to the corporate server to perform safety measure(s) as described in this patent document.
(33) FIG. 4 shows an example flow diagram to determine whether a person located in front of a kiosk is engaging normal or abnormal behavior. At the receiving operation 402, a first server (e.g., an image processing server) can receive a plurality of images from a camera located on or in an electronic kiosk. The camera is positioned to record images of a person located in an area in front of the electronic kiosk, where the images includes one or more hands of the person that perform an action in front of the electronic kiosk. Some examples of an action performed in front of the electronic kiosk include a passcode being entered using the keypad, an electronic card being inserted using the card reader, cash withdrawal, cash or check deposit, or installation of an unauthorized device on the electronic kiosk. The electronic kiosk can include an automated teller machine (ATM) or a gas station fuel dispenser.
Emphasis provided.
Regarding Claim 9, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein the identified indication of a malicious activity comprises an installation of an unauthorized card reading device at the ATM, as taught by Krebs, in Cuan’s computing device and system for the purpose of increasing security by eliminating unauthorized installation of devices.
Regarding Claim 2, see the rejection of Claim 9.
Claim(s) 6, 7, 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Ebrahimi Afrouzi et al (US 2022/0066456 A1), further in view of Krebs et al (US 11,631,068 B1) and further in view of Lewis et al (US 8,640,947 B1).
Regarding Claims 6, 7, 13 and 14, Cuan teaches the system as described above.
Regarding Claim 13, Cuan does not expressly teach wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM; and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user.
Regarding Claim 13, Cuan does not expressly teach, but Krebs teaches wherein an object of interest within the field of view of the lidar device, i.e., images generated from camera (320), as illustrated in figures 3c and 3d and as mentioned at col. 6, lines 18-30, is an automatic teller machine (ATM), i.e., the front (316) of the atm, and wherein the instructions further cause the computing device (510) to:
determine, based on the lidar feed, an identification of a user of the ATM; and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user.
Regarding Claim 13, Cuan does not expressly teach, but Lewis teaches wherein an object of interest within the field of view of the camera/lidar device, i.e., images generated from camera (34, 131), as illustrated in figures 3 and 10, and as mentioned at col. 7, lines 31-51, is an automatic teller machine (ATM), i.e., the bezel (517) of the atm/automated banking machine (10), as illustrated in figures 1, 26 and 27, and as mentioned in col. 42, lines 25-42, and wherein the instructions further cause the computing device (72, 510), as mentioned at col. 47, lines 33-44, and terminal controller (464) and as illustrated in figures 3 and 4, for example, to:
determine, based on the camera/lidar feed, an identification of a user of the ATM (10, 517); and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user, as mentioned at col. 25, line 57-col. 26, line 8.
Regarding Claim 13, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM; and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user, as taught by Lewis, in Cuan’s computing device and system for the purpose of providing additional security through linking of both the unauthorized device and the user/customer whom installed it.
Regarding Claim 14, Cuan does not expressly teach wherein the secure identifier of the user comprises a biometric identifier of the user.
Regarding Claim 14, Cuan does not expressly teach, but Lewis teaches wherein the secure identifier of the user comprises a biometric identifier of the user, as mentioned at col. 7, lines 31-49, col. 38, lines 54-59, col. 45, line 41-col. 46-line 2 and col. 54, lines 47-67.
Regarding Claim 6, see the rejection of Claim 13.
Regarding Claim 7, see the rejection of Claim 14.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Ebrahimi Afrouzi et al (US 2022/0066456 A1), further in view of Krebs et al (US 11,631,068 B1) and further in view of Weis et al (US 2022/0406092 A1).
Regarding Claim 15, Cuan teaches the system as described above.
Regarding Claim 15, Cuan does not expressly teach wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM;
determine, based on the lidar feed, a distance between the user of the ATM a second individual near the user; and
generate, based on the distance between the user of the ATM and the individual, an alert.
Regarding Claim 15, Cuan does not expressly teach, but Weis teaches wherein an object of interest within the field of view of the camera/lidar device is an automatic teller machine (ATM), as taught by Krebs, and wherein the instructions further cause the computing device, i.e., processor (110) as illustrated in figure 1, to:
determine, based on the camera/lidar feed, an identification of a user of the ATM (100) as illustrated in figure 1;
determine, based on the camera/lidar feed, a distance between the user of the ATM (100) a second individual near the user; and
generate, based on the distance between the user of the ATM and the individual, an alert, as mentioned at paragraphs 16, 33, 64, 72, 75 and 77, for example, noting that Weis Claim 5 mentions “the display device displays information that the first face is not the second digital image” and noting that paragraph 76, last two sentences mention “[t]he display device can be configured to provide the information regarding the security state 510, for example to display it on a screen, for example to output an acoustic indication, such as an acoustic signal, for example” and “[t]o put it another way, the display device can be configured to make a user aware of the security state 510, i.e. for example the open money dispensing tray having the money, by means of displaying or outputting the information on a screen and/or by means of outputting an acoustic indication”. See paragraphs 16, 33, 64, 72, 75 and 77, which state as follows.
[0016] The at least one processor can be configured to determine the first distance of the first person using the at least one digital image. The at least one processor can be configured to determine the first distance using a first interocular distance of the first person. The at least one processor can be configured to determine the first interocular distance using the extracted first biometric data. The at least one processor can be configured to determine the second distance of the second person using the at least one digital image. The at least one processor can be configured to determine the second distance using a second interocular distance of the second person. The at least one processor can be configured to determine the second interocular distance using the extracted second biometric data. The features described in this paragraph in combination with the eleventh example form a twelfth example.
[0033] The first distance of the first person can be determined using the at least one digital image. The first distance can be determined using a first interocular distance of the first person. The first interocular distance can be determined using the extracted first biometric data. The second distance of the second person can be determined using the at least one digital image. The second distance can be determined using a second interocular distance of the second person. The second interocular distance can be determined using the extracted second biometric data. The features described in this paragraph in combination with the twenty-eighth example form a twenty-ninth example.
Description
[0064] FIG. 4 illustrates a processing system 400 in accordance with various embodiments. The processing system 400 can comprise facial recognition 402. The facial recognition 402 can substantially correspond to the processing system for facial recognition 200, wherein the facial recognition 402 can process at least one digital image 404, which can comprise more than one face. The at least one digital image 404 can comprise a first face 406 of a first person and a second face 408 of a second person, and the facial recognition 402 can be configured to determine or to recognize the first face 406 of the first person and the second face 408 of the second person in the at least one digital image 404. The facial recognition 402 can furthermore be configured to determine or to extract the biometric features of the first face 406 and/or the biometric features of the second face 408. The at least one processor 110 can be configured to determine whether a first criterion 410 is satisfied. The first criterion 410 can be assigned to the first face 406 of the first person. The processor 110 can be configured to determine whether a second criterion 412 is satisfied. The second criterion 412 can be assigned to the second face 408 of the second person. The first criterion 410 and/or the second criterion 412 can each be a facial recognition criterion, wherein the facial recognition criterion, as described above, can comprise information of whether a face is hidden, or information of whether a face is not hidden. The processor 110 can be configured to determine a first distance between the first person and the self-service terminal 100 and can furthermore be configured to determine a second distance between the second person and the self-service terminal 100. The processor 110 can be configured to determine the first distance and/or the second distance using the at least one digital image 404. The processor 110 can be configured to determine a first interocular distance using the biometric features of the first face 406 and can furthermore be configured to determine the first distance of the first person using the first interocular distance determined. The processor 110 can be configured to determine a second interocular distance using the biometric features of the second face 408 and can furthermore be configured to determine the second distance of the second person using the second interocular distance determined.
[0072] By way of example, the determined first distance of the first person can be less than a first distance criterion and the determined second distance of the second person can be less than a second distance criterion, wherein the first distance of the first person is less than the second distance of the second person. In this case, the processor 110 can determine, for example, that the first person is a user of the self-service terminal 100 and that the second person is not a user of the self-service terminal 100. The processor 110 can determine, for example, that the second person is not complying with a personal space distance. Furthermore, the first facial recognition criterion and the second facial recognition criterion can be satisfied, that is to say that the processor 110 determines that the first person and the second person are not disguised. The processor 110 can be configured to determine that a security violation might be present. The self-service terminal 100 can be an automated teller machine and the security violation may be, for example, that the second person, who was not determined as a user of the self-service terminal 100, is attempting to covertly observe the PIN (personal identification number) of the first person, who was determined as a user of the self-service terminal 100. The processor 110 can be configured to communicate the indication 414 that a security violation might be present to the display device, and the display device can be configured to provide the first person, i.e. the user of the self-service terminal 100, with information. The display device can be configured to provide the first person with the at least one digital image 404 comprising the face 406 of the first person and the face 408 of the second person, for example to display it on the screen. The processor 110 can furthermore be configured to provide the display device with the first distance of the first person and the second distance of the second person. The display device can be configured to mark the first face 406 of the first person displayed on the screen as a user, for example to represent it with a first colored frame, such as a green frame, for example, and can furthermore be configured to mark the second face 408 of the second person displayed on the screen as a potential threat, for example to represent it with a second colored frame, such as a blue, yellow or red frame, for example, wherein a respective marking, such as, for example, a blue frame or a red frame, can be selected depending on the determined second distance of the second person. The display device can furthermore be configured to provide the first person with additional information, for example in the form of text, as sound, etc. In accordance with various embodiments, the self-service terminal 100 furthermore comprises a user interface, wherein the user interface can comprise the display device and wherein the user interface can furthermore comprise an input device. The input device can be configured to enable the user of the self-service terminal 100 to interact with the self-service terminal 100. The input device can be for example a keypad, for example a keyboard. The display device and the input device can be integrated as a touch-sensitive screen (touchscreen). The information provided by means of the display device can enable the first person to deny the potential threat by means of the input device. That is to say that the first person can indicate by means of the user interface that the second person is not a threat (the second person may be, for example, a friend, a partner or a family member). The processor 110 can be configured to block a use of the self-service terminal 100 until the first person, i.e. the user, has denied the potential threat.
[0075] By way of example, the processor 110 can determine that the first digital image 502 comprises a first face 506 of a first person. The processor 110 can furthermore determine a first distance of the first person. The processor 110 can be configured to determine the first person as a user of the self-service terminal using the determined first face 506 of the first person and optionally using the first distance of the first person. The processor 110 can determine, for example, that the second digital image 504 does not comprise a face of a person and the processor 110 can be configured to control the self-service terminal 100 in such a way that the self-service terminal 100 transitions to a security state 510. The processor 110 can be configured to control the self-service terminal in such a way that the self-service terminal 100 transitions to a security state 510 after a predefined time interval (for example after 5 seconds, for example after 10 seconds, for example after more than 10 seconds) after the detection of the second digital image 504 that does not comprise a face of a person. The security state 510 can also be the standby state if the self-service terminal 100 is in a non-critical state (for example is in a state in which a second person could not cause damage) at the time of the detection of the second digital image 504.
[0077] By way of example, the processor 110 can determine that the first digital image 502 comprises a first face 506 of a first person. The processor 110 can furthermore determine a first distance of the first person. The processor 110 can be configured to determine the first person as a user of the self-service terminal using the determined first face 506 of the first person and optionally using the first distance of the first person. The processor 110 can be configured to determine first biometric features of the first face 506 of the first person or to extract them from the first digital image 502. The processor 110 can determine, for example, that the second digital image 504 comprises a second face 508 of a second person. The processor 110 can be configured to determine second biometric features of the second face 508 of the second person or to extract them from the second digital image 504. The processor 110 can determine, for example, that the second face 508 is different than the first face 506. To put it another way, the first biometric features determined can be different than the second biometric features determined. The processor 110 can be configured to control the self-service terminal 100 in such a way that the self-service terminal 100 transitions to the security state 510 if said processor determines that the first biometric features and the second biometric features are different. That is to say that the second digital image 504 comprises a different person than the first digital image 502. The self-service terminal 100 can be an automated teller machine and the second digital image 504 can be detected at a time at which the money dispensing tray is open and has money; the security state 510 can comprise for example the money dispensing tray being closed. The self-service terminal 100 can be an automated teller machine and the second digital image 504 can be detected at a time at which the PIN of the user has already been entered; the security state 510 can comprise for example the self-service terminal 100, i.e. the automated teller machine, being blocked so that no money can be withdrawn. For example, renewed entry of a PIN can be requested in this case. The self-service terminal 100 can comprise the user interface and the processor 110 can furthermore be configured to communicate information regarding the security state 510 to the display device of the user interface. The display device can be configured to provide the information regarding the security state 510, for example to display it on a screen.
Emphasis provided.
Regarding Claim 15, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM;
determine, based on the lidar feed, a distance between the user of the ATM a second individual near the user; and
generate, based on the distance between the user of the ATM and the individual, an alert, as taught by Weis, in Cuan’s computing device and system for the purpose of providing additional security through linking of both the image of the second user with the first user/customer.
Claim(s) 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Ebrahimi Afrouzi et al (US 2022/0066456 A1), further in view of Krebs et al (US 11,631,068 B1), further in view of Weis et al (US 2022/0406092 A1) and further in view of Wurmfeld (US 2023/0025391 A1).
Regarding Claims 16-18, Cuan teaches the system as described above.
Regarding Claim 16, Cuan does not expressly teach wherein the instructions, further cause the computing device to:
determine, based on the lidar feed, an identification of the second individual; and
initiate, based on a match between the identification of the second individual and a historical record of individuals previously determined to perform malicious activity near the ATM, a security response.
Regarding Claim 16, Cuan does not expressly teach, but Wurmfeld teaches wherein the instructions, further cause the computing device to:
determine, based on the camera/lidar feed, an identification of the second individual, as taught by Weis; and
initiate, based on a match between the identification of the second individual and a historical record of individuals previously determined to perform malicious activity near the ATM, a security response, as mentioned at paragraphs 60 and 61 which state as follows.
[0060] In decision block 610, the system may determine whether current first user activity data matches stored historical activity data. Current first user activity data may be data may be determined in response to receiving first level authentication from a first user as described with respect to step 305 in FIG. 3.
[0061] When the current first user activity data matches historical activity data beyond a predetermined threshold of similarity, the system may confirm that the current location of the first user is within the predetermined proximity of the first computing device (e.g., computer interface 130) in block 615. When the current first user activity data does not match historical activity data beyond the predetermined threshold of similarity, the method may end.
Emphasis provided.
Note also the mention at step (325), i.e., “one or more objects associated with a human”, the mention at step (330), i.e., “trigger a security measure”, step (335) which states “transmit an indication of the triggered security measure to the first computing device” and step (340) which states “transmit instructions to the first user device configured to cause the first user device to provide an audible or vibrational alert to the first user” as illustrated at figure 3.
Regarding Claim 16, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein the instructions, further cause the computing device to:
determine, based on the lidar feed, an identification of the second individual; and
initiate, based on a match between the identification of the second individual and a historical record of individuals previously determined to perform malicious activity near the ATM, a security response, as taught by Wurmfeld, in Cuan’s computing device and system for the purpose of providing additional security through linking of both the image of the second user with the first user/customer.
Regarding Claim 17, Cuan does not expressly teach wherein the security response comprises generation of a message to security personnel, the message comprising a location of the ATM and a lidar image of the second individual captured by the at least one lidar device
Regarding Claim 17, Cuan does not expressly teach, but Wurmfeld teaches, wherein the security response comprises generation of a message to security personnel, the message comprising a location of the ATM and a lidar image of the second individual captured by the at least one lidar device, as mentioned at Wurmfeld, paragraphs 60 and 61 and figures 3-7, for example.
Regarding Claim 18, see the rejection of Claims 16 and 17, above.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Ebrahimi Afrouzi et al (US 2022/0066456 A1) and further in view of Masuda (US 2022/0084342 A1).
Regarding Claim 20, Cuan teaches the system as described above.
Regarding Claim 20, Cuan does not expressly teach wherein an object of interest within the field of view of the lidar device is a security gate, and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of an individual adjacent to the security gate; and
generate, based on the identification of the individual, an alert when the identification of the individual fails to match a secure identifier that the individual uses in an attempt to proceed through the security gate.
Regarding Claim 20, Cuan does not expressly teach, but Masuda teaches wherein an object of interest, i.e., a user or customer, within the field of view of the camera/lidar device, i.e., imaging unit (52), as mentioned in paragraph 23, is a security gate, (10, 20, 30), and wherein the instructions further cause the computing device, i.e., CPU (55), as mentioned at paragraph 37, to:
determine, based on the camera/lidar feed, an identification of an individual adjacent to the security gate (10, 20, 30); and
generate, based on the identification of the individual, i.e., first biometric information as mentioned at paragraph 20 and second biometric information as mentioned at paragraph 20, an alert when the identification of the individual fails to match a secure identifier that the individual uses in an attempt to proceed through the security gate, as mentioned at Masuda Claims 6, 12 and 19, which states as follows.
6. The gate control device according to claim 1, wherein the processor is further configured to transmit a notification via the communication interface to a manager terminal device when the received second biometric information fails to match the stored biometric information.
12. The gate control system according to claim 7, wherein the processor is further configured to transmit a notification via the communication interface to a manager terminal device when the received second biometric information fails to match the stored biometric information.
19. The gate control method according to claim 13, further comprising: sending a notification to a manager terminal when the received second biometric information fails to match the retained first biometric information.
Emphasis provided.
Regarding Claim 20, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein an object of interest within the field of view of the lidar device is a security gate, and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of an individual adjacent to the security gate; and
generate, based on the identification of the individual, an alert when the identification of the individual fails to match a secure identifier that the individual uses in an attempt to proceed through the security gate, as taught by Masuda, in Cuan’s computing device and system for the purpose of providing additional security through contacting/alerting local security and law enforcement officials.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-18 and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant is encouraged to contact the Examiner should there be any questions about this rejection or in an endeavor to explore potential amendments or potential allowable subject matter.
Frossard ‘610 is cited as teaching Lidar systems and and LIDAR point cloud data in paragraphs 20, 23, 24, 29, with detection of three-dimensional images/space by the LIDAR system at paragraphs 52 and 53, LIDAR data being mentioned in paragraphs 70-72, and probability and matching score as mentioned at paragraph 81, for example. Paragraphs 98 and 99 mentions the LIDAR sensors being used in various types of systems such as robotics and user computing devices, noting that an ATM can be construed to be a computing device, for example. Paragraph 68, second sentence, mentions “[t]he object associations component 208 can receive data about the determined objects and analyze prior object instance data to determine a most likely association of each determined object with a prior object instance, or in some cases, determine if the potential object is a new object instance”.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFREY ALAN SHAPIRO whose telephone number is (571)272-6943. The examiner can normally be reached Monday-Friday generally between 8:30AM and 6:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anita Y Coupe can be reached at 571-270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFREY A SHAPIRO/Primary Examiner, Art Unit 3619
September 13, 2026
Prosecution Insights