DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/1/26 has been entered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 3-5, 8 and 10-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1) and further in view of Pandya et al (US 11,803,955 B1).
Regarding Claim 8, Cuan teaches a computing device comprising:
a processor, i.e., processor (102) of server (101) as illustrated in figure 1 and as mentioned at col. 4, lines 5-37, client computing device (109) as mentioned at col. 4, lines 38-59, transacting device (195) with processor (170) as mentioned at col. 4, line 66-col. 5, line 20, for example; and
non-transitory memory storing instructions, i.e., stored in memory (104, 180) as mentioned at col. 5, lines 21-39, the mention of a memory in mobile device circuitry (162) as mentioned at col. 5, lines 43-67, that, when executed by the processor (102, 109, 195, 170), cause the computing device to:
capture, in real time, lidar information captured by at least one lidar device, i.e, the camera device, as mentioned at col. 7, line 57-col. 8, line 5 and col. 10, lines 29-47, for example;
monitor, in real time based on a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device, i.e., noting the pulling of camera footage in col. 11, line 52-col. 12, line 9;
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach cause/causing the computing device to:
capture, in real time, lidar information captured by at least one lidar device;
monitor, in real time based on a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device;
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach, but Sumpter teaches cause/causing the computing device, i.e., processor (113) of transaction terminal (110) or processor (121) of server (120) or processor (131) of review terminal (130) as illustrated in figure 1, to:
capture, in real time, lidar/camera information, i.e., noting step (220) of the flowchart in figure 2 which states “ATM activates camera and begins recording activity”, captured by at least one lidar/camera device, i.e., camera (111);
monitor, in real time based on a lidar/camera stream received from the at least one lidar/camera device, i.e., camera (111), activities within a field of view of the at least one lidar/camera device, i.e., camera (111);
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar/camera device, i.e., camera (111), an indication of a malicious activity, i.e., noting steps (231, 232, 233, 234, 250, 251, 260 and 262, for example;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar/camera device, i.e., camera (111), noting paragraph 49, which states as follows.
[0049] When fraud is potential, at 233, fraud manager 125 determines from the image/video, at 240, based on comparison of a fraud score calculated and compared to the fraud threshold, to notify fraud review interface 133 on review terminal 130. A fraud analyst operates fraud review interface 133 to inspect elements in the images/video, at 241, to decide as to whether fraud is present or not. Fraud review interface 133 based on options selected by the fraud analyst sends a notification to fraud agent 116. When the notification indicates no fraud, at 260, the ATM 110 completes the transaction on behalf of the user and analysis is complete at 262. When the notification indicates fraud, at 260, fraud agent 116 disables the transaction and/or the ATM 110 and alerts are sent to bank personnel and/or bank personnel-operated devices, at 261.
Emphasis provided.
Regarding Claim 8, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided cause/causing the computing device to:
capture, in real time, lidar information captured by at least one lidar device;
monitor, in real time based on a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device;
identify, in real time and based on a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity;
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device, as taught by Sumpter, in Cuan’s computing device and system for the purpose of enabling the lidar/camera data to be interpreted and identified probable fraud to be communicated to users.
Regarding Claim 8, Cuan does not expressly teach at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach, but Goetz teaches at least one lidar device, i.e., sensors (220) as mentioned at col. 11, line 14-col. 12, line 9 and in particular, col. 11, lines 46-58, which states as follows.
The camera 224 may include a photographic camera, a video camera, a thermal camera, etc., which may be used to prevent collisions between the unmanned vehicle 104 and objects (e.g., trees, buildings, people, etc.) encountered during the delivery path (e.g., the flight path 105). The camera 224 may also be used to scan or capture images of CACs on the customer user devices 106A-106D and to confirm biometrics of the customers 107A-107D retrieving cash from the cash container device 140. The sensors 220 may include proximity sensors, GPS sensors, a Light Imaging Detection and Ranging (LIDAR) device, force sensors for sensing pressure on the display screen 228, ambient sensors that detect surrounding conditions (e.g., sound sensors, light detectors, the microphone 226, other sound detectors, and so on) and biometric sensors (e.g., fingerprint readers, a heart monitor that detects cardiovascular signals, an iris scanner, a face scanner, and so forth).
Emphasis provided.
Regarding Claim 8, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided at least one lidar device, as taught by Goetz, in Cuan’s computing device and system for the purpose of providing a common proximity type device, as is common in the art.
Regarding Claim 8, Cuan does not expressly teach capture in real time and using at least one lidar feed generated from a laser of at least one lidar device, lidar feed information captured by the at least one lidar device, wherein the lidar feed information provides three dimensional details of objects within a field of view;
monitor, in real time based on features extracted from a lidar stream received from the at least one lidar device, activities within a field of view of the at least one lidar device;
identify, in real time and based on the features extracted from the lidar stream and a historical record of activities within the field of view of the at least one lidar device, an indication of a malicious activity; and
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view of the at least one lidar device, wherein the identified indication of the malicious activity comprises a real-time indication of malicious activity captured in three dimensions.
Regarding Claim 8, Cuan does not expressly teach, but Darrer teaches capture, i.e., via processor (not shown), as mentioned in paragraph 170, fourth sentence, in real time and using at least one lidar feed generated from a laser, i.e., light source (109a) with laser driver (110), as mentioned at paragraph 72, noting also the mention of five laser sources (109a) as mentioned at paragraph 73 and as illustrated at figure 1, of at least one lidar device (100), as mentioned at paragraphs 32 and 170, for example, lidar feed information captured by the at least one lidar device (100), wherein the lidar feed information provides three dimensional details of objects within a field of view, as mentioned in the last two sentences of paragraph 170, for example;
monitor, in real time based on features extracted from a lidar stream received from the at least one lidar device (100, 500), which includes control equipment (101), for example, as mentioned in paragraph 35, stating “[t]he control equipment 101 may obtain (e.g., sequentially, in real time) feedback motion measurements 152 associated to the mirror 122, to adaptively synchronize the motion mirror control signal 172 with a reference signal (not shown) on the basis of feedback motion measurements 152 associated to the motion of the motion of the mirror 122”, activities within a field of view of the at least one lidar device (500), as mentioned in paragraph 170, second, fifth and sixth sentences, i.e., “LIDAR is a remote sensing technique that uses light in the form of a pulsed laser (e.g., 112, 114, 312, 314) to measure ranges (variable distances) to one or more objects in a field of view (e.g., 522)” and “[a]rrays of photodetectors 515 receive reflections from objects illuminated by the light (114, 314) irradiated by the photodiodes 109a of the illumination unit 109, and the time necessary for the reflections to arrive at various sensors in the photodetector array 515 is determined” and “[t]hus, the time-of-flight computations can create distance and depth maps, which may be used to generate images”;
identify, in real time and based on the features extracted from the lidar stream (100, 500) and a historical record of activities within the field of view of the at least one lidar device (100, 500), an indication of a malicious activity, as taught by Cuan; and
communicate, in real time, an alert corresponding to an identified indication of a malicious activity within the field of view, as taught by Cuan, of the at least one lidar device (100, 500), wherein the identified indication of the malicious activity comprises a real-time indication of malicious activity captured in three dimensions, as taught by Cuan as well as at Darrer, paragraph 170, last sentence, i.e., “[b]y emitting successive light pulses in different directions (scanning directions) established by the mirror (122, 322, an area can be scanned, a three-dimensional image can be generated, and objects within the area can be detected” for example. See Darrer’s paragraphs 32-34, 72, 73 and 170, which state as follows.
[0032] The laser scanning control system 100 may be used, for example in a light detection and ranging (LIDAR) system for synchronizing light pulses (e.g., laser pulses) 112 with a moving mirror 122, e.g., so that the generated light pulses 112 impinge the mirror 122 at predetermined positions 123 (e.g., angles θ). The laser scanning control system 100 may include control equipment 101 (e.g., including circuitry). The control equipment 101 may perform a coordinated scanning control to control:
[0033] through a motion mirror control signal 172, a mirror driver 121, which may sequentially drive the mirror 122 along preselected mirror positions 123 (e.g., angles θ.sub.1, θ.sub.2, θ.sub.3 . . . ); and/or
[0034] through a pulse trigger control signal 132, a laser driver 110, which may cause the generation light pulses 112 so that the light pulses 112 are impinged onto the mirror 122 at the preselected mirror positions 123.
[0072] A laser driver 110 may be part of an illumination unit 109 for generating the light pulses 112. The illumination unit 109 (a structural layout of which is schematized in FIG. 5) may include at least one light source 109a (e.g., a plurality of light sources 109b. The at least one light source 109a may include laser diode(s) or light emitting diode(s). A plurality of light sources 109a may be configured as an array (or matrix) of linearly aligned light sources 109a. The light pulses 112 emitted by the light source(s) 109a may be infrared light, although light with other wavelength might also be used in some examples. As can be seen from FIG. 5, the shape of the light emitted by the light sources 109a may be spread in a direction perpendicular to a transmission direction to form a light beam with an oblong shape perpendicular to a transmission. The illumination light transmitted from the light source(s) 109a may be directed towards the mirror 122.
[0073] Between the light source(s) 109a and the mirror 122, a transmitter optics 511 may be interposed. The transmitter optics 511 may be configured to focus each the laser light 112 onto the mirror 122. The transmitter optics 511 may be, for example, a lens or a prism. When reflected by the mirror 12, the rays of the light emitted from the light sources 109a may be aligned vertically to form a one-dimensional vertical scanning line of infrared light or a vertical bar of infrared light. Each light source(s) 109a (e.g., photodiode) of the illumination unit 109 may contribute to a different vertical region of a vertical scanning line 523a. While five laser sources 109a are shown in FIG. 5, it will be appreciated that the number of laser sources 109a are not limited thereto. For example, the vertical scanning line 523a may be generated by a single laser source 109a, two laser sources 109a, and so on. It will also be appreciated that the light sources 109a may be arranged in a matrix formation.
[0170] With reference to FIG. 5, examples above may be used, for example in a light detection and ranging (LIDAR) system 500. LIDAR is a remote sensing technique that uses light in the form of a pulsed laser (e.g., 112, 114, 312, 314) to measure ranges (variable distances) to one or more objects in a field of view (e.g., 522). In particular, the mirror 122 or 322 (e.g., a MEMS mirror) may be used to scan light across the field of view 522. A reception unit 514 may be used to receive a response light beam. A processor (not shown) may be used to perform a ranging on the basis of the response light beam. Arrays of photodetectors 515 receive reflections from objects illuminated by the light (114, 314) irradiated by the photodiodes 109a of the illumination unit 109, and the time necessary for the reflections to arrive at various sensors in the photodetector array 515 is determined. This is also referred to as measuring time-of-flight (TOF). The LIDAR system 500 forms depth measurements and makes distance measurements by mapping the distance to objects based on the time-of-flight computations. Thus, the time-of-flight computations can create distance and depth maps, which may be used to generate images. (The operations discuss above for generating the light pulses 112 and for reflecting them towards scanning areas of the field of view 522 are here not repeated.) The photodetector array 515 can be any of a number of photodetector types; including avalanche photodiodes (APD), photocells, and/or other photodiode devices. Imaging sensors such as charge-coupled devices (CCDs) can be the photodetectors. In the examples provided herein, the photodetector array 515 is a two-dimensional (2D) APD array that comprises an array of APD pixels. In other embodiments, the photodetector array 515 may be a 1D array that includes a single column of photodiodes. The activation of the photodiodes 515 may be synchronized with light pulses emitted by the illumination unit 109. In examples, for each distance sampling, a microcontroller triggers a laser pulse from each of the light sources 109a of the illumination unit 509 and also starts a timer in a Time-to-Digital Converter (TDC) Integrated Circuit (IC). The laser pulse (112, 114, 312, 314) is propagated through the transmission optics, reflected by the target field, and captured by an APD of the APD array 515. The APD array 515 may emit a short electrical pulse which may then be amplified by an electrical signal amplifier. A comparator IC may recognize the pulse and send a digital signal to the TDC to stop the timer. The TDC uses a clock frequency to calibrate each measurement. The TDC may send the serial data of the differential time between the start and stop digital signals to the microcontroller, which filters out any error reads, averages multiple time measurements, and calculates the distance to the target at that particular field position. By emitting successive light pulses in different directions (scanning directions) established by the mirror (122, 322, an area can be scanned, a three-dimensional image can be generated, and objects within the area can be detected.
Emphasis provided.
Regarding Claim 8, Cuan does not expressly teach capture/capturing, by a camera, an image;
generate/generating, based on a combination of the image captured by the camera and the at least one lidar feed, an enhanced lidar stream;
monitor in real time based on the features extracted from the enhanced lidar stream, activities within a field of view of the at last one lidar device;
identify, in real time and based on the features extracted from the enhanced lidar stream and a historical record of activities within the field of view of the at least one lidar device.
Regarding Claim 8, Cuan does not expressly teach, but Pandya teaches capture/capturing, by a camera/vision based system (110), as illustrated in figure 1 and as mentioned at col. 10, line 48-col. 11, line 32, for example, an image;
generate/generating, based on a combination of the image captured by the camera (110) and the at least one lidar feed, i.e,. LIDAR system (130), as illustrated in figure 1 and as mentioned at col. 10, lines 22-47 and col. 11, lines 4-18, for example, an enhanced lidar stream, i.e., via an intelligent fusion framework, as mentioned at col. 1, lines 45-54, col. 2, lines 1-25, col. 3, lines 29-43 and using real-time locating system (RTLS), as mentioned at col. 3, lines 6-17, col. 8, lines 1-13, col. 9, line 25-col. 10, line 21, and noting computer vision system (201), LIDAR system (204) and RTLS system (205), as mentioned at col. 18, lines 12-36, and ;
monitor in real time based on the features extracted from the enhanced lidar stream, activities within a field of view of the at last one lidar device (130, 204);
identify, in real time and based on the features extracted from the enhanced lidar stream and a historical record of activities, i.e., “each individual’s or worksite’s historical data, training datasets, data about a predictive model (e.g., parameters, hyper-parameters, model architecture, performance metrics, threshold, rules, etc.), data generated by a predictive model (e.g., intermediary results, output of a model, latent features, input and output of a component of the model system, etc.), incident report, record and user provided information (e.g., user information such as name, credential, etc.), safety law related data, algorithms, and the like“ as mentioned at col. 23, lines 8-17, within the field of view of the at least one lidar device (130, 204).
Regarding Claim 8, Regarding Claim 8, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided capture/capturing, by a camera, an image;
generate/generating, based on a combination of the image captured by the camera and the at least one lidar feed, an enhanced lidar stream;
monitor in real time based on the features extracted from the enhanced lidar stream, activities within a field of view of the at last one lidar device;
identify, in real time and based on the features extracted from the enhanced lidar stream and a historical record of activities within the field of view of the at least one lidar device, as taught by Pandya, in Cuan’s computing device and system for the purpose of providing a more accurate and complete model of the three dimensional area around the ATM, based upon combined sensor data from both computer vision/imaging/camera data along with LIDAR sensor three dimensional data along with RTIS location data.
Regarding Claim 10, Cuan does not expressly teach wherein the at least one lidar feed comprises a signal generated by a lidar device co-located with an object of interest.
Regarding Claim 10, Cuan does not expressly teach, but Goetz teaches wherein the at least one lidar feed comprises a signal generated by a lidar device, as taught by Goetz, i.e., the feed from the sensor (220) in the form of a lidar device, co-located with an object of interest, i.e., the camera device which is aimed at an atm, as mentioned at col. 7, line 57-col. 8, line 5 and col. 10, lines 29-47, for example.
Regarding Claim 11, Cuan does not expressly teach wherein the at least one lidar feed comprises a signal generated by a lidar device, as taught by Goetz, i.e., the feed from the sensor (220) in the form of a lidar device, remote from an object of interest, noting that Goetz’ cameras are located on unmanned vehicle, (104) as illustrated at figure 2, for example, wherein the object of interest lies within a field of view of the lidar device (220) .
Regarding Claim 12, Cuan does not expressly teach wherein the instructions further cause the computing device to:
receive, from a video capture device, a video feed of an object of interest within the field of view of the lidar device; and
generate, by the lidar processing server based on the lidar feed and the video feed, a real-color image of the object of interest.
Regarding Claim 12, Cuan does not expressly teach, but Sumpter teaches wherein the instructions further cause the computing device (100, 200) to:
receive, from a video capture device, i.e., camera (111), a video feed of an object of interest within the field of view of the lidar device, as taught by Goetz; and
generate, by the lidar processing server (120) based on the lidar feed and the video feed, a real-color image of the object of interest, as mentioned at paragraphs 28 and 30, for example.
Regarding Claim 1, see the rejection of Claim 8.
Regarding Claim 3, see the rejection of Claim 10.
Regarding Claim 4, see the rejection of Claim 11.
Regarding Claim 5, see the rejection of Claim 12.
Claim(s) 2 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1) and further in view of Krebs et al (US 11,631,068 B1).
Regarding Claims 2 and 9, Cuan teaches the system as described above.
Regarding Claim 9, Cuan teaches, wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), i.e., transacting device/terminal (195, 802a, 802b, 802n), as illustrated in figures 1 and 5.
Regarding Claim 9, Cuan does not expressly teach wherein the identified indication of a malicious activity comprises an installation of an unauthorized card reading device at the ATM.
Regarding Claim 9, Cuan does not expressly teach, but Krebs teaches, wherein the identified indication of a malicious activity comprises an installation of an unauthorized card reading device at the ATM, i.e., kiosk (100), as mentioned at col. 2, line 62-col. 3, line 17, col. 3, line 51-col. 4, line 6, col. 6, lines 31-63 and col. 8, lines 36-51, which states as follows.
(13) The kiosk 100 also includes a camera 102 that is positioned to record a plurality of images of a person located in an area in front of the kiosk 100. As shown in FIG. 1, the camera 102 may be positioned on top of the kiosk and pointing down to record images (e.g., still or moving images) of one or more hands of a person that perform an action in front of the kiosk 100. Some examples of an action performed in front of the kiosk 100 include a person entering a passcode using the keypad 108, inserting an electronic card using the card reader 106, withdrawing cash, depositing a cash or check, or installing an unauthorized device (e.g., shimmer, skimmer, or additional camera(s)) on the kiosk 100. The camera 102 is positioned on the kiosk in a way that allows the camera 102 to record images as a person performs action(s) with respect to the card reader 106, the keypad 108, or the chassis of the kiosk 100. Thus, in some embodiments, the camera 102 is positioned above the card reader 106 or the keypad 108 so that the field of view of the camera 102 captures the images needed for the image processing techniques described in this patent document. In some embodiments, the camera 102 can be placed next to the display 104 and pointing at an angle to record images as a person in front of the kiosk 100.
(16) FIGS. 3A and 3B shows two example scenarios where a person is extending his or her hand(s) towards the kiosk to perform an action. In FIG. 3A, the person is entering his or her passcode using the keypad 308, and in FIG. 3B, the person is trying to install an unauthorized device 314 (e.g., shimmer or skimmer device) on a card reader 318. Using the example techniques described in this patent document, the image processing server 202 can receive an image of the persons in FIG. 3A or 3B from a camera, and the image processing server 202 can determine one or more areas (302, 304 in FIGS. 3A and 310, 312 in FIG. 3B) occupied by a person's hand(s) in the received image. In some embodiments, the image processing server 202 may determine that a received image includes a single area where a person has extended one of his or her hands towards the kiosk. In some embodiments, the image processing server 202 may determine that one or more hands of a person are extended towards the kiosk by determining a shortest distance from the one or more areas occupied by the person's hands and a location on the image associated with the kiosk (e.g., an edge of the front plate of the kiosk, shown as 306 in FIGS. 3A and 316 in FIG. 3B) and by determining that the shortest distance calculated is less than a pre-determined value.
(26) FIG. 3E shows an enlarged image of a keypad area where an example image processing technique is used to determine whether an unauthorized device has been installed on a kiosk. A kiosk camera can obtain an image that includes the image of the keypad area as shown in FIG. 3E. Using the image received from the kiosk camera, an image processing server may determine a location of a card reader and may use an array of points (or pixels) around the card reader area for reference (shown in dashed line in FIG. 3E). In some embodiments, an area associated with the location of a card reader may be previously stored on the image processing server by a person (e.g., a technician or maintenance worker). Using the array or set of points, the image processing server may obtain a shape of a card reader and can thus determine a presence of an unauthorized device on top of or in front of the card reader. For example, on the left-side of FIG. 3E shows an unaltered enlarged image of the keypad area where an image processing server can determine a first array of points associated with an outline of the card reader. The right-side of FIG. 3E shows an enlarged image of a card reader on a kiosk that has been tampered with where an unauthorized device (e.g., skimmer device or shimmer device) has been attached to the card reader. The image processing server can determine that a kiosk has been tampered with by comparing a second array of points associated with an outline of the card reader and the unauthorized device. By determining that the two arrays of points are different in location or are related to or comprise different pixels or describe different areas, the image processing server can determine that an unauthorized device has been installed and can send a message to the corporate server to perform safety measure(s) as described in this patent document.
(33) FIG. 4 shows an example flow diagram to determine whether a person located in front of a kiosk is engaging normal or abnormal behavior. At the receiving operation 402, a first server (e.g., an image processing server) can receive a plurality of images from a camera located on or in an electronic kiosk. The camera is positioned to record images of a person located in an area in front of the electronic kiosk, where the images includes one or more hands of the person that perform an action in front of the electronic kiosk. Some examples of an action performed in front of the electronic kiosk include a passcode being entered using the keypad, an electronic card being inserted using the card reader, cash withdrawal, cash or check deposit, or installation of an unauthorized device on the electronic kiosk. The electronic kiosk can include an automated teller machine (ATM) or a gas station fuel dispenser.
Emphasis provided.
Regarding Claim 9, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein the identified indication of a malicious activity comprises an installation of an unauthorized card reading device at the ATM, as taught by Krebs, in Cuan’s computing device and system for the purpose of increasing security by eliminating unauthorized installation of devices.
Regarding Claim 2, see the rejection of Claim 9.
Claim(s) 6, 7, 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Krebs et al (US 11,631,068 B1) and further in view of Lewis et al (US 8,640,947 B1).
Regarding Claims 6, 7, 13 and 14, Cuan teaches the system as described above.
Regarding Claim 13, Cuan does not expressly teach wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM; and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user.
Regarding Claim 13, Cuan does not expressly teach, but Krebs teaches wherein an object of interest within the field of view of the lidar device, i.e., images generated from camera (320), as illustrated in figures 3c and 3d and as mentioned at col. 6, lines 18-30, is an automatic teller machine (ATM), i.e., the front (316) of the atm, and wherein the instructions further cause the computing device (510) to:
determine, based on the lidar feed, an identification of a user of the ATM; and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user.
Regarding Claim 13, Cuan does not expressly teach, but Lewis teaches wherein an object of interest within the field of view of the camera/lidar device, i.e., images generated from camera (34, 131), as illustrated in figures 3 and 10, and as mentioned at col. 7, lines 31-51, is an automatic teller machine (ATM), i.e., the bezel (517) of the atm/automated banking machine (10), as illustrated in figures 1, 26 and 27, and as mentioned in col. 42, lines 25-42, and wherein the instructions further cause the computing device (72, 510), as mentioned at col. 47, lines 33-44, and terminal controller (464) and as illustrated in figures 3 and 4, for example, to:
determine, based on the camera/lidar feed, an identification of a user of the ATM (10, 517); and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user, as mentioned at col. 25, line 57-col. 26, line 8.
Regarding Claim 13, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM; and
generate, based on the identification of the user, an alert when an image of the user fails to match a secure identifier of the user, as taught by Lewis, in Cuan’s computing device and system for the purpose of providing additional security through linking of both the unauthorized device and the user/customer whom installed it.
Regarding Claim 14, Cuan does not expressly teach wherein the secure identifier of the user comprises a biometric identifier of the user.
Regarding Claim 14, Cuan does not expressly teach, but Lewis teaches wherein the secure identifier of the user comprises a biometric identifier of the user, as mentioned at col. 7, lines 31-49, col. 38, lines 54-59, col. 45, line 41-col. 46-line 2 and col. 54, lines 47-67.
Regarding Claim 6, see the rejection of Claim 13.
Regarding Claim 7, see the rejection of Claim 14.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Krebs et al (US 11,631,068 B1) and further in view of Weis et al (US 2022/0406092 A1).
Regarding Claim 15, Cuan teaches the system as described above.
Regarding Claim 15, Cuan does not expressly teach wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM;
determine, based on the lidar feed, a distance between the user of the ATM a second individual near the user; and
generate, based on the distance between the user of the ATM and the individual, an alert.
Regarding Claim 15, Cuan does not expressly teach, but Weis teaches wherein an object of interest within the field of view of the camera/lidar device is an automatic teller machine (ATM), as taught by Krebs, and wherein the instructions further cause the computing device, i.e., processor (110) as illustrated in figure 1, to:
determine, based on the camera/lidar feed, an identification of a user of the ATM (100) as illustrated in figure 1;
determine, based on the camera/lidar feed, a distance between the user of the ATM (100) a second individual near the user; and
generate, based on the distance between the user of the ATM and the individual, an alert, as mentioned at paragraphs 16, 33, 64, 72, 75 and 77, for example, noting that Weis Claim 5 mentions “the display device displays information that the first face is not the second digital image” and noting that paragraph 76, last two sentences mention “[t]he display device can be configured to provide the information regarding the security state 510, for example to display it on a screen, for example to output an acoustic indication, such as an acoustic signal, for example” and “[t]o put it another way, the display device can be configured to make a user aware of the security state 510, i.e. for example the open money dispensing tray having the money, by means of displaying or outputting the information on a screen and/or by means of outputting an acoustic indication”. See paragraphs 16, 33, 64, 72, 75 and 77, which state as follows.
[0016] The at least one processor can be configured to determine the first distance of the first person using the at least one digital image. The at least one processor can be configured to determine the first distance using a first interocular distance of the first person. The at least one processor can be configured to determine the first interocular distance using the extracted first biometric data. The at least one processor can be configured to determine the second distance of the second person using the at least one digital image. The at least one processor can be configured to determine the second distance using a second interocular distance of the second person. The at least one processor can be configured to determine the second interocular distance using the extracted second biometric data. The features described in this paragraph in combination with the eleventh example form a twelfth example.
[0033] The first distance of the first person can be determined using the at least one digital image. The first distance can be determined using a first interocular distance of the first person. The first interocular distance can be determined using the extracted first biometric data. The second distance of the second person can be determined using the at least one digital image. The second distance can be determined using a second interocular distance of the second person. The second interocular distance can be determined using the extracted second biometric data. The features described in this paragraph in combination with the twenty-eighth example form a twenty-ninth example.
Description
[0064] FIG. 4 illustrates a processing system 400 in accordance with various embodiments. The processing system 400 can comprise facial recognition 402. The facial recognition 402 can substantially correspond to the processing system for facial recognition 200, wherein the facial recognition 402 can process at least one digital image 404, which can comprise more than one face. The at least one digital image 404 can comprise a first face 406 of a first person and a second face 408 of a second person, and the facial recognition 402 can be configured to determine or to recognize the first face 406 of the first person and the second face 408 of the second person in the at least one digital image 404. The facial recognition 402 can furthermore be configured to determine or to extract the biometric features of the first face 406 and/or the biometric features of the second face 408. The at least one processor 110 can be configured to determine whether a first criterion 410 is satisfied. The first criterion 410 can be assigned to the first face 406 of the first person. The processor 110 can be configured to determine whether a second criterion 412 is satisfied. The second criterion 412 can be assigned to the second face 408 of the second person. The first criterion 410 and/or the second criterion 412 can each be a facial recognition criterion, wherein the facial recognition criterion, as described above, can comprise information of whether a face is hidden, or information of whether a face is not hidden. The processor 110 can be configured to determine a first distance between the first person and the self-service terminal 100 and can furthermore be configured to determine a second distance between the second person and the self-service terminal 100. The processor 110 can be configured to determine the first distance and/or the second distance using the at least one digital image 404. The processor 110 can be configured to determine a first interocular distance using the biometric features of the first face 406 and can furthermore be configured to determine the first distance of the first person using the first interocular distance determined. The processor 110 can be configured to determine a second interocular distance using the biometric features of the second face 408 and can furthermore be configured to determine the second distance of the second person using the second interocular distance determined.
[0072] By way of example, the determined first distance of the first person can be less than a first distance criterion and the determined second distance of the second person can be less than a second distance criterion, wherein the first distance of the first person is less than the second distance of the second person. In this case, the processor 110 can determine, for example, that the first person is a user of the self-service terminal 100 and that the second person is not a user of the self-service terminal 100. The processor 110 can determine, for example, that the second person is not complying with a personal space distance. Furthermore, the first facial recognition criterion and the second facial recognition criterion can be satisfied, that is to say that the processor 110 determines that the first person and the second person are not disguised. The processor 110 can be configured to determine that a security violation might be present. The self-service terminal 100 can be an automated teller machine and the security violation may be, for example, that the second person, who was not determined as a user of the self-service terminal 100, is attempting to covertly observe the PIN (personal identification number) of the first person, who was determined as a user of the self-service terminal 100. The processor 110 can be configured to communicate the indication 414 that a security violation might be present to the display device, and the display device can be configured to provide the first person, i.e. the user of the self-service terminal 100, with information. The display device can be configured to provide the first person with the at least one digital image 404 comprising the face 406 of the first person and the face 408 of the second person, for example to display it on the screen. The processor 110 can furthermore be configured to provide the display device with the first distance of the first person and the second distance of the second person. The display device can be configured to mark the first face 406 of the first person displayed on the screen as a user, for example to represent it with a first colored frame, such as a green frame, for example, and can furthermore be configured to mark the second face 408 of the second person displayed on the screen as a potential threat, for example to represent it with a second colored frame, such as a blue, yellow or red frame, for example, wherein a respective marking, such as, for example, a blue frame or a red frame, can be selected depending on the determined second distance of the second person. The display device can furthermore be configured to provide the first person with additional information, for example in the form of text, as sound, etc. In accordance with various embodiments, the self-service terminal 100 furthermore comprises a user interface, wherein the user interface can comprise the display device and wherein the user interface can furthermore comprise an input device. The input device can be configured to enable the user of the self-service terminal 100 to interact with the self-service terminal 100. The input device can be for example a keypad, for example a keyboard. The display device and the input device can be integrated as a touch-sensitive screen (touchscreen). The information provided by means of the display device can enable the first person to deny the potential threat by means of the input device. That is to say that the first person can indicate by means of the user interface that the second person is not a threat (the second person may be, for example, a friend, a partner or a family member). The processor 110 can be configured to block a use of the self-service terminal 100 until the first person, i.e. the user, has denied the potential threat.
[0075] By way of example, the processor 110 can determine that the first digital image 502 comprises a first face 506 of a first person. The processor 110 can furthermore determine a first distance of the first person. The processor 110 can be configured to determine the first person as a user of the self-service terminal using the determined first face 506 of the first person and optionally using the first distance of the first person. The processor 110 can determine, for example, that the second digital image 504 does not comprise a face of a person and the processor 110 can be configured to control the self-service terminal 100 in such a way that the self-service terminal 100 transitions to a security state 510. The processor 110 can be configured to control the self-service terminal in such a way that the self-service terminal 100 transitions to a security state 510 after a predefined time interval (for example after 5 seconds, for example after 10 seconds, for example after more than 10 seconds) after the detection of the second digital image 504 that does not comprise a face of a person. The security state 510 can also be the standby state if the self-service terminal 100 is in a non-critical state (for example is in a state in which a second person could not cause damage) at the time of the detection of the second digital image 504.
[0077] By way of example, the processor 110 can determine that the first digital image 502 comprises a first face 506 of a first person. The processor 110 can furthermore determine a first distance of the first person. The processor 110 can be configured to determine the first person as a user of the self-service terminal using the determined first face 506 of the first person and optionally using the first distance of the first person. The processor 110 can be configured to determine first biometric features of the first face 506 of the first person or to extract them from the first digital image 502. The processor 110 can determine, for example, that the second digital image 504 comprises a second face 508 of a second person. The processor 110 can be configured to determine second biometric features of the second face 508 of the second person or to extract them from the second digital image 504. The processor 110 can determine, for example, that the second face 508 is different than the first face 506. To put it another way, the first biometric features determined can be different than the second biometric features determined. The processor 110 can be configured to control the self-service terminal 100 in such a way that the self-service terminal 100 transitions to the security state 510 if said processor determines that the first biometric features and the second biometric features are different. That is to say that the second digital image 504 comprises a different person than the first digital image 502. The self-service terminal 100 can be an automated teller machine and the second digital image 504 can be detected at a time at which the money dispensing tray is open and has money; the security state 510 can comprise for example the money dispensing tray being closed. The self-service terminal 100 can be an automated teller machine and the second digital image 504 can be detected at a time at which the PIN of the user has already been entered; the security state 510 can comprise for example the self-service terminal 100, i.e. the automated teller machine, being blocked so that no money can be withdrawn. For example, renewed entry of a PIN can be requested in this case. The self-service terminal 100 can comprise the user interface and the processor 110 can furthermore be configured to communicate information regarding the security state 510 to the display device of the user interface. The display device can be configured to provide the information regarding the security state 510, for example to display it on a screen.
Emphasis provided.
Regarding Claim 15, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein an object of interest within the field of view of the lidar device is an automatic teller machine (ATM), and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of a user of the ATM;
determine, based on the lidar feed, a distance between the user of the ATM a second individual near the user; and
generate, based on the distance between the user of the ATM and the individual, an alert, as taught by Weis, in Cuan’s computing device and system for the purpose of providing additional security through linking of both the image of the second user with the first user/customer.
Claim(s) 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Krebs et al (US 11,631,068 B1), further in view of Weis et al (US 2022/0406092 A1) and further in view of Wurmfeld (US 2023/0025391 A1).
Regarding Claims 16-18, Cuan teaches the system as described above.
Regarding Claim 16, Cuan does not expressly teach wherein the instructions, further cause the computing device to:
determine, based on the lidar feed, an identification of the second individual; and
initiate, based on a match between the identification of the second individual and a historical record of individuals previously determined to perform malicious activity near the ATM, a security response.
Regarding Claim 16, Cuan does not expressly teach, but Wurmfeld teaches wherein the instructions, further cause the computing device to:
determine, based on the camera/lidar feed, an identification of the second individual, as taught by Weis; and
initiate, based on a match between the identification of the second individual and a historical record of individuals previously determined to perform malicious activity near the ATM, a security response, as mentioned at paragraphs 60 and 61 which state as follows.
[0060] In decision block 610, the system may determine whether current first user activity data matches stored historical activity data. Current first user activity data may be data may be determined in response to receiving first level authentication from a first user as described with respect to step 305 in FIG. 3.
[0061] When the current first user activity data matches historical activity data beyond a predetermined threshold of similarity, the system may confirm that the current location of the first user is within the predetermined proximity of the first computing device (e.g., computer interface 130) in block 615. When the current first user activity data does not match historical activity data beyond the predetermined threshold of similarity, the method may end.
Emphasis provided.
Note also the mention at step (325), i.e., “one or more objects associated with a human”, the mention at step (330), i.e., “trigger a security measure”, step (335) which states “transmit an indication of the triggered security measure to the first computing device” and step (340) which states “transmit instructions to the first user device configured to cause the first user device to provide an audible or vibrational alert to the first user” as illustrated at figure 3.
Regarding Claim 16, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein the instructions, further cause the computing device to:
determine, based on the lidar feed, an identification of the second individual; and
initiate, based on a match between the identification of the second individual and a historical record of individuals previously determined to perform malicious activity near the ATM, a security response, as taught by Wurmfeld, in Cuan’s computing device and system for the purpose of providing additional security through linking of both the image of the second user with the first user/customer.
Regarding Claim 17, Cuan does not expressly teach wherein the security response comprises generation of a message to security personnel, the message comprising a location of the ATM and a lidar image of the second individual captured by the at least one lidar device
Regarding Claim 17, Cuan does not expressly teach, but Wurmfeld teaches, wherein the security response comprises generation of a message to security personnel, the message comprising a location of the ATM and a lidar image of the second individual captured by the at least one lidar device, as mentioned at Wurmfeld, paragraphs 60 and 61 and figures 3-7, for example.
Regarding Claim 18, see the rejection of Claims 16 and 17, above.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1), further in view of Krebs et al (US 11,631,068 B1), further in view of Weis et al (US 2022/0406092 A1), further in view of Wurmfeld (US 2023/0025391 A1) and further in view of Burris et al (US 2023/0252803 A1).
Regarding Claim 19, Cuan teaches the system as described above.
Regarding Claim 19, Cuan does not expressly teach wherein the security response comprises generation of a message to a local law enforcement office, the message comprising location information of the ATM and a lidar image of the second individual generated by the lidar processing device.
Regarding Claim 19, Cuan does not expressly teach, but Burris teaches wherein the security response comprises generation of a message to a local law enforcement office, the message comprising location information of the ATM and a lidar image of the second individual generated by the lidar processing device, as mentioned at paragraphs 40-45, which state as follows.
[0040] In an embodiment, one camera 120 and/or 137 is used to perform the card slot tampering detection and images obtained other cameras 120 and/or 137 are correlated via time stamps. Card tampering detection manager 113 then assembles the correlated video associated with a card slot tampering detection event into a video clip that is provided as a link for viewing with the messages and/or alerts sent to ATM/security manager 143, governmental authority systems, security systems, and/or security agent 136. The video clip may comprise an image of the operator's face or other distinguishing features of the operator that may prove use during investigation of the card slot tampering event.
[0041] In an embodiment, system 100 operates on any transaction terminal that comprises a card reader with card slot 132, such as Self-Service Terminals (SSTs) operated for self-checkouts and/or Point-Of-Sale (POS) terminals operated by cashiers of a retailer during customer-assisted checkouts. Thus, system 100 can comprises a plurality of different types of transaction terminals beyond just the ATM 130 illustrated in FIG. 1 of system 100.
[0042] In an embodiment, the ATM 130 comprises a microphone as one of the other modules and/or the cameras 120 and/or 130 comprise a microphone. The microphone can capture audio while the operator is at the ATM including noises related to use of tools and speech of the operator. Predefined keywords may be listened for such as “hurry up get it in there,” we are going to get caught,” “skimmer,” “shimmer,” etc. An audio clip may be captured of the noises and/or speech and included with the alter or notification and included within any link to an audio clip.
[0043] In an embodiment, at least one camera 120 and/or 130 captures visual features of the individual (operator), such as facial features, body features, eye features, etc. The facial or body features are provided with the video clip and with the alert or the notification. This information can be used to comparison against other events associated with other ATMs 130 where skimmers and/or shimmers were installed.
[0044] In an embodiment, the hand actions monitored may be respect to other modules 133 may be tracked and monitored beyond just the card reader 132. For example, actions and activities with respect to the ATM's Personal Identification Number (PIN) pad. Any available camera 137 affixed to the modules 133 can be used when evaluating the images for objects in the hands and hand actions of the user. Any other module 133 associated with these hand actions or objects may also be identified in the alert or the notification, such that a service technician can perform tests on such modules to ensure they have not been tampered with by the individua/operator.
[0045] In an embodiment, wireless transceivers (types of other modules 133) may be activated to scan for wireless devices in possession of the individua/operator at the ATM 130. The transceivers can scan for wireless device identifiers that are within wireless range of the transceivers, and the device identifiers may be recorded for purposes of linking a device of the individual to other past skimmer/shimmer attacks or for purposes of allowing law enforcement to track the location of the mobile device in possession of the individual. The device identifier of the individual/operator may be sent with the alert or the notification.
Emphasis provided.
Regarding Claim 19, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein the security response comprises generation of a message to a local law enforcement office, the message comprising location information of the ATM and a lidar image of the second individual generated by the lidar processing device, as taught by Burris, in Cuan’s computing device and system for the purpose of providing additional security through contacting local security and law enforcement officials.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuan et al (US 11,062,248 B1) in view of Sumpter et al (US 2021/0097540 A1), further in view of Goetz et al (US 11,526,861 B1), further in view of Darrer et al (US 2020/0386867 A1), further in view of Pandya et al (US 11,803,955 B1) and further in view of Masuda (US 2022/0084342 A1).
Regarding Claim 20, Cuan teaches the system as described above.
Regarding Claim 20, Cuan does not expressly teach wherein an object of interest within the field of view of the lidar device is a security gate, and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of an individual adjacent to the security gate; and
generate, based on the identification of the individual, an alert when the identification of the individual fails to match a secure identifier that the individual uses in an attempt to proceed through the security gate.
Regarding Claim 20, Cuan does not expressly teach, but Masuda teaches wherein an object of interest, i.e., a user or customer, within the field of view of the camera/lidar device, i.e., imaging unit (52), as mentioned in paragraph 23, is a security gate, (10, 20, 30), and wherein the instructions further cause the computing device, i.e., CPU (55), as mentioned at paragraph 37, to:
determine, based on the camera/lidar feed, an identification of an individual adjacent to the security gate (10, 20, 30); and
generate, based on the identification of the individual, i.e., first biometric information as mentioned at paragraph 20 and second biometric information as mentioned at paragraph 20, an alert when the identification of the individual fails to match a secure identifier that the individual uses in an attempt to proceed through the security gate, as mentioned at Masuda Claims 6, 12 and 19, which states as follows.
6. The gate control device according to claim 1, wherein the processor is further configured to transmit a notification via the communication interface to a manager terminal device when the received second biometric information fails to match the stored biometric information.
12. The gate control system according to claim 7, wherein the processor is further configured to transmit a notification via the communication interface to a manager terminal device when the received second biometric information fails to match the stored biometric information.
19. The gate control method according to claim 13, further comprising: sending a notification to a manager terminal when the received second biometric information fails to match the retained first biometric information.
Emphasis provided.
Regarding Claim 20, before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have provided wherein an object of interest within the field of view of the lidar device is a security gate, and wherein the instructions further cause the computing device to:
determine, based on the lidar feed, an identification of an individual adjacent to the security gate; and
generate, based on the identification of the individual, an alert when the identification of the individual fails to match a secure identifier that the individual uses in an attempt to proceed through the security gate, as taught by Masuda, in Cuan’s computing device and system for the purpose of providing additional security through contacting/alerting local security and law enforcement officials.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant is encouraged to contact the Examiner should there be any questions about this rejection or in an endeavor to explore potential amendments or potential allowable subject matter.
Guo ‘989 teaches an image/LIDAR fusion module (250) as mentioned at paragraphs 54-61 and as illustrated in figures 1-7, for example.
Liang ‘457 teaches a three-dimensional object detection device, as mentioned at paragraphs 11 and 13, and as illustrated in figures 1-18, for example.
Lee ‘369 teaches a sensor fusion system, as mentioned at abstract and paragraphs 6 and 88, as well as illustrated in figures 4-7, for example.
MEI ‘990 is cited as teaching a system for using multimodal sensor data using a fusion module (250), as mentioned at paragraphs 54-61 and as illustrated in figures 2 and 3, for example.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFREY ALAN SHAPIRO whose telephone number is (571)272-6943. The examiner can normally be reached Monday-Friday generally between 8:30AM and 6:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anita Y Coupe can be reached at 571-270-3614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFREY A SHAPIRO/Primary Examiner, Art Unit 3619
December 24, 2025
Prosecution Insights