Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 4-6, 8, 9, 11, 12, 14-16, 18 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schillen (US 20250094553 A1), in view of Vargas (US 11829460 B2), in view of Maizels (US 20240073219 A1) and further in view of Thubert (US 20220368547 A1).
Regarding claims 1 and 11, Schillen discloses actively irradiating a scene in front of an irradiation sensor using a structured irradiation projector ([0051] the pattern illumination is a regular pattern and/or a periodic pattern and/or a dot pattern and/or speckle pattern, in particular, with at least 2, preferably at least 6, more preferably at least 20 pattern features and/or dots and/or speckles per face area at a distance of 5-200 cm, preferably 30-60 cm, more preferably 15-80 cm.);
generating ([0131] In an embodiment, the patterned infrared illumination may comprise a periodic and/or regular pattern, preferably, a hexagonal dot pattern with 1000-2000 dots per image or pseudo-random structured light.);
projecting the dot pattern onto a user's face using the structured irradiation projector ([0207] In a further embodiment of the invention, the pattern is a dot or speckle pattern, preferably with at least 20 dots or speckles per face area at a distance of 20-60 cm or 15-80 cm.);
capturing a reflected pattern from the user's face using an irradiation sensor to produce a depth map of the user's face ([0131] In an embodiment, the method further comprises the step of generating depth map from infrared image data, e.g. a heat map (as described below) for which the parameters relate to the depth and/or the depth distribution of the object shown by the image.);
Schillen does not disclose verifying the identity of the user by comparing the depth map with stored biometric data; rejecting the biometric verification if the comparison with the depth map does not match.
In a similar field of endeavor of biometric authentication, Vargas teaches verifying the identity of the user by comparing the depth map with stored biometric data (col 12, lines 20-28: In the instance wherein lidar and/or infrared cameras (or similar imaging technology are used), the unique depth map may be compared against previously stored biometric information, such as a biometric signature refined over time by the user performing an authentication process. Again, the unique depth map and biometric signature may be a mathematical representations, as such, the comparison may require that the unique depth map and biometric signature match within a certain threshold.);
rejecting the biometric verification if the comparison with the depth map does not match (col 12, lines 9-19: For example, the unique depth map may be a mathematical representation of the captured measurements, and in furtherance of step 406, wherein the unique data is converted into a template, in order to authenticate the user, the template may be compared against a biometric signature or biometric data already known about the user. The comparison may require that the template match the biometric signature within a certain pre-determined threshold in order for the user to be authenticated. In instances where the template does not exceed the pre-determined threshold, the user authentication may be denied.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen’s disclosure dot pattern illumination, with Vargas’ teaching of depth map verification, in order to enable a user authentication via contactless physiological biometric authentication processes, behavioral authentication processes, and passive data authentication processes, that do not require a user to remove personal protective equipment. Schillen and Vargas do not disclose continuous biometric verification of a video feed during video calls, characterized by comprising: ensuring the transmission of genuine real-time media streams from authenticated users during the video call by revoking the identity of the user if the depth map does not match, or if the challenge is not responded.
In a similar field of endeavor of continuous video authentication, Maizels teaches continuous biometric verification of a video feed during video calls, characterized by comprising ([0357] “Continuous authentication” refers to authentication for more than a single instant in time. For example, continuous authentication may be provided by uninterrupted authentication for a contiguous length of time or time period. The time period may be any amount of time (e.g., seconds, minutes, hours, days, or any other extent of time depending on the specific implementation). As another example, continuous authentication may be provided by authentication for multiple spaced-apart time periods. The multiple time periods may be spaced apart by any amount of time.):
ensuring the transmission of genuine real-time media streams from authenticated users during the video call by revoking the identity of the user if the depth map does not match, or if the challenge is not responded ([0259] In this context, the term “continuously compares” means constantly or regularly compares new facial skin micromovements with the plurality of facial skin micromovements in the memory over a period of time (e.g., during a phone call). In this context, continuous comparison includes intervals between comparisons such as multiple times a second or multiple times a minute. The term “instantaneous level of certainty” refers to a degree of confidence in an identity of individual associated with the new facial skin micromovements. For example, during a phone call with a banker, the system may regularly compare new facial skin micromovements to make sure that the same authorized individual remains on the line. Consistent with some disclosed embodiments, when the instantaneous certainty level is below a threshold, the at least one processor is configured to initiate an associated action. The fact that the instantaneous certainty level is below a threshold means that there is a risk that someone else—other than the identified individual—is responsible for the new facial skin micromovements. The associated action refers to an action associated with the fact that the instantaneous certainty level is now below a threshold and may include the second action or stopping the first action. Specifically, in some embodiments, after initiating the first action, when the instantaneous certainty level is below a threshold, the at least one processor is configured to stop the first action. For example, the first action may be authorizing a transaction in the bank by speaking with a banker over the phone and providing the banker with ongoing confirmation of the identity of the individual over the phone. But, once the instantaneous certainty level drops below the threshold, which may indicate that someone other than the individual is talking to the banker, the transaction may be stopped. In some cases, the second action may include stopping the first action.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen and Vargas’s disclosure of depth map verification with Maizels’ teaching of video authentication streams, in order to detect facial skin micromovements of an individual, use the detected facial skin micromovements to identify the individual, and determine an action to initiate based on the identification of the individual (0229). Schillen, Vargas and Maizels do not explicitly state after initiation of a video call a pseudo-random, time-variant dot pattern as a session-specific challenge in a challenge-response mechanism ([0015] In light of these observations, the disclosed embodiments ensure validity of a video conference participant by generating a session unique and modulating pattern of encoded data and transmitting the encoded data to a participant user device. The data pattern is modulated such that validation can be continuous throughout the video conference or as needed. In some embodiments, the pattern is a series of dots or dashes projected onto a face of the video participant.); determining, from the captured reflected pattern, a response corresponding to the generated dot pattern, and verifying the response as a function of that generated dot pattern.
In a similar field of endeavor of video teleconference validation, Thubert teaches generating after initiation of a video call, a pseudo-random, time-variant dot pattern as a session-specific challenge in a challenge-response mechanism ([0015] In light of these observations, the disclosed embodiments ensure validity of a video conference participant by generating a session unique and modulating pattern of encoded data and transmitting the encoded data to a participant user device. The data pattern is modulated such that validation can be continuous throughout the video conference or as needed. In some embodiments, the pattern is a series of dots or dashes projected onto a face of the video participant.);
determining, from the captured reflected pattern, a response corresponding to the generated dot pattern ([0044] If the two sets of modulated data are within a threshold tolerance, operation 530 determines that the data match. Otherwise, operation 530 determines the data does not match. As some embodiments encode a message in the continuously modulated data transmitted in operation 520, operation 530 attempts to decode the message from the continuously modulated data derived from the video stream.),
and verifying the response as a function of that generated dot pattern ([0015] The data pattern is modulated such that validation can be continuous throughout the video conference or as needed. In some embodiments, the pattern is a series of dots or dashes projected onto a face of the video participant. In some cases, the pattern is encoded as luminosity or color changes to an image displayed on a video screen that is proximate to the participant. In some embodiments, the pattern is projected using infrared wavelengths, while other embodiments use visible light wavelengths. By both projecting a pattern onto a face of the participant, and varying the pattern over time, the disclosed embodiments are able to verify both identity and a time currency of a video image of the participant.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the known system of using pseudo-random dot pattern for depth map verification, as disclosed by Schillen, Vargas, and Maizels with the known teaching of with’ teaching of session specific modulation patterns, in order to yield the predictable results of enhancing security from risks by continuously validating the light pattern throughout a video conference.
Regarding claims 2 and 12, Schillen discloses actively illuminating the scene in front of the irradiation sensor using near-infrared light ([0285] The sensor may be or may comprise at least one inorganic photodiode which are sensitive in the infrared spectral range, preferably in the range of 700 nm to 3.0 micrometers. Specifically, the sensor may be sensitive in the part of the near infrared region where silicon photodiodes are applicable specifically in the range of 700 nm to 1100 nm.),
and capturing the reflected pattern using an infrared camera or a LIDAR/Tof irradiation sensor ("[0054] Image of the object captured while the object is being illuminated with the patterned infrared illumination may also be referred to as pattern image. Pattern image is capture while object is being illuminated with patterned infrared illumination. Preferably, the pattern image arises as the reflection of the light pattern when the light pattern is reflected from the object.
[0284] For example, the camera 203 may include both IR sensors and RGB (red, green, and blue) sensors, e.g. RGB camera 205a.").
Regarding claims 4 and 14, Schillen discloses creating the depth map using a triangulation algorithm processing data from the infrared camera ([0127] Preferably, material information may be derived by using a NN and/or depth information may be derived from a beam profile analysis via one or more appropriate filters. More preferably, in contrast to the prior art using triangulation in structured light approach, in an embodiment, patter analysis, preferably, regular pattern analysis, of the pattern, may be used for depth recognition.).
Regarding claims 5 and 15, Schillen discloses capturing a ([0297] One or more neural network modules 310 may be used to operate the machine learning models. The neural network module 310 may be located in IP 308 and/or SP 309. SP 309 may compare the image characteristics with stored templates for each type of image to generate an authentication score depending on a matching score or other ranking of matching between the user in the captured image and in the stored templates. The authentication scores for the images such as the flood IR and patterned illumination images may be combined to decide on the identity of the user and, if authenticated, allow the user to use device e.g., unlock the device.). Schillen and Vargas do not disclose a video feed.
In a similar field of endeavor of continuous video authentication, Maizels teaches a video feed ([0391] Consistent with some disclosed embodiments, the first period of time, the second period of time, and the third period of time are part of a single communication session, and wherein the communication session is at least one of: a phone call, a teleconference, a video conference, or a real-time virtual communication.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen and Vargas’s disclosure of depth map authentication with Maizels’ teaching of video authentication streams, in order to detect facial skin micromovements of an individual, use the detected facial skin micromovements to identify the individual, and determine an action to initiate based on the identification of the individual (0229).
Regarding claims 6 and 16, Schillen discloses continuously projecting the dot pattern onto the user's face and capturing the reflected light in a pseudo-random manner to verify the presence of a live person ("[0131] In an embodiment, the patterned infrared illumination may comprise a periodic and/or regular pattern, preferably, a hexagonal dot pattern with 1000-2000 dots per image or pseudo-random structured light.
[0207] In a further embodiment of the invention, the pattern is a dot or speckle pattern, preferably with at least 20 dots or speckles per face area at a distance of 20-60 cm or 15-80 cm.").
Regarding claims 8 and 18, Schillen and Vargas do not disclose but Maizels teaches preventing injection of deepfake media by detecting and blocking any discrepancies between the depth map and the video feed ([0333] Similarly, when the comparison 1512 indicates a mismatch between the real-time signals 1508 and the reference signals 1502, the notification 1514 may block or prevent the individual from conducting the transaction.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen and Vargas’s disclosure of depth map verification with Maizels’ teaching of video authentication streams, in order to detect facial skin micromovements of an individual, use the detected facial skin micromovements to identify the individual, and determine an action to initiate based on the identification of the individual (0229).
Regarding claims 9 and 19, Schillen and Vargas do not disclose but Maizels teaches providing feedback to the user and the system administrator if an attempt to inject fake media is detected ("[0332] Some disclosed embodiments involve, upon authentication, notifying the institution that the particular individual is authenticated. The term “notifying” (and other related constructs such as notify, notification, etc.) refers to informing someone of something.
[0339] With reference to FIGS. 15, if after notifying 1514 the institution of the successful authentication of the individual, system 1500 detects that the real-time signals 1508 received at a subsequent time does not match the stored reference signals 1502 of the individual, system 1500 may report the mismatch to institution 1400 (and/or other authorities). The institution (and/or authentication system 1500) may terminate the individual's access to the account and/or take other protective measures based on the reporting of the mismatch.").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen and Vargas’s disclosure of depth map verification, with Maizels’ teaching of video streams, in order to detect facial skin micromovements of an individual, use the detected facial skin micromovements to identify the individual, and determine an action to initiate based on the identification of the individual (0229).
Claim(s) 3, 7, 13 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schillen (US 20250094553 A1), in view of Vargas (US 11829460 B2), in view of Maizels (US 20240073219 A1), in view of Thubert (US 20220368547 A1), and further in view of Tang (US 20230274453 A1).
Regarding claims 3 and 13, Schillen, Vargas, Maizels and Thubert do not disclose or teach creating a disparity map that corresponds to depth based on a spatial offset between the projector and the sensor.
In a similar field of endeavor of face depth imaging, Tang teaches creating a disparity map that corresponds to depth based on a spatial offset between the projector and the sensor ([0201] The closer the object distance is to the lens 160, the larger disparity and higher density on the depth information may be generated for the object. The greater amount of disparity and density of the depth information for close objects may offset the loss in accuracy.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen, Vargas, Maizels and Thubert disclosure of continuous video authentication streams with Tang’s teaching of disparity maps, in order to increase the speed and accuracy of 3D reconstruction by enabling a downscaled structured light image to be generated with a sufficient number of depth pixels (0010).
Regarding claims 7 and 17, Schillen and Vargas do not disclose but in a similar field of endeavor of continuous video authentication, Maizels teaches the video feed ([0391] Consistent with some disclosed embodiments, the first period of time, the second period of time, and the third period of time are part of a single communication session, and wherein the communication session is at least one of: a phone call, a teleconference, a video conference, or a real-time virtual communication.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen and Vargas’s disclosure of depth map authentication with Maizels’ teaching of video authentication streams, in order to detect facial skin micromovements of an individual, use the detected facial skin micromovements to identify the individual, and determine an action to initiate based on the identification of the individual (0229).
Schillen, Vargas, Maizels and Thubert do not disclose or teach but Tang teaches transmitting the depth map and the video feed securely to a remote server for comparison with biometric data stored in a biometric database ([0075] In some embodiments, the signal VIDOUT may comprise the data extracted from the video frames (e.g., the results of the computer vision), and the results may be communicated to another device (e.g., a remote server, a cloud computing system, etc.) to offload analysis of the results to another device (e.g., offload analysis of the results to a cloud computing service instead of performing all the analysis locally).
[0150] The reference images 304 may be used for computer vision operations, 3D reconstruction, depth map generation, etc. The reference images 304 may enable 3D reconstruction to be performed when the monocular lens 160 is implemented. The memory 150 may be configured to store multiple reference images 304 of various reference locations and/or objects (e.g., faces) at different distances to have a basis for comparison of the structured light pattern SLP captured in the live video frames 302a-302n. The reference images 304 may comprise images in an IR YUV format. The reference images 304 may comprise dot patterns associated with various distances to a lens and/or various types of objects.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen, Vargas, Maizels and Thubert’s disclosure of continuous video authentication streams with Tang’s teaching of disparity maps, in order to increase the speed and accuracy of 3D reconstruction by enabling a downscaled structured light image to be generated with a sufficient number of depth pixels (0010).
Claim(s) 10 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schillen (US 20250094553 A1), in view of Vargas (US 11829460 B2), in view of Maizels (US 20240073219 A1), in view of Thubert (US 20220368547 A1) and further in view of Aragon (US 20230162534 A1).
Regarding claims 10 and 20, Schillen, Vargas, Maizels and Thubert do not explicitly disclose or teach moving critical components, such as a dot pattern generator or face identification algorithm, to an external server for enhancing security.
In a similar field of endeavor of face depth imaging, Aragon teaches, moving critical components, such as a dot pattern generator or face identification algorithm, to an external server for enhancing security ([0075] However, it can also be provided that not all steps of the method are performed on a mobile device but, for example, only the images are actually taken by the mobile device and are then forwarded to an external server that performs the further processing and then only provides the result of this processing back to the mobile device where, then, for example the identification of the user or the extraction of the biometric characteristic can be performed.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine Schillen, Vargas, Maizels and Thubert's disclosure of continuous video authentication streams with Aragon’s teaching of external servers, in order to performing an identification algorithm to find a biometric characteristic for distinguishing between a real user and a spoof image (abstract).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. However, upon further consideration, a new ground(s) of rejection is made in view of Thubert (US 20220368547 A1).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 20220222464 A1 with respect to claims 2 and 12 ([0068] When NIR illumination, subject to inverse square law reduction in intensity, illuminates an object, the light can be reflected by specular reflection or diffuse reflection.
[0075] Examples of passive distance measuring technologies include distance estimation using light polarization, lidar, and ultrasound. For example, lidar can determine a distance from a sensor to an object in a scene by measuring time required for a pulse of light to travel from a sensor to an object and back. Polarization techniques can measure a difference in reflected light polarization between a background and an object in an NIR image. Ultrasound sensors can measure time required for a pulse of ultrasound energy to travel from a transducer to an object and back. A distance value determined by light polarization, lidar or ultrasound can be averaged with an estimated distance value determined by techniques discussed herein to generate an estimated relative distance measure.).
US 20210350769 A1 with respect to claim 1: [0018] The processor may be configured generate a depth map using the scan, and determine a similarity score between the depth map and one or more identity maps of a set of stored biometric identity maps that are associated with a registered user.
[0062] The additional depth map is then compared with the stored biometric identity depth map. If the additional depth map shares a similarity score with the stored biometric identity map, the user is identified and/or authenticated and permitted access to the electronic device 100.
(if the user is identified when the depth map shares a similarity with the identity map, then it would be obvious to one of ordinary skill in the art to not identify the person if the depth map is not similar with the identity map)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED A NASHER whose telephone number is (571)272-1885. The examiner can normally be reached Mon - Fri 0800 - 1700.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED A NASHER/Examiner, Art Unit 2675
/ANDREW M MOYER/Supervisory Patent Examiner, Art Unit 2675
Prosecution Insights