DETAILED ACTION
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the “extract the first image part from the image, wherein the first image part is greater in size than an reference image part used to authenticate the target person, the reference image part includes both eyebrows, both eye corners, and a chin area of the face, and the first image part includes the reference image part and the face area in the image.” as in claim 7; “determine whether or not the target person is a living body by a third determination method, wherein the third determination method determines whether or not the target person is a living body in response to the first image part being input; and determine whether or not the target person is a living body by the first determination method in response to the input image having the face area that is not covered by a mask being inputted; and determine whether or not the target person is a living body by the second determination method in response to the input image having the face area that is covered by a mask being inputted.” as in claim 8 must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 7 and 8 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Regarding claim 7, which recites “extract the first image part from the image, wherein the first image part is greater in size than an reference image part used to authenticate the target person, the reference image part includes both eyebrows, both eye corners, and a chin area of the face, and the first image part includes the reference image part and the face area in the image.” is not disclosed in the specification as originally filed. Regarding claim 8 which recites “determine whether or not the target person is a living body by a third determination method, wherein the third determination method determines whether or not the target person is a living body in response to the first image part being input; and determine whether or not the target person is a living body by the first determination method in response to the input image having the face area that is not covered by a mask being inputted; and determine whether or not the target person is a living body by the second determination method in response to the input image having the face area that is covered by a mask being inputted” is not disclosed in the specification as originally filed.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tsai et al. (2019/0392129) in view of Vemulapalli et al. (2023/0063229) and Streit (2021/0377298).
Regarding claim 1, Tsai teaches a biometric determination apparatus comprising: at least one memory (4; Fig 1) that is configured to store instructions (para [0033] The storage medium 4 stores programs and materials for identity authentication executed by the processing unit 2 using the face image and the fingerprint image.); and at least one processor (2; Fig 1) that is configured to execute the instructions (para [0033] The storage medium 4 stores programs and materials for identity authentication executed by the processing unit 2 using the face image and the fingerprint image.) to: obtain an input image including a face area of a face of a target person (para [0033] The camera 6 is used for capturing a face image.); determine whether (i) a face area of the target person is at least partially covered by a mask (para [0035] After the captured face image and the fingerprint image are transmitted to the processing unit 2, the processing unit 2 may perform some preprocessing procedures to the face image and the fingerprint image, such as adjusting the size, orientation, scale of the images and so on, for the following face recognition and fingerprint recognition. In the step S20, the processing unit 2 determines whether a cover object, such as a mask or a pair of sunglasses, is presented in the face image. The cover object covers a part of a face in the face image. Artificial intelligence or image analysis technique may be applied to determine whether a cover object is presented in the face image; para [0036] When the processing unit 2 determines that a cover object is presented in the face image in the step S20, the step S21 is proceeded to select a non-covered area from the face image) or (ii) the face area of the target person is not covered (para [0041] When the processing unit 2 determines that the face image has no cover object in the step S20, the step S24 is executed.); in response to determining (ii) further determine whether the target person is authenticate by a first determination method using the input image, wherein the first determination method is based on face not covered (S21-S23, S40, S50; Fig 2; para [0041] When the processing unit 2 determines that the face image has no cover object in the step S20, the step S24 is executed. The step S24 is to compare the face image obtained in the step S10 with full face enrollment information, such as the full face enrollment information H, to generate a first value X2. para [0046] In the step S40, the processing unit 2 calculates an output value S according to the first value generated in the step S23 or S24 and the second value generated in the step S32 or S33. The step S50 is to verify the user's identity according to the output value S generated in the step S40, so as to determine whether the face image and the fingerprint image obtained in the step S10 match the user enrolled in the electronic device A.); and is response to determining (i), further determine whether the target person is authenticate by a second determination method using the input image , wherein the second determination method is different from the first determination method (S24; S40, S50; Fig 2; para [0036] When the processing unit 2 determines that a cover object is presented in the face image in the step S20, the step S21 is proceeded to select a non-covered area from the face image. The selected non-covered area does not overlap the cover object. It means that the step S21 is to choose other parts of the face image that are not covered by the cover object.; para [0041] When the processing unit 2 determines that the face image has no cover object in the step S20, the step S24 is executed. The step S24 is to compare the face image obtained in the step S10 with full face enrollment information, such as the full face enrollment information H, to generate a first value X2. In the step S24, the processing unit 2 converts the face image into face information to be verified first, and then calculates the similarity between the face information to be verified and the full face enrollment information to generate the first value X2. In the FIG. 2, the first values as described in the steps S23 and S24 represent the recognition result of the face image, and does not means that the first values generated in the steps S23 and S24 are the same.).
Tsai fails to teach, determine whether the target person is a living body by a first determination method using the input image; wherein the first determination method is constructed by machine learning using facial images of faces that are not covered by masks; and determine whether the target person is a living body by a second determination method that uses the input image; wherein the first determination method is different from the first determination method and is constructed by machine learning using facial images of faces that are covered by masks; as claimed.
Vemulapalli teaches the biometric determination apparatus; wherein the first determination method is a method of determining whether or not the target person is a living body, by using a first image part of the image including at least a part of the face image (para [0026] One or more images captured using the camera 111 can be analyzed to determine whether the images captured using the camera 111 correspond to an actual live person or if the images correspond to an alternative representation of the live person. For example, an alternative representation of a live person can be a 2D representation (e.g., a photograph) or a 3D representation (e.g., a mask) of the live person displaying one or more facial expressions. Para [0035] In some implementations, the region of interest can be the entire face of the user. The regions of interest 222 can be compared with a template region of interest 224 extracted from an image 204d captured during an enrollment process), and the second determination method is a method of determining whether or not the target person is a living body, by using a second image part in which at least a part of a covered area is removed from the first image part (para [0026]; para [0035] In some implementations, one or more points of interest (or landmarks) can be used to extract or crop a region of interest 220a, 220b, 220c (220, in general) from each captured image. The region of interest 220 can depend on the particular facial expression being used. For example, for a liveness detection process that analyzes a smile, the region of interest 220a, 220b, 220c, 220d can be a region around the mouth of a subject, as illustrated in FIG. 2A. In another example, for a liveness detection process that analyzes facial features associated with raised eyebrows, the region of interest can be a region 222a, 222b, 222c (222, in general) around the eyes or eyebrows of a subject. Examples of such regions of interest 222 are shown in FIG. 2B.).
It would have been obvious to one of ordinary skill in the art before the filing date of present application to have modified the apparatus of Tsai with the teachings of Vemulapalli, because controlling access to the secure system can improve performance of an underlying liveness detection system by filtering out spoof attacks at a front end, and thereby reducing waste of computational resources (Vemulapalli: para [0058]).
Tsai and Vemulapalli fails to teach, wherein the first determination method is constructed by machine learning using facial images of faces that are not covered by masks; wherein the first determination method is different from the first determination method and is constructed by machine learning using facial images of faces that are covered by masks; as claimed.
Streit teaches a system comprising: a first determination method (215; Fig 2) and a second determination method (216; Fig 2); wherein the first determination method is constructed by machine learning using facial images of faces that are not covered by masks (para [0011] where sets of helper networks can be trained to acquire and transform biometric values or types of biometrics to improve biometric capture, increase accuracy, reduce training time for embedding and/or classification networks, eliminate vulnerabilities (e.g., liveness checking and validation); para [0059] Once an input is validated the inputs can undergo further processing, including, identification, authentication, enrollment, etc. For example, the input can be processed by a respective embedding network in stage 240. For example, a face embedding DNN 215 can process user face images; Fig 2); wherein the first determination method is different from the first determination method and is constructed by machine learning using facial images of faces that are covered by masks (para [0056] the authentication system can use the state determination, wearing mask, to select a respective embedding DNN (e.g., 216—an embedding network trained on images with users wearing masks).para [0059] In further example, a face with mask embedding network 216 can process images of users wearing masks.; Fig 2).
It would have been obvious to one of ordinary skill in the art before the filing date of present application to have modified the apparatus of Tsai and Vemulapalli with the teachings of Streit, because this will provide authentication systems that are configured to leverage machine learning approaches in the context of pre-processing data for use in subsequent tasks, for example, recognition tasks (including e.g., recognition by machine learning models that support identification and/or authentication). The inventors have further realized that, unlike prior solutions, it is possible to create lightweight models (e.g., small file size models) that provide sufficient accuracy (e.g., >90%) in identifying features or states of input identification/authentication data to serve as a gateway for further processing (Streit: para [0004]).
Regarding claim 2, Tsai teaches the biometric determination apparatus according to claim 1, wherein the first determination method is a method of determining whether or not the target person is authenticate, by using a first image part of the image including at least a part of the face image (para [0047] Since the step S24 executes the face recognition with the full face image), and the second determination method is a method of determining whether or not the target person is a living body, by using a second image part in which at least a part of a covered area is removed from the first image part (para [0038] For example, the image P1 shown in FIG. 3A is a face image to be verified. When the processing unit 2 analyzes that a mask 30 exists in the face image P1, the processing unit 2 selects an upper area 11 of the face image P1 that is not covered by the mask 30 in the step S21, and selects a face partition enrollment information H1 according to the upper area 11 including the eyes in the step S22. One way to select the upper area 11 is to use the facial landmark detection to identify the two eyes from the face image P1 first, and then to extend a region of a predetermined size outwardly from a center of the two eyes to cover at least the two eyes. The upper area 11 includes the two eyes. The contents of the face partition enrollment information H1 at least including the two eyes. In the step S23, the processing unit 2 compares the image of the upper area 11 with the face partition enrollment datum H1 to generate the first value X1. Para [0039]).
Tsai fails to teach, wherein the first determination method is a method of determining whether or not the target person is a living body, by using a first image part of the image including at least a part of the face image, and the second determination method is a method of determining whether or not the target person is a living body, by using a second image part in which at least a part of a covered area is removed from the first image part; as claimed.
Vemulapalli teaches the biometric determination apparatus; wherein the first determination method is a method of determining whether or not the target person is a living body, by using a first image part of the image including at least a part of the face image (para [0026] One or more images captured using the camera 111 can be analyzed to determine whether the images captured using the camera 111 correspond to an actual live person or if the images correspond to an alternative representation of the live person. For example, an alternative representation of a live person can be a 2D representation (e.g., a photograph) or a 3D representation (e.g., a mask) of the live person displaying one or more facial expressions. Para [0035] In some implementations, the region of interest can be the entire face of the user. The regions of interest 222 can be compared with a template region of interest 224 extracted from an image 204d captured during an enrollment process), and the second determination method is a method of determining whether or not the target person is a living body, by using a second image part in which at least a part of a covered area is removed from the first image part (para [0026]; para [0035] In some implementations, one or more points of interest (or landmarks) can be used to extract or crop a region of interest 220a, 220b, 220c (220, in general) from each captured image. The region of interest 220 can depend on the particular facial expression being used. For example, for a liveness detection process that analyzes a smile, the region of interest 220a, 220b, 220c, 220d can be a region around the mouth of a subject, as illustrated in FIG. 2A. In another example, for a liveness detection process that analyzes facial features associated with raised eyebrows, the region of interest can be a region 222a, 222b, 222c (222, in general) around the eyes or eyebrows of a subject. Examples of such regions of interest 222 are shown in FIG. 2B.).
It would have been obvious to one of ordinary skill in the art before the filing date of present application to have modified the apparatus of Tsai with the teachings of Vemulapalli, because controlling access to the secure system can improve performance of an underlying liveness detection system by filtering out spoof attacks at a front end, and thereby reducing waste of computational resources (Vemulapalli: para [0058]).
Regarding claim 3, Tsai teaches the biometric determination apparatus according to claim 2, wherein the at least one processor is configured to execute the instructions to extract the second image part from the image on the basis of a distance between eyes included in the face (Fig 3A; para [0038] For example, the image P1 shown in FIG. 3A is a face image to be verified. When the processing unit 2 analyzes that a mask 30 exists in the face image P1, the processing unit 2 selects an upper area 11 of the face image P1 that is not covered by the mask 30 in the step S21, and selects a face partition enrollment information H1 according to the upper area 11 including the eyes in the step S22. One way to select the upper area 11 is to use the facial landmark detection to identify the two eyes from the face image P1 first, and then to extend a region of a predetermined size outwardly from a center of the two eyes to cover at least the two eyes. The upper area 11 includes the two eyes. The contents of the face partition enrollment information H1 at least including the two eyes. In the step S23, the processing unit 2 compares the image of the upper area 11 with the face partition enrollment datum H1 to generate the first value X1.).
Regarding claim 4, Tsai teaches the biometric determination apparatus according to claim 3, wherein the at least one processor is configured to execute the instructions to decide a size of the second image part to be extracted, and a position of the second image part, on the basis of the distance between the eyes included in the face (Fig 3A; para [0038] For example, the image P1 shown in FIG. 3A is a face image to be verified. When the processing unit 2 analyzes that a mask 30 exists in the face image P1, the processing unit 2 selects an upper area 11 of the face image P1 that is not covered by the mask 30 in the step S21, and selects a face partition enrollment information H1 according to the upper area 11 including the eyes in the step S22. One way to select the upper area 11 is to use the facial landmark detection to identify the two eyes from the face image P1 first, and then to extend a region of a predetermined size outwardly from a center of the two eyes to cover at least the two eyes. The upper area 11 includes the two eyes. The contents of the face partition enrollment information H1 at least including the two eyes. In the step S23, the processing unit 2 compares the image of the upper area 11 with the face partition enrollment datum H1 to generate the first value X1).
Regarding claim 5, Tsai teaches the biometric determination apparatus according to claim 2, wherein the at least one processor is configured to execute the instructions to extract the second image part from the image on the basis of both side parts of a lower part of a dorsum of a nose included in the face (12; Fig 3B; para [0039] As the embodiment shown in FIG. 3B, the image P2 is a face image to be verified. When the processing unit 2 analyzes that a pair of sunglasses 31 exists in the face image P2, the processing unit 2 selects a lower area 12 of the face image P2 that is not covered by the pair of sunglasses 31 in the step S21, and selects a face partition enrollment information H2 according to the lower area 12 including the mouth in the step S22. One way to select the lower area 12 is to use the facial landmark detection to identify the mouth from the face image P2 first, and then to extend a region of a predetermined size outwardly from a center of the mouth to cover at least the mouth.).
Regarding claim 6, Tsai teaches the biometric determination apparatus according to claim 5, wherein the at least one processor is configured to execute the instructions to decide a lower limit part of the second image part to be extracted, on the basis of the both side parts of the lower part of the dorsum of the nose included in the face (12; Fig 3B; para [0039] As the embodiment shown in FIG. 3B, the image P2 is a face image to be verified. When the processing unit 2 analyzes that a pair of sunglasses 31 exists in the face image P2, the processing unit 2 selects a lower area 12 of the face image P2 that is not covered by the pair of sunglasses 31 in the step S21, and selects a face partition enrollment information H2 according to the lower area 12 including the mouth in the step S22. One way to select the lower area 12 is to use the facial landmark detection to identify the mouth from the face image P2 first, and then to extend a region of a predetermined size outwardly from a center of the mouth to cover at least the mouth.).
Regarding claim 13, which is a method claim and is similar in scope to claim 1, thus claim 13 is also rejected same as claim 1 as explained above.
Regarding claim 14, which is a claim directed towards a non-transitory recording medium on which a computer program that allows a computer to execute a biometric determination method is recorded (para [0033] The storage medium 4 stores programs and materials for identity authentication executed by the processing unit 2 using the face image and the fingerprint image) and is further similar in scope to claim 1, thus claim 14 is also rejected same as claim 1 as explained above.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tsai et al. (2019/0392129) in view of Vemulapalli et al. (2023/0063229) and Streit (2021/0377298) as applied to claim 1 above, and further in view of Henson et al. (2021/0256281).
Regarding claim 9, Tsai, Vemulapalli and Streit teaches the biometric determination apparatus as explained for claim 1 above.
Tsai, Vemulapalli and Streit fails to teach, wherein the image is an infrared image; as claimed.
Henson teaches a biometric determination apparatus; wherein an image is an infrared image (para [0025] the captured image data may include data representing captured visible light and data representing captured infrared (IR) light. In some embodiments, the captured image data may include data representing captured infrared light in two non-overlapping wavelength bands. In general, the techniques described herein may be applied to captured image data representing captured light in any combination of non-overlapping wavelength bands. An example process for liveness detection in which visible light is captured using an RGB camera and infrared light is captured using an IR camera is illustrated in FIG. 3 and described below.).
It would have been obvious to one of ordinary skill in the art before the filing date of present application to have modified the apparatus of Tsai, Vemulapalli and Streit with the teachings of Henson, because the process to merge the two images in different spectral bands to create a combined image that accurately passes or fails a liveness test when implied three-dimensional (3D) features, such as shadowing, are present or absent, respectively. (Henson: para [0037]).
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tsai et al. (2019/0392129) in view of Vemulapalli et al. (2023/0063229) and Streit (2021/0377298) as applied to claim 1 above, and further in view of Miu et al. (2019/0205680).
Regarding claim 12, Tsai, Vemulapalli and Streit teaches the biometric determination apparatus as explained for claim 1 above.
Tsai, Vemulapalli and Streit fails to teach, wherein the at least one processor is configured to acquire a plurality of person images generated by imaging the target person using a plurality of different lighting conditions, and the image is at least one of the plurality of persons images; as claimed.
Miu teaches a biometric determination apparatus comprising: at least one processor is configured to acquire a plurality of person images generated by imaging the target person using a plurality of different lighting conditions, and the image is at least one of the plurality of persons images (Fig 4A; Fig 4B; para [0035] Referring to FIGS. 4A-4D, examples of verifying liveliness are illustrated when authenticating a facial biometric as submitted by a user from a mobile device. Here, the facial portraits 402 to 406 illustrate either live or static portrait under lighting or without lighting. These representative facial portraits may be derived from a scanning sequence in which a number of shots of the subject's face are taken from varying angles on the same scanning path, for example, during a selfie session using a mobile device, such as an iPhone. In these examples, the facial portraits from a live session reflect the face of the subject at various angles on this arc or line segment. Due to slight changes in illumination angles of various facial features from the facial portrait (e.g., the forehead, the eye brows, the eyes, the eye corners, the lips, and the mouths), the appearance of shadows may vary. Due to changes in lighting patterns, each landmark feature may be associated with one commensurate changes in shadow appearance. As illustrated, facial portraits 402 is taken without lighting while facial portrait 404 is taken with lighting on; Fig 5).
It would have been obvious to one of ordinary skill in the art before the filing date of present application to have modified the apparatus of Tsai, Vemulapalli and Streit with the teachings of Miu, because by employing variations in lighting patterns to inject another dimension of robustness for liveness determination. (Miu: para [0039]).
Response to Arguments
Applicant's arguments filed 06/18/2026 have been fully considered but they are not persuasive. Remarks on page 7-8 are directed towards amended claim limitations of claims 1, 13 and 14 which were not presented earlier in the manner they are now. This changes the scope of the claim and required further consideration and updated search. Newly cited prior art Streit (2021/0377298) in combination with other previously relied upon prior art Tsai and Vemulapalli; teaches all the claim limitations of claim 1, 13 and 14 as explained in the office action above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PREMAL PATEL whose telephone number is (571)270-5892. The examiner can normally be reached Mon-Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATTHEW EASON can be reached at 571-270-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PREMAL R PATEL/Primary Examiner, Art Unit 2624
Prosecution Insights