Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4, 6-8, 11, 13-14, 16 and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process (concept performed in a human mind, including as observation, evaluation, judgment, opinion, organizing human activity and mathematical concepts and calculations). The claims recite performing liveness detection. This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved .The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally using paper/pencil, solving mathematical problem and no additional features in the claims would preclude them from being performed as such except for the generic computer elements at high level of generality (i.e., processor, memory) .
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claims 1, 5 and 6 are directed to an abstract idea as shown below:
Regarding claims 1, 5 and 6
STEP 1: Do the claims fall within one of the statutory categories?
YES.
Claim(s) 1, 13 and 20 are directed to a method, a computer device and a non-transitory computer readable medium storing instruction, i.e. process, system and manufacture.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?
YES.
The claims are directed toward a mental process and solving mathematical problem (i.e. abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
claims 1, 13 and 20 comprise a mental process that can be practicably performed in the human mind and solving mathematical problem (or generic computers or components configured to perform the process) and, therefore, an abstract idea.
Regarding Claims 1, 13 and 20 (representative claim 1):
A liveness detection , performed by a computer device (generic computing system), comprising:
acquiring a facial image of an object and a reference feature set, the reference feature set including a plurality of reference features conforming to a pre-set feature distribution (data collection activity which is extra solution activity of acquiring face images and refence feature);
extracting an image feature from the facial image (mental process of extracting face feature based on observation and judgement using collected face image data);
performing liveness feature mapping on the plurality of reference features to obtain a plurality of corresponding reference liveness features (mental process using paper pencil and collected refence feature of mapping i.e. registration and matching of collected reference feature on the mathematical function to obtain reference feature of liveness i.e. mathematical problem);
performing, based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain a discrimination result (mental process of discriminating by comparing feature of collected face image feature and obtained reference features by solving mathematical function);
performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result (fitting by matching the distribution of reference features to a model function of distribution of features i.e. mathematical problem solving); and
performing liveness detection on the object based on the discrimination result and the distribution fitting result (mental process of observation and judgement liveness detection based on the mathematical problem solving using paper pencil).
The above limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human intelligence and solving mathematical process. Furthermore limitations, “ acquiring a facial image of an object and a reference feature set, the reference feature set including a plurality of reference features conforming to a pre-set feature distribution (data collection activity which is extra solution activity of acquiring face images and refence feature), extracting an image feature from the facial image (mental process of extracting face feature based on observation and judgement using collected face image data), performing liveness feature mapping on the plurality of reference features to obtain a plurality of corresponding reference liveness features (mental process using paper pencil and collected refence feature of mapping i.e. registration and matching of collected reference feature on the mathematical function to obtain reference feature of liveness i.e. mathematical problem), performing, based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain a discrimination result (mental process of discriminating by comparing feature of collected face image feature and obtained reference features by solving mathematical function), performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result (fitting by matching the distribution of reference features to a model function of distribution of features i.e. mathematical problem solving); and performing liveness detection on the object based on the discrimination result and the distribution fitting result (mental process of observation and judgement liveness detection based on the mathematical problem solving using paper pencil)” are insignificant.
The Examiner notes that under MPEP 2106.04(A) (2) (III), the courts consider a mental process (thinking, human intelligence) that can be performed in the mind/intelligence using a paper and pencil to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[Mental processes and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978).
Furthermore the Examiner also notes that even if you combined the math with the mental process, a combination of abstract ideas don't make a claim eligible. See MPEP 2106.04(II)(A)(2): Because a judicial exception is not eligible subject matter, Bilski, 561 U.S. at 601, 95 USPQ2d at 1005-06 (quoting Chakrabarty, 447 U.S. at 309, 206 USPQ at 197 (1980)), if there are no additional claim elements besides the judicial exception, or if the additional claim elements merely recite another judicial exception, that is insufficient to integrate the judicial exception into a practical application. See, e.g., RecogniCorp, LLC v. Nintendo Co., 855 F.3d 1322, 1327, 122 USPQ2d 1377 (Fed. Cir. 2017) ("Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract").
Other than generic and well-known computer hardware recited in the independent claims 1, 13 and 20 disclosed in the specification, nothing in the claims 1, 8 and 20 elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process using paper/pencil. The liveness detection/discrimination recited in independent claims 1, 13 and 20 is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing hardware is recited as just to automate the mental process of mathematical problem solving(Step 2A, prong 1 Test Abstract idea = Yes).
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
[YES/NO].
The claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim(s) 1, 13 and 20 do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. Claim(s) 1, 13 and 20 recite(s) the limitations of:
A liveness detection , performed by a computer device (generic computing system), comprising:
acquiring a facial image of an object and a reference feature set, the reference feature set including a plurality of reference features conforming to a pre-set feature distribution (data collection activity which is extra solution activity of acquiring face images and refence feature);
extracting an image feature from the facial image (mental process of extracting face feature based on observation and judgement using collected face image data);
performing liveness feature mapping on the plurality of reference features to obtain a plurality of corresponding reference liveness features (mental process using paper pencil and collected refence feature of mapping i.e. registration and matching of collected reference feature on the mathematical function to obtain reference feature of liveness i.e. mathematical problem);
performing, based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain a discrimination result (mental process of discriminating by comparing feature of collected face image feature and obtained reference features by solving mathematical function);
performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result (fitting by matching the distribution of reference features to a model function of distribution of features i.e. mathematical problem solving); and
performing liveness detection on the object based on the discrimination result and the distribution fitting result (mental process of observation and judgement liveness detection based on the mathematical problem solving using paper pencil).
These limitations are recited at a high level of generality (i.e. as a general action or calculation being taken based on the results of the acquiring step) and amounts to mere post solution actions, which is a form of insignificant extra-solution activity without further detail. Further, the claims 1, 13 and 20 are claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Other than generic and well-known computer hardware recited in the independent claims 1, 13 and 20 disclosed in the specification, nothing in the claims 1, 13 and 20 elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process using paper/pencil. The liveness detection/discrimination recited in independent claims 1, 13 and 20 is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing hardware is recited as just to automate the mental process of mathematical problem solving (Step 2A, prong 2 Test Abstract idea = Yes).
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
NO.
The claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
As stated above Other than generic and well-known computer hardware recited in the independent claims 1, 13 and 20 disclosed in the specification, nothing in the claims 1, 13 and 20 elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process using paper/pencil. The liveness detection/discrimination recited in independent claims 1, 13 and 20 is conventional and well known and is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing hardware is recited as just to automate the mental process of mathematical problem solving (
Thus, since Claim(s) 1, 13 and 20 are: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that Claim(s) 1, 13 and 20 are not eligible subject matter under 35 U.S.C 101 (Step 2B, Test Abstract idea = Yes).
Regarding dependent claims 2, 4, 6-8, 11, 14, 16 and 18-19 : the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. Claims 2, 4 6-7, 11, 14. 16 and 18-19 further limit the abstract idea of independent claims 1 and 13. The limitations of these dependent claims full under (mental process including observation and evaluation, and judgement and mathematical problem solving which can be done mentally in the human mind) OR (insignificant pre/post-solution extra activity of generating/gathering data, performing mathematical calculation) OR (generic computers or components configured to perform the process) and the generic machine learning/training model recited in the dependent claims and as disclosed in the specification is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6-7, 11, 13-14, and 18-20 are rejected under 35 USC 103 as being unpatentable over of Wang (From RGB to Depth: Domain Transfer Network for Face Ant-Spoofing, IEEE 1556-602, IEEE TRANSACTION ON INFORMATION FORENSICS AND SECURITY, VOL. 16, 2021, pages 4280-4290, USPTO-892 ).
Regarding claims 1, 13 and 20 Wang disclose a liveness detection method, a computer device and a non-transitory computer readable medium storing instructions (Wang Abstract, page 4280, section I INTROCTION, Figs 1 and 3, page 4285, section Implementation detail), comprising:
acquiring a facial image of an object and a reference feature set, the reference feature set including a plurality of reference features conforming to a pre-set feature distribution (Wang Figs. 3-5, page 4283, section III THE PRPOSED FRAMEWORK acquiring facial image and features extracted by ResNet50 in projected 3D space of RGB face images of Replay-attack and RGB images CASA-FASD case set and depth images of Replay-attack and RGB images CASA-FASD test data set, as shown Fig 4(a)-(d) see description below Fig. 4, i.e. acquiring facial image and reference images extracting face features and reference features and page 4283 left column disclose distribution of face features and reference features and obviously image feature and reference are conforming to preset distribution as shown in Fig. 4);
extracting an image feature from the facial image (Wang Abstract, Figs. 3-5, page 4283, section III THE PRPOSED FRAMEWORK acquiring facial image and features extracted by ResNet50 in projected 3D space of RGB face images of Replay-attack and RGB images CASA-FASD case set and depth images of Replay-attack and RGB images CASA-FASD test data set, as shown Fig 4(a)-(d) see description below Fig. 4, i.e. acquiring facial image and reference images extracting face features and reference features as shown in Fig. 4 i.e. extracting depth features by domain transfer);
performing liveness feature mapping on the plurality of reference features to obtain a plurality of corresponding reference liveness features (Wang Abstract, page 4283, Fig. 4, description below Fig. 4 , shows Visualization of CNN features extracted by ResNet50 in a projected 3D space using PCA. The ResNet50 model was trained on the Replay Attack dataset. We compare the distributions of CNN features extracted from (a) the RGB images of the Replay-Attack test set; (b) the RGB images of the CASIA-FASD test dataset; (c) the depth images of the Replay-Attack test set; and (d) the depth images of the CASIA-FASD test dataset. This obviously corresponds to performing liveness feature mapping on the plurality of reference features to obtain a plurality of corresponding reference liveness features and pages 4283-4284, section System Overview, domain-transfer network to obtain depth features);
performing, based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain a discrimination result (Wang page 4284, right-column shows Algorithm I Domain Transfer show m image pair and discriminator of liveness feature and section Classification below Algorithm I, Wang disclose the classifier aims to decide whether the input face image is a live or spoofing face. As shown in Fig. 3, we utilize multiple convolutional layers, with one additional fully connected layer that converts the feature map into 0 or 1 for binary classification. As aforementioned, the input comes from one part of the proposed DTN branch. Specifically, it uses the feature maps generated by Enet1. Since the feature maps are generated during the domain transfer process, they can be regarded as features of the depth domain. As discussed earlier, compared with the information extracted from the RGB domain, the proposed method can gradually learn the shared features supervised by the depth domain transfer network. We optimize the classifier by minimizing the cross-entropy loss. This obviously corresponds to performing, based on the e plurality of reference liveness features, liveness discrimination on the image feature to obtain a discrimination result of classification);
performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result (Wang Fig. 6, section Visualization and Analysis, page 4288 left column, second- paragraph Wang disclose To verify the discrimination of the extracted CNN features for face anti-spoofing, we use PCA to project the CNN features used for the classifier into a 3D subspace, as shown in Fig. 6. In the left sub-figure, we plot the feature distributions of the live and spoofing faces of the CASIA-FASD test set, using the model trained on the CASIA-FASD training set. For the right sub-figure, we plot the feature distributions of the live and spoofing faces of the Replay-Attack test set, using the model trained on the CASIA-FASD training set as well. We can see that the distributions of live and spoofing faces are very similar between the two sub-figures. Although the distributions in the right sub-figure are slightly shifted from the left ones, we can still easily distinguish real and spoofing faces. The shift is probably caused by the facial appearance variations in illumination and image resolution. As compared with Fig. 4-(a) and Fig. 4-(b), the proposed method has improved the discrimination of CNN features significantly. The proposed DTN architecture converts RGB faces to the depth domain, which leads to similar feature distributions of real and spoofing faces among different datasets thus improves the generalization capability of the proposed method. This obviously corresponds to performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result); and
performing liveness detection on the object based on the discrimination result and the distribution fitting result (Wang page 4284, right-column shows Algorithm I Domain Transfer show m image pair and discriminator of liveness feature and section Classification below Algorithm I, Wang disclose the classifier aims to decide whether the input face image is a live or spoofing face. As shown in Fig. 3, we utilize multiple convolutional layers, with one additional fully connected layer that converts the feature map into 0 or 1 for binary classification. As aforementioned, the input comes from one part of the proposed DTN branch. Specifically, it uses the feature maps generated by Enet1. Since the feature maps are generated during the domain transfer process, they can be regarded as features of the depth domain. As discussed earlier, compared with the information extracted from the RGB domain, the proposed method can gradually learn the shared features supervised by the depth domain transfer network. We optimize the classifier by minimizing the cross-entropy loss and Wang Fig. 6, section Visualization and Analysis, page 4288 left column, second- paragraph Wang disclose To verify the discrimination of the extracted CNN features for face anti-spoofing, we use PCA to project the CNN features used for the classifier into a 3D subspace, as shown in Fig. 6. In the left sub-figure, we plot the feature distributions of the live and spoofing faces of the CASIA-FASD test set, using the model trained on the CASIA-FASD training set. For the right sub-figure, we plot the feature distributions of the live and spoofing faces of the Replay-Attack test set, using the model trained on the CASIA-FASD training set as well. We can see that the distributions of live and spoofing faces are very similar between the two sub-figures. Although the distributions in the right sub-figure are slightly shifted from the left ones, we can still easily distinguish real and spoofing faces. The shift is probably caused by the facial appearance variations in illumination and image resolution. As compared with Fig. 4-(a) and Fig. 4-(b), the proposed method has improved the discrimination of CNN features significantly. The proposed DTN architecture converts RGB faces to the depth domain, which leads to similar feature distributions of real and spoofing faces among different datasets thus improves the generalization capability of the proposed method. This obviously corresponds to performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result. All this obviously corresponds to performing liveness detection on the object based on the discrimination result and the distribution fitting result).
Therefore it would have been obvious to one of ordinary skill in the art, before the claimed invention was filed to acquire facial image of an object and a reference feature set, the reference feature set including a plurality of reference features conforming to a pre-set feature distribution, extract an image feature from the facial image, perform liveness feature mapping on the plurality of reference features to obtain a plurality of corresponding reference liveness features, perform based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain a discrimination result, performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result and perform liveness detection on the object based on the discrimination result and the distribution fitting result as shown by Wang because such a system/process provides automate system to detect face spoofing attacks as stated by Wang in Abstract and INTRODUCTION.
Regarding claims 2 and 14 Wang disclose performing gradient operation on the plurality of reference features to obtain gradient information of each reference feature; converging, based on the image feature, the gradient information of each reference feature to obtain converged information of each reference feature; and determining the distribution fitting result based on the converged information of each reference feature (Wang page 4284, right-column shows Algorithm I Domain Transfer show m image pair and discriminator of liveness feature and section Classification below Algorithm I, Wang disclose the classifier aims to decide whether the input face image is a live or spoofing face. As shown in Fig. 3, we utilize multiple convolutional layers, with one additional fully connected layer that converts the feature map into 0 or 1 for binary classification. As aforementioned, the input comes from one part of the proposed DTN branch. Specifically, it uses the feature maps generated by Enet1. Since the feature maps are generated during the domain transfer process, they can be regarded as features of the depth domain. As discussed earlier, compared with the information extracted from the RGB domain, the proposed method can gradually learn the shared features supervised by the depth domain transfer network. We optimize the classifier by minimizing the cross-entropy loss and Wang Fig. 6, section Visualization and Analysis, page 4288 left column, second- paragraph Wang disclose to verify the discrimination (classification) of the extracted CNN features for face anti-spoofing, we use PCA to project the CNN features used for the classifier into a 3D subspace, as shown in Fig. 6. In the left sub-figure, we plot the feature distributions of the live and spoofing faces of the CASIA-FASD test set, using the model trained on the CASIA-FASD training set. For the right sub-figure, we plot the feature distributions of the live and spoofing faces of the Replay-Attack test set, using the model trained on the CASIA-FASD training set as well. We can see that the distributions of live and spoofing faces are very similar between the two sub-figures. Although the distributions in the right sub-figure are slightly shifted from the left ones, we can still easily distinguish real and spoofing faces. The shift is probably caused by the facial appearance variations in illumination and image resolution. As compared with Fig. 4-(a) and Fig. 4-(b), the proposed method has improved the discrimination of CNN features significantly. The proposed DTN architecture converts RGB faces to the depth domain, which leads to similar feature distributions of real and spoofing faces among different datasets thus improves the generalization capability of the proposed method. This obviously corresponds to performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result. All this obviously corresponds to performing gradient operation on the plurality of reference features to obtain gradient information of each reference feature; converging, based on the image feature, the gradient information of each reference feature to obtain converged information of each reference feature; and determining the distribution fitting result based on the converged information of each reference feature).
Regarding claims 6 and 18 Wang disclose performing full connection on the plurality of reference features to obtain fully connected information corresponding to the plurality of reference features; classifying, based on the fully connected information, the plurality of reference features to obtain classification results corresponding to the plurality of reference features (Wang page 4284, right-column shows Algorithm I Domain Transfer show m image pair and discriminator of liveness feature and section Classification below Algorithm I, Wang disclose the classifier aims to decide whether the input face image is a live or spoofing face. As shown in Fig. 3, we utilize multiple convolutional layers, with one additional fully connected layer that converts the feature map into 0 or 1 for binary classification. As aforementioned, the input comes from one part of the proposed DTN branch. Specifically, it uses the feature maps generated by Enet1. Since the feature maps are generated during the domain transfer process, they can be regarded as features of the depth domain. As discussed earlier, compared with the information extracted from the RGB domain, the proposed method can gradually learn the shared features supervised by the depth domain transfer network. We optimize the classifier by minimizing the cross-entropy loss); and
mapping, based on the classification results, the plurality of reference features into a pre-set liveness feature space to obtain the plurality of reference liveness features (Wang Fig. 6, section Visualization and Analysis, page 4288 left column, second- paragraph Wang disclose to verify the discrimination (classification) of the extracted CNN features for face anti-spoofing, we use PCA to project the CNN features used for the classifier into a 3D subspace, as shown in Fig. 6. In the left sub-figure, we plot the feature distributions of the live and spoofing faces of the CASIA-FASD test set, using the model trained on the CASIA-FASD training set. For the right sub-figure, we plot the feature distributions of the live and spoofing faces of the Replay-Attack test set, using the model trained on the CASIA-FASD training set as well. We can see that the distributions of live and spoofing faces are very similar between the two sub-figures. Although the distributions in the right sub-figure are slightly shifted from the left ones, we can still easily distinguish real and spoofing faces. The shift is probably caused by the facial appearance variations in illumination and image resolution. As compared with Fig. 4-(a) and Fig. 4-(b), the proposed method has improved the discrimination of CNN features significantly. The proposed DTN architecture converts RGB faces to the depth domain, which leads to similar feature distributions of real and spoofing faces among different datasets thus improves the generalization capability of the proposed method. This obviously corresponds to performing, based on the image feature, distribution fitting on the plurality of reference features to obtain a distribution fitting result. All this corresponds to mapping, based on the classification results, the plurality of reference features into a pre-set liveness feature space to obtain the plurality of reference liveness features).
Regarding claims 7 and 19 Wang disclose extracting the image feature from the facial image includes extracting the image feature from the facial image by using a pre-set feature extraction model (Wang Abstract, Figs. 3-5, page 4283, section III THE PRPOSED FRAMEWORK acquiring facial image and features extracted by ResNet50 in projected 3D space of RGB face images of Replay-attack and RGB images CASA-FASD case set and depth images of Replay-attack and RGB images CASA-FASD test data set, as shown Fig 4(a)-(d) see description below Fig. 4, i.e. acquiring facial image and reference images extracting face features and reference features as shown in Fig. 4 i.e. extracting depth features by domain transfer. This corresponds to extracting the image feature from the facial image includes extracting the image feature from the facial image by using a pre-set feature extraction model);
performing liveness feature mapping on the plurality of reference features to obtain the plurality of corresponding reference liveness features includes performing liveness feature mapping on the plurality of reference features by using a pre-set feature generation model to obtain the plurality of corresponding reference liveness features ((Wang Abstract, page 4283, Fig. 4, description below Fig. 4 , shows Visualization of CNN features extracted by ResNet50 in a projected 3D space using PCA. The ResNet50 model was trained on the Replay Attack dataset. We compare the distributions of CNN features extracted from (a) the RGB images of the Replay-Attack test set; (b) the RGB images of the CASIA-FASD test dataset; (c) the depth images of the Replay-Attack test set; and (d) the depth images of the CASIA-FASD test dataset.);
performing, based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain the discrimination result includes performing, based on the plurality of reference liveness features, liveness discrimination on the image feature by using a pre-set discrimination module to obtain the discrimination result (Wang page 4284, right-column shows Algorithm I Domain Transfer show m image pair and discriminator of liveness feature and section Classification below Algorithm I, Wang disclose the classifier aims to decide whether the input face image is a live or spoofing face. As shown in Fig. 3, we utilize multiple convolutional layers, with one additional fully connected layer that converts the feature map into 0 or 1 for binary classification. As aforementioned, the input comes from one part of the proposed DTN branch. Specifically, it uses the feature maps generated by Enet1. Since the feature maps are generated during the domain transfer process, they can be regarded as features of the depth domain. As discussed earlier, compared with the information extracted from the RGB domain, the proposed method can gradually learn the shared features supervised by the depth domain transfer network. We optimize the classifier by minimizing the cross-entropy loss. This obviously corresponds to performing, based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain the discrimination result includes performing, based on the plurality of reference liveness features, liveness discrimination on the image feature by using a pre-set discrimination module to obtain the discrimination result ); and
performing, based on the image feature, distribution fitting on the plurality of reference features to obtain the distribution fitting result includes performing, based on the image feature, distribution fitting on the plurality of reference features by using a pre-set distribution fitting module to obtain the distribution fitting result (Wang Fig. 6, section Visualization and Analysis, page 4288 left column, second- paragraph Wang disclose To verify the discrimination of the extracted CNN features for face anti-spoofing, we use PCA to project the CNN features used for the classifier into a 3D subspace, as shown in Fig. 6. In the left sub-figure, we plot the feature distributions of the live and spoofing faces of the CASIA-FASD test set, using the model trained on the CASIA-FASD training set. For the right sub-figure, we plot the feature distributions of the live and spoofing faces of the Replay-Attack test set, using the model trained on the CASIA-FASD training set as well. We can see that the distributions of live and spoofing faces are very similar between the two sub-figures. Although the distributions in the right sub-figure are slightly shifted from the left ones, we can still easily distinguish real and spoofing faces. The shift is probably caused by the facial appearance variations in illumination and image resolution. As compared with Fig. 4-(a) and Fig. 4-(b), the proposed method has improved the discrimination of CNN features significantly. The proposed DTN architecture converts RGB faces to the depth domain, which leads to similar feature distributions of real and spoofing faces among different datasets thus improves the generalization capability of the proposed method. This obviously corresponds to performing, based on the plurality of reference liveness features, liveness discrimination on the image feature to obtain the discrimination result includes performing, based on the plurality of reference liveness features, liveness discrimination on the image feature by using a pre-set discrimination module to obtain the discrimination result).
Regarding claim 11 Wang disclose before acquiring the facial image of the object: receiving a liveness detection trigger instruction (Wang Abstract, Figs 1 and 3, page 4284 , right-column, section 2) Classification of input image is live/spoofing face. In the computing system of Wang it would be obvious to receive a liveness detection trigger instruction to classify the received face image as live or spoofing face);
collecting a video including a face of the object based on the liveness detection trigger instruction (Wang Figs 1 and 3, page 4283 right-column last paragraph disclose collecting images sequence including face image and Wang page 4281, right-column, section Traditional Method i.e., prior art Wang disclose collecting video lines 20-23 to enhance the prior empirical knowledge, Komulainen et al. [24] concatenated the features of sequence frames to improve the performance);
converting the video into at least one image (Wang Figs 1 and 3, page 4283 right-column last paragraph disclose collecting images sequence including face image and Wang page 4281, right-column, section Traditional Method i.e., prior art Wang disclose collecting video frames lines 20-23 to enhance the prior empirical knowledge, Komulainen et al. [24] concatenated the features of sequence frames to improve the performance. By concatenating the features of sequence frames it would be obvious to obtain representative image frame );
performing facial detection on the at least one image to obtain a detection result; and determining the facial image from the at least one image based on the detection result (Wang, Abstract, Fig. 1 and 3 page 4284 , right-column, section 2) Classification of input image is live/spoofing face which obviously corresponds to performing facial detection on the at least one image to obtain a detection result ).
Claim Objection
Claims 3, 5, 9-10, 12, 15 and 17 are objected as being dependent on the rejected base claim but would be allowable over the prior art of record if rewritten in the independent form including limitations of the base claim and any intervening claims.
Communication
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHRAT I SHERALI whose telephone number is (571)272-7398. The examiner can normally be reached Monday-Friday 8:00AM -5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ISHRAT I. SHERALI
Examiner
Art Unit 2667
/ISHRAT I SHERALI/Primary Examiner, Art Unit 2667