DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-3, and 8 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the comparison result" in line 9 of Claim 1. There is insufficient antecedent basis for this limitation in the claim. The examiner suggests rewriting Claim 1 to recite “comparing the first three-dimensional depth image with a pre-generated second three- dimensional depth image corresponding to the face image to obtain a comparison result; and adjusting the anti-spoofing analysis result based on the comparison result”.
Claim 2 recites the limitation "the recognition result" in line 4 of Claim 2. There is insufficient antecedent basis for this limitation in the claim. The examiner suggests rewriting Claim 1 to recite “performing face recognition on the face image and obtaining a recognition result, and drawing a first box in the face image according to the recognition result”.
Claim 3 recites the limitation "the short side” in line 4 of Claim 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites the limitation "the sigmoid function" in line 4. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 7, 10, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Xu (SG 10202002506Y), hereinafter Xu, in view of Sun et al. (CN 111507131A), hereinafter Sun, and further in view of Hong (CN 108388889A), hereinafter Hong.
As to Claim 1, Xu teaches a face anti-spoofing method (see paragraph [006], “Embodiments seek to provide a user authentication method which involves a combination of different authentication methods such as computing face similarity”), comprising
extracting features from a face image (see paragraph [0036], “The step of generating the third score (S3) may comprise extracting first features (F1) corresponding to the first human face image using a trained CNN”)
calculating an anti-spoofing analysis result of the face image in a predetermined manner based on the features of the face image (see paragraph [0035], “first score (S1) associated with the user authentication image is generated using a trained image spoof model”).
Xu fails to teach performing a feature visualization process on the features of the face image to obtain a first three-dimensional depth image. However, Sun teaches that depth information can be predicted from an image using a neural network (see paragraph [0088], “In some embodiments, in step S12, the image to be detected may be subjected to depth prediction processing by a depth prediction network, where the depth prediction network may be a neural network such as a BP neural network, a recurrent neural network, or a convolutional neural network, and the present disclosure does not limit the type of the depth prediction network”). Sun further teaches that this depth information can be predicted based on features extracted from the image (see paragraph [0089], “In an example, the target object of the image to be detected is a person, or a picture such as a photo of the person, and the depth prediction network may predict depth information of the target object according to feature data extracted from the image to be detected, where a type of the extracted feature data may be learned in a training process of the depth prediction network, which is not limited in this disclosure.”
Xu is combinable with Sun because both are from analogous fields of user authentication through image analysis. Thus, it would have been obvious to one of ordinary skill in the art to combine the user authentication method taught by Xu with the three dimensional depth image generation taught by Sun. The motivation for doing so would be to be to better identify whether a face image is from a living person. Sun teaches in [0002], “it is necessary to determine whether a face image in front of a camera comes from a real person by means of living body detection so as to improve the safety of face recognition. In the related art, a target object may be detected in vivo in combination with a depth map”.
Xu fails to teach that comparing the first three-dimensional depth image with a pre-generated second three- dimensional depth image corresponding to the face image. Xu teaches comparing two two-dimensional facial images in order to obtain another authentication score (see paragraph [007]). However, Sun teaches that a generated three dimensional depth image can be compared to another generated depth image corresponding to the image (see paragraph [0089], “depth values of a plurality of pixel points in the image to be detected corresponding to the depth image and depth predicted values of the plurality of pixel points, determining depth difference information of the plurality of pixel points; and determining a living body detection result according to the depth difference information”). Thus, it would have been obvious to combine the depth image comparison taught by Sun with the user authentication method taught by Sun. The motivation for doing so would
Xu teaches adjusting the anti-spoofing analysis result based on a comparison result (see paragraph [007], “According to one embodiment, there is provided a user authentication method, comprising: extracting, from a user enrollment image comprising a first human face image and a first background image, the first human face image and the first background image; extracting, from a user authentication image comprising a second human face image and a second background image, the second human face image and the second background image; generating, using trained image spoof model, a first score (S1) associated with the user authentication image; generating a second score (S2) corresponding to a similarity between the first background image and the second background image; generating a third score (S3) corresponding to a similarity between the first human face image and the second human face image; and authenticating the user based on the first score (S1), the second score (S2) and the third score (S3)”).
Xu fails to explicitly teach that the face anti-spoofing method can be used to determine liveness. However, Sun teaches determining whether or not the face image is from a living body (see paragraph [0010] “In this way, the depth difference information can be used to determine the result of the living body detection, and the robustness of the living body detection can be improved.”)
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the user authentication and scoring method taught by Xu, with the three dimensional image generation and comparison taught by Sun. The motivation for doing so would be to improve the robustness of living body detection, as taught by Sun in paragraph [0010].
Sun fails to explicitly teach that the depth images generated are three dimensional. However, Hong teaches a method in which a three-dimensional model of a face can be created from a planar image, and then be used to extract depth information (see paragraph [0053], “the execution body may input the face plane image into a pre-built 3D Morphable Model (3DMM), so that the average face and face in the 3D deformation model can be analyzed…The three-dimensional face model generated here can approximate the real face structure of the target object. Then, according to the three-dimensional face model, the execution body can extract the face depth data of the target object in the preset posture, thereby generating simulated face depth data”). Hong further teaches that this constructed depth data e can be compared to previously generated depth data (see paragraph [0056], “the execution body may also input the simulated face depth data and the face depth image into the pre-trained network model, thereby performing matching analysis on the two through the network model. The network model can be used to perform matching analysis on the simulated face depth data and the face depth image having the same posture, and generate a matching analysis result.” Thus, it would have been obvious to combine the three-dimensional model generation taught by Hong with the teachings of Xu and Sun. The motivation for doing would be to enhance spoof detection by allowing comparison of artifacts caused by three-dimensional features. Hong teaches in paragraph [0060], “It can be understood that since the real face has a three-dimensional human face structure, when the face image is collected from different angles, the occlusion area, the shadow area, and the like formed in the image may be different, especially the nose portion. Therefore, the simulated face depth data can reflect the three-dimensional structure of the face. The simulated face depth data and the face depth image are both in the same preset posture, so it is also possible to perform matching analysis on the contour regions, occlusion positions, shadow positions, and areas of the same feature, thereby determining the acquired face depth”. Thus, it would have been obvious to combine the teachings of Xu, Sun, and Hong in order to obtain the invention as claimed in Claim 1.
As to Claim 7, Xu fails to teach calculating a difference between the first three-dimensional depth image and the second three-dimensional depth image and adjusting the anti-spoofing analysis result based on the comparison result comprises: setting a weight based on the difference between the first three-dimensional depth image and the second three-dimensional depth image, and adjusting the anti-spoofing analysis result.
However, Sun teaches that a network trained to calculate a difference between a first depth image and second depth image (see paragraph [0089]),
and adjusting network parameters based on the difference between the first depth image and the second depth image (see paragraph [0093], “In some embodiments, the network loss of the depth prediction network may be determined from a depth prediction map of a sample image and a sample depth image output by the depth prediction network” and paragraph [0094]“In some embodiments, network parameters of the deep prediction network may be adjusted based on network losses, in an example, the network parameters may be adjusted in a direction that minimizes network losses”, where a network parameter can include weight). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Sun with the teachings of Xu and Hong. The motivation for doing so would be to increase the reliability of the anti-spoofing network by minimizing network losses as taught by Sun. Thus, it would have been obvious to combine the teachings of Xu, Sun and Hong in order to obtain the invention as claimed in Claim 1.
As to Claim 10, Claim 10, claims one or more computer-readable storage media comprising a plurality of computer readable media instructions stored thereon (see Xu, paragraph [0045]), the instructions being adapted to be loaded and run by a processor (see Xu, Fig. 3, processor 304) to perform the same operations disclosed in Claim 1. Therefore, the rejection and rationale are similar to that of Claim 1.
As to Claim 16, Claim 16, claims in a device comprising a memory (see Xu, Fig. 3, memory 308) and a processor (see Xu, Fig. 3, processor 304) , the memory storing one or more instructions that, once executed by the processor, cause the processor to perform the same operations claimed in Claim 1. Therefore, the rejection and rationale are similar to that of Claim 1.
Claims 2-3, 11-12, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (SG 10202002506Y), hereinafter Xu, in view of Sun et al. (CN 111507131A), hereinafter Sun, further in view of Hong (CN 108388889A), hereinafter Hong, and further in view of Tang (CN 112348069A), hereinafter Tang.
As to Claim 2, Xu in view of Sun teaches method according to Claim 1, before extracting features from the face image, the method further comprises,
performing face recognition on the face image and drawing a first box in the face image according to the recognition result (see Xu, paragraph [0032], “The step of extracting the first human face image 202 and the first background image 204 from the user enrollment image may comprise applying a face detection method to the user enrollment image to generate a face bounding box 206 that demarcates the first human face image 202 such that an area of the user enrollment image within the face bounding box 206”).
Xu further teaches extracting features from the face detection box (see paragraph [0036], “The step of generating the third score (S3) may comprise extracting first features (F1) corresponding to the first human face image using a trained CNN”, where this is done after the first face is extracted by using the bounding box)
Xu in view of Sun fail to teach processing the first box to obtain a second box according to a predetermined expansion ratio. However, Tang teaches expanding a first box to obtain a second box (paragraph [0012], "the image expansion module is used for expanding the minimum bounding box outwards to obtain a training image set for training a target model, wherein the target model is a model for estimating hand gestures").
Tan further teaches extracting keypoints from within the expanded box (see paragraph [0084], “As shown in fig. 7, the hand image is extracted according to the expanded square, and the coordinates of the labeled key points of each hand are recalculated,”)
Thus, it would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Tang with the teachings of Xu and Sun. The motivation for doing would be to increase the field captured by the bounding box, and thus increase the spoofing detection model’s robustness. Tang teaches in paragraph [0098], “the Receptive Field (Receptive Field) of the model to the data is increased, the trained model is stronger in robustness”. Thus, it would have been obvious to combine the teachings of Tang with the teachings of Xu, Sun, and Hong to obtain the invention as claimed in Claim 2.
As to Claim 3, Xu in view of Sun and Hong fails to teach generating a square third box based on the short side of the first box; and enlarging the third box according to the expansion ratio to obtain the second box. However, Tang teaches generating a square box based on a side of a box, and then enlarging the box to obtain another box (see paragraph [0010], “Determine each set of extended parameters of the minimum bounding box, where each set of extended parameters includes taking the center point of the minimum bounding box as a reference position , and the length of the long side of the minimum bounding box as the reference length. The first multiple of the expansion in the first direction of the long side of the minimum bounding box, the second multiple of the expansion in the second direction of the long side of the minimum bounding box, the first multiple of the short side of the minimum bounding box A third multiple of expansion in three directions, and a fourth multiple of expansion in the fourth direction of the short side of the minimum bounding box”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the bounding box expansion method taught by Tang and combine with the teachings of Xu, Sun, and Hong. The motivation for doing would be to increase the field captured by the bounding box, and thus increase the training model’s robustness. Tang teaches in paragraph [0098], “the Receptive Field (Receptive Field) of the model to the data is increased, the trained model is stronger in robustness”.
As to Claim 11, Claim 11 claims the same limitation as claimed in Claim 2 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 2.
As to Claim 12, Claim 12 claims the same limitation as claimed in Claim 3 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 3.
As to Claim 17, Claim 17 claims the same limitation as claimed in Claim 2 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 2.
As to Claim 18, Claim 18 claims the same limitation as claimed in Claim 3 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 3.
Claim 4, 13, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (SG 10202002506Y), hereinafter Xu, in view of Sun et al. (CN 111507131A), hereinafter Sun, and further in view of Tang (CN112348069A), and further in view of Pegg (US Pub No 20120069007), hereinafter Pegg.
As to Claim 4, Xu in view of Sun, Hong, and Tang fails to teach generating the second three-dimensional depth image based on a region of the face image located within the second box. Sun teaches that the second generated image is generated by using a depth sensor. However, Pegg teaches a method in which a bounding box is used to identify a face, and the data in the box is used to create a depth model (see paragraph [0049], “The enhancement can be seen in the example images shown in FIG. 6. FIG. 6 a shows a 2D image with bounding boxes from a face detector superimposed. It can be seen that the face detector has identified the face of the subject in the image. Using a 2D to 3D conversion process the depth map as shown in FIG. 6 b is determined”).
Pegg is combinable with Xu, Sun, Hong, and Tang since all four are from the analogous field of image analysis. Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the 3D depth generation method taught by Pegg with the teachings of Xu, Sun, Hong, and Tang. The motivation for doing so would be to enhance 3D model generation by using a bounding box to only include pertinent objects. Pegg teaches in paragraph [0031], “The current invention describes a method of targeting faces as meaningful objects. This information is used as a basis for further analyzing of an image and enhancing the associated depth map, therefore leading to improved real-time 2D to 3D conversion.” Thus, it would have been obvious to combine the teachings of Pegg with the teachings of Xu, Sun, Hong, and Tang in order to obtain the invention as claimed in Claim 4.
As to Claim 13, Claim 13 claims the same limitation as claimed in Claim 4 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 4.
As to Claim 19, Claim 19 claims the same limitation as claimed in Claim 4 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 4.
Claims 5-6, 14-15, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (SG 10202002506Y), hereinafter Xu, in view of Sun et al. (CN 111507131A), hereinafter Sun, and further in view of Tang (CN112348069A), and further in view of Pegg (US Pub No 20120069007), hereinafter Ye, and further in view of Cusano et al. (WO 2008119368), hereinafter Cusano.
As to Claim 5, Xu in view of Sun, Hong, Tang, and Pegg fails to teach generating the second three-dimensional depth image based on a region of the face image located within the second box, calculating a second position of the binary rectangular mask in the second three- dimensional depth image based on the first position of the first box in the face image; and processing the second three-dimensional depth image with the binary rectangular mask based on the second position. However, Cusano teaches that the position of a binary mask can be calculated with respect to a bounding box, and that an image can be processed with the binary mask (see page 14, lines 1-4, “A binary mask 102 is thus created in step 2a, the dimensions of which are preferably equal to those of the rescaled input image 100. In mask 102, all pixels belonging to a region classified as a face region (the bounding box BB) have the value of "1" (which corresponds to white pixels), while all pixels outside the face regions have the value of "0" (which corresponds to black pixels)”). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Harvill with reteaching of Xu in view of Sun, Hong, Tang and Pegg. The motivation for doing so would have been to isolate the face from the background, as taught in page 14. Thus, by using a binary mask, noise caused by the environment can be removed, and lead to better 3D depth image reconstructions. Thus, it would have been obvious to combine the teachings of Cusano with the teachings of Xu, Sun, Hong, Tang and Pegg in order to obtain the invention as claimed in Claim 5.
As to Claim 6, Xu in view of Sun, Hong, Tang and Pegg, and Cusano teaches setting area covered by the binary rectangular mask in the second three- dimensional depth image to 1, and setting area uncovered by the binary rectangular mask in the second three-dimensional depth image to 0 (see page 14, lines 1-4, “A binary mask 102 is thus created in step 2a, the dimensions of which are preferably equal to those of the rescaled input image 100. In mask 102, all pixels belonging to a region classified as a face region (the bounding box BB) have the value of "1" (which corresponds to white pixels), while all pixels outside the face regions have the value of "0" (which corresponds to black pixels)”).
As to Claim 14, Claim 14 claims the same limitation as claimed in Claim 5 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 5.
As to Claim 15, Claim 10 claims the same limitation as claimed in Claim 6 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 6.
As to Claim 20, Claim 20 claims the same limitation as claimed in Claim 5 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 5.
As to Claim 21, Claim 18 claims the same limitation as claimed in Claim 6 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale are similar to that of Claim 6.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Xu et al. (CN 112613345A), hereinafter Xu, in view of Sun et al. (CN 111507131A), hereinafter Sun, and further in view of Hong, (CN 108388889A), hereinafter Hong, and further in view of Rowe et al. (US Pub No 20210110018), hereinafter Rowe.
As to Claim 8, Xu in view of Sun fails to teach calculating the anti-spoofing analysis result of the face image in the predetermined manner comprises: using the sigmoid function to calculate the anti-spoofing analysis result.
However, Rowe teaches a spoofing detection method (see [0004]) which uses sigmoid to determine a spoofing result (see paragraph [0100], “A nonlinear operation may then be applied to the output planes of the convolution. The nonlinearity may be a sigmoid”, and see paragraph [0101], “The final layer of the network may have a small number of nodes that correspond to a classification result. For example, in the case of spoof detection, the network may culminate in a single node that produces values that range from 0.0 to 1.0 (e.g., sigmoid) where 0.0 is a strong indicator of a genuine person and 1.0 is a strong indicator of a spoof.”)
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the sigmoid function taught by Rowe with the anti-spoofing method and depth image comparison taught by Xu, Sun, and Hong. The motivation for doing so would be to would be to easily quantify the output of the anti-spoofing network, which can then be used to determine if a spoofing attach has occurred. Rowe teaches in paragraph [0103], “As shown at 512, the neural network 510 generates a spoof-detection result 514, which represents a determination made by the neural network 510 as to whether the biometric-authentication subject…is a real face or a spoof specimen”. Thus, it would have been obvious to combine the teachings of Rowe with the teachings of Xu and Sun in order to obtain the invention as claimed in Claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOUMYA THOMAS whose telephone number is (571)272-8639. The examiner can normally be reached M-F 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.T./ Examiner, Art Unit 2664
/JENNIFER MEHMOOD/ Supervisory Patent Examiner, Art Unit 2664