DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Objections
Claims 1, 13 and 14 are objected to because of the following informalities: the term “organ” in the phrase “verification organ” should be changed to a term with a plain meaning that more accurately reflects the described embodiments. The description at paragraph 58, for examples, lists “an eye, a mouth, a head, or the like” as examples of a “verification organ”. However, these are more accurately described as body parts or in the case of a mouth, a body cavity surrounding by body parts, e.g., lips. Body parts that are specifically labeled as “organs” typically include the heart, lungs, stomach, liver, kidney, skin, spleen, uterus, and so forth. Using the term “organ” may confuse a reader into misunderstanding Applicant’s claimed invention as being directed to images of internal body structures like x-ray images, which does not appear to be the intended meaning of images that contain “verification organs”.
Claim 16 is objected to because of the following informalities: “for each frame of reference image” in line 4 should be changed to “for each frame of the reference images” to be consistent with the preamble.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
This application includes a claim limitation that does not use the word “means,” but is nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitation uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation is:
“storage apparatus configured to store at least one program” in claim 13.
Because this claim limitation is being interpreted under 35 U.S.C. 112(f), it is being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this limitation interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation to avoid it being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation recites sufficient structure to perform the claimed function so as to avoid it being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4, 8-14, 16-18 and 22 are rejected under 35 U.S.C. 101 because the claimed inventions are directed to a judicial exception without significantly more.
[Claim 1] A method for image processing, comprising:
acquiring an image sequence of a target object in response to a live body detection trigger operation, wherein the image sequence comprises an image to be detected;
determining, for the image to be detected, multiple frames of reference images associated with the image to be detected in acquisition time from the image sequence;
determining a risk factor of the image to be detected based on a region to be detected in the multiple frames of reference images, wherein the region to be detected at least comprises a region where a verification organ performing a preset verification action is located, and the risk factor indicates whether the target object is a live body; and
determining a live body detection result corresponding to the target object based on the risk factor of the image to be detected.
[claim 13] An electronic device, comprising: at least one processor; a storage apparatus configured to store at least one program; wherein the at least one program, when executed by the at least one processor, causes the at least one processor to [perform limitations (a) – (d)].
[claim 14] A non-transitory storage medium containing computer- executable instructions, wherein the computer-executable instructions, when executed by a computer processor, causing the computer processor to: [perform limitations (a) – (d)].
Claim Interpretation
Under the broadest reasonable interpretation, the terms of the claims are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP 2111. Based on the plain meaning of the words in the independent claims, the broadest reasonable interpretation of the independent claims is a liveness detection method with steps of limitations (a) - (d), a device having a processor and storage (i.e., a general purpose computer) that stores a program to be executed by the processor to perform the method of limitations (a) - (d), and a non-transitory computer containing instructions executed by a processor to perform the method of limitations (a) - (d).
Regarding limitation (a), the claims do not put any substantial limits on the image acquisition device that originally created the image sequence. Merely “acquiring” an image sequence does not require an image acquisition device, such as a digital still image camera or a video camera. Stored images can be acquired from a memory without the direct involvement of a camera.
Regarding limitation (b), the claims do not put any substantial limits on the criteria or method by which a specific set of “multiple frames of reference images” is determined.
Regarding limitation (c), the claims do not put any substantial limits on the “risk factor” beyond being “based on” a region in the reference images containing an organ used for liveness detection. A verification action being “preset” merely means that it is known ahead of time or predetermined and does not require involvement of a computer. For example, a person may view a live video to assess whether the face therein is a real face by looking for expected patterns indicative of a real face like a blinking pattern, a range of motion, and so forth.
Regarding limitation (d), the claims do not put any substantial limits on how the “risk factor” is actually used to determine liveness beyond it “corresponding to the target object”.
Step 1: do the claims fall within any statutory category?
Claim 1 recites a series of steps and therefore, is a process. Claim 13 recites a device comprising a storage and a processor and therefore, is a machine. Claim 14 recites a non-transitory computer readable storage medium. The disclosure gives magnetic media such as RAM and ROM as non-limiting examples of a computer-readable medium. See par. 156. The broadest reasonable interpretation of claim 14 covers only statutory embodiments of a computer-readable storage medium and not a transitory signal. A non-transitory computer readable storage medium falls within the “manufacture” category of invention. See MPEP 2106.03. (Step 1: YES).
Step 2A, Prong One: do the claims recite a judicial exception?
The subject matter of the independent claims generally describes liveness detection in an image sequence. The “verification organ” is described at a high level of generality and is so broadly recited that it amounts to an attempt to claim a liveness detection method from any anatomical region depicted in reference images. Even though phrases like “image sequence”, “electronic device”, “processor”, “storage”, “program” and “instructions” evoke a computer environment where the method is performed, the bulk of the claimed subject matter is directed to the evaluation and subsequent decisions made with respect to the image sequence.
The method represented by limitations (a) - (d) is embodied in each independent claim, where claim 1 is the method itself and each step of the method is reasonably interpreted as a process of human review in a computer environment, as in the case of the generic computer components of claims 13 and 14. For example, a person is capable of viewing a series of images displayed on a computer screen and making an assessment. The recitation of physical computer-related structures in the claims, e.g., “processor”, does not negate the mental nature of these process limitations because the claims merely use the structural components as tools to carry out the aims and decisions of the person. See MPEP 2106.04(a)(2), subsection III.C.
Under the broadest reasonable interpretation, the claims describe the observations, evaluations, judgments, and opinions of a person assessing appearances changes of an anatomical region over time via a series of images, which falls within the mental process grouping of abstract ideas. See MPEP 2106.04(a)(2), subsection III. (Step 2A, Prong One: YES).
Step 2A, Prong Two: Do the claims as a whole integrate the recited judicial exception into a practical application of the exception?
Claims 1, 13 and 14 recite additional elements: (i) “a live body detection trigger operation”, (ii) “processor” specifically in claims 13 and 14, and (iii) “A non-transitory computer readable storage medium containing instructions” specifically in claim 14. Individually, the additional element (i) provides no detail as to the nature of the “operation”, as it could be a computer-based operation like an automated audio prompt telling a user to make a certain facial gesture, an on-screen visual prompt, or merely a condition that a person or a computer waits to encounter before proceeding to more image processing. This additional element only recites the idea of a trigger or condition that initiates or signals proceeding to liveness detection. The additional elements (ii) and (iii) describe generic computer components and/or operations thereof (e.g., using a pre-built computer or a commercially available smartphone) recited at a high level of generality and do not amount to any of the relevant considerations for evaluating whether additional limitations integrate a judicial exception into a practical application provided in MPEP 2106.04(d), subsection I. The additional elements (ii) and (iii) amount to merely including instructions to implement the abstract idea on a computer or merely using a computer as a tool to perform the abstract idea. See MPEP 2106.05(f). The additional elements of the claims invoke computers merely as a tool to perform an existing process: liveness detection (which can be performed by a person alone, a computer alone, or a combination). See MPEP 2106.05(f)(2).
Claim 2 includes the additional element “determining, for each frame of reference image, a binarized image of a region to be detected in the reference image”. Image binarization is the process of converting a color or grayscale image into a binary image, which is not a manual process and requires a computer.
The disclosure identifies a technical problem encountered in liveness detection where local disturbances like a pen being waved around can be utilized to attack/spoof a liveness detection method. See par. 66. The disclosure identifies a technical solution which considers dynamic changes in multiple frames of reference image. See par. 67. The dynamic changes are determined from pixel statistics values computed from binarized reference images and specifically, calculating a variance of the statistics. See par. 89.
Considering the limitations of claims 2-4 and 8-11 individually and as a whole, there is no additional element that integrates the recited exception into a practical application. However, claim 5, which introduces the variance concept to the claims that is part of the technical solution described, does integrate the recited exception into a practical application and therefore is eligible. Dependent claims 6 and 7 are eligible by virtue of dependency. Unlike claim 5, claims 2-4 and 8-11 do not include the technical solution to the technical problem and thus fail to add an inventive concept to the claims. See MPEP 2106.05(h). Although claim 8 includes the “preset risk factor threshold” as an additional element and claim 9 includes the “fluctuation value” as an additional element, these elements to do not reflect the described technical solution. At best, these additional elements amount to merely including instructions to implement the abstract idea on a computer or merely using a computer as a tool to perform the abstract idea. See MPEP 2106.05(f).
Claims 19-21 are eligible for the same reasons provided for claims 5-7. Claims 16-18 and 22 are not eligible for the same reasons as claims 2-4 and 8-11. It should be noted that because the courts have made it clear that mere physicality or tangibility of an additional element or elements is not a relevant consideration in the eligibility analysis, the physical nature of any underlying hardware in the additional elements, e.g., “processor”, “non-transitory storage medium” and “storage apparatus” does not affect this analysis. See MPEP 2106.05(I).
Whether evaluated individually or in combination, the additional elements in claims 2-4, 8-11, 16-18 and 22 do not integrate the recited judicial exception into a practical application and therefore, are directed to the judicial exception. (Step 2A, Prong Two: NO). As indicated above, claims 5-7 and 19-21 are eligible. (Step 2A, Prong Two: YES).
Step 2B: do the claims as a whole amount to significantly more than the judicial exception?
Additional elements of the pending claims that amount to performing the abstract idea using a computer as a tool to perform an existing process (retrieve data and manipulate data at a user’s direction) cannot provide an inventive concept. See MPEP 2106.05(f). Based on the high-level of specify of the technical aspects of limitations in claims 1-4, 8-11, 13, 14, 16-18 and 22, there is no additional element or combination of additional elements that constitutes an improvement to the functioning of a computer. At best, these claims amount to mere instructions to “apply” the abstract idea using well-known computer processes like using thresholds, cropping images, and performing image binarization. See the cited prior art below for evidence that these concepts are well-known. Even considering each of claims 1-4, 8-11, 13, 14, 16-18 and 22 as a whole, the claims do not amount to significantly more than the recited judicial exception and fail to encompass an inventive concept (Step 2B: NO). Claims 1-4, 8-11, 13, 14, 16-18 and 22, therefore, are not eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 8, 13 and 14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chinese Appl. Pat. Appl. Pub. No. 109977839 (published 20 March 2019, using the translation from Google Patents provided by the examiner) to Wang (hereinafter “Wang”).
Regarding claim 1, Wang teaches a method for image processing, comprising:
acquiring an image sequence of a target object in response to a live body detection trigger operation (In Step 201, an authentication trigger is detected, a preset action instruction is output, and video of a target face object is captured. See Wang at pg. 4), wherein the image sequence comprises an image to be detected (Wang, pg. 4, “face object”);
determining, for the image to be detected, multiple frames of reference images associated with the image to be detected in acquisition time from the image sequence (In Step 203, if the action was performed as determined in Step 202, then a target number of video frames from the video are extracted. See Wang at pg. 4);
determining a risk factor (Wang calculates a liveness value for each of the target number of video frames. See Wang, pg. 4, “living body value Face object for characterizing corresponding video frame is the probability of living body faces”, pg. 5, “respectively”) of the image to be detected based on a region to be detected in the multiple frames of reference images (Key points are tracked through multiple frames where a specific facial region is detected, such as the eyes or the mouth. See Wang at pg. 5), wherein the region to be detected at least comprises a region where a verification organ (Wang, pg. 5, “blink of face object, open one’s mouth, shake the head, point head, etc.”) performing a preset verification action is located (Wang, pg. 5, “preset action command”), and the risk factor indicates whether the target object is a live body (In Step 205, a frame is selected as an authentication image based on the liveness values, which correspond to risk factor values that indicate the likelihood of trusting that the selected image is genuine and live. See Wang, pg. 5, “maximum video frame of corresponding living body value … Alternatively … body value … is more than or equal to default living body threshold”); and
determining a live body detection result corresponding to the target object based on the risk factor of the image to be detected (In Step 205, a frame is selected as an authentication image based on the liveness values. Either the image with the highest value is chosen or an image having a value greater than or equal to a liveness threshold is selected. See Wang, pg. 5, “maximum video frame of corresponding living body value … Alternatively … body value … is more than or equal to default living body threshold”).
Regarding claim 8, Wang teaches the method for image processing of claim 1, wherein determining a live body detection result corresponding to the target object based on the risk factor of the image to be detected comprises: determining the live body detection result corresponding to the target object based on the risk factor of the image to be detected and a preset risk factor threshold (In Step 205, a frame is selected as an authentication image based on the liveness values. Either the image with the highest value is chosen or an image having a value greater than or equal to a liveness threshold is selected. See Wang, pg. 5, “maximum video frame of corresponding living body value … Alternatively … body value … is more than or equal to default living body threshold”).
Claim 13 substantially correspond to claim 1 by reciting an electronic device, comprising: at least one processor (Wang, pg. 7, “processor”); a storage apparatus (Wang, pg. 7, “ROM”) configured to store at least one program (Wang, pg. 7, “programs”); wherein the at least one program, when executed by the at least one processor, causes the at least one processor to perform the method of claim 1.
Claim 13 is rejected for the same reasons provided for claim 1.
Claim 14 substantially corresponds to claim 1 by reciting a non-transitory storage medium (Wang, pg. 7, “ROM”) containing computer-executable instructions (Wang, pg. 7, “programs”), when executed by a computer processor (Wang, pg. 7, “processor”), causing the computer processor to perform the method of claim 1.
Claim 14 is rejected for the same reasons provided for claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2, 5-7, 16 and 19-22 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of U.S. Pat. Appl. Pub. No. 20160379042 to Bourlai et al. (hereinafter “Bourlai”) and in further view of U.S. Pat. Appl. Pub. No. 20180096212 to Lin (hereinafter “Lin”).
Regarding claim 2, Wang teaches the method for image processing of claim 1 (In the description of Step 405 on page 6, Wang discloses using a pre-established lookup table derived from statistics of multiple video frames and quality values.), but does not teach that which is explicitly taught by Bourlai
Bourlai teaches determining, for each frame of reference image (“plurality of current images”. See par. 91, claim 3), a binarized image (Binarization is the process of converting a color or grayscale image into a binary image using a threshold to classify pixel intensities into binary categories, i.e., 1 or 0.) of a region to be detected in the reference image (“binarize each of the plurality of current images”. See claim 3. See also par. 46); and
counting a quantity of pixel points with a same pixel value in the binarized image to obtain a pixel point statistics value (Bourlai, par. 96, “a contiguous grouping of bright pixels that exceeds 10% of the total pixels in the image indicates that an undesirable reflection that may, for example, be caused by a screen, exist in the image and result in a determination that a reflection is detected at 214.” Exceeding a percentage of all the pixels requires knowing the count of pixels in the contiguous group. “group contiguous pixels in each of the binarized plurality of current image into segments; identify features of the ocular region including one of the detected eyes in each of the plurality of current images; remove segments having fewer than a predetermined number of contiguous pixels from the features of the ocular region including one of the detected eyes”. See claim 3. See also par. 52).
Wang discloses liveness detection that tracks facial regions using detected keypoints. Thus, Wang shows that it was known in the art before the effective filing date of the claimed invention to isolate and track specific regions to determine liveness, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, determining a metric of change indicative of liveness for an image region over time. Bourlai discloses liveness detection that segments image regions for assessing liveness using binarization. Thus, Bourlai shows that it was known in the art before the effective filing date of the claimed invention to binarize a face image to segment a particular region for assessment, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, determining a metric of change indicative of liveness for an image region over time.
A person of ordinary skill in the art would have been motivated to combine the keypoint detection disclosed by Wang with the binarization disclosed by Bourlai to thereby segment a region of interest, e.g., a mouth, an eye, both eyes. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of reducing computational load on the underlying computer/processor.
Wang in view of Bourlai does not teach that which is explicitly taught by Lin.
Lin teaches determining the risk factor of the image to be detected (Lin, par. 104, “if the image difference data is null (e.g., the first image and the second image have the same respective image data, and there is no image difference data), the determination can be made that the target object is a virtual object. Conversely, if the image difference data is not null (e.g., the first image and the second image do not have entirely identical respective image data (or the corresponding image data for the first image and the second image are not within a threshold degree of similarity), and there exists image difference data), the determination can be made that the target object is a living object.”). based on pixel point statistics values of at least two frames of reference images in the multiple frames of reference images (Lin, par. 70, “processor 230 is configured to determine that the target object is a virtual object if the first degree of variation is the brightness variation value and/or the color variation value of the entire face (e.g., when the projection information is (or includes) projection brightness and/or projection color), and the difference between the first degree of variation and the second degree of variation falls within a first predetermined range of variation. The brightness variation or the color variation can be computed in various manners. … The target object can correspond to a face; by comparing two images before and after an updating of the projection information, a difference between the area corresponding to the face in a first image and the area corresponding to the face in a second image can be computed. The difference between the face in a first image and the area corresponding to the face in a second image can correspond to a difference in brightness, color, content, or one or more other characteristics. The variance of the target object can correspond to a difference in brightness, color, content, or one or more other characteristics of the target object.”).
Wang and Bourlai are analogous to the claimed invention for the reasons provided above. Lin discloses detecting attacks using virtual faces by comparing changes across images over time and calculating a variance. Thus, Lin shows that it was known in the art before the effective filing date of the claimed invention to use variance as an indicator of liveness, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, determining a metric of change indicative of liveness for an image region over time.
A person of ordinary skill in the art would have been motivated to combine the liveness value probability determination disclosed by Wang in view of Bourlai with the variance calculation disclosed by Lin, to thereby compute the variance between two binarized face regions and determine that the face in the image to be determined is live and not virtual/fake by comparing that variance to a threshold. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of reducing attacks by virtual objects.
Regarding claim 5, Wang in view of Bourlai and in further view of Lin teaches the method for image processing of claim 2, wherein determining a risk factor of the image to be detected based on pixel point statistics values of at least two frames of reference images in the multiple frames of reference images comprises:
calculating a variance of pixel point statistics values of the at least two frames of reference images in the multiple frames of reference images (Lin, par. 70, “processor 230 is configured to determine that the target object is a virtual object if the first degree of variation is the brightness variation value and/or the color variation value of the entire face (e.g., when the projection information is (or includes) projection brightness and/or projection color), and the difference between the first degree of variation and the second degree of variation falls within a first predetermined range of variation. The brightness variation or the color variation can be computed in various manners. … The target object can correspond to a face; by comparing two images before and after an updating of the projection information, a difference between the area corresponding to the face in a first image and the area corresponding to the face in a second image can be computed. The difference between the face in a first image and the area corresponding to the face in a second image can correspond to a difference in brightness, color, content, or one or more other characteristics. The variance of the target object can correspond to a difference in brightness, color, content, or one or more other characteristics of the target object.”); and
determining the risk factor of the image to be detected based on the variance (Lin, par. 104, “if the image difference data is null (e.g., the first image and the second image have the same respective image data, and there is no image difference data), the determination can be made that the target object is a virtual object. Conversely, if the image difference data is not null (e.g., the first image and the second image do not have entirely identical respective image data (or the corresponding image data for the first image and the second image are not within a threshold degree of similarity), and there exists image difference data), the determination can be made that the target object is a living object.”).
The rationale for obviousness is the same as provided for claim 2.
Regarding claim 6, Wang in view of Bourlai and in further view of Lin teaches the method for image processing of claim 5, wherein calculating a variance of pixel point statistics values of the at least two frames of reference images in the multiple frames of reference images comprises:
obtaining at least two frames of reference images within a preset acquisition time range in the multiple frames of reference images (Lin, par. 6, “The user is prompted to perform one or more actions (such as nodding or shaking the head, eye-blinking, or opening the mouth), and the system receives the corresponding actions performed by the user. The person's actions are computed and analyzed using the consecutive images that were taken.”); and
calculating the variance of pixel point statistics values of the at least two frames of reference images (See Lin at pars. 70 and 101).
The rationale for obviousness is the same as provided for claim 2.
Regarding claim 7, Wang in view of Bourlai and in further view of Lin teaches the method for image processing of claim 5, wherein calculating a variance of pixel point statistics values of the at least two frames of reference images in the multiple frames of reference images comprises:
obtaining a preset quantity of reference images in the multiple frames of reference images to calculate the variance of pixel point statistics values of the preset quantity of reference images (The system captures a first image and then captures a second image, thereby capturing a predetermined number of images, i.e., two. See Lin at pars. 70, 74 and 10. A difference is compared to a threshold to determine liveness. See Lin at pars. 67-68. The difference can be a variance. See Lin at par. 70.).
The rationale for obviousness is the same as provided for claim 2.
Claims 16 and 19-22 substantially correspond to claims 5-8 by reciting an electronic device, comprising: at least one processor (Wang, pg. 7, “processor”); a storage apparatus (Wang, pg. 7, “ROM”) configured to store at least one program (Wang, pg. 7, “programs”); wherein the at least one program, when executed by the at least one processor, causes the at least one processor to perform the method of claims 5-8. Claim 22 differs from claim 8 in that claim 8 depends directly from claim 1 whereas claim 16 intervenes claims 13 and 22. The difference in dependency does not change how the cited art is applied.
The rationale for obviousness is the same as provided for claims 5-8.
Claims 3, 4, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Bourlai, in view of Lin and in further view of European Pat. Appl. Pub. no. 3933666 to Liljestrand et al. (hereinafter “Liljestrand”).
Regarding claim 3, Wang in view of Bourlai and in further view of Lin teaches the method for image processing of claim 2 (Wang, pg. 5, “blink of face object, open one’s mouth, shake the head, point head, etc.”), wherein determining a binarized image of a region to be detected in the reference image comprises:
performing binarization on the image of the region to be detected to obtain the binarized image (Bourlai, “binarize each of the plurality of current images”. See claim 3. See also par. 46).
The rationale for obviousness is the same as provided for claim 2.
Wang in view of Bourlai and in further view of Lin does not teach that which is explicitly taught by Liljestrand.
Liljestrand teaches cropping the region to be detected in the reference image to obtain an image of the region to be detected (Liljestrand, par. 63, “In S7 a region of interest is determined in the set of at least two images 110. This region of interest could for example be the eye region of the target 50, 9, or the entire face region. This step may be referred to as frame preparation, in which consecutive image frames may be for example cropped, rotated, translated, etc. to include the region of interest in a pre-determined desired size. Note that, the frame [preparation], e.g. cropping is typically done based on the motion-based results. The tracker may provide for example facial feature coordinates which may be used to determine the crop.”).
Wang, Bourlai and Lin are analogous to the claimed invention for the reasons provided above. Liljestrand discloses an anti-spoofing system that compares face regions including the eyes and mouth to detect spoofing attempts. Before detecting a spoofing attempt, a region of interest is cropped. Thus, Liljestrand shows that it was known in the art before the effective filing date of the claimed invention to prepare images for liveness detection by cropping regions of interest to subsequently evaluate, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, determining a metric of change indicative of liveness for an image region over time.
A person of ordinary skill in the art would have been motivated to combine the image processing pipeline of the liveness detection system disclosed by Wang in view of Bourlai and in further view of with the cropping disclosed by Liljestrand, to thereby crop regions of interest and then binarize the regions or binarize the entire image and crop the binarized region as a pre-processing step. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of reducing the amount of data to process.
Regarding claim 4, Wang in view of Wang in view of Bourlai and in further view of Lin teaches the method for image processing of claim 2, performing binarization on the image of the region to be detected to obtain the binarized image (Bourlai, “binarize each of the plurality of current images”. See claim 3. See also par. 46). The rationale for obviousness is the same as provided for claim 2.
Wang in view of Bourlai and in further view of Lin does not teach that which is explicitly taught by Liljestrand.
Liljestrand teaches cropping the region (See Liljestrand at pars. 63-65).
The rationale for obviousness is the same as provided for claim 3.
Claims 17 and 18 substantially correspond to claims 3 and 4 by reciting an electronic device, comprising: at least one processor (Wang, pg. 7, “processor”); a storage apparatus (Wang, pg. 7, “ROM”) configured to store at least one program (Wang, pg. 7, “programs”); wherein the at least one program, when executed by the at least one processor, causes the at least one processor to perform the method of claims 3 and 4.
The rationale for obviousness is the same as provided for claims 3 and 4.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Lin.
Regarding claim 9, Wang teaches the method for image processing of claim 1, but does not teach that which is explicitly taught by Lin.
Lin teaches determining a fluctuation value (variance) corresponding to the multiple frames of images to be detected based on risk factors of the multiple frames of images to be detected (Lin, par. 70, “The target object can correspond to a face; by comparing two images before and after an updating of the projection information, a difference between the area corresponding to the face in a first image and the area corresponding to the face in a second image can be computed.”); and
determining the live body detection result corresponding to the target object based on the fluctuation value and a preset fluctuation threshold (The system captures a first image and then captures a second image, thereby capturing a predetermined number of images, i.e., two. See Lin at pars. 70, 74 and 10. A difference is compared to a threshold to determine liveness. See Lin at pars. 67-68. The difference can be a variance. See Lin at par. 70.).
Wang and Lin are analogous to the claimed invention for the reasons provided above.
A person of ordinary skill in the art would have been motivated to combine the liveness value probability determination disclosed by Wang with the variance calculation disclosed by Lin, to thereby compute the variance between two binarized face regions and determine that the face in the image to be determined is live and not virtual/fake by comparing that variance to a threshold. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of reducing attacks by virtual objects.
Claims 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Exploiting Temporal and Depth Information for Multi-Frame Face Anti-Spoofing to Wang et al. (hereinafter “Wang2”).
Regarding claim 10, Wang teaches the method for image processing of claim 1, wherein the multiple frames of reference images are a preset quantity of images (In step S203, a predetermined target number of frames is extracted. See Wang at pg. 4. In an example, the predetermined number of images is two, where each image comes from a different region. See Wang at pg. 5), but does not teach that which is explicitly taught by Wang2.
Wang2 teaches preceding images of the image to be detected that are closest to the image to be detected in acquisition time (Wang2, Figure 5, “The inputs are consecutive frames in a fixed interval.”).
Wang is analogous to the claimed invention for the reasons provided above. Wang2 discloses an anti-spoofing architecture that acquires multiple consecutive images at a fixed temporal spacing and determines if a face presented is a live body, where the face is included in each of the images, including one images and the two immediately preceding images. Thus, Wang2 shows that it was known in the art before the effective filing date of the claimed invention to determine liveness of a face image based on multiple preceding images in time, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, determining a metric of change indicative of liveness for an image region over time.
A person of ordinary skill in the art would have been motivated to combine the image acquisition and liveness determination disclosed by Wang with the multi-image processing disclosed by Wang2, to thereby acquire at least three consecutive and equally spaced images and determine if the face in the third image is a living body based on the preceding images. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of incorporating temporal proximity into the liveness determination, which would reduce the chances of an attack succeeding because of the fine-grained analysis.
Regarding claim 11, Wang teaches the method for image processing of claim 1, but does not teach that which is explicitly taught by Wang 2.
Wang2 teaches wherein the multiple frames of reference images are images obtained by equally spaced sampling of a preset quantity of images in preceding images of the image to be detected that are closest to the image to be detected in acquisition time (Wang2, Figure 5, “The inputs are consecutive frames in a fixed interval.” The number of images to acquire is predetermined by the spacing because changing the spacing changes the number of consecutive frames that are extracted. Furthermore, the pipeline is automated and deterministic. The number of images to be extracted is thus predetermined before the pipeline begins its processing.).
The rationale for obviousness is the same as provided for claim 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN P POTTS whose telephone number is (571)272-6351. The examiner can normally be reached M-F, 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN P POTTS/Examiner, Art Unit 2672
Prosecution Insights