DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 11, 2026 has been entered. Claims 1-7, 9-19 and 21-22 remain pending in the application.
Claim Objections
Claims 5, 7, 13 and 21-22 are objected to because of the following informalities:
In claim 5 line 2, “the second, higher resolution” should read “a second, higher resolution”
In claim 7 line 15, “head identification score” should read “head identification confidence score”
In claim 7 line 18, “head dentification score” should read “head identification confidence score”
In claim 13 line 23, “head identification score” should read “head identification confidence score”
In claim 13 line 26, “head dentification score” should read “head identification confidence score”
In claim 21 line 2, “at the first resolution” should be deleted
In claim 22 line 10, “a network input” should read “a network input size”
In claim 22 line 14, “to provide the head identification confidence score” should be deleted
Appropriate correction is required.
Applicant is advised that should claim 13 be found allowable, claim 14 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6 and 21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. All of the claims are method claims under Step 1, but under Step 2A all of these claims recite abstract ideas and specifically mental processes—concepts performed in the human mind including observation, evaluation, judgement and opinion; these mental processes are more particularly:
Recited in claim 1 as:
performing face detection on the image
providing a face detection confidence score of a candidate face
evaluating a face detection confidence score to a lower threshold
based on the face detection confidence score being below the lower threshold, discarding the candidate face
evaluating the face detection confidence score to a higher threshold
based on the face detection confidence score being above the higher threshold, accepting the candidate face
evaluating the face detection confidence score to the lower threshold and the higher threshold
based on the face detection confidence score being between the lower threshold and the higher threshold performing head identification on the candidate face using the image
providing a head identification confidence score for the candidate face
based on the head identification confidence score, accepting or discarding the candidate face
Recited in claim 2 as:
evaluating the provided head identification confidence score to a third threshold
discarding the candidate face with the head identification confidence score below the third threshold
accepting the candidate face with the head identification confidence score above the third threshold
Recited in claim 3 as:
provides a bounding box for the candidate face
enlarging the bounding box of the candidate face
Recited in claim 4 as:
determining the size of the image in the enlarged bounding box
comparing the size of the image in the enlarged bounding box with a predetermined size
cropping the image in the enlarged bounding box to the predetermined size when the size exceeds the predetermined size
providing the cropped image for the head identification performance
Recited in claim 5 as:
sampling the image in the enlarged bounding box from a second, higher resolution image when the image size in the enlarged bounding box is below the predetermined size
providing the sampled image for head identification performance
It is noted that the above analysis is according to the 2019 Revised Patent Subject Matter Eligibility Guidance published in the Federal Register (84 FR 50) on January 7, 2019 and MPEP 2106.04(a)(2)(III).
Consider also that “If a claim recites a limitation that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper, the limitation falls within the mental processes grouping, and the claim recites an abstract idea” as per MPEP 2106.04(a)(2)(III)(B). See also footnotes 14 and 15 of the Federal Register Notice. The limitations detailed above may be practically performed in the human mind with the use of a physical aid such as pen and paper (e.g. drawing the bounding boxes for each candidate face with a pen).
Under Step 2A, this judicial exception is not integrated into a practical application because each of claims 1-6 and 21 do not recite additional elements that integrate the exception into a practical application. The additional element (receiving an image in claim 1) is adding insignificant pre-solution activity of acquiring image data to the judicial exception, which is not indicative of integration into a practical application as per MPEP 2106.05(g). The additional element (videoconferencing endpoint in claim 6) is generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of integration into a practical application as per MPEP 2106.05(h). The additional element (machine learning model in claim 21) is recited at a high level of generality and merely equate to “apply it” or otherwise merely uses a generic computer as a tool to perform an abstract idea which are not indicative of integration into a practical application as per MPEP 2106.05(f). See also MPEP 2106.04(a)(2)(III) with respect to Mental Processes: “Nor do the courts distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer”. See also MPEP 2106.04(a)(2)(III)(C)(3) Using a computer as tool to perform a mental process and MPEP 2106.04(a)(2)(III)(D) as well as the case law cited therein.
Under Step 2B, each of claims 1-6 and 21 do not recite additional elements that are indicative of an inventive concept. The additional elements detailed above are simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception as per MPEP 2106.05(d) and 2106.07(a)III. In other words, the additional elements do not amount to significantly more than the judicial exception.
For all of the above reasons, taken alone or in combination, claims 1-6 and 21 recite a non-statutory mental process.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by Matsimanis et al. (US 2018/0239977 A1).
Regarding claim 1, Matsimanis discloses a method of face detection comprising: receiving an image (Matsimanis paragraph 0012: “receiving an image”); performing face detection on the image (Matsimanis paragraph 0012: “processing the image for characteristics associated with a presence of at least one candidate face with a spatial dimension”), and providing a face detection confidence score of a candidate face (Matsimanis paragraph 0012: “receiving an initial confidence value”), the face detection confidence score indicating the confidence of a presence of a face for the candidate face in the image (Matsimanis paragraph 0012: “indicating the presence of a candidate face having a spatial dimension”); and at least one of: evaluating a face detection confidence score to a lower threshold and based on the face detection confidence score being below the lower threshold, discarding the candidate face (Matsimanis paragraphs 0013-0014: “a confidence value based on the face detection engine alone can be below the lower threshold, indicating a false positive”); evaluating the face detection confidence score to a higher threshold and based on the face detection confidence score being above the higher threshold, accepting the candidate face (Matsimanis paragraphs 0013-0014: “a confidence value based on the face detection engine alone can be above the higher threshold, indicating a true positive state”); or evaluating the face detection confidence score to the lower threshold and the higher threshold, and based on the face detection confidence score being between the lower threshold and the higher threshold (Matsimanis paragraph 0014: “face detection engine can initially provide a confidence value that is indeterminate, being between the lower and upper thresholds”) performing head identification on the candidate face using the image and providing a head identification confidence score for the candidate face and based on the head identification confidence score, accepting or discarding the candidate face (this limitation is disclosed in an alternative clause and thus, read only on the first and second limitations).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over Matsimanis in view of Andalo et al. (US 2019/0278894 A1) and in further view of Fan et al. (US 2011/0274315 A1).
Regarding claim 2, Matsimanis discloses the method of claim 1. However, Matsimanis fails to disclose based on the face detection confidence score being between the lower threshold and the higher threshold performing head identification on the candidate face using the image and providing a head identification confidence score for the candidate face and evaluating the provided head identification confidence score to a third threshold; and at least one of: discarding the candidate face with the head identification confidence score below the third threshold; or accepting the candidate face with the head identification confidence score above the third threshold.
In the related art of face detection and identification, Andalo discloses based on the face detection confidence score being between the lower threshold and the higher threshold (Andalo FIG. 5, paragraph 0039: “If the first probability metric lies between the first and second thresholds”) performing face identification on the candidate face using the image and providing a face identification confidence score for the candidate face (Andalo paragraph 0040: “a second classifier 180 is employed to generate a second probability metric of the probe image data being associated with an authorized user”) and evaluating the provided face identification confidence score to a third threshold (Andalo paragraph 0041: “the second probability metric is compared to a third threshold (Thr3)”); and at least one of: discarding the candidate face with the face identification confidence score below the third threshold (Andalo paragraph 0041: “Otherwise, the biometric authentication request is denied”); or accepting the candidate face with the face identification confidence score above the third threshold (Andalo paragraph 0041: “If the second probability metric is greater than the third threshold in method block 540, the biometric authentication request is approved”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Matsimanis to incorporate the teachings of Andalo to reduce false negatives and improve the overall accuracy of the facial recognition (Andalo paragraphs 0023-0024). Examiner notes under broadest reasonable interpretation, head identification would include face identification. However, for means of compact prosecution, Andalo still fails to explicitly disclose the second classifier performing head identification.
In the related art of face and head detection, Fan discloses performing head identification in addition to face detection (Fan FIG. 2, paragraphs 0043-0045: “a human face list 202, a human head list 203…these lists serve as respective detection results…if a human face a in the human face list 202 overlaps with a human head b in the human head list 203, then that implies that a possibility of a human being existing is very high”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Matsimanis and Andalo to incorporate the teachings of Fan to improve the detection accuracy and reduce the detection error rate (Fan paragraph 0011).
Claim(s) 3-4 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Matsimanis, Andalo and Fan in view of Liu et al. (US 2018/0039853 A1).
Regarding claim 3, Matsimanis, modified by Andalo and Fan, discloses the method of claim 2. However, Matsimanis fails to disclose providing a bounding box for the candidate face; and enlarging the bounding box of the candidate face before performing the head identification. In the related art of object detection, Liu discloses providing a bounding box for the candidate face (Liu paragraph 0002: “detect and localize all instances of pre-defined object classes in the form of bounding boxes with confidence values for given input images”); and enlarging the bounding box of the candidate face before performing the head identification (Liu FIG. 4A, paragraph 0028: “enlarges the proposal box 15 by seven times in both x and y directions to obtain the context box 20…The resized context image 21 is transmitted to the second DCNN 220”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Matsimanis to incorporate the teachings of Liu to improve detecting small objects because extracting features from greater areas in the image helps to incorporate context information (Liu paragraph 0028).
Regarding claim 4, Matsimanis, modified by Andalo, Fan and Liu, discloses the method of claim 3, further comprising: determining the size of the image in the enlarged bounding box before performing head identification (Liu FIG. 4A, paragraph 0030: the context box 20 has a width w’ and a height h’); comparing the size of the image in the enlarged bounding box with a predetermined size (Liu FIG. 4A, paragraph 0025: “the predetermined identical size may be 227x227 (224x224 for VGG16) patches (pixels)”); cropping the image in the enlarged bounding box to the predetermined size when the size exceeds the predetermined size (Liu FIG. 4A, paragraph 0025: “the context region image corresponding to the context box 20 is resized, using the resize module 14, to a resized context image 21 having the predetermined size”); and providing the cropped image for the head identification performance (Liu FIG. 4A, paragraphs 0025-0026: “and transmitted to the ContexNet 250…As a result, the object detection of the target object image corresponding to the proposal box 15 is obtained”).
Regarding claim 22, Matsimanis discloses the method of claim 1, further comprising: receiving the image at a first resolution and at a second resolution higher than the first resolution (Matsimanis paragraph 0016: “the different resolution of the two cameras [rear and front cameras]”). However, Matsimanis fails to disclose providing a bounding box around the candidate face in the image at the first resolution; increasing the size of the bounding box around the candidate face in the image at the first resolution, based on the face detection confidence score being between the lower threshold and the higher threshold; and at least one of: based on the increased bounding box size being larger than a network input, performing head identification on the increased bounding box of the image at the first resolution to provide the head identification confidence score; or based on the increased bounding box size being smaller than the network input size to provide the head identification confidence score, performing head identification on the increased bounding box of the image at the second resolution to provide the head identification confidence score.
In related art, Liu discloses providing a bounding box around the candidate face in the image at the first resolution (Liu paragraph 0002: “detect and localize all instances of pre-defined object classes in the form of bounding boxes with confidence values for given input images”); increasing the size of the bounding box around the candidate face in the image at the first resolution (Liu FIG. 4A, paragraph 0028: “enlarges the proposal box 15 by seven times in both x and y directions to obtain the context box 20…The resized context image 21 is transmitted to the second DCNN 220”); and at least one of: based on the increased bounding box size being larger than a network input, performing identification on the increased bounding box of the image at the first resolution (Liu FIG. 4A, paragraphs 0025-0026: “and transmitted to the ContexNet 250…As a result, the object detection of the target object image corresponding to the proposal box 15 is obtained”); or based on the increased bounding box size being smaller than the network input size, performing identification on the increased bounding box of the image at the second resolution (this limitation is disclosed in an alternative clause and thus, read only on the first limitation). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Matsimanis to incorporate the teachings of Liu to improve detecting small objects because extracting features from greater areas in the image helps to incorporate context information (Liu paragraph 0028). However, Matsimanis, modified by Liu, still fails to disclose based on the face detection confidence score being between the lower threshold and the higher threshold, performing head identification to provide the head identification confidence score.
In related art, Andalo discloses based on the face detection confidence score being between the lower threshold and the higher threshold (Andalo FIG. 5, paragraph 0039: “If the first probability metric lies between the first and second thresholds”), performing face identification to provide the face identification confidence score (Andalo paragraph 0040: “a second classifier 180 is employed to generate a second probability metric of the probe image data being associated with an authorized user”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Matsimanis to incorporate the teachings of Andalo to reduce false negatives and improve the overall accuracy of the facial recognition (Andalo paragraphs 0023-0024). Examiner notes under broadest reasonable interpretation, head identification would include face identification. However, for means of compact prosecution, Andalo still fails to explicitly disclose the second classifier performing head identification.
In related art, Fan discloses performing head identification in addition to face detection (Fan FIG. 2, paragraphs 0043-0045: “a human face list 202, a human head list 203…these lists serve as respective detection results…if a human face a in the human face list 202 overlaps with a human head b in the human head list 203, then that implies that a possibility of a human being existing is very high”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Matsimanis and Andalo to incorporate the teachings of Fan to improve the detection accuracy and reduce the detection error rate (Fan paragraph 0011).
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Matsimanis, Andalo, Fan and Liu in view of Bigioi et al. (US 2009/0080713 A1).
Regarding claim 5, Matsimanis, modified by Andalo, Fan and Liu, discloses the method of claim 4. However, Matsimanis and Liu fail to disclose sampling the image in the enlarged bounding box from a second, higher resolution image when the image size in the enlarged bounding box is below the predetermined size; and providing the sampled image for head identification performance. In the related art of face detection, Bigioi discloses sampling the image in the enlarged bounding box from a second, higher resolution image when the image size in the enlarged bounding box is below the predetermined size (Bigioi paragraph 0012: “one or more identified small sized face regions in the higher resolution version of the image”); and providing the sampled image for head identification performance (Bigioi paragraph 0045: “retests the face regions identified by the relaxed small face classifier on the larger (higher resolution) main image…with a high quality classifier”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Matsimanis and Liu to incorporate the teachings of Bigioi to achieve more accurate results while optimizing the efficiency of the face detection (Bigioi paragraphs 0027-0028).
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Matsimanis in view of Fan.
Regarding claim 6, Matsimanis discloses the method of claim 1. However, Matsimanis fails to disclose the method is performed in a videoconferencing endpoint. In related art, Fan discloses the method is performed in a videoconferencing endpoint (Fan paragraph 0041: “in a case of human detection, in a video conference scene”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Matsimanis to incorporate the teachings of Fan to improve the detection accuracy and reduce the detection error rate (Fan paragraph 0011) since accurate object detection techniques such as human face detection techniques, etc., are the foundations of various video application systems, for example, video conference systems (Fan paragraph 0004).
Claim(s) 7 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Andalo in view of Fan and in further view of Bigioi.
Regarding claim 7, Andalo discloses a non-transitory computer readable medium storing instructions (Andalo paragraph 0052: “the software instructions may be transferred from a non-transitory computer readable storage medium to a memory, such as the memory 120”) that when executed by a processor cause the processor to perform a method of face detection (Andalo paragraph 0035: “method 500 for performing a multiple-tiered facial recognition technique”), the method comprising: receiving an image at a first resolution (Andalo paragraph 0035: “probe image data 210 associated with a biometric authentication request is received in the mobile device 100”); performing face detection on the image at the first resolution (Andalo paragraph 0015: “detects the face in the probe image”) and providing a face detection confidence score of a candidate face (Andalo paragraph 0036: “a first classifier 175 is employed to generate a first probability metric of the probe image data being associated with an authorized user”); evaluating the provided face detection confidence score to a lower threshold (Andalo paragraph 0038: “The first probability metric is compared to a second threshold (Thr2-true negative)”) and a higher threshold (Andalo paragraph 0037: “The first probability metric is compared to a first threshold (Thr1-true positive)”); based on the face detection confidence score being between the lower threshold and the higher threshold (Andalo FIG. 5, paragraph 0039: “If the first probability metric lies between the first and second thresholds”), performing face identification on the candidate face and providing a face identification confidence score for the candidate face (Andalo paragraph 0040: “a second classifier 180 is employed to generate a second probability metric of the probe image data being associated with an authorized user”); evaluating the provided face identification confidence score to a third threshold (Andalo paragraph 0041: “the second probability metric is compared to a third threshold (Thr3)”); and at least one of: based on the face identification confidence score being below the third threshold, discarding the candidate face with the face identification confidence score below the third threshold (Andalo paragraph 0041: “Otherwise, the biometric authentication request is denied”); or based on the face identification confidence score being above the third threshold, accepting the candidate face (Andalo paragraph 0041: “If the second probability metric is greater than the third threshold in method block 540, the biometric authentication request is approved”). However, Andalo fails to explicitly disclose the second classifier performing head identification; receiving the image at a second resolution higher than the first resolution; and based on the face detection confidence score being between the lower threshold and the higher threshold, performing head identification on the candidate face using the image at the second resolution.
In related art, Fan discloses performing head identification in addition to face detection (Fan FIG. 2, paragraphs 0043-0045: “a human face list 202, a human head list 203…these lists serve as respective detection results…if a human face a in the human face list 202 overlaps with a human head b in the human head list 203, then that implies that a possibility of a human being existing is very high”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Andalo to incorporate the teachings of Fan to improve the detection accuracy and reduce the detection error rate (Fan paragraph 0011). However, Andalo, modified by Fan, still fails to disclose receiving the image at a second resolution higher than the first resolution; and based on the face detection confidence score being between the lower threshold and the higher threshold, performing head identification on the candidate face using the image at the second resolution.
In related art, Bigioi discloses receiving the image at a second resolution higher than the first resolution (Bigioi paragraphs 0012, 0034: “A relatively high resolution image of nominally the same scene is also received” where “the apparatus 10 automatically captures and stores a series of images at close intervals so that sequential images are nominally of the same scene. Such a series of images may include a series of preview images [low resolution images], post-view images, or a main acquired image [higher resolution image]”); and based on the face detection confidence score being between the lower threshold and the higher threshold (Andalo teaches in paragraph 0039, if the first probability metric lies between the first and second thresholds, a high confidence decision cannot yet be made; in Bigioi, a high confidence decision cannot be made in the case of smaller sized faces since they comprise fewer pixels and “As such, detection of smaller sized faces is inherently less reliable” – see Bigioi paragraph 0030), performing head identification on the candidate face using the image at the second resolution (Bigioi paragraph 0016: “The second processor is arranged to receive a relatively high resolution image of nominally the same scene and to apply at least one high quality face classifier to at least one identified small sized face region in the higher resolution version of the image”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Andalo to incorporate the teachings of Bigioi to achieve more accurate results while optimizing the efficiency of the face detection (Bigioi paragraphs 0027-0028).
Regarding claim 12, Andalo, modified by Fan and Bigioi, discloses the non-transitory computer readable medium of claim 7, wherein the non-transitory computer readable medium and the processor are used in a videoconferencing endpoint (Fan paragraph 0041: “in a case of human detection, in a video conference scene”).
Regarding claim 13, Andalo discloses a computing device (Andalo paragraph 0012: “computing system 110”) comprising: an image memory for storing an image at a first resolution (Andalo paragraphs 0013, 0015: “store information in the memory 120, such as the results of the executed instructions” which involves “captur[ing] a current image of the individual seeking to authenticate”); a processor coupled to the image memory (Andalo FIG. 1, paragraph 0012: “processor 115”); a non-transitory memory for storing program instructions coupled to the processor (Andalo paragraph 0052: “the software instructions may be transferred from a non-transitory computer readable storage medium to a memory, such as the memory 120”), the instructions causing the processor to perform a method of face detection (Andalo paragraph 0035: “method 500 for performing a multiple-tiered facial recognition technique”), the method comprising: performing face detection on a stored image at the first resolution (Andalo paragraph 0015: “detects the face in the probe image”) and providing face detection confidence scores of each candidate face (Andalo paragraph 0036: “a first classifier 175 is employed to generate a first probability metric of the probe image data being associated with an authorized user”); evaluating each of the provided face detection confidence scores to a lower threshold (Andalo paragraph 0038: “The first probability metric is compared to a second threshold (Thr2-true negative)”) and a higher threshold (Andalo paragraph 0037: “The first probability metric is compared to a first threshold (Thr1-true positive)”); for each face detection confidence score being below the lower threshold, discarding the candidate face (Andalo paragraph 0038: “If the first probability metric is less than the second threshold (i.e., high negative confidence) in method block 525, the biometric authentication request is denied in method block 530”); for each face detection confidence score being above the higher threshold, accepting the candidate face (Andalo paragraph 0037: “If the first probability metric is greater than the first threshold (i.e., high positive confidence) in method block 515, the biometric authentication request is approved in method block 520”); and for each face detection confidence score being between the lower threshold and the higher threshold (Andalo FIG. 5, paragraph 0039: “If the first probability metric lies between the first and second thresholds”), performing face identification on the candidate face and providing a face identification confidence score for the candidate face (Andalo paragraph 0040: “a second classifier 180 is employed to generate a second probability metric of the probe image data being associated with an authorized user”); evaluating the provided face identification confidence score to a third threshold (Andalo paragraph 0041: “the second probability metric is compared to a third threshold (Thr3)”); and at least one of: based on the face identification confidence score being below the third threshold, discarding the candidate face with the face identification confidence score below the third threshold (Andalo paragraph 0041: “Otherwise, the biometric authentication request is denied”); or based on the face identification confidence score being above the third threshold, accepting the candidate face (Andalo paragraph 0041: “If the second probability metric is greater than the third threshold in method block 540, the biometric authentication request is approved”). However, Andalo fails to explicitly disclose the second classifier performing head identification; storing the image at a second resolution higher than the first resolution; and for each face detection confidence score being between the lower threshold and the higher threshold, performing head identification on the candidate face using the image at the second resolution.
In related art, Fan discloses performing head identification in addition to face detection (Fan FIG. 2, paragraphs 0043-0045: “a human face list 202, a human head list 203…these lists serve as respective detection results…if a human face a in the human face list 202 overlaps with a human head b in the human head list 203, then that implies that a possibility of a human being existing is very high”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Andalo to incorporate the teachings of Fan to improve the detection accuracy and reduce the detection error rate (Fan paragraph 0011). However, Andalo, modified by Fan, still fails to disclose storing the image at a second resolution higher than the first resolution; and for each face detection confidence score being between the lower threshold and the higher threshold, performing head identification on the candidate face using the image at the second resolution.
In related art, Bigioi discloses storing the image at a second resolution higher than the first resolution (Bigioi paragraph 0016: “the second processor is arranged to store the image…[and] receive a relatively high resolution image of nominally the same scene”); and for each face detection confidence score being between the lower threshold and the higher threshold (Andalo teaches in paragraph 0039, if the first probability metric lies between the first and second thresholds, a high confidence decision cannot yet be made; in Bigioi, a high confidence decision cannot be made in the case of smaller sized faces since they comprise fewer pixels and “As such, detection of smaller sized faces is inherently less reliable” – see Bigioi paragraph 0030), performing head identification on the candidate face using the image at the second resolution (Bigioi paragraph 0016: “The second processor is arranged to receive a relatively high resolution image of nominally the same scene and to apply at least one high quality face classifier to at least one identified small sized face region in the higher resolution version of the image”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Andalo to incorporate the teachings of Bigioi to achieve more accurate results while optimizing the efficiency of the face detection (Bigioi paragraphs 0027-0028).
Regarding claim 14, Andalo, modified by Fan and Bigioi, discloses the computing device of claim 13, the method further comprising: evaluating each provided head identification confidence score to a third threshold; discarding each candidate face with a head identification confidence score below the third threshold; and accepting each candidate face with a head identification confidence score above the third threshold (as claimed in claim 13).
Claim(s) 9-11 and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Andalo, Fan and Bigioi in view of Liu.
Regarding claim 9, Andalo, modified by Fan and Bigioi, discloses the non-transitory computer readable medium of claim 7. However, Andalo fails to disclose providing a bounding box for the candidate face; and enlarging the bounding box of the candidate face before performing the head identification on the image defined by the bounding box. In related art, Liu discloses providing a bounding box for the candidate face (Liu paragraph 0002: “detect and localize all instances of pre-defined object classes in the form of bounding boxes with confidence values for given input images”); and enlarging the bounding box of the candidate face before performing the head identification on the image defined by the bounding box (Liu FIG. 4A, paragraph 0028: “enlarges the proposal box 15 by seven times in both x and y directions to obtain the context box 20…The resized context image 21 is transmitted to the second DCNN 220”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Andalo to incorporate the teachings of Liu to improve detecting small objects because extracting features from greater areas in the image helps to incorporate context information (Liu paragraph 0028).
Regarding claim 10, Andalo, modified by Fan, Bigioi and Liu, discloses the non-transitory computer readable medium of claim 9, the method further comprising: determining the size of the image in the enlarged bounding box before performing head identification (Liu FIG. 4A, paragraph 0030: the context box 20 has a width w’ and a height h’); comparing the size of the image in the enlarged bounding box with a predetermined size (Liu FIG. 4A, paragraph 0025: “the predetermined identical size may be 227x227 (224x224 for VGG16) patches (pixels)”); cropping the image in the enlarged bounding box to the predetermined size when the size exceeds the predetermined size (Liu FIG. 4A, paragraph 0025: “the context region image corresponding to the context box 20 is resized, using the resize module 14, to a resized context image 21 having the predetermined size”); and providing the cropped image for the head identification performance (Liu FIG. 4A, paragraphs 0025-0026: “and transmitted to the ContexNet 250…As a result, the object detection of the target object image corresponding to the proposal box 15 is obtained”).
Regarding claim 11, Andalo, modified by Fan, Bigioi and Liu, discloses the non-transitory computer readable medium of claim 10, the method further comprising: receiving the image at a second, higher resolution (Bigioi paragraph 0012: “A relatively high resolution image of nominally the same scene is also received”); sampling the image in the enlarged bounding box from the second, higher resolution image when the image size in the enlarged bounding box is below the predetermined size (Bigioi paragraph 0012: “one or more identified small sized face regions in the higher resolution version of the image”); and providing the sampled image for head identification performance (Bigioi paragraph 0045: “retests the face regions identified by the relaxed small face classifier on the larger (higher resolution) main image…with a high quality classifier”).
Regarding claim 15, Andalo, modified by Fan and Bigioi, discloses the computing device of claim 14. However, Andalo fails to disclose providing a bounding box for each candidate face; and enlarging the bounding box of each candidate face before performing the head identification on the image defined by the bounding box. In related art, Liu discloses providing a bounding box for each candidate face (Liu paragraph 0002: “detect and localize all instances of pre-defined object classes in the form of bounding boxes with confidence values for given input images”); and enlarging the bounding box of each candidate face before performing the head identification on the image defined by the bounding box (Liu FIG. 4A, paragraph 0028: “enlarges the proposal box 15 by seven times in both x and y directions to obtain the context box 20…The resized context image 21 is transmitted to the second DCNN 220”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Andalo to incorporate the teachings of Liu to improve detecting small objects because extracting features from greater areas in the image helps to incorporate context information (Liu paragraph 0028).
Regarding claim 16, Andalo, modified by Fan, Bigioi and Liu, discloses the computing device of claim 15, the method further comprising: determining the size of the image in each enlarged bounding box before performing the head identification (Liu FIG. 4A, paragraph 0030: the context box 20 has a width w’ and a height h’); comparing the size of the image in each enlarged bounding box with a predetermined size (Liu FIG. 4A, paragraph 0025: “the predetermined identical size may be 227x227 (224x224 for VGG16) patches (pixels)”); cropping the image in the enlarged bounding box to the predetermined size when the size exceeds the predetermined size (Liu FIG. 4A, paragraph 0025: “the context region image corresponding to the context box 20 is resized, using the resize module 14, to a resized context image 21 having the predetermined size”); and providing the cropped image for the head identification performance (Liu FIG. 4A, paragraphs 0025-0026: “and transmitted to the ContexNet 250…As a result, the object detection of the target object image corresponding to the proposal box 15 is obtained”).
Regarding claim 17, Andalo, modified by Fan, Bigioi and Liu, discloses the computing device of claim 16, the method further comprising: sampling the image in the enlarged bounding box from the second, higher resolution image when the image size in the enlarged bounding box is below the predetermined size (Bigioi paragraph 0012: “one or more identified small sized face regions in the higher resolution version of the image”); and providing the sampled image for the head identification performance (Bigioi paragraph 0045: “retests the face regions identified by the relaxed small face classifier on the larger (higher resolution) main image…with a high quality classifier”).
Claim(s) 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Andalo, Fan, Bigioi and Liu in view of Fukuda (US 2012/0328155 A1).
Regarding claim 18, Andalo, modified by Fan, Bigioi and Liu, discloses the computing device of claim 17, wherein the computing device is a videoconferencing endpoint (Fan paragraph 0041: “in a case of human detection, in a video conference scene”), the computing device further comprising: a camera (Andalo FIG. 1: camera 150) coupled to the image memory (Andalo FIG. 1: memory 120). However, Andalo fails to explicitly disclose storing the image at a third resolution higher than the second, higher resolution; developing the images at the first and second, higher resolutions from the image stored at the third resolution and storing the first and second resolution images in the image memory. In the related art of object detection, Fukuda discloses storing the image at a third resolution higher than the second, higher resolution (Fukuda paragraph 0035: “An image magnification varying unit 110 changes (at least reduces) a magnification ratio of an image stored in the image storage unit 101”); developing the images at the first (Fukuda FIG. 5, paragraph 0044: “the image 802 at low resolution”) and second, higher resolutions (Fukuda FIG. 5, paragraph 0044: “the image 801 at intermediate resolution”) from the image stored at the third resolution (Fukuda FIG. 5, paragraphs 0012, 0044: “the image 710 at high resolution” where “each of images at different resolutions…are generated from the input image”) and storing the first and second resolution images in the image memory (Fukuda paragraph 0035: “The image magnification varying unit 110 is also configured to output the image with its magnification ratio changed to the image storage unit 101”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Andalo to incorporate the teachings of Fukuda to detect detection targets in various sizes, such that the detection algorithm has a high generalization property (Fukuda paragraph 0045).
Regarding claim 19, Andalo, modified by Fan, Bigioi and Liu, discloses the computing device of claim 17, wherein the computing device is a videoconferencing endpoint (Fan paragraph 0041: “in a case of human detection, in a video conference scene”), the computing device further comprising: a camera (Andalo FIG. 1: camera 150) coupled to the image memory (Andalo FIG. 1: memory 120). However, Andalo fails to explicitly disclose storing the image at a third resolution higher than the first resolution; developing the image at the first resolution from the image stored at the third resolution and storing the first resolution image in the image memory. In related art, Fukuda discloses storing the image at a third resolution higher than the first resolution (Fukuda paragraph 0035: “An image magnification varying unit 110 changes (at least reduces) a magnification ratio of an image stored in the image storage unit 101”); developing the image at the first resolution (Fukuda FIG. 5, paragraph 0044: “the image 802 at low resolution”) from the image stored at the third resolution (Fukuda FIG. 5, paragraphs 0012, 0044: “the image 710 at high resolution” where “each of images at different resolutions…are generated from the input image”) and storing the first resolution image in the image memory (Fukuda paragraph 0035: “The image magnification varying unit 110 is also configured to output the image with its magnification ratio changed to the image storage unit 101”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Andalo to incorporate the teachings of Fukuda to detect detection targets in various sizes, such that the detection algorithm has a high generalization property (Fukuda paragraph 0045).
Claim(s) 21 is rejected under 35 U.S.C. 103 as being unpatentable over Matsimanis in view of Andalo.
Regarding claim 21, Matsimanis discloses the method of claim 1. However, Matsimanis fails to explicitly disclose providing the candidate face to a machine learning model that outputs the face detection confidence score for the candidate face. In related art, Andalo discloses
providing the candidate face to a machine learning model (Andalo paragraph 0036: “The first classifier 175 may include multiple components, such as the HOG SVM 175A and the LRPCA SVM 175B” where Support Vector Machine (SVM) is a known supervised machine learning algorithm) that outputs the face detection confidence score for the candidate face (Andalo paragraph 0036: “a first classifier 175 is employed to generate a first probability metric of the probe image data being associated with an authorized user”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Matsimanis to incorporate the teachings of Andalo to reduce false negatives and improve the overall accuracy of the facial recognition (Andalo paragraphs 0023-0024).
Response to Arguments
Applicant's arguments with respect to the 101 rejection of independent of claim 1 have been fully considered but they are not persuasive.
Regarding the argument that “the use of multiple thresholds for face detection, followed by head detection results is an improvement in face detection, particularly faces being reliably detected at farther distances. Therefore, independent claim 1 recites additional elements beyond the alleged abstract idea and those additional elements integrate the alleged abstract idea into a practical application”, MPEP 2106.05(a) discloses “It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements…[or] by the additional element(s) in combination with the recited judicial exception”. The limitations of independent claim 1 that provide the alleged improvement are encompassed by the judicial exception alone. Applicant does not specify what additional elements in the claim are providing the improvement or how additional elements in the claim are combined with the exception to provide the improvement. Instead, the additional elements identified by the Examiner do not integrate the exception into a practical application (as detailed in the 101 rejection above). Furthermore, MPEP 2106.04(d)(1) discloses “the claim must be evaluated to ensure that...the claim includes the components or steps of the invention that provide the improvement described in the specification”. The claims fail to include all necessary steps to provide sufficient detail for the improvement to be apparent to a person of ordinary skill in the art.
Applicant’s arguments with respect to the 101 rejection of independent claims 7 and 13 have been fully considered and are persuasive. The 101 rejection of claims 7 and 9-19 has been withdrawn.
Applicant’s arguments with respect to independent claim(s) 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments with respect to independent claims 7 and 13 have been fully considered but they are not persuasive.
Regarding the argument that “Bigioi applies a face classifier to both the low resolution image (to detect large sized faces) and the high resolution image (to detect small sized faces)”, Bigioi teaches processing first a low resolution image to identify small sized face regions. Then, a second high resolution image is processed selectively for the small sized face regions (Bigioi paragraph 0016). This selective image processing is further emphasized by the advantages discussed in paragraphs 0027-0028 of Bigioi.
Regarding the argument that “Nowhere in the disclosure of Bigioi, including the cited portion thereof, makes any mention of performing head identification on the candidate face nevertheless using the image at the second resolution”, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). The combination of Andalo, Fan and Bigioi teaches “based on the face detection confidence score being between the lower threshold and the higher threshold, performing head identification on the candidate face using the image at the second resolution and providing a head identification confidence score for the candidate face”, as delineated in the above rejection.
Regarding the argument that “it would not be obvious that one would combine Andalo with Bigioi, as asserted by the Office, because there would be no high or low threshold to be selected, since Andalo specifically only teaches 128x128 images”, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). The combined teachings of Andalo and Bigioi would have suggested to one of ordinary skill in the art that when a high confidence decision cannot yet be made (Andalo paragraph 0039), additional processing needs to be performed, such as using the image at a second, higher resolution (Bigioi paragraph 0016).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTINE ZHAO whose telephone number is (703)756-5986. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.Z./Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights