DETAILED ACTION
Notice of Pre-AIA or AIA Status.
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/24/2025 has been entered.
3. In the applicant’s submission, claims 1, 6, and 10-11 were amended; claims 5 and 9 were cancelled. Accordingly, claims 1-4, 6-8, and 10-20 are pending and being examined. Claims 1, 6, and 11 are independent form.
4. The claim rejections under 35 USC § 112 (a) made in the previous office action have been withdrawn in view of applicant’s remarks.
Claim Rejections - 35 USC § 103
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1-4, 6-8, 10-20 are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al (“Robust RGB-D Face Recognition Using Attribute-Aware Loss”, 2020, hereinafter “Jiang”) in view of Meyer et al (US 2017/0046827, 2014, hereinafter “Meyer”).
Regarding claim 1, Jiang discloses an electronic device (the RGB-D based face recognition; see the title and abstract), comprising: an image sensor; a depth sensor; and a controller (these hardware related features are inherent in the method of Jiang) to:
receive, from the image sensor, red-green-blue (RGB) information of an image in separate R, G, and B channels; receive, from the depth sensor, depth information of the image (see sec. 3.4, para.1, lines 1-4: “we use RGB-D facial images as the training data, to improve robustness to illumination conditions compared with RGB facial images. The RGB-D data are collected using low-cost sensors such as PrimeSense.” It would be noticed that RGB-D is a 4-d channel input as claimed by claim 11);
perform facial detection to identify a face in the RGB information (see sec.3.4, para.1 lines 4-5: “Using the RGB part of a facial image”);
truncate the depth information to exclude information for depth points not within a threshold see sec.3.4, para.1 lines 10-13 “we extract a face region from the corresponding depth image by transferring the RGB face region... We find the nose tip and crop the point cloud in the face region within an empirically set radius of 90mm.”); and
process the R, G, and B channels of the image and the truncated depth information according to a trained machine learning process to segment a foreground of the image from a background of the image (see fig.8a and sec.4.6, para.1, lines 1-3: “we concatenate RGB and depth [i.e., the cropped point cloud] into a six channel data as input to the 28-layer ResNet, which is a signal-level fusion scheme for RGB-D face recognition (see Fig. 8a).” It would be noticed that the depth information, i.e., the cropped point cloud, is segmented only from “the face region” as disclosed in sec.3.4, para.1, lines 9-13), the trained machine learning process to receive the R, G, and B channels of the image and the truncated depth information as input and to output a segmentation result (see fig.8a. It should be noticed that: the top row includes a RGB image and the corresponding depth information (the copped point cloud in the face region; the 3rd row is a CNN “ResNet-28” which performs face recognition to each of input pairs (i.e., a RBG and the corresponding face point cloud. See Table 5 and sec.4.6); and
Jiang does not explicitly disclose “exclude[ing] depth points which are not located in the identified face using a threshold distance” and “form[ing] an image mask by processing the RGB information and the truncated depth information according to the machine learning process” recited by the claim. However, in the same field of endeavor, that is, in the field of detecting head/face pose based RGB and depth (D) images, Meyer, para.71 teaches “analyz[ing] a depth image 405 to determine the silhouette of a user (e.g., via thresholding). A binary mask is then defined to identify pixels that are located inside of the boundary of the user's silhouette”, wherein the binary mask may be defined based on Equation 1”. In other words, the depth distances of the points within the head (face) mask are thresholding by dm<d(i, j)<dM ). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was made to incorporate the teachings of Meyer into the teachings of Jiang and further analyze the depth image and form a head/face mask based the threshold values. Suggestion or motivation for doing so would have been to solve the issue of “unsatisfactory results when images are acquired in poor lighting conditions” caused by traditional RBG-based only techniques, see para.6 of Meyer. Therefore, the combination of Jiang and Meyer suggests or teaches all the limitations recited in claim 1, and the claim is unpatentable over Jiang in view of Meyer.
Regarding claim 2, 13, the combination of Jiang and Meyer discloses, wherein the controller is to truncate the depth information according to Euclidean distance clustering (Meyer, using the depth distance thresholding to cluster the face region; see Eq(1) and para.71).
Regarding claim 3, 12, the combination of Jiang and Meyer discloses, wherein the controller is to sample the depth information within a bounding box that bounds the identified face to determine the depth of the identified face (Jiang, see training RGB-D facial data shown in the left col. of fig.3; see sec.3.4, para.1 lines 10-13 “we extract a face region from the corresponding depth image by transferring the RGB face region... We find the nose tip and crop the point cloud in the face region within an empirically set radius of 90mm.).
Regarding claim 4, the combination of Jiang and Meyer discloses the electronic device of claim 1, wherein the controller is to process the R, G, and B channels of the image and the truncated depth information according to a convolutional neural network (Jiang, see fig.3 (training phase) and fig.8 (testing phase)).
Regarding claim 6, Jiang discloses an electronic device, comprising controller to implement an image segmentation process to:
obtain color information of an image in separate color channels; obtain depth information of the image (see sec. 3.4, para.1, lines 1-4: “we use RGB-D facial images as the training data, to improve robustness to illumination conditions compared with RGB facial images. The RGB-D data are collected using low-cost sensors such as PrimeSense.” It would be noticed that RGB-D is a 4-d channel input as claimed by claim 11); determine a depth of a face represented in the color information (see the separate RGB face image and the depth face image represented by the point cloud shown in the 1st col. of fig.3); provide the color information of the image in the separate color channels and the depth of the face to a trained machine learning process to separate a foreground of the image from a background of the image (wherein the DCNN shown by fig.3 is by the training data; see sec, 3.34), the trained machine learning process to receive the color channels of the image and the truncated depth information as input and to output a segmentation result (the trained DCNN outputs the identified segmented faces according to the input face data; see fig.6), and
Jiang does not explicitly disclose “perform a depth cutoff of points of the depth information and separate the foreground of the image from the background of the image by processing the image and the cutoff depth information according to the trained machine learning process to form an image mask” recited by the claim. However, in the same field of endeavor, that is, in the field of detecting head/face pose based RGB and depth (D) images, Meyer, para.71 teaches “analyz[ing] a depth image 405 to determine the silhouette of a user (e.g., via thresholding). A binary mask is then defined to identify pixels that are located inside of the boundary of the user's silhouette”, wherein the binary mask may be defined based on Equation 1”. In other words, the head (face) mask are thresholding by dm<d(i, j)<dM. Also see the head region mask 520 in fig.5D. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was made to incorporate the teachings of Meyer into the teachings of Jiang and further analyze the depth image and form a head/face mask based the threshold values. Suggestion or motivation for doing so would have been to solve the issue of “unsatisfactory results when images are acquired in poor lighting conditions” caused by traditional RBG-based only techniques, see para.6 of Meyer. Therefore, the combination of Jiang and Meyer suggests or teaches all the limitations recited in the claim, and the claim is unpatentable over Jiang in view of Meyer.
Regarding claim 7, the combination of Jiang and Meyer discloses the electronic device of claim 6, wherein the controller is to perform facial detection on the color channels of the image to define a region of the image including the face and sample the depth information of the image within the region to determine the depth of the face (Jiang, sec.3.4, para.1: “Using the RGB part of a facial image, we first detect the face region and five landmarks... Afterward, we extract a face region from the corresponding depth image by transferring the RGB face region.”).
Regarding claim 8, the combination of Jiang and Meyer discloses the electronic device of claim 7, wherein the controller is to perform a depth cutoff of points of the depth information that have a greater distance from a viewpoint than the depth of the face plus a threshold value (Meyer, using the depth distance thresholding to cluster the face region; see Eq(1) and para.71).
Regarding claim 10, the combination of Jiang and Meyer discloses the electronic device of claim 9, wherein the controller is to apply the image mask to the image to segment the foreground of the image from the background of the image (Meyer, using the depth distance thresholding to cluster the face region; see Eq(1) and para.71).
Regarding claim 11, Jiang discloses a non-transitory computer-readable instructions which, when executed by a controller of an electronic device, cause the controller to:
obtain color information of an image in separate red, green, and blue color channels;
obtain depth information of the image (see sec. 3.4, para.1, lines 1-4: “we use RGB-D facial images as the training data, to improve robustness to illumination conditions compared with RGB facial images. The RGB-D data are collected using low-cost sensors such as PrimeSense.” It would be noticed that RGB-D is a 4-d channel input as claimed by claim 11); determine a depth of a face present in the image (see the separate RGB face image and the depth face image represented by the point cloud shown in the 1st col. of fig.3);
provide a 4-channel input to a machine learning process to process the image according to the cutoff depth information and the color information to separate a foreground of the image from a background of the image, the 4-channel input including the cutoff depth information, the red color channel, the green color channel, and the blue color channel (wherein the DCNN shown by fig.3 is by the training data, each of them includes three RGB face images and one corresponding depth face image shown in the left col of fig.3; see fig.3 and sec, 3.34), the trained machine learning process to receive the 4-channel input and to output a segmentation result (the trained DCNN outputs the identified segmented faces according to the input face data; see fig.6),), and
Jiang does not explicitly disclose “perform a depth cutoff of the depth information for points having greater than a threshold distance from the depth of the face” and “form an image mask by processing the 4-channel input according to the machine learning process” recited by the claim. However, in the same field of endeavor, that is, in the field of detecting head/face pose based RGB and depth (D) images, Meyer, para.71 teaches “analyz[ing] a depth image 405 to determine the silhouette of a user (e.g., via thresholding). A binary mask is then defined to identify pixels that are located inside of the boundary of the user's silhouette”, wherein the binary mask may be defined based on Equation 1”. In other words, the head (face) mask are thresholding by dm<d(i, j)<dM. Also see the head region mask 520 in fig.5D. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was made to incorporate the teachings of Meyer into the teachings of Jiang and further analyze the depth image and form a head/face mask based the threshold values. Suggestion or motivation for doing so would have been to solve the issue of “unsatisfactory results when images are acquired in poor lighting conditions” caused by traditional RBG-based only techniques, see para.6 of Meyer. Therefore, the combination of Jiang and Meyer suggests or teaches all the limitations recited in the claim, and the claim is unpatentable over Jiang in view of Meyer.
Regarding claims 14, 15, 17, 18, the combination of Jiang and Meyer discloses, wherein execution of the executable code causes the controller to overlay the foreground over a manipulated representation of the image (Jiang, see “sacked images” in fig.3).
Regarding claims 16, 19, 20, the combination of Jiang and Meyer discloses, wherein execution of the executable code causes the controller to concatenate the RGB information red, green, and blue color channels with the truncated depth information to generate the 4-channel input for the trained machine learning process (Jiang, see RGB-D 4-d images shown in the top row in fig.8).
Response to Arguments
8. Applicant's arguments submitted on 11/04/2025 have been considered but are moot in view of the new ground(s) of rejection. As explained in the claim rejections above, Meyer teaches the features missed by Jiang. The claimed inventions are unpatentable over Jiang in view of Meyer.
Conclusion
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RUIPING LI whose telephone number is (571)270-3376. The examiner can normally be reached 8:30am--5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HENOK SHIFERAW can be reached on (571)272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov; https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center, and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RUIPING LI/Primary Examiner, Ph.D., Art Unit 2676