DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed on August 5, 2026, has been entered. Claims 1–26 are currently pending. Claims 1, 5, 9, 13, 18, and 22 have been amended.
Response to Arguments
Applicant’s amendment and arguments concerning the rejection of claims 18–26 under 35 U.S.C. § 101 are persuasive with respect to the previously identified inclusion of transitory signals. Applicant amended independent claim 18 to expressly recite a “non-transitory computer readable medium”. Claims 19–26 incorporate that limitation through their dependence on claim 18. Accordingly, the amendment resolves the statutory-category deficiency identified in the previous Office Action, and the rejection of claims 18–26 under 35 U.S.C. § 101 on that basis is withdrawn.
Applicant’s amendment and arguments, in the Remarks pages 8–11, concerning the rejection of claims 1–20 under 35 U.S.C. § 112(a), the rejection of claims 5, 13, and 22 under 35 U.S.C. § 112(b), and the previous rejections under 35 U.S.C. § 103 have been reconsidered. Accordingly, the previously stated rejections under 35 U.S.C. § 112(a), 35 U.S.C. § 112(b) and 35 U.S.C. § 103 are withdrawn.
However, new grounds of rejection under 35 U.S.C. §103 are made in this Office action. The revised rejections additionally apply Saban and Ye to claims 1–3, 8–11, 16–20, and 25–26; and further in view of Yang in the rejection of claims 4–5, 12–13, and 21–22; and further in view of Tang in the rejection of claims 6–7, 14–15, and 23–24. These substantive limitations narrow the claimed combination beyond the subject matter addressed by the previous action. Because Applicant’s substantive amendments necessitated the new grounds of rejection presented in this Office action, this action is properly made final in accordance with MPEP §706.07(a).
Based on these facts, this action is made FINAL.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–3, 8–11, 16–20 and 25-26 are rejected under 35 U.S.C. §103 as being unpatentable over Choukroun (Choukroun et al, US 2016/0166145 A1, 2016) in view of Foley (Foley et al., US 6,535,223 B1, 2003), further in view of Saban (Saban et al, US 2022/0395176 A1, 2022), and further in view of Ye (Ye et al, US 2015/0146169 A1, 2015).
Regarding claim 1, Choukroun teaches a system for measuring facial features ([abstract], using an image sensor for determining at least one ocular measurement), the system comprising:
a camera configured to produce output signals on a channel corresponding to one or more images
( [0127], Choukroun discloses a system consisting of a camera (webcam, handheld camera, tablet, depth-map camera), a screen and along with other channel medium to acquire at least one image of the head of the user [0010]. )
a memory including computer-executable instructions; and a processor coupled to the memory and operative to execute the computer-executable instructions, the computer-executable instructions causing the processor to perform operations including:
( Choukroun discloses a system comprising: a processor as part of the system architecture [0129], a software package allowing the image capture and the calculations to be carried out may take the form of an application and/or an Internet site/plug-in for a web browser [0189], e.g. smartphone [0071]. )
(i) performing 3D face alignment of a user's face using a face landmark model to identify positions of facial features, including 2D iris landmarks, wherein using the 3D landmark locations, a 3D pose of the user's face is estimated, wherein the 3D pose includes roll, pitch, yaw, x, y, and z coordinates;
( Choukroun teaches metric 3D facial detection and tracking to optimize the pose of the face for image capture [0166]. The system uses a 3D facial model matched to 2D facial landmarks for tracking [0073] and real-time feedback on facial orientation is provided [0070]. Further, Choukroun discusses transforming 3D points using rotation and translation matrices Rm and Tm between the camera and facial components like the gauge and pupils [0090–0108]. These transformations are the mathematical equivalent of estimating 3D pose, where: Rm (rotation matrix) corresponds to roll, pitch, and yaw, whereas Tm (translation vector) corresponds to x, y, z position. This combination is a standard formulation for 6-DOF pose estimation in computer vision. Choukroun does not literally list "roll, pitch, yaw, x, y, z", Choukroun describes the underlying mathematical operations that any person of ordinary skill in the art would recognize as implementation of those parameters in pose estimation. )
(ii) performing reference card placement and detection, wherein the reference card is positioned on the user’s forehead and identified in the captured image; and
( Choukroun teaches a rectangular positioning guide displayed in the image to assist the user in validating facial position and depth relative to the camera [0171], corresponding to a rectangular ROI in this mapping. Choukroun further identifies a credit card as a reference gauge [0140], permits its placement on the forehead [0146], and describes identifying its image boundaries by fitting an adjustable rectangle to the card image [0207–0208]. )
(iii) performing one or more facial measurement calculations, including converting the size of the detected reference card in pixels to real-world dimensions, forming a pixel-to-distance ratio, and using the calculated pixel-to-distance ratio to convert distance between landmarks in pixels to actual facial measurements in metric units, wherein given an estimated camera FOV, a distance between the user and the camera is determined, and calculating one or more facial measurements, wherein the one or more facial measurements are scaled according to the user's distance from the camera
( [0177-0188]: Choukroun teaches calculating the actual size of facial features using the known physical size of a rigid object, e.g. credit card or eyeglasses, visible in the image. The equation D = dP × tMM / tMP uses the pixel size of the reference card (tMP) and its known size (tMM) to derive a scaling ratio. Choukroun describes using the above ratio to convert pixel distance between pupils or other facial features into real-world millimeters. If focal length is not known, it may be estimated using the apparent size of the reference object; camera-to-user distance dC is derived using iris or gauge size and perspective. Pupillary distance (PD) and face width are explicitly calculated using this scaling method. Pupillary distance (PD) and face width are explicitly calculated using this scaling method. Choukroun adjusts final measurements based on camera distance dC, applying corrections when the distance is small, e.g. <60 cm. )
Choukroun teaches calculating facial measurements by comparing image dimensions with a reference of known physical size [0178–0182] and identifies facial dimensions as a possible reference [0142]. Foley further teaches using the iris as a dimensional reference for the following calculation:
performing calculation, including estimating the user's pupillary distance (PD) using an average camera field of view (FOV) and estimated iris diameter based on the 2D iris landmarks, and deriving a rough PD scale from the estimated PD
( [Figs. 1, 7–8], [col. 6, ll. 10–63], [col. 7, final paragraph–col. 8, ll. 1–5]: Foley determines image scale by comparing the physical width of a reference with its pixel width, then applies that scale to image pupil separation to determine PD. Its fourth embodiment uses the iris as the reference, assuming an approximately 12 mm physical diameter. The reference’s image width is determined from two-dimensional endpoint coordinates, corresponding to iris-edge landmarks when the iris is used as the reference. )
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Foley’s iris-reference calculation into Choukroun’s facial-image processing to obtain a preliminary PD estimate before placement of the reference card. Choukroun already processes facial images before card placement [0158–0161], while Foley expressly identifies obtaining measurements without an additional reference object as an advantage of using the iris. Both techniques determine physical dimensions by scaling image distances against a reference dimension. Applying Foley’s iris reference to Choukroun’s available facial image would therefore predictably provide an approximate image scale and PD estimate without first requiring the card to appear in the image.
Choukroun [as modified by Foley] teaches estimating PD from image measurements. Saban further teaches using the estimated PD as a calibration reference for subsequent image measurements:
deriving a rough PD scale from the estimated PD wherein the derived rough PD scale is used to convert pixel distances into physical distances.
( [0017], [0093], [0108]: Saban describes initially obtaining PD through iris-based calibration and subsequently using that PD as a physical calibration length. A further calibration factor is calculated by dividing the physical calibration length by the corresponding pixel distance in a subsequent image. When PD is selected as the calibration length, this factor provides a PD-derived scale for converting pixel distances into physical distances. )
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Saban’s PD-based calibration technique into the method of Choukroun [as modified by Foley] so that the initially estimated PD could provide a calibration reference for subsequent images. The reason for doing so would have been to maintain image calibration when the user moves farther from the camera and iris measurements become less reliable, as explained in Saban’s paragraph [0093]. A reasonable expectation of success follows because the modification uses the estimated physical PD and the corresponding image pixel distance in the disclosed ratio calculation. This applies a known calibration technique to a similar image-measurement method to obtain the predictable result of converting pixel distances into physical distances.
Choukroun [as modified by Foley and Saban] provides an estimated PD and a PD-derived calibration scale. Ye further teaches displaying a rectangular ROI to guide reference-card placement and verifying that the card is positioned within the ROI. Applying Ye’s rectangular ROI guidance and detection techniques to Choukroun’s forehead placement, together with Saban’s scaling technique, would provide:
wherein the derived rough PD scale is used to form a rectangular region-of-interest (ROI) on the user's forehead in an image captured by the camera a size of the rectangular ROI in the image being determined by the derived rough PD scale, wherein the rectangular ROI is displayed to the user overlaid on the user's forehead and illustrates correct placement of the reference card on the user's forehead, and wherein the reference card is positioned in the rectangular ROI and detected by the camera; and
( [Figs. 12–14]; [0054–0065], [0066–0071]: Ye displays a rectangular card window based on facial tracking, calculates its dimensions and location from facial landmarks, and determines whether the card is positioned within the window. Its detection procedure extracts the card’s black band and checks whether the detected band’s four corner points lie inside the window. These disclosures support the displayed placement guide and image-based card detection. Ye’s illustrated window is located near the mouth and is sized using eye-corner distances; the forehead location and PD-derived sizing are supplied by the proposed modifications described below. )
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Ye’s displayed rectangular ROI and card-placement verification into Choukroun [as modified by Foley and Saban]’s method, and to position the ROI over the user’s forehead. Choukroun [as modified by Foley and Saban] expressly permits placement of the reference object on the forehead in paragraph [Choukroun > 0146]. Applying Ye’s guidance and detection techniques at that disclosed location would provide a visible target corresponding to the intended card position and permit verification that the card is within the ROI before proceeding with card-based measurement. Choukroun [as modified by Foley and Saban] describes deriving a calibration factor from a previously determined PD in paragraphs [Saban > 0017, 0093, and 0108]. Choukroun [as modified by Foley and Saban] additionally describes scaling virtual glasses according to measured PD in paragraph [Saban > 0072] and proportionately sizing objects superimposed on a user’s image in paragraph [Saban > 0091]. These teachings would have motivated applying the available PD-derived scale to Ye’s rectangular ROI so that the displayed guide approximately corresponds to the expected image size of the known reference card at Choukroun [as modified by Foley and Saban]’s disclosed forehead location.
Regarding claim 2, Choukroun [as modified by Foley, Saban and Ye] teaches the system of claim 1, wherein the one or more calculated facial measurements include a pupillary distance (PD).
( [0010]: Choukroun explicitly discloses measuring pupillary distance (PD), which is the primary measurement determined by Choukroun's system. )
Regarding claim 3, Choukroun [as modified by Foley, Saban and Ye] teaches the system of claim 1, wherein the one or more calculated facial measurements include a face width (FW).
( [0182-0183], Choukroun teaches using a reference object to determine a pixel-to-metric scaling ratio and then applying this ratio to various facial dimensions, including pupillary distance and distances between other facial landmarks. Foley reinforce use of facial landmarks for real-world metric calculations [Page. 11, Col. 6, Line 35-40]. )
Regarding claim 8, Choukroun [as modified by Foley, Saban and Ye] teaches the system of claim 1, wherein the reference object comprises a reference card.
( [0146]: Choukroun explicitly discloses using a reference card, such as a credit card, placed on the user’s forehead for scale calibration. )
Regarding claims 9–11 and 16, 18–20 and 25, the rationale provided for claims 1–3 and 8 is incorporated herein. In addition, the system for measuring facial features of claim 1–3 and 8 corresponds to the method of claims 9–11 and 16, as well as the computer-readable storage medium of claims 18–20 and 25, and performs the steps disclosed herein. Therefore, the claims are all rejected.
Regarding claim 17, Choukroun [as modified by Foley, Saban and Ye] teaches the method of claim 9, wherein detecting the reference object comprises edge detection.
( [0208] & [0211], Choukroun teaches how the card’s silhouette or outline is used to identify its projected shape: see the line / outline formed [corresponding to the edge detection] between these points and thus follow the silhouette of the card exactly, that form the card's perimeter; using this geometric shape to extract dimensions for scaling; tracking the projective deformation of the card outline in the image. )
Regarding claim 26, the rationale provided for claim 17 is incorporated herein. In addition, the method of using a camera for measuring point distances of facial features of claim 17 corresponds to the computer-readable storage medium of claim 26, and performs the steps disclosed herein. Therefore, the claim is rejected.
Claims 4–5, 12–13 and 21–22 are rejected under 35 U.S.C. §103 as being unpatentable over Choukroun [as modified by Foley, Saban and Ye], and further in view of Yang (Yang et al., US 2014/0160123 A1, 2014).
Regarding claim 4, Choukroun [as modified by Foley, Saban and Ye] fails to explicitly disclose where Yang teaches the system of claim 1, wherein the camera comprises an RGB camera including an RGB sensor, wherein the RGB camera is configured to produce output signals on an RGB channel corresponding to one or more RGB images.
( [0005], Yang explicitly teaches using an RGB camera including an RGB sensor for facial/ head image capture and 3D modeling. Use of an RGB camera was well-known and expected in such systems, almost all standard consumer and professional digital cameras are RGB cameras. )
It would have been prima facie obvious to a POSITA, before the effective filing date of the claimed invention, to implement Choukroun [as modified by Foley, Saban and Ye]’s facial measurement system using an RGB camera as taught by Yang, because Yang demonstrates that RGB cameras are well-suited and commonly used for facial tracking and analysis. Substituting Yang’s explicitly disclosed RGB camera into Choukroun [as modified by Foley, Saban and Ye]’s system is a predictable use of prior art elements according to their established functions.
Regarding claim 5, Choukroun [as modified by Foley, Saban and Ye] fails to explicitly disclose where Yang teaches the system of claim 4, wherein the one or more images comprise a plurality of frames of video from the RGB camera.
( [0005] & [0006], Yang explicitly teaches capturing a plurality of RGB video frames from an RGB camera for facial imaging. )
It would have been prima facie obvious to a POSITA, before the effective filing date of the claimed invention, to implement Choukroun [as modified by Foley, Saban and Ye]’s facial measurement system using video frames from an RGB camera, as taught by Yang, in order to improve robustness and accuracy of facial tracking and alignment. The combination merely applies Yang’s known RGB video acquisition technique to Choukroun [as modified by Foley, Saban and Ye]’s facial measurement workflow and yields predictable results.
Regarding claims 12–13, 21–22, the rationale provided for claims 4–5 is incorporated herein. In addition, the system for measuring facial features of claim 4–5 corresponds to the method of claims 12–13, as well as the computer-readable storage medium of claims 21–22, and performs the steps disclosed herein. Therefore, the claims are all rejected.
Claims 6–7, 14–15 and 23–24 are rejected under 35 U.S.C. §103 as being unpatentable over Choukroun [as modified by Foley, Saban and Ye], and further in view of Tang (Tang et al., WO 2016/026063 A1, 2016).
Regarding claim 6, Choukroun [as modified by Foley, Saban and Ye] fails to explicitly disclose where Tang teaches the system of claim 1, wherein the face landmark model comprises a convolutional neural network.
( [0016]: Tang discloses a face landmark system comprises a convolutional neural network.)
It would have been prima facie obvious to a POSITA, before the effective filing date of the claimed invention, to incorporate Tang’s convolutional detection neural network into Choukroun [as modified by Foley, Saban and Ye]’s facial measurement system in order to enhance accuracy and robustness of facial landmark localization. Tang teaches a multi-layer convolutional neural network (CNN) architecture trained to extract facial features and predict landmark positions, which includes deep layers and auxiliary tasks, [0025] & [0053]. The combination is a predictable integration of Tang’s known deep-learning-based facial landmark models with Choukroun [as modified by Foley, Saban and Ye]’s image-based facial measurement system, improving precision in face alignment and supporting landmark detection under varied lighting and pose conditions.
Regarding claim 7, Choukroun [as modified by Foley, Saban and Ye] fails to explicitly disclose where Tang teaches the system of claim 1, wherein the face landmark model comprises a deep landmark detection network.
( [0025]: Tang discloses a face landmark detection system comprises a convolutional neural network, "a plurality of (for example, three) convolution-pooling layers comprising one or more (for example, three) convolutional layers and one or more (for example, three) pooling layers, one convolutional layer and one fully connected layer". A training unit optimizes multiple auxiliary tasks alongside landmark detection, creating a multi-task deep learning structure. )
It would have been prima facie obvious to a POSITA, before the effective filing date of the claimed invention, to implement Tang’s deep landmark detection network into Choukroun [as modified by Foley, Saban and Ye]’s facial measurement system in order to improve accuracy and stability of facial landmark identification used for measurement and alignment. The combination is a predictable integration of Tang’s known deep-learning-based landmark detector into Choukroun [as modified by Foley, Saban and Ye]’s image-based facial measurement system.
Regarding claims 14–15 and 23–24, the rationale provided for claims 6-7 is incorporated herein. In addition, the system for measuring facial features of claims 6-7 corresponds to the method of claims 14-15, as well as the computer-readable storage medium of claims 23-24, and performs the steps disclosed herein. Therefore, the claims are all rejected.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEN KUDO whose telephone number is (571)272-4498. The examiner can normally be reached M-F 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KEN KUDO
Examiner
Art Unit 2671
/KEN KUDO/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
