DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 34 is objected to because of the following informalities: claim 34 line 10 cites “which one or more extracted landmarks form one or more target physical features” which should be “which one or more extracted landmarks that form one or more target physical features”. Appropriate correction is required.
Response to Arguments
Applicant's arguments filed 06/24/2026 have been fully considered but they are not persuasive.
Applicant argues: Chen (US 20190035149 A1) does not establish anticipation. (claim 34)
Examiner replies: In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., anticipation of the particular specification-driven mesh deformation sequence ) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Applicant argues: Chen does not disclose extracting landmark from the 3D polygonal mesh. (claim 34)
Examiner replies: Chen discloses 3D scan of the user’s face. A 3D scan creates a polygonal mesh by capturing millions of individual points (a point cloud) and connecting them together with tiny flat shapes, usually triangles.
(FIG. 6; para. [0069], “ (i) acquiring at least one 3D scan of the user's face;”; para. [0209], " To reconstruct a user's 3D face, we first analyze the input image and extract the shape features of the user's face. To achieve that, we detect the 2D facial landmarks automatically in our pipeline by integrating a 2D face landmark detector, which, in an example, can be provided by an open source image processing and computer vision library, e.g. DLib or OpenCV….” para. [0300] - [0304], " 2.3 From a Raw 3D Face Scan; The third stream of approaches for 3D personalised face reconstruction is to create a user's 3D head model from an arbitrary depth scan or a 3D scan of their face. A raw 3D scan of a user's face can be obtained from three sources: Source 1: from an image-based 3d reconstruction process using the techniques of structure from motion (SfM) [14] or simultaneous localisation and mapping (SLAM) [12]. Source 2: from a depth scan captured by a commercial depth camera (e.g. Microsoft Kinect, Intel RealSense F200/R200); Source 3: from a full 3D scan, captured using a 3D scanner, e.g. Artec Eva. Below we describe the detailed processes for creating the 3D geometry and appearance model from the face scan input”)
Applicant argues: Chen does not disclose a Landmark Extraction Specification that species physical feature using semantic labels. (claim 34)
Examiner replies: Chen discloses the landmark detector based on a regression forest algorithm which detects 68 face landmarks. Chen also discloses the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc that correspond to semantic labels.
(FIG. 6; para. [0069], “ (i) acquiring at least one 3D scan of the user's face;”; para. [0209], " To reconstruct a user's 3D face, we first analyze the input image and extract the shape features of the user's face. To achieve that, we detect the 2D facial landmarks automatically in our pipeline by integrating a 2D face landmark detector, which, in an example, can be provided by an open source image processing and computer vision library, e.g. DLib or OpenCV. In an example, the detector we adopted is implemented based on a regression forest algorithm [18]. It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc. This detector is proved to be reasonably robust against input images with different lighting conditions, head pose changes, and facial expression. The module can, however, be replaced by other more sophisticated 2D or 3D face landmark detectors or trackers, e.g. the 3D Constraint Local Model (CLM) based facial landmark detector”)
Applicant argues: Chen does not disclose determining extracted landmarks that form target physical features indicated by a mesh editing specification and deforming target physical features in a way specified by a mesh editing specification. (claim 34)
Examiner replies: Chen discloses extracted landmarks that form target physical features indicated by a mesh editing specification and deforming target physical features in a way specified by a mesh editing specification. Training a regressor R gives mapping landmark positions to model parameters of 3D morphable head model. By defining intuitive face biometrics based on the facial landmarks as a variant of face regression, predict the full 3D head model. Deforming the initial head model to generate 3D head model that much more similar to the user’s face.
(para. [0212], "Generate an Approximate Geometry Using Shape Priors: In the first stage, we find an approximate head geometry as an initialisation using a generative shape prior that models shape variation of an object category (i.e. the face) in the low dimensional subspace with a dimension reduction method. The full head geometry of the user can be reconstructed from this low dimensional shape prior with a small number of parameters. A representable approach for head modelling is to learn a 3D morphable head model (3DMHM) [8], in which principal component analysis (PCA) is used to capture dominant modes of human head shape variation. In 3DMHM the parameters are the PCA weights w={w.sub.p}.sub.p=1.sup.P."; para. [0216], "1. predicting the PCA weights of the 3DMHM from 2D landmark positions. This is done by training a regressor R that gives a mapping from the M normalised 2D landmark positions {circumflex over (L)}={Î.sub.i}.sub.i=1.sup.M to the underlying model parameters {w.sub.p}.sub.p=1.sup.P of the 3D morphable head model, where each 2D landmark position I.sub.i=(l.sub.i,x, l.sub.i,y) is first normalised by the dimension H.sub.i, W.sub.i, and the centre c.sub.i=(c.sub.i,x, c.sub.i,y) of the face detection bounding box."; para. [0217], "From the morph weights {w.sub.p}.sub.p32 1.sup.P, we can reconstruct the full 3D head geometry X.sub.user of the user as follows."; para.[0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."; para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.”; para. [0220], "In the algorithm, we first find a similarity transform T* (a 3×2 matrix) based on the layout of the 2D image landmarks {I.sub.i}.sub.i=1.sup.M to the frontal projection {circumflex over (L)}.sub.i=(L.sub.i,x, L.sub.i,y) of the corresponding 3D landmarks L.sub.i=(L.sub.i,x, L.sub.i,y, L.sub.i,z) (i=1, 2, . . . , M) of the 3D head model X.sub.user obtained from the shape regression above. This can be obtained by solving the following least squares problem in (4).”; para. [0221], " We then use a 3D thin-plate spline (TPS) deformation model [9] to rectify the 3D geometry of the regressed head model X.sub.user to achieve better geometric similarity. To implement that, we define the M source and sink control point pairs {(s.sub.i, t.sub.i)}.sub.i=1.sup.M of the 3D TPS model as:
PNG
media_image1.png
98
677
media_image1.png
Greyscale
”)
Applicant argues: Chen does not disclose editing other parts of the 3D polygonal mesh as specified by the mesh editing specification. (claim 34)
Examiner’s replies: Chen disclose clipping (editing) the mesh of the user’s head model using the mesh clipping algorithm.
(para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”)
Applicant argues: The combination (Chen and Joyner (US 20240048932 A1)) does not teach generating an HR filter from the edited 3D polygonal mesh. (Claim 44)
Examiner’s replies: Chen discloses generating 3D polygonal mesh.
(para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”; para.[0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters.")
Joyner discloses generating personalized HR filter based on 3D scans.
(Abstract, “An apparatus and method of generating personalized HRTFs. The system is prepared by calculating a model for HRTFs described as the relationship between a finite example set of input data, namely anthropometric measures and demographic information for a set of individuals, and a corresponding set of output data, namely HRTFs numerically simulated using a high-resolution database of 3D scans of the same set of individuals”; para. [0009], “According to an embodiment, a method generates head-related transfer functions (HRTFs). The method includes generating an HRTF calculation system, and generating a personalized HRTF for a user using the HRTF calculation system”)
Chen and Joyner are from the same filed of endeavor, generating 3D polygonal mesh, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include generating an HR filter from the edited 3D polygonal mesh by Chen according to the teaching of Joyner in order to generate custom audio based on listener's anthropometric characteristic (para. [0002] of Joyner).
Chen and Joyner are reasonably pertinent to the problem which is obtaining personalized HR filter to provide custom audio based on listener's anthropometric characteristic. See MPEP § 2141.01(A).
Applicant argues: The combination (Chen and Joyner (US 20240048932 A1)) does not teach generating multiple HR filters by deforming the same 3D polygonal mesh according to multiple different mesh editing specifications. (Claim 45)
Examiner’s replies: Chen discloses deforming the same 3D polygonal mesh. Finding the most suitable shape or appropriate 3D shape from a library corresponds to the same 3D polygonal mesh when it’s selected for the different multiple users.
(FIG. 10; Examiner’s note: FIG. 10 corresponds to different anatomical objects.; para. [0215], "2. using a shape prior selection process to find the most suitable shape prior from a library using selection criteria such as the user's ethnicity, gender, age, and other attributes. The system diagram example of this solution is illustrated in FIG. 7. In the pipeline, we introduce an additional machine-learning-based attribute classifier, which can be implemented by e.g. a deep convolutional neural network (CNN) [19, 21, 29], to analyze the user's photo, and predict attributes (e.g. ethnicity, gender, and age) from the appearance information (i.e. skin colour, hair colour and styles, etc.) in the image. We select the appropriate 3D shape prior from the library based on matching a user's attributes with those defined for each shape prior. This method can recover the missing depth information of the user's face more accurately. It is useful for building a product that will work across different ethnic regions. See FIG. 8 for an example of image-based face attribute classification implemented by CNN classifiers."; para.[0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."; para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.”; para. [0220], "In the algorithm, we first find a similarity transform T* (a 3×2 matrix) based on the layout of the 2D image landmarks {I.sub.i}.sub.i=1.sup.M to the frontal projection {circumflex over (L)}.sub.i=(L.sub.i,x, L.sub.i,y) of the corresponding 3D landmarks L.sub.i=(L.sub.i,x, L.sub.i,y, L.sub.i,z) (i=1, 2, . . . , M) of the 3D head model X.sub.user obtained from the shape regression above. This can be obtained by solving the following least squares problem in (4).”; para. [0221], " We then use a 3D thin-plate spline (TPS) deformation model [9] to rectify the 3D geometry of the regressed head model X.sub.user to achieve better geometric similarity. To implement that, we define the M source and sink control point pairs {(s.sub.i, t.sub.i)}.sub.i=1.sup.M of the 3D TPS model as:"; para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”; Examiner's note: the same 3D shape prior can be selected for different multiple users based on their ethnicity, gender, age and other attributes.)
Joyner discloses generating multiple HR filters.
(Abstract, “An apparatus and method of generating personalized HRTFs. The system is prepared by calculating a model for HRTFs described as the relationship between a finite example set of input data, namely anthropometric measures and demographic information for a set of individuals, and a corresponding set of output data, namely HRTFs numerically simulated using a high-resolution database of 3D scans of the same set of individuals”; para. [0009], “According to an embodiment, a method generates head-related transfer functions (HRTFs). The method includes generating an HRTF calculation system, and generating a personalized HRTF for a user using the HRTF calculation system”)
As both Chen and Joyner are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include generating multiple HR filters personalized in the context of 3D polygonal mesh deformation, by Chen according to the teaching of Joyner in order to generate custom audio based on listener's anthropometric characteristic (para. [0002] of Joyner).
Chen and Joyner are reasonably pertinent to the problem which is obtaining personalized HR filter to provide custom audio based on listener's anthropometric characteristic. See MPEP § 2141.01(A).
Regarding the remaining arguments: Applicant argues with respect to the claim language, which is fully addressed in the prior art rejections set forth below.
Conclusion: The rejections set forth in the previous Office Action are shown to have been proper, and the claims are rejected below. Therefore, the present Office Action is made final.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 34-36, 38-41, 46-48, and 50-53 are rejected under 35 U.S.C. 102(a)(1)/(a)(2)as being anticipated by Chen et al. (US 20190035149 A1; IDS REF) (hereinafter Chen).
Regarding claim 34, Chen discloses A method performed by computing equipment for deforming a three- dimensional (3D) polygonal mesh, the method comprising: (Tittle, "Methods of generating personalized 3D head models…")
extracting, from the 3D polygonal mesh, one or more landmarks that form one or more physical features specified by a landmark extraction specification, wherein the landmark extraction specification specifies the one or more physical features by including one or more semantic labels of the one or more physical features, (FIG. 6; para. [0069], “ (i) acquiring at least one 3D scan of the user's face;”; para. [0209], " To reconstruct a user's 3D face, we first analyze the input image and extract the shape features of the user's face. To achieve that, we detect the 2D facial landmarks automatically in our pipeline by integrating a 2D face landmark detector, which, in an example, can be provided by an open source image processing and computer vision library, e.g. DLib or OpenCV. In an example, the detector we adopted is implemented based on a regression forest algorithm [18]. It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc. This detector is proved to be reasonably robust against input images with different lighting conditions, head pose changes, and facial expression. The module can, however, be replaced by other more sophisticated 2D or 3D face landmark detectors or trackers, e.g. the 3D Constraint Local Model (CLM) based facial landmark detector"; para. [0300] - [0304], " 2.3 From a Raw 3D Face Scan; The third stream of approaches for 3D personalised face reconstruction is to create a user's 3D head model from an arbitrary depth scan or a 3D scan of their face. A raw 3D scan of a user's face can be obtained from three sources: Source 1: from an image-based 3d reconstruction process using the techniques of structure from motion (SfM) [14] or simultaneous localisation and mapping (SLAM) [12]. Source 2: from a depth scan captured by a commercial depth camera (e.g. Microsoft Kinect, Intel RealSense F200/R200); Source 3: from a full 3D scan, captured using a 3D scanner, e.g. Artec Eva. Below we describe the detailed processes for creating the 3D geometry and appearance model from the face scan input."; para. [0212], "Generate an Approximate Geometry Using Shape Priors: In the first stage, we find an approximate head geometry as an initialisation using a generative shape prior that models shape variation of an object category (i.e. the face) in the low dimensional subspace with a dimension reduction method.”)
PNG
media_image2.png
549
825
media_image2.png
Greyscale
wherein the one or more semantic labels are associated with one or more parameters according to which the one or more landmarks are to be extracted; (FIG. 6; para. [0029], " In an example, the detector we adopted is implemented based on a regression forest algorithm [18]. It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc.)
determining which one or more extracted landmarks form one or more target physical features that a mesh editing specification indicates are to be deformed;
deforming the one or more target physical features in a way specified by the mesh editing specification by manipulating the one or more determined landmarks as one or more handles; and (para. [0212], "Generate an Approximate Geometry Using Shape Priors: In the first stage, we find an approximate head geometry as an initialisation using a generative shape prior that models shape variation of an object category (i.e. the face) in the low dimensional subspace with a dimension reduction method. The full head geometry of the user can be reconstructed from this low dimensional shape prior with a small number of parameters. A representable approach for head modelling is to learn a 3D morphable head model (3DMHM) [8], in which principal component analysis (PCA) is used to capture dominant modes of human head shape variation. In 3DMHM the parameters are the PCA weights w={w.sub.p}.sub.p=1.sup.P."; para. [0216], "1. predicting the PCA weights of the 3DMHM from 2D landmark positions. This is done by training a regressor R that gives a mapping from the M normalised 2D landmark positions {circumflex over (L)}={Î.sub.i}.sub.i=1.sup.M to the underlying model parameters {w.sub.p}.sub.p=1.sup.P of the 3D morphable head model, where each 2D landmark position I.sub.i=(l.sub.i,x, l.sub.i,y) is first normalised by the dimension H.sub.i, W.sub.i, and the centre c.sub.i=(c.sub.i,x, c.sub.i,y) of the face detection bounding box."; para. [0217], "From the morph weights {w.sub.p}.sub.p32 1.sup.P, we can reconstruct the full 3D head geometry X.sub.user of the user as follows."; para.[0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."; para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.”; para. [0220], "In the algorithm, we first find a similarity transform T* (a 3×2 matrix) based on the layout of the 2D image landmarks {I.sub.i}.sub.i=1.sup.M to the frontal projection {circumflex over (L)}.sub.i=(L.sub.i,x, L.sub.i,y) of the corresponding 3D landmarks L.sub.i=(L.sub.i,x, L.sub.i,y, L.sub.i,z) (i=1, 2, . . . , M) of the 3D head model X.sub.user obtained from the shape regression above. This can be obtained by solving the following least squares problem in (4).”; para. [0221], " We then use a 3D thin-plate spline (TPS) deformation model [9] to rectify the 3D geometry of the regressed head model X.sub.user to achieve better geometric similarity. To implement that, we define the M source and sink control point pairs {(s.sub.i, t.sub.i)}.sub.i=1.sup.M of the 3D TPS model as:
PNG
media_image1.png
98
677
media_image1.png
Greyscale
”)
editing one or more other parts of the 3D polygonal mesh as specified by the mesh editing specification, to account for deformation of the one or more target physical features. (para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”)
As discussed above, Chen discloses (1) extracting landmarks from a 3D polygonal mesh; (2) a landmark extraction specification that specifies physical features using semantic labels; (3) semantic labels associated with parameters according to which landmarks are extracted; (4) determining extracted landmarks that form target physical features indicated by a mesh editing specification; (5) deforming those target
physical features in a way specified by the mesh editing specification by manipulating the determined landmarks as handles; and (6) editing other mesh parts as specified by the mesh editing specification to account for the deformation.
Regarding claim 35, Chen discloses The method according to claim 34,
wherein the landmark extraction specification specifies the one or more parameters according to which the one or more landmarks are to be extracted. (FIG 6; para. [0209], " To reconstruct a user's 3D face, we first analyze the input image and extract the shape features of the user's face. To achieve that, we detect the 2D facial landmarks automatically in our pipeline by integrating a 2D face landmark detector, which, in an example, can be provided by an open source image processing and computer vision library, e.g. DLib or OpenCV. In an example, the detector we adopted is implemented based on a regression forest algorithm [18]. It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc. This detector is proved to be reasonably robust against input images with different lighting conditions, head pose changes, and facial expression. The module can, however, be replaced by other more sophisticated 2D or 3D face landmark detectors or trackers, e.g. the 3D Constraint Local Model (CLM) based facial landmark detector")
Regarding claim 36, Chen discloses The method according to claim 34, wherein the one or more parameters according to which the one or more landmarks are to be extracted include, for each of the one or more physical features, one or more of: (see supra rejection of claim 34)
a view of the 3D polygonal mesh from which a two-dimensional (2D) outline of the 3D polygonal mesh is to be extracted; (FIG. 6; FIG. 5; para. [0203], "A typical end-to-end process for the single-view-based 3D face reconstruction is illustrated for example in FIG. 5"; para. [0209], "It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc."
PNG
media_image3.png
561
826
media_image3.png
Greyscale
a resolution of points that are to form the 2D outline; or
a range of points on the 2D outline within which to search for one or more landmarks that form the physical feature.
Regarding claim 38, Chen discloses The method according to claim 34, wherein the mesh editing specification indicates the one or more target physical features (see supra rejection of claim 34) by indicating one or more semantic labels of the one or more target physical features. (para. [0212], "Generate an Approximate Geometry Using Shape Priors: In the first stage, we find an approximate head geometry as an initialisation using a generative shape prior that models shape variation of an object category (i.e. the face) in the low dimensional subspace with a dimension reduction method. The full head geometry of the user can be reconstructed from this low dimensional shape prior with a small number of parameters. A representable approach for head modelling is to learn a 3D morphable head model (3DMHM) [8], in which principal component analysis (PCA) is used to capture dominant modes of human head shape variation. In 3DMHM the parameters are the PCA weights w={w.sub.p}.sub.p=1.sup.P."; para. [0216], "1. predicting the PCA weights of the 3DMHM from 2D landmark positions. This is done by training a regressor R that gives a mapping from the M normalised 2D landmark positions {circumflex over (L)}={Î.sub.i}.sub.i=1.sup.M to the underlying model parameters {w.sub.p}.sub.p=1.sup.P of the 3D morphable head model, where each 2D landmark position I.sub.i=(l.sub.i,x, l.sub.i,y) is first normalised by the dimension H.sub.i, W.sub.i, and the centre c.sub.i=(c.sub.i,x, c.sub.i,y) of the face detection bounding box."; para.[0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."
Regarding claim 39, Chen discloses The method according to claim 34, wherein the mesh editing specification specifies the way that the one or more target physical features are to be deformed by specifying, for each target physical feature, one or more of:
an amount, ratio, or coefficient by which the target physical feature is to be moved or scaled; or (para.[ 0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."; para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.” para. [0220], "In the algorithm, we first find a similarity transform T* (a 3×2 matrix) based on the layout of the 2D image landmarks {I.sub.i}.sub.i=1.sup.M to the frontal projection {circumflex over (L)}.sub.i=(L.sub.i,x, L.sub.i,y) of the corresponding 3D landmarks L.sub.i=(L.sub.i,x, L.sub.i,y, L.sub.i,z) (i=1, 2, . . . , M) of the 3D head model X.sub.user obtained from the shape regression above. This can be obtained by solving the following least squares problem in (4).
PNG
media_image4.png
54
670
media_image4.png
Greyscale
”;
para. [0221], " We then use a 3D thin-plate spline (TPS) deformation model [9] to rectify the 3D geometry of the regressed head model X.sub.user to achieve better geometric similarity. To implement that, we define the M source and sink control point pairs {(s.sub.i, t.sub.i)}.sub.i=1.sup.M of the 3D TPS model as:
PNG
media_image5.png
58
399
media_image5.png
Greyscale
”)
an angle by which the target physical feature is to be rotated.
Regarding claim 40, Chen discloses The method according to claim 34, wherein manipulating the one or more handles comprises re-locating the one or more handles in the 3D polygonal mesh as needed to move, scale, and/or rotate the one or more target physical features to an extent specified by the mesh editing specification, and (para.[ 0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."; para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.” para. [0220], "In the algorithm, we first find a similarity transform T* (a 3×2 matrix) based on the layout of the 2D image landmarks {I.sub.i}.sub.i=1.sup.M to the frontal projection {circumflex over (L)}.sub.i=(L.sub.i,x, L.sub.i,y) of the corresponding 3D landmarks L.sub.i=(L.sub.i,x, L.sub.i,y, L.sub.i,z) (i=1, 2, . . . , M) of the 3D head model X.sub.user obtained from the shape regression above. This can be obtained by solving the following least squares problem in (4).
PNG
media_image4.png
54
670
media_image4.png
Greyscale
”;
para. [0221], " We then use a 3D thin-plate spline (TPS) deformation model [9] to rectify the 3D geometry of the regressed head model X.sub.user to achieve better geometric similarity. To implement that, we define the M source and sink control point pairs {(s.sub.i, t.sub.i)}.sub.i=1.sup.M of the 3D TPS model as:
PNG
media_image5.png
58
399
media_image5.png
Greyscale
”;
(Examiner's note: face landmark positions correspond to handles.) wherein editing the one or more other parts of the 3D polygonal mesh comprises editing the one or more other parts of the 3D polygonal mesh according to an algorithm specified by the mesh editing specification, constrained by the one or more handles as re-located. (para. [0221], " We then use a 3D thin-plate spline (TPS) deformation model [9] to rectify the 3D geometry of the regressed head model X.sub.user to achieve better geometric similarity. To implement that, we define the M source and sink control point pairs {(s.sub.i, t.sub.i)}.sub.i=1.sup.M of the 3D TPS model as:
PNG
media_image5.png
58
399
media_image5.png
Greyscale
”;
para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”)
Regarding claim 41, Chen discloses The method according to claim 34,
further comprising extracting, as a function of the one or more landmarks, one or more regions of interest from the 3D polygonal mesh according to a region of interest extraction specification that specifies one or more parameters according to which the one or more regions of interest are to be extracted, wherein each of the one or more regions of interest is a region within which the one or more target physical features are to be deformed, and (FIG. 6; para. [0069], “ (i) acquiring at least one 3D scan of the user's face;”; para. [0209], " To reconstruct a user's 3D face, we first analyze the input image and extract the shape features of the user's face. To achieve that, we detect the 2D facial landmarks automatically in our pipeline by integrating a 2D face landmark detector, which, in an example, can be provided by an open source image processing and computer vision library, e.g. DLib or OpenCV. In an example, the detector we adopted is implemented based on a regression forest algorithm [18]. It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc. This detector is proved to be reasonably robust against input images with different lighting conditions, head pose changes, and facial expression. The module can, however, be replaced by other more sophisticated 2D or 3D face landmark detectors or trackers, e.g. the 3D Constraint Local Model (CLM) based facial landmark detector"; para. [0300] - [0304], " 2.3 From a Raw 3D Face Scan; The third stream of approaches for 3D personalised face reconstruction is to create a user's 3D head model from an arbitrary depth scan or a 3D scan of their face. A raw 3D scan of a user's face can be obtained from three sources: Source 1: from an image-based 3d reconstruction process using the techniques of structure from motion (SfM) [14] or simultaneous localisation and mapping (SLAM) [12]. Source 2: from a depth scan captured by a commercial depth camera (e.g. Microsoft Kinect, Intel RealSense F200/R200); Source 3: from a full 3D scan, captured using a 3D scanner, e.g. Artec Eva. Below we describe the detailed processes for creating the 3D geometry and appearance model from the face scan input."; para. [0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."; para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.”; Examiner's note: 3D face landmark detector indicates the landmark detection from the 3D polygonal mesh and landmarks correspond to regions of interest.) wherein the one or more semantic labels are associated with one or more parameters according to which the one or more landmarks are to be extracted; (para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”)
Regarding claim 46, 52, and 53, the claims are computing equipment claims of method claims 34, 40, and 41 respectively. The claims are similar scope to claims 34, 40, and 41 respectively so they are rejected under similar rationales as claims 34, 40, and 41 respectively.
Regarding claims 47, 48, 50, and 51, the claims are computing equipment claims of 35, 36, 38, and 39 respectively except the processing circuity (para. [0013], processor may be may be configured to perform steps (i) to (v) of the method, or steps (ii) to (v) of the method.). The claims are similar scope to claims 35, 46, 38, and 39 respectively so they are rejected under similar rationales as claims 35, 46, 38, and 39 respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 37 and 49 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US 20190035149 A1; IDS REF) (hereinafter Chen) in view of Ng-Thow-Hing (US 20180095276 A1) (hereinafter Ng).
Regarding Claim 37, Chen discloses The method according to claim 34, wherein extracting the one or more landmarks comprises: (see supra rejection of claim 34)
for each of one or more views specified by the landmark extraction specification, extracting a 2D outline of the 3D polygonal mesh from a perspective of the view and at a resolution specified by the landmark extraction specification; and (FIG. 5; para. [0203], "A typical end-to-end process for the single-view-based 3D face reconstruction is illustrated for example in FIG. 5"; para. [0209], " To reconstruct a user's 3D face, we first analyze the input image and extract the shape features of the user's face. To achieve that, we detect the 2D facial landmarks automatically in our pipeline by integrating a 2D face landmark detector, which, in an example, can be provided by an open source image processing and computer vision library, e.g. DLib or OpenCV. In an example, the detector we adopted is implemented based on a regression forest algorithm [18]. It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc. This detector is proved to be reasonably robust against input images with different lighting conditions, head pose changes, and facial expression. The module can, however, be replaced by other more sophisticated 2D or 3D face landmark detectors or trackers, e.g. the 3D Constraint Local Model (CLM) based facial landmark detector")
searching for the one or more landmarks within one or more ranges of points on the 2D outline specified by the landmark extraction specification. (FIG. 6; para. [0209], " To reconstruct a user's 3D face, we first analyze the input image and extract the shape features of the user's face. To achieve that, we detect the 2D facial landmarks automatically in our pipeline by integrating a 2D face landmark detector, which, in an example, can be provided by an open source image processing and computer vision library, e.g. DLib or OpenCV. In an example, the detector we adopted is implemented based on a regression forest algorithm [18]. It is able to detect N.sub.L=68 face landmarks from the image (see FIG. 6 for example), which characterise the positions and silhouettes of eyes, eyebrows, nose, mouth, lips, the jaw line, etc. This detector is proved to be reasonably robust against input images with different lighting conditions, head pose changes, and facial expression. The module can, however, be replaced by other more sophisticated 2D or 3D face landmark detectors or trackers, e.g. the 3D Constraint Local Model (CLM) based facial landmark detector"; Examiner’s note: FIG. 6 indicates one of more ranges of points on the 2D outline.)
PNG
media_image2.png
549
825
media_image2.png
Greyscale
Chen does not explicitly disclose extracting a 2D outline of the 3D polygonal mesh from a perspective of the view and at a resolution specified.
However, Ng more explicitly teaches extracting a 2D outline of the 3D polygonal mesh from a perspective of the view and at a resolution specified. (FIG. 6; FIG. 8; para. [0064], "At step 404, the system 300 (e.g., the image processor 310 thereof) generates a planar polygon mesh from the received images/videos. Referring briefly to FIG. 6, it illustrates planar polygon/triangular meshes 602, according to one embodiment. One or more planar polygon meshes (e.g., the planar polygon/triangular meshes 602 as shown in FIG. 6) may be formed from the images/videos/3-D point clouds captured or constructed from the physical environment to delineate and/or represent surfaces in the physical environment. In one embodiment, a marching cubes algorithm is used to generate a nonplanar mesh from the received images/videos, and the planar polygon mesh 602 is derived from the nonplanar mesh."; para. [0068]-[0069], "At step 408, the system 300 (e.g., the image processor 310 thereof) generates a convex hull from the extracted boundary polygon. Referring briefly to FIG. 8, it illustrates a convex hull 806 generated from an extracted boundary polygon 804, according to one embodiment. The convex hull 806 around the boundary polygon 804 is a mathematical function analogous to wrapping the boundary polygon 804 with a tight rubber band or tight plastic wrap. The result is that no line on the convex hull 806 penetrates the boundary polygon 804 to reach inside of the boundary polygon 804. In one embodiment, the convex hull 806 is generated from the boundary polygon 804 using a Graham-Scan algorithm. Generating the convex hull 806 from the boundary polygon 804 may include drawing lines to connect the “high points” of the boundary polygon 804 (e.g., the points of the boundary polygon 804 farthest away from a center thereof). Both the boundary polygon 804 and the convex hull 806 represent a vertical surface of a real-world physical whiteboard 802.")
PNG
media_image6.png
541
817
media_image6.png
Greyscale
PNG
media_image7.png
542
642
media_image7.png
Greyscale
As both Chen and Ng are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include extracting a 2D outline of the 3D polygonal mesh from a perspective of the view and at a resolution specified in the context of 3D polygonal mesh deformation, by Chen according to the teaching of Ng in order to define the extent of the surface more accurately (para. [0012] of Ng).
Regarding claim 49, the claims is a computing equipment claim of method claims 37. The claim is similar scope to claim 37 so it is rejected under similar rationales as claims 37.
Claims 42, 43, 44, and 45 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US 20190035149 A1; IDS REF) (hereinafter Chen) in view of Joyner et al. (US 20240048932 A1) (hereinafter Joyner).
Regarding claim 42, Chen discloses The method according to claim 34,
wherein the 3D polygonal mesh is a 3D polygonal mesh of an anatomical object, (Fig. 5; para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries.
PNG
media_image8.png
659
948
media_image8.png
Greyscale
wherein the one or more physical features are one or more anatomical features, and wherein the one or more target physical features are one or more target anatomical features. (para. [0212], "Generate an Approximate Geometry Using Shape Priors: In the first stage, we find an approximate head geometry as an initialisation using a generative shape prior that models shape variation of an object category (i.e. the face) in the low dimensional subspace with a dimension reduction method. The full head geometry of the user can be reconstructed from this low dimensional shape prior with a small number of parameters. A representable approach for head modelling is to learn a 3D morphable head model (3DMHM) [8], in which principal component analysis (PCA) is used to capture dominant modes of human head shape variation. In 3DMHM the parameters are the PCA weights w={w.sub.p}.sub.p=1.sup.P."; para. [0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."
Chen does not explicitly disclose wherein the one or more physical features are one or more anatomical features, and wherein the one or more target physical features are one or more target anatomical features.
However, Joyner more explicitly teaches wherein the one or more physical features are one or more anatomical features, and wherein the one or more target physical features are one or more target anatomical features. (para. [0002], "The present disclosure relates to audio processing, and in particular, to generating custom audio according to the listener's anthropometric and demographic characteristics."
para. [0024], "to generate a 3D landmark set that corresponds to anthropometric landmarks of the user identified using the structural image set and the transformed plurality of camera transforms. The anthropometry component is configured to receive the 3D landmark set, and to generate anthropometric data from the 3D landmark set, wherein the anthropometric data corresponds to a set of distances and angles measured between individual landmarks of the 3D landmark set. The server device is configured to generate the personalized HRTF for the user by inputting the anthropometric data into the HRTF calculation system."; para. [0045], " Anthropometric data will refer to a set of salient geometric measurements that can be used to describe a person's acoustic anatomy. Demographic data will refer to demographic information provided by a person that may include their sex, age, race, height, and weight. Generative data will refer to a combined set of complete or partial anthropometric data and demographic data that can collectively be used to estimate a person's HRTF. "; para. [0048], "Once prepared in the training environment, the HRTF calculation system may be used to generate personalized HRTFs for any user without the need for mesh data or acoustic measurement. The system queries the user for demographic data and uses a series of photogrammetry, computer vision, image processing, and neural network techniques to extract anthropometric data from structural imagery. For the purpose of this description, the term structural imagery refers to a plurality of images, which may be a series of images or be derived from “burst” images or video footage, in which the user's acoustic anatomy is visible.”; Examiner's note: anatomical data corresponds to physical features and target physical features because it used to generate a person's acoustic anatomy.)
As both Chen and Joyner are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include one or more anatomical features and one or more target anatomical features in the context of 3D polygonal mesh deformation, by Chen according to the teaching of Joyner in order to generate custom audio based on listener's anthropometric characteristic (para. [0002] of Joyner).
Regarding claim 43, Chen discloses The method according to claim 42, wherein the anatomical object includes a head, one or more ears, and/or an upper torso. (para. [0212], "Generate an Approximate Geometry Using Shape Priors: In the first stage, we find an approximate head geometry as an initialisation using a generative shape prior that models shape variation of an object category (i.e. the face) in the low dimensional subspace with a dimension reduction method. The full head geometry of the user can be reconstructed from this low dimensional shape prior with a small number of parameters. A representable approach for head modelling is to learn a 3D morphable head model (3DMHM) [8], in which principal component analysis (PCA) is used to capture dominant modes of human head shape variation. In 3DMHM the parameters are the PCA weights w={w.sub.p}.sub.p=1.sup.P.")
Regarding claim 44, Chen discloses The method according to claim 34, further comprising generating, from the edited 3D polygonal mesh, a head-related, HR, filter personalized to a deformed anatomical object that comprises the anatomical object with the one or more target physical features deformed. (para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.; para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”; para.[0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters.")
Chen does not disclose generating, from the edited 3D polygonal mesh, a head-related, HR, filter personalized.
Joyner more explicitly teaches generating, from the edited 3D polygonal mesh, a head-related, HR, filter personalized. (Abstract, “An apparatus and method of generating personalized HRTFs. The system is prepared by calculating a model for HRTFs described as the relationship between a finite example set of input data, namely anthropometric measures and demographic information for a set of individuals, and a corresponding set of output data, namely HRTFs numerically simulated using a high-resolution database of 3D scans of the same set of individuals”; para. [0009], “According to an embodiment, a method generates head-related transfer functions (HRTFs). The method includes generating an HRTF calculation system, and generating a personalized HRTF for a user using the HRTF calculation system”)
As both Chen and Joyner are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include generating, from the edited 3D polygonal mesh, a head-related, HR, filter personalized in the context of 3D polygonal mesh deformation, by Chen according to the teaching of Joyner in order to generate custom audio based on listener's anthropometric characteristic (para. [0002] of Joyner).
As discussed above, the combination of Chen and Joyner teach generating an HR filter from the edited 3D polygonal mesh.
Regarding claim 45, Chen discloses The method according to claim 44, further comprising generating multiple HR filters personalized to different anatomical objects, by deforming the same 3D polygonal mesh according to multiple different mesh editing specifications that specify different ways to deform the one or more target physical features and/or different target physical features to deform. (FIG. 10; Examiner’s note: FIG. 10 corresponds to different anatomical objects.;
PNG
media_image9.png
467
665
media_image9.png
Greyscale
para. [0215], "2. using a shape prior selection process to find the most suitable shape prior from a library using selection criteria such as the user's ethnicity, gender, age, and other attributes. The system diagram example of this solution is illustrated in FIG. 7. In the pipeline, we introduce an additional machine-learning-based attribute classifier, which can be implemented by e.g. a deep convolutional neural network (CNN) [19, 21, 29], to analyze the user's photo, and predict attributes (e.g. ethnicity, gender, and age) from the appearance information (i.e. skin colour, hair colour and styles, etc.) in the image. We select the appropriate 3D shape prior from the library based on matching a user's attributes with those defined for each shape prior. This method can recover the missing depth information of the user's face more accurately. It is useful for building a product that will work across different ethnic regions. See FIG. 8 for an example of image-based face attribute classification implemented by CNN classifiers."; para.[0218], "2. Predicting the full 3D head model from the face measurements by defining intuitive face biometrics based on the 2D facial landmarks (i.e., eye distance, eye-to-mouth distance, nose height, nose length, jaw width, etc.), as a variant of face regression. Then a linear regression can be applied to map these biometric measurements to PCA morph parameters."; para. [0219], "Landmark-Driven Geometry Refinement: In the second stage, we improve the head geometry for better realism by deforming the initial head model. We rectify the face landmark positions of the 3D model in the directions within the image plane (i.e. X and Y directions), so that the projection of the facial landmarks on the 3D face model will be a similarity transform of the corresponding 2D facial landmarks {I.sub.i}.sub.i=1.sup.M in the image. This process will make the generated 3D head model appear much more similar to the user's face in the input photo, particularly in near-frontal views. This refinement stage will not change the depth (i.e. the Z direction) of the head or face model.”; para. [0220], "In the algorithm, we first find a similarity transform T* (a 3×2 matrix) based on the layout of the 2D image landmarks {I.sub.i}.sub.i=1.sup.M to the frontal projection {circumflex over (L)}.sub.i=(L.sub.i,x, L.sub.i,y) of the corresponding 3D landmarks L.sub.i=(L.sub.i,x, L.sub.i,y, L.sub.i,z) (i=1, 2, . . . , M) of the 3D head model X.sub.user obtained from the shape regression above. This can be obtained by solving the following least squares problem in (4).”; para. [0221], " We then use a 3D thin-plate spline (TPS) deformation model [9] to rectify the 3D geometry of the regressed head model X.sub.user to achieve better geometric similarity. To implement that, we define the M source and sink control point pairs {(s.sub.i, t.sub.i)}.sub.i=1.sup.M of the 3D TPS model as:"; para. [0223], "Finally, the 3D face model of the user is generated by clipping the mesh of the user's head model with a 3D plane (defined by 2 vertices on the forehead and 1 vertex on the jaw) using the mesh-clipping algorithm described in [28]. This will yield a 3D mesh of the user's face with smooth boundaries. The 3D face model above is further refined into a watertight 3D face mask mesh using an off-the-shelf mesh solidification algorithm. The result can be used for 3D printing of personalised 3D face masks.”; Examiner's note: the same 3D shape prior can be selected for different multiple users based on their ethnicity, gender, age and other attributes)
Chen does not disclose generating multiple HR filters personalized to different anatomical objects (Abstract, “An apparatus and method of generating personalized HRTFs. The system is prepared by calculating a model for HRTFs described as the relationship between a finite example set of input data, namely anthropometric measures and demographic information for a set of individuals, and a corresponding set of output data, namely HRTFs numerically simulated using a high-resolution database of 3D scans of the same set of individuals”; para. [0009], “According to an embodiment, a method generates head-related transfer functions (HRTFs). The method includes generating an HRTF calculation system, and generating a personalized HRTF for a user using the HRTF calculation system”)
As both Chen and Joyner are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include generating multiple HR filters personalized in the context of 3D polygonal mesh deformation, by Chen according to the teaching of Joyner in order to generate custom audio based on listener's anthropometric characteristic (para. [0002] of Joyner).
As discussed above, the combination of Chen and Joyner teach generating multiple HR filters by deforming the same 3D polygonal mesh according to multiple different mesh editing specifications.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hyorim Park/
Examiner, Art Unit 2615
/JAMES A THOMPSON/Primary Examiner, Art Unit 2615
Prosecution Insights