DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed February 25th, 2026 has been entered. Claims 1-2, 4-15, 17-19, and 21-22 remain pending in the application. Claims 1, 4, 14, 17 and 18 have been amended. Claims 3, 16, and 20 have been canceled. Claims 21-22 are newly added.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4-5, 10, 14, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wood (A 3D Morphable Eye Region Model for Gaze Estimation, 2016), in view of Ploumpis (Towards a complete 3D morphable model of the human head, 2019) and Ciuc (Pub. No. US 2019/0279393 A1).
Regarding Claim 1, Wood teaches A method for generating a three-dimensional model, comprising:
Obtaining one or more images of the head, wherein the head includes eyes (Section 1, An Eye Region 3DMM: “We constructed a 3DMM of the facial eye region by carefully registering a set of high-quality 3D head scans”, where the image and scans are visualized in Figure 1);
retrieving a parametric model for the eyes that includes a set of parameters (Section 2.1: “A 3D morphable model is a statistically-derived generative model, parameterized by shape and texture coefficients”; Section 4: “While this model is simple, we found it to be sufficient. If we considered a larger facial region, or fit models to both eyes at once, we would explore more advanced material or illumination models”, where extending the model to both eyes is suggested to be an anticipated and largely possible extension);
assigning values for each parameter in the set of parameters of the parametric model for the eyes based on the one or more images (Figure 3: “An overview our fitting process: We localize landmarks L in an image, and use them to initialze our 3DMM. We then use analysis-by-synthesis to render a [synthesized image] that best matches [an observed image]”, where Figure 3 clearly depicts using the Landmarks to fill the initial parameters);
generating eye patch areas of areas surrounding the eyes based on the values of the parameters in the set of parameters of the parametric model for the eyes (Section 1, An Eye Region 3DMM: “We constructed a 3DMM of the facial eye region by carefully registering a set of high-quality 3D head scans, and extracting modes of shape and texture variation using PCA”, where Figure 4 and Figure 5 visualize the eye patch area model);
wherein the eye patch areas are generated using a gradient descent algorithm applied to raw data from the one or more images and eye data from a database of heads (Wood, Section 1, Analysis-by-Synthesis: “We iteratively fit our model using gradient descent with numerical derivatives efficiently calculated with a tailored GPU rasterizer”; Figure 7 also demonstrates how the model appears after each iteration, where each iteration visualizes a model in which loss is minimized via gradient descent. Notes: The broadest reasonable interpretation of applying a gradient descent algorithm to raw data from an image is utilizing gradient descent as a means for minimizing loss between the output of a generation model and a target image, where the raw data includes image data, which is subsequently used for image processing)
wherein each head in the database of heads includes eyes (Wood, Figure 9 shows examples of datasets Eyediap and Columbia, which both include images of heads with eyes), and
generating the 3D model of the eyes and the eye patch areas (Section 1, An Eye Region 3DMM: “We constructed a 3DMM of the facial eye region by carefully registering a set of high-quality 3D head scans, and extracting modes of shape and texture variation using PCA. We combined this with an anatomy-based eyeball model that can be posed separately to simulate changes in eye gaze”, where Figure 4 and Figure 5 visualize the eye patch area model and Figure 6 visualizes the eye model”).
Wood does not teach generating the 3D model of the head.
However, Ploumpis teaches generating the 3D model of the head with the eye and eye patch areas (Abstract: “Abstract—Three-dimensional Morphable Models (3DMMs) are powerful statistical tools for representing the 3D shapes and textures of an object class. Here we present the most complete 3DMM of the human head to date that includes face, cranium, ears, eyes, teeth and tongue. To achieve this, we propose two methods for combining existing 3DMMs of different overlapping head parts: i. use a regressor to complete missing parts of one model using the other, ii. use the Gaussian Process framework to blend covariance matrices from multiple models. Thus we build a new combined face-and-head shape model that blends the variability and facial detail of an existing face model (the LSFM) with the full head modelling capability of an existing head model (the LYHM). Then we construct and fuse a highly-detailed ear model to extend the variation of the ear shape. Eye and eye region models are incorporated into the head model … We use our model to reconstruct full head representations from single, unconstrained images allowing us to parameterize craniofacial shape and texture, along with the ear shape, eye gaze and eye color”, where Figures 7-9 clearly depict a 3DMM model of the eye and eye patch area similar to that of Wood; the eye and eye patch area models are integrated into the 3D model of the head).
Wood and Ploumpis are considered analogous in the art with respect to working with 3DMMs for generating eye and eye patch areas. Given that the method for generating a 3D head model of Ploumpis integrates 3DMM models of the eyes and eye patch areas, it is not a novel concept to integrate models of the eyes and eye patch areas (such as those of Wood) into a 3D head model; the motivation for doing so would be to create a more detailed and accurate 3D head model, especially with regards to how the region around the eyes, and the eyes themselves, visually change as a person looks in different directions.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of generating eye and eye patch area models of Wood with the 3D model generation of the head with eye and eye patch areas of Ploumpis; Doing so would yield the predictable result of the generation of a 3D head model with specifically fitted eye and eye patch areas.
Wood as modified does not teach that for each head in the database of heads, the eyes are normalized to be spaced a fixed distance apart from one another, nor does it teach that the eyes of the 3D model are normalized to be spaced a fixed distance apart from one another, and wherein a size of the head is scaled based on the fixed distance between the eyes.
However, Ciuc teaches modifying a head from a database of heads with eyes such that the eyes are normalized to be spaced a fixed distance apart from one another for each head, and for each 3D model, wherein a size of the head in the 3D model is scaled based on the fixed distance between the eyes (Paragraph [0065]: “In an implementation, a database of 3D models of human heads 108, such as the set of 35, is artificially generated using a modeler… In an implementation, the models 108 are scaled such that the interpupillary distance is 63 mm, which is the average distance for an adult person”; Paragraph [0064]: “In an implementation, the example features tracker 100 operates with averaged 3D models 108 of the head, with facial landmarks represented by feature points at average positions, on which is applied a set of transformations to match a 3D model 108 to the head of the subject face 102 in the current image 104”. Notes: the database of heads are derived from images containing faces, where each head in the database is scaled such that the interpupillary distance, which describes the distance between the eyes, is fixed at 63 mm).
Wood as modified and Ciuc are considered analogous in the art with respect to modeling with respect to a human face from an image. A common motivation in the art is to normalize faces to be modeled such that faces can be more easily generalized for modeling purposes.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of 3D head model generation with eyes of Wood as modified with the eye space normalization and head scaling of Ciuc; Doing so would yield the predictable result of a more easily generated 3D head model.
Regarding Claim 4, the method of Claim 1 is rejected over Wood as modified.
Wood as modified teaches normalizing the eyes of each head in the database by spacing the distance between the eyes using the average distance between the eyes (See Ciuc, “Paragraph [0065]: “In an implementation, a database of 3D models of human heads 108, such as the set of 35, is artificially generated using a modeler… In an implementation, the models 108 are scaled such that the interpupillary distance is 63 mm, which is the average distance for an adult person”).
However, Wood as modified does not explicitly teach computing the average distance between the eyes.
While Wood as modified does not explicitly teach computing the average distance between the eyes, calculating the average distance between the eyes is clearly known in the art through the established average for an adult person; said established average would have been calculated from a collection of human heads, wherein the collection is akin to a database. Therefore, through the broadest reasonable interpretation of an average, the established average distance between the eyes would have been calculated in a manner analogous with the method of calculating an average distance between the eyes of a database of heads in the claimed invention. A motivation for applying the average to a specific database of heads not necessarily representative of the general human population would be to obtain an average representative of the specific database of heads.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to integrate calculating average eye spacing for heads in a database with the database of heads and use of an average distance between the eyes of Wood as modified; doing so would yield the predictable result of a plurality of anatomically accurate 3D models of heads with an eye spacing representative of the average eye spacing of the heads within the database.
Regarding Claim 5, the method of Claim 1 is rejected over Wood, in view of Ploumpis and Ciuc.
Wood as modified teaches assigning the values for each parameter in the set of parameters of the parametric model for the eyes based on the one or more images, wherein doing so comprises:
Obtaining an initial set of parameter values for the set of parameters of the parametric model for the eyes based on the one or more images (Wood, Figure 3 demonstrates that an initial set of parameters
PNG
media_image1.png
1
1
media_image1.png
Greyscale
𝜙 is derived from an input image); and
assigning the values for each parameter in the set of parameters of the parametric model by performing gradient descent on the initial set of parameter values to optimize the parameter values (Wood, Figure 3 demonstrates the optimization of the energy function that describes fitting the model to the eye of the subject; Wood, Section 5.2 further describes the process of using gradient descent to optimize the initial parameter set 𝜙, as demonstrated in Wood, Equation 13-14).
Regarding Claim 10, the method of Claim 1 is rejected over Wood, in view of Ploumpis and Ciuc.
Wood as modified teaches rendering an image of the 3D model of the head using differential rendering (Wood, 5.2, Optimization Procedure: “Computing our gradients is expensive, requiring rendering and differencing two images per parameter. Their efficient computation is possible with our tailored GPU DirectX rasterizer that can render [a synthesized image] at over 5000fps”; Notes: Differential rendering within the context of generating images from a 3D model, in its broadest reasonable interpretation, is iteratively comparing a synthesized image with an observed image, and computing a gradient from the difference of the two images. Differential rendering is often combined with parametric models describing the 3D model. In Wood, the difference between the synthesized image and the observed image is defined as the energy function E in Wood, Equation 10; the gradients are defined in Wood, Equation 13 and 14, where Wood, Equation 14 describes how the gradient of the difference E itself is computed using the current parameter values, and Wood, Equation 13 describes how the parameter set is optimized using the acquired gradient E (gradient descent equation). The result of each iteration of gradient descent can be observed in Wood, Fig 7, wherein each iteration generated an image of the eye and eye patch region in accordance with the set of parameters that were updated through gradient descent; With regards to generating an image of the head, it was established in the rejection of Claim 1 that integrating the eye and eye patch models into a 3D head model is not novel. It is noted that while the claim language does not specify a parametric model for the head, a parametric model of the head is present in Ploumpis as indicated by the Abstract: “The new model achieves state-of-the-art performance. We use our model to reconstruct full head representations from single, unconstrained images allowing us to parameterize craniofacial shape and texture, along with the ear shape, eye gaze and eye color”).
Claim 14, being similar in scope to Claim 1, is rejected under the same rationale.
Claim 17, being similar in scope to Claim 4, is rejected under the same rationale.
Claim 18, being similar in scope to Claim 1, is rejected under the same rationale.
Claims 2, 15 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wood in view of Ploumpis and Ciuc as applied to Claim 1 above, and further in view of Kuang (Towards an Accurate 3D Deformable Eye Model for Gaze Estimation, 2022) and Crouch (Parametric Eye Models, 2007).
Regarding Claim 2, the method of Claim 1 is rejected over Wood as modified.
Wood as modified teaches a set of parameters of the parametric model for the eyes (Section 2.1: “A 3D morphable model is a statistically-derived generative model, parameterized by shape and texture coefficients”; Section 4: “While this model is simple, we found it to be sufficient. If we considered a larger facial region, or fit models to both eyes at once, we would explore more advanced material or illumination models”) that broadly encompass geometric parameters such as shape, texture, and pose.
Wood does not teach a set of parameters of the parametric model for the eyes that include iris diameter, cornea radius of curvature, eye width, eye axial length, and iris depth.
However, Kuang teaches a model defined by parameters such as iris diameter, cornea radius of curvature, eye width, and iris depth (Table 1 lists the aforementioned parameters for a 3D eyeball model, with Figure 1 illustrating the model and associated parameters; Notes: iris radius is treated as being analogous to iris diameter, considering the ease in which diameter can be derived from radius. Cornea radius of curvature is analogous to cornea radius, as supported by Figure 1. Eye width is synonymous with Eyeball radius as demonstrated by Figure 1. Lastly, iris depth can easily be derived as the eyeball radius parameter value summed with the distance between eyeball center parameter value and cornea center parameter value).
Crouch teaches a parametric eye model including the parameter of eye axial length (“The eye model presented contains analytically defined shape equations that produce models matching user-specified physical measurements such as … eye axial length”).
Eye modeling is well established in the art. There are numerous parameters to define a parametric eye model, many of which are well documented characteristics of the eyes, or are otherwise easily derived from existing well-established parameters. One ordinarily skilled in the art would be motivated to select certain parameters to best define an eye model, or increase the volume of parameters to have a more detailed and accurate eye model. Combining the parameters used by others in the field is a known approach for defining a parameter set. Consequently, combining the parameter set of Kuang and the parameter set of Crouch would result in a set of parameters containing iris diameter, cornea radius of curvature, eye width, iris depth, and eye axial length without significant experimentation.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date to combine the set of parameters for the parametric eye models of Kuang and Crouch together; doing so would yield the predictable result of a more detailed parametric eye model defined by parameters including iris diameter, cornea radius of curvature, eye width, iris depth, and eye axial length.
Claim 15, being similar in scope to Claim 2, is rejected under the same rationale.
Claim 19, being similar in scope to Claim 2, is rejected under the same rationale.
Claims 6-7 and Claim 9 are rejected under 35 U.S.C. 103 as being unpatentable over Wood in view of Ploumpis and Ciuc as applied to Claim 1 above, and further in view of Berard (Lightweight eye capture using a parametric model, 2016).
Regarding Claim 6, the method of Claim 1 is rejected over Wood as modified.
Wood as modified teaches generating an iris texture for an iris of the eyes (Wood, Section 4, Parametric eyeball model: “We used a collection of aligned high-resolution iris photos to build a generative model… of iris texture using PCA”).
Wood as modified does not teach modeling the iris using polar coordinates.
However, Berard teaches generating an iris texture for an iris of the eyes, wherein the iris is modeled using polar coordinates (Section 6.1: “The structure of an iris is arranged radially around the pupil. Operating in polar coordinates (angle/radius) unwraps the radial structure (Fig. 5) and presents itself well for synthesis with rectangular patches”, where Figure 5 shows the iris modeled in polar coordinates).
Wood as modified and Berard are considered analogous in the art, as they both generate iris textures for an eye model. One ordinarily skilled in the art of 3D modeling of heads and their associated organs and components would be generally motivated to 3D model a head more realistically, as well as with more detail. Combining the more detailed method of iris generation of Berard with the general 3D modeling method of a head with eyes of Wood as modified would be a logical and obvious step for generating a more detailed and realistic 3D model of a head.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of iris texture generation using the polar coordinates of said iris of Berard with the 3D modeling method of a head with eyes of Wood as modified; doing so would yield the predictable result of a more detailed and realistic 3D model of a head, especially with regards to the iris texturing.
Regarding Claim 7, the method of Claim 1 is rejected over Wood as modified.
Wood as modified teaches receiving an image of an iris, wherein the iris is circular, and generating a texture for the iris (Wood, Section 4, Parametric eyeball model: “We used a collection of aligned high-resolution iris photos to build a generative model… of iris texture using PCA”).
Wood as modified does not teach transforming the iris to polar coordinates, wherein the iris transformed to polar coordinates can be represented by a rectangle, and subsequently used to generate a texture for the iris based on the polar coordinates.
However, Berard teaches transforming the iris to polar coordinates, wherein the iris transformed to polar coordinates can be represented by a rectangle, and subsequently used to generate a texture for the iris based on the polar coordinates (Section 6.1: “The structure of an iris is arranged radially around the pupil. Operating in polar coordinates (angle/radius) unwraps the radial structure (Fig. 5) and presents itself well for synthesis with rectangular patches”, where Figure 5 shows the iris modeled in polar coordinates, as well as the texture generated from the iris represented in polar coordinates).
The motivation for combining Wood as modified with Berard from Claim 6 is applied in Claim 7.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of iris texture generation using the polar coordinates represented as a rectangle of said iris of Berard with the 3D modeling method of a head with eyes of Wood as modified; doing so would yield the predictable result of a more detailed and realistic 3D model of a head, especially with regards to the iris texturing.
Regarding Claim 9, the method of Claim 7 is rejected by Wood as modified.
Wood as modified teaches a texture for the iris that includes a diffuse color map and a height map for the eyes (See Berard, Figure 14 demonstrates the height maps (left most image); Berard, Figure 5 demonstrates a control map: “Synthesizing an iris consists of capturing initial textures (a), from which control maps are generated by removing specular highlights (b)”. Notes: the iris geometry (left most images) are consistent with the definition of a height map, as they demonstrate the vertical changes through 3D representation of the iris. The control map is considered synonymous with diffuse color map. A diffuse color map, in its broadest reasonable interpretation, is a diffuse map when dealing with non-metallic materials; a diffuse map is an image where specular highlights pertaining to lighting are removed. Figure 5a demonstrates an initial image, in which an image Figure 5b is derived by removing specular highlights, therefore resulting in a diffuse map, and consequently a diffuse color map representative of the iris).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Wood in view of Ploumpis, Ciuc and Berard as applied to Claim 7 above, and further in view of Benalcazar (A 3D iris Scanner From a Single Image Using Convolutional Neural Networks, 2020) and Paumard (Image Reassembly Combining Deep Learning and Shortest Path Problem, 2018).
Regarding Claim 8, the method of Claim 7 is rejected over Wood as modified.
Wood as modified teaches a rectangle that represents the iris transformed to polar coordinates (Section 6.1: “The structure of an iris is arranged radially around the pupil. Operating in polar coordinates (angle/radius) unwraps the radial structure (Fig. 5) and presents itself well for synthesis with rectangular patches”, where Figure 5 shows the iris modeled in polar coordinates, as well as the texture generated from the iris represented in polar coordinates).
However, Benalcazar teaches dividing the rectangle that represents the iris transformed to polar coordinates into slices (Section E: “We reconstructed the 3D rubber sheet from the 3D model in Figure 12c by obtaining one 2D slice every 1 [degree]”, where Figure 15 demonstrates the manner in which the slices are concatenated such that they result in an elongated rectangle: “3D Rubber Sheet obtained from 360 slices of the 3D model in Figure 12c. The iris image of the subject is shown on the bottom left corner along with the 0 [degree] line of the slicing process”. Notes: The method of representing the iris in polar coordinates is commonly referred to as the rubber sheet model. The 3D rubber sheet is a 3D version of the rectangle resulting from transforming a 3D representation of the iris to polar coordinates. The method of transformation is identical save for the iris being represented in the 2D or 3D space. Benalcazar demonstrates that in performing the transformation to polar coordinates for an iris, the result would be 360 slices (one for each degree); in other words, the slices are inherent in forming the rectangular representation of the iris by transforming the iris to polar coordinates).
Paumard teaches a machine learning model that represents an image based on the slices of the image (Conclusion: “In this paper, we tackled the image reassembly problem where given a unordered list of image fragments, we want to recover the original image. To that end, we proposed a deep neural network architecture that predicts the relative position of a given pair of fragments. Then, we cast the reassembly problem into a shortest path in a graph algorithm for which we propose several construction algorithms depending on whether the puzzle is complete or if there are missing pieces”, where the reconstruction is visualized in Figure 1; Notes: Considering Benalcazar teaches dividing the iris into segments as claimed in the invention, the task of representing the iris in its broadest reasonable interpretation includes representing the rectangular representation of the iris after transformation to polar coordinates. Therefore, given the slices of the rectangular representation of the iris, the method of Wood as modified can be utilized to reconstruct the rectangular representation of the iris).
Benalcazar explicitly teaches what Wood as modified implies, in that transforming an iris to polar coordinates, a rectangle representative of a slice of the iris can be derived, of which said slices can be sequentially joined together to form a single rectangle representative of the iris. Paumard teaches a machine learning model that can reconstruct an image; Therefore, Benalcazar and Paumard are considered analogous in the art of image reconstruction. One ordinarily skilled in the art would be motivated to automate the process of reconstructing an image of an iris using the slices derived from Wood as modified (implicitly) and Benalcazar, as doing so would eliminate the need to manually piece together the slices to form the rectangular representation of an iris image.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the transformation of an iris to slices of Benalcazar and Wood as modified with the image reconstruction machine learning model of Paumard; doing so would yield the predictable result of a representation of the iris as an output from the machine learning model of Paumard, with the slices derived as specified by Benalcazar and Wood as modified as in input into the machine learning of Paumard.
Claims 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Wood in view of Ploumpis and Ciuc as applied to Claim 1 above, and further in view of Ablavatski (Pub. No. US 2021/0104096 A1).
Regarding Claim 11, the method of Claim 1 is rejected over Wood as modified.
Wood as modified teaches generating eye patch areas, and obtaining initial eye patch vertex locations based on the values of the parameters in the set of parameters of the parametric model of the eyes (Wood, Section 4: “the facial eye regions are represented as a combination of 3D shape s (n vertices) and 2D texture t (m texels), encoded as 3n and 3m dimensional vectors respectively”; Wood, Section 4: “Facial eye region shapes s and textures t can then be generated from shape and texture coefficients”, where the shape and texture coefficients are used to calculate the shape and texture in Wood, Equations 5 and 6, and the coefficients are described in the passage between Equation 4 and Equation 5 as being “the average 3D shape and 2D texture”, as well as “the Gaussian distributions of each shape and texture basis function”. Notes: Parameters are defined broadly within Wood as encompassing shape and texture. Considering shape and texture are defined by the above values, the above values are considered parameters);
wherein the initial eye patch vertex locations are subdivided into groups (Wood, Section 4, Morphable facial eye region model: “Additionally, we maintain correspondences for detailed parts, e.g. the interior eyelid margins, which are poorly defined for previous models” Notes: Woods explicitly states that the correspondence between the scan vertices and the simplified initial vertex set is maintained for the eyelid area; hence, the eyelids are considered to be a distinct subgroup within the initial vertex set; Wood, Section 4, Posing our multi-part model: “each eyelid vertex is rotated about the inter-eye-corner axis, with rotational amounts chosen to match measurements from an anatomical study. As our multi-part model contains disjoint parts, we also “shrinkrwap” the eyelid skin to the eyeball, projecting eyelid vertices onto the eyeball mesh to avoid gaps and clipping issues”; Notes: Wood clearly differentiates between the eyelid vertices and other eye patch area vertices, in that the eyelid vertices group has to conform to the eyeball vertices, while the eye patch area vertices group does not).
Wood as modified does not teach generating the eye patch areas based on optimizing the initial eye patch vertex locations using Catmull-Clark subdivision surface equations.
However, Ablavatski teaches generating a face model with optimized face vertex locations using Catmull-Clark subdivision surface equations on an initial set of face vertex locations (Paragraph [0032]: “The mesh representation 140 enables the building a plausible smooth surface representation 130 of the human face using algorithmic or machine leaning techniques. For example, Catmull-Clark subdivision can be used, resulting in the representation 150”, with corresponding Figures 1A and 1B).
While Ablavatski does not teach generating eye patch areas being generated from initial eye patch vertex locations being optimized using Catmull-Clark subdivision, it does demonstrate that using Catmull-Clark sub-division within the realm of 3D modeling of faces for smoothing is well established. There is common motivation within the art to smooth edges of a model to make it look more appealing and realistic to the user. With regards to a specific part of a head, such as the eye patch areas, one ordinarily skilled in the art would use Catmull-Clark subdivision to populate the 3D model of the eye patch areas with more vertices or otherwise optimize the vertices to smooth the 3D model of the eye patch areas and make them look more realistic. It is also worth noting that using Catmull-Clark sub-division for subdivision surface smoothing on any 3D mesh is well established to the point that 3D modeling software applications such as Blender incorporate it as a tool.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the initial eye patch vertex locations of Wood as modified with the use of Catmull-Clark subdivision to smooth a 3D model of a face of Ablavatski; doing so would yield the predictable result of generating eye patch areas that have optimized eye patch vertex locations, resulting in smoothed eye patch areas that are realistic and appealing to look at.
Regarding Claim 12, the method of Claim 11 is rejected over Wood as modified.
Wood as modified teaches a first group of initial eye patch vertex locations defining vertices of an eyelid and a second group of initial eye patch vertex locations defining vertices of an eyeball, and where in optimizing the initial eye patch vertex locations comprises constraining the vertices in the first group of initial eye patch vertex locations that defines vertices of the eyelid to approximate a curvature of the vertices in the second group of initial eye patch vertex locations that defines vertices of the eyeball (Wood, Section 4, Posing our multi-part model: “each eyelid vertex is rotated about the inter-eye-corner axis, with rotational amounts chosen to match measurements from an anatomical study. As our multi-part model contains disjoint parts, we also “shrinkrwap” the eyelid skin to the eyeball, projecting eyelid vertices onto the eyeball mesh to avoid gaps and clipping issues”; Notes: eyeball mesh by nature of being a mesh is a set of vertices defining the eyeball. Therefore, we have a group of vertices pertaining to the eyelid that are positioned such that they are constrained to the vertices of the eyeball, such that the eyelid vertices are projected onto the eyeball mesh vertices, which approximates a curvature of the vertices of the eyeball by nature of projection).
Regarding Claim 13, the method of Claim 1 is rejected over Wood as modified.
Wood as modified teaches subdividing vertices of an iris of an eye in the 3D model to obtain a dense sampling of vertices of the iris (Ablavatski, Paragraph [0032]: “The mesh representation 140 enables the building a plausible smooth surface representation 130 of the human face using algorithmic or machine leaning techniques. For example, Catmull-Clark subdivision can be used, resulting in the representation 150”, with corresponding Figures 1A and 1B);
Computing refraction values for the iris based on the dense sampling of vertices (Wood, Section 4, Parametric eyeball model: “We model changes in the iris size geometrically, by scaling vertices on the iris boundary about the 3D iris centre as specified by iris diameter… This can be used to generate new iris textures… we avoid explicitly modelling this by computing refraction effects in texture-space”); and
rendering the iris based on applying a texture to vertices of the iris and the refraction values for the iris (Wood, Figure 6 demonstrates applying a texture to an eyeball 3D mesh, where a mesh is a collection of vertices; note that the mean texture is actually both the vertices and the 2d texture combined, as specified with the symbol
PNG
media_image1.png
1
1
media_image1.png
Greyscale
𝜇 representing “the average 3D shape and 2D texture” (Wood, Section 4, Morphable facial eye region model). Note that iris variations and refraction are captured in Wood, Figure 6 as well).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Wood in view of Ploumpis, Ciuc, and Ablavatski as applied to Claim 13 above, and further in view of Wood.B (Learning an Appearance-Based Gaze Estimator from One Million Synthesised Images, 2016), Lyu (Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects, 2020) and Laine (Modular Primitives for High-Performance Differentiable Rendering, 2020).
Regarding Claim 21, the method of Claim 13 is rejected over Wood as modified
Wood as modified teaches subdividing an iris to generate multiple vertices (Ablavatski, Paragraph [0032]: “The mesh representation 140 enables the building a plausible smooth surface representation 130 of the human face using algorithmic or machine leaning techniques. For example, Catmull-Clark subdivision can be used, resulting in the representation 150”, with corresponding Figures 1A and 1B. Notes: Refer to obviousness for applying Catmull-Clark subdivision for smoothing body components).
Wood as modified does not teach generating the texture using differentiable refraction by computing texture coordinates with gradients based on view rays interacting with the cornea that refract at each vertex of the multiple vertices of the subdivided vertices.
However, Wood.B teaches generating the texture with regards to a refraction at a cornea vertex with regards to view rays (Figure 3: “We model iris refraction by altering texture look-ups. In (a), a viewed pixel is refracted correctly to show black (pupil) instead of blue (geometry surface). Example renders with (top) and without (bottom) refraction are shown in (b)”; Section 3, Approximate eyeball model, Physically based Refraction; “In reality, the iris is a flat disk of muscle that appears distorted through the refractive corneal bulge. This phenomenon is particularly apparent when the eye is viewed at an angle (Figure 3b), so it was important to model it”; refer to Figure 3 for an illustration of calculating refraction).
Wood as modified and Wood.B are considered analogous in the art with respect to the modeling of the eye; specifically, Wood references Wood.B for eye modeling. A common motivation in modeling is to improve modeling detail, and would therefore utilize refraction modeling of eyes in Wood.B to improve modeling of the eyes of Wood.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the eye modeling method of Wood as modified with the refraction modeling of view rays passing through the cornea of an eye of Wood.B; Doing so would yield the predictable result of more accurately modeled eyes.
Wood as modified does not teach the use of differentiable refraction with gradients.
However, Lyu teaches performing differentiable refraction with respect to view rays using gradients (Figure 4 clearly illustrates a view ray passing through a curved surface, resulting in the refraction, and is differentiable refraction because the loss between Q and Q’, represented by Equation 2, is minimized; Section 4.1, Refraction Loss: “Q′, obtained by intersecting the ray with the background monitor, is also a function of the associated vertices. Since all the operations to obtain Q′ are differentiable, the gradient of Eq. (2) is easily calculated using the chain rule. The effect of the refraction loss is visualized in Fig. 5”).
Wood as modified and Lyu are considered analogous in the art with respect to modeling refraction with respect to view rays. A common motivation in the art is to use differentiable modeling to improve predictability and efficiency. One would be motivated to utilize differentiable refraction when seeking to decrease time and/or resources spent modeling refractive effects on a model.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of refraction modeling the eye using view rays of Wood as modified with the differentiable refraction method utilizing gradients of Lyu; Doing so would yield the predictable result of increased efficiency and predictability when modeling refractive effects in an eye model.
Wood as modified does not explicitly teach computing texture coordinates utilizing differentiable refraction.
However, Laine teaches computing texture coordinates with gradients through differentiable rendering (Section 3.4, Interpolation: “Generally, vertex attributes can be used for arbitrary purposes. One of their typical uses, however, is to provide 2D coordinates for texture mapping”; Refer to Figure 1, which clearly illustrates the use of gradients for calculating texture coordinates via differentiable rendering. Notes: Differentiable refraction is a form of differentiable rendering)
Wood as modified and Laine are considered analogous in the art with respect to the appearance of models with respect to light effects (texture effects). A common motivation in the art is to utilize differentiable rendering to efficiently model complex visual effects; this is apparent with differentiable refraction, which takes into account how a view ray may change direction when interacting with curved surfaces or materials with different refractive effects.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method differentiable refraction of Wood as modified with the computation of texture coordinates with gradients through differentiable rendering of Laine; Doing so would yield the predictable result of generating textures with specific values at certain texture coordinates, providing a dynamic, realistic view of the model from different angles.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Wood in view of Ploumpis and Ciuc as applied to Claim 1 above, and in further view of Ascust (3DMM-Fitting-Pytorch, 2021) and Game Development Stack Exchange (Should a mesh consist of triangles or quads?, 2015).
Regarding Claim 22, the method of Claim 1 is rejected over Wood as modified.
Wood as modified teaches generating textures for the head, the eye patch areas, and the eyes by combining the head, the eye patch areas, and the eyes to generate a mesh model (Ploumpis, Introduction: “Therefore, we present a general approach that can be employed to combine 3DMMs from different parts of an object class into a single 3DMM. Due to their widespread use in the computer vision community, we fuse 3DMMs of the human face and the full human head as our exemplar. We add detailed models of the ears, eyes and eye regions to our head model, along with a basic model of the oral cavity, tongue and teeth”; Ploumpis, Figure 10: “Head texture completion given an unseen facial texture”, where Ploumpis, Figure 13 demonstrates the overall head texture for a model; Ploumpis, Figure 7: “The bank of iris textures in our model along with our eye mesh structure”; Ploumpis, Figure 9: “Illustration of the first five components of the eye region shape model”; Wood, Introduction, An Eye Region 3DMM: “We constructed a 3DMM of the facial eye region by carefully registering a set of high-quality 3D head scans, and extracting modes of shape and texture variation using PCA”. Notes: 3DMM models contain meshes (groups of vertices)).
Wood as modified does not teach generating a differentiable model that can be rasterized and shaded.
However, Ascust teaches that 3DMM models are differentiable (3DMM model fitting using Pytorch: “This is a fitting framework implemented in Pytorch for reconstructing the face in an image or a video using a 3DMM model. The framework only uses Pytorch modules and a differentiable renderer from pytorch3d. The whole module is differentiable and can be integrated into other systems for the gradient propagation”. Notes: note that Ploumpis teaches combining 3DMM models into a single 3DMM model. Furthermore, rasterization and shading are well known in the art as being achieved by differential rendering).
Wood as modified and Ascust are considered analogous in the art with respect to the use of 3DMM models. A common motivation in the art is to utilize differentiable 3D models for machine learning tasks; this is evident in Ascust, as a 3DMM undergoes differentiable rendering to integrate into videos.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the combination of multiple models and associated textures into a 3DMM mesh model of Wood as modified with the method of differentiable rendering of 3DMM models of Ascust; Doing so would yield the predictable result of enabling machine learning tasks utilizing gradient propagation via differentiable rendering.
Wood as modified does not teach generating a triangle mesh.
However, Game Development Stack Exchange teaches that generating triangle meshes is common and necessary in the art (user 1430: “However, modern graphics cards only work with triangles, so at some point the mesh data must be converted to triangles”).
Wood as modified and Game Development Stack Exchange are considered analogous in the art with respect to the use of mesh models. A common motivation for representing mesh models as triangle mesh models is that they work with modern graphics cards, as is pointed out in Game Development Stack Exchange.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the mesh model generation of Wood as modified with the conversion of mesh models to triangle mesh models of Game Development Stack Exchange; Doing so would yield the predictable result of allowing graphics cards to work with the mesh models.
Response to Arguments
Applicant's arguments filed February 25th, 2026, have been fully considered but they are not persuasive.
Applicant asserts that Claim 1 as amended to include the limitations of previously presented dependent Claim 3 (“generating eye patch areas of areas surrounding the eyes based on the values of the parameters in the set of parameters of the parametric model for the eyes, wherein the eye patch areas are generated using a gradient descent algorithm applied to raw data from the one or more images and eye data from a database of heads, wherein each head in the database of heads includes eyes, and wherein the eyes of each head in the database of heads are normalized to be spaced the fixed distance apart from one another for each head”) is not taught by Wood, Ploumpis, or Ciuc. Applicant notes that Wood only broadly describes gaze estimation by fitting a 3D morphable model to an input image, and fitting their model using gradient descent with numerical derivatives efficiently calculated with a tailored GPU rasterizer, and does not describe using it or applying to raw data.
However, in the broadest reasonable interpretation of raw data from an image, raw data includes image data that is utilized for image processing. Additionally, the broadest reasonable interpretation of applying gradient descent to image data from one or more images is applying gradient descent for the purpose of fitting a model to the image, in which the difference between the model and the image, in the form of the parameters (which include the difference between the model and the image in various ways) are minimized (loss function). While the applicant’s assertion that gradient descent is not applied directly to raw data, the Examiner notes that the language of Claim 1 does not necessitate the direct application (“wherein the eye patch areas are generated using a gradient descent algorithm applied to raw data from the one or more images and eye data from a database of heads”). While there is support in the specification for “direct” application, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Therefore, the broadest reasonable interpretation of applying gradient descent to raw data is through the adjustment of parameters resulting in an output in which loss minimization can be performed; to this end, Wood’s application of gradient descent by fitting a 3D morphable model to an input image, and fitting their model using gradient descent with numerical derivatives efficiently calculated with a tailored GPU rasterizer, is sufficient for the limitation as described in amended Claim 1.
Similarly, Applicants following assertion that Wood does not describe applying gradient descent to directly to the raw data is refuted under the same rationale as above.
The Applicant asserts that Wood as modified does not teach having eyes that are normalized to be spaced a fixed distance apart from one another for each head in the database of heads. The Examiner directs the Applicant to Ciuc, which is utilized in combination to Wood and Ploumpis, and previously presented in rejection of both previously presented independent Claim 1 and previously presented dependent Claim 3, from which the specified limitations of amended Claim 1 are derived from: (Ciuc, Paragraph [0065]: “In an implementation, a database of 3D models of human heads 108, such as the set of 35, is artificially generated using a modeler… In an implementation, the models 108 are scaled such that the interpupillary distance is 63 mm, which is the average distance for an adult person”; Ciuc, Paragraph [0064]: “In an implementation, the example features tracker 100 operates with averaged 3D models 108 of the head, with facial landmarks represented by feature points at average positions, on which is applied a set of transformations to match a 3D model 108 to the head of the subject face 102 in the current image 104”. Notes: the database of heads are derived from images containing faces, where each head in the database is scaled such that the interpupillary distance, which describes the distance between the eyes, is fixed at 63 mm). Ciuc clearly establishes a database of human head models, where the head models are scaled, such that the eyes are normalized to have an interpupillary distance fixed at 63 mm, where interpupillary distance is the distance between the eyes. It would have been obvious to a person having ordinary skill in the art to normalize a database of heads to have the eyes fixed a certain distance apart from one another for each head, and to subsequently utilize the database of heads for generating eye patch areas. Similarly, applying gradient descent is discussed above. For a more detailed analysis, refer to the rejection of amended Claim 1. Therefore, amended Claims 1, 14, and 18 are rejected, and all dependent claims of amended Claims 1, 14, and 18 are not considered allowable based solely on the allowability of their respective independent claims.
New Claims 21 and 22 are rejected (refer to the Claim Rejections), and are not considered allowable at least because of the allowability of independent Claim 1, from which Claims 21 and 22 ultimately depend on, due to the rejection of Claim 1.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights