DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8, 10-11, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lee (US 20170071706 A1) and in view of Wang (US 20200051326 A1).
Regarding to claim 1, Lee discloses a computer-implemented method for generating a shape model, the method comprising ([0009]: the reconstructed landmark points are used to build a full arch tooth model with individual crown models; Fig. 1; [0021]: the software is loaded onto a computer processor; a computer processor executes the instructions to implement the method; the instruction memory stores the instructions that are executed by the processor and includes computer readable storage media in the form of volatile and/or nonvolatile memory such as random access memory (RAM), read only memory (ROM), cache, Flash memory, a hard disk, or any other suitable storage component; Fig. 1; [0023]: the reconstructed landmark points are used to build a full arch tooth model with individual crown models; Fig. 3; [0030]: reconstruct any arbitrary shape of a full arch that consists of average ACMs with enough constraints):
generating, via execution of a set of mathematical models based on a plurality of shapes associated with an object, a set of attributes associated with a set of anatomical constraints for the object ([0005]: enforce anatomical constraints between the consecutive restorative units; [0007]: the model generates and represents location and size of each tooth in the arch, the alignment of cusps, and occlusal surface that would reflect dental anatomical features, i.e. a set of attributes; [0020]: PCA has been used to generate a statistical tooth model and to produce denture parts including a set of attributes; PCA defines those elements of the sample that best characterize differences across the samples; [0023]: create a linear subspace of the feature points with the basis of principal components found during the PCA procedure; Fig. 2; [0024]: constraints in the overall shape of the arch and relative positioning among teeth in the arch; the full geometry, i.e. attribute, of an arbitrary dental arch is generated and constructed by the computational counterpart of FAM using the second part of FAM; [0027]: the 4 or more points on the occlusal surface of an individual tooth in the arch are used to find a similarity transform from the corresponding feature points on the ACM; analyze the distribution of a set of shapes by optimally superimposing the shapes);
computing, based on the set of attributes, a set of positions of a set of points on the object ([0009]: compute a correct bite registration of individual upper and lower arch scans; an arbitrary set of landmark points on a pair of upper and lower arches are reconstructed by a linear combination of the statistical modeling coefficients in PCA to create a linear subspace of arch feature points within a reasonable range of variations captured from the training samples; [0022]: compute the correct bite registration of the individual upper and lower arch scans; the relative position and the number of scans are varied; Fig. 2; [0024]: constraints in the overall shape of the arch and relative positioning among teeth in the arch; compute and construct the full geometry of an arbitrary dental arch by the computational counterpart of FAM using the second part of FAM; [0034]: compute and find this minimal set of corresponding feature points automatically utilizing some 3D feature detection algorithms.); and
generating a three-dimensional (3D) model of the object based on the set of positions of the set of points ([0006]: the upper and lower jaw in the arch model includes a set of complete 3D geometry of individual tooth models; [0009]: the multiple sets of digitized dental arches include a set of complete 3D geometry of individual tooth models; the reconstructed landmark points are used to build a full arch tooth model with individual crown models; [0020]: generate a statistical shape model for medical objects using PCA where a template shape is developed and all objects to be analyzed; [0021]: create a full arch model; Fig. 1; [0023]: an arbitrary set of landmark points on a pair of upper and lower arches are reconstructed by a linear combination of the principal components within a reasonable range of variations captured from the training samples, when PCA is used as the statistical model; Fig. 3; [0030]: generate and reconstruct any arbitrary shape of a full arch that consists of average ACMs with enough constraints that would make the entire arch look natural;
PNG
media_image1.png
600
674
media_image1.png
Greyscale
; [0034]: align two 3D arch models correctly along the occlusal axis of the arch, at least two corresponding 3D feature points are desired.).
Lee fails to explicitly disclose mathematical models are neural networks.
In same field of endeavor, Wang teaches:
mathematical models are neural networks ([0034]: perform a deep learning process to generate a disparity map DP by the depth calculation neural network; [0038]: the three-channel feature map TFP is processed by the weighting calculation neural network 320 to generate a plurality of blend-shape weightings; [0043]: a three-dimensional facial expression is modeled according to the blend-shape weightings).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee to include mathematical models are neural networks as taught by Wang. The motivation for doing so would have been to perform a deep learning process to generate a disparity map DP by the depth calculation neural network; to perform convolution on the two-dimensional images IM1 and IM2 to generate the disparity map DP that includes the depth information of the facial expression; to process the three-channel feature map TFP to generate a plurality of blend-shape weightings by the weighting calculation neural network 320; to generate a more accurate modeling of the facial expression as taught by Wang in paragraphs [0034-0035], [0038], and [0047]. This would be considered common general knowledge to a person of ordinary skill in the art as replacing PCA statistical modelling which is discussed in Lee with neural networks for non-linear shape modelling in well-known.
Regarding to claim 8, Lee in view of Wang discloses the computer-implemented method of claim 1, wherein the object comprises a face (Wang; Fig. 1; [0032]: the facial expression modeling apparatus; face;
PNG
media_image2.png
196
144
media_image2.png
Greyscale
; [0039]: different parts of the face play different roles due to distinct movement of facial muscles).
Same motivation of claim 1 is applied here.
Regarding to claim 10, Lee in view of Wang discloses the computer-implemented method of claim 8, wherein the plurality of shapes comprises a neutral facial expression associated with the face and one or more non-neutral facial expressions associated with the face (Wang; [0039]: for different facial expressions, different parts of the face play different roles due to distinct movement of facial muscles; the blend-shape weightings WE are associated with different facial regions for different expressions; [0040]: distinguish more details of different facial expressions; [0043]: a three-dimensional facial expression is modeled according to the blend-shape weightings WE; [0047]: generate a more accurate modeling of the facial expression).
Regarding to claim 11, Lee discloses one or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising ([0009]: the reconstructed landmark points are used to build a full arch tooth model with individual crown models; Fig. 1; [0021]: a computer processor executes the instructions to implement the method; the instruction memory stores the instructions that are executed by the processor and includes computer readable storage media in the form of volatile and/or nonvolatile memory such as random access memory (RAM), read only memory (ROM), cache, Flash memory, a hard disk, or any other suitable storage component; Fig. 1; [0023]: the reconstructed landmark points are used to build a full arch tooth model with individual crown models fit to a given set of arch points; Fig. 3; [0030]: reconstruct any arbitrary shape of a full arch that consists of average ACMs with enough constraints):
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 11.
Regarding to claim 16, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 11, wherein computing the set of positions of the set of points on the object (same as rejected in claim 1) comprises:
for each point in the set of points, computing a first position of the point on a baseline shape for the object based on a first subset of the set of attributes (Lee; [0009]: compute a correct bite registration of individual upper and lower arch scans; an arbitrary set of landmark points on a pair of upper and lower arches are reconstructed by a linear combination of the statistical modeling coefficients in PCA to create a linear subspace of arch feature points within a reasonable range of variations captured from the training samples; [0022]: compute the correct bite registration of the individual upper and lower arch scans; the relative position and the number of scans are varied; Fig. 2; [0024]: constraints in the overall shape of the arch and relative positioning among teeth in the arch; compute and construct the full geometry of an arbitrary dental arch by the computational counterpart of FAM using the second part of FAM; [0034]: compute and find this minimal set of corresponding feature points automatically utilizing some 3D feature detection algorithms); and
computing a second position of the point on an additional shape included in the plurality of shapes based on the first position of the point and a second subset of the set of attributes (Lee; [0011]: modify the respective parameter to reduce the penalty function, iterating to find a lowest penalty function, and using the respective parameters to update the full arch tooth model; Fig. 2; [0024]: constraints in the overall shape of the arch and relative positioning among teeth in the arch; compute and construct the full geometry of an arbitrary dental arch by the computational counterpart of FAM using the second part of FAM; [0034]: compute and find this minimal set of corresponding feature points automatically utilizing some 3D feature detection algorithms).
Regarding to claim 17, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 16, wherein the second position of the point is computed using a linear blend skinning operation and a corrective displacement associated with the second subset of the set of attributes (Wang; [0034]: perform a deep learning process to generate a disparity map DP by the depth calculation neural network; [0038]: the three-channel feature map TFP is processed by the weighting calculation neural network 320 to generate a plurality of blend-shape weightings; [0039]: for different facial expressions, different parts of the face play different roles due to distinct movement of facial muscles; the blend-shape weightings WE are associated with different facial regions for different expressions; [0040]: distinguish more details of different facial expressions; [0043]: a three-dimensional facial expression is modeled according to the blend-shape weightings WE; [0047]: generate a more accurate modeling of the facial expression).
Same motivation of claim 1 is applied here.
Regarding to claim 18, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 11, wherein the object comprises a face (Wang; Fig. 1; [0032]: the facial expression modeling apparatus; face;
PNG
media_image2.png
196
144
media_image2.png
Greyscale
; [0039]: different parts of the face play different roles due to distinct movement of facial muscles).
Same motivation of claim 1 is applied here.
Regarding to claim 19, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 18, wherein the set of points lie on a surface of the face (Wang; Fig. 1; [0032]: the facial expression modeling apparatus; face;
PNG
media_image2.png
196
144
media_image2.png
Greyscale
; [0039]: different parts of the face play different roles due to distinct movement of facial muscles; there are points in different parts of the face).
Regarding to claim 20, Lee discloses a system ([0009]: the reconstructed landmark points are used to build a full arch tooth model with individual crown models; Fig. 1; [0021]: the software is loaded onto a computer processor, i.e. a system; a computer processor executes the instructions to implement the method; the instruction memory stores the instructions that are executed by the processor and includes computer readable storage media in the form of volatile and/or nonvolatile memory such as random access memory (RAM), read only memory (ROM), cache, Flash memory, a hard disk, or any other suitable storage component; Fig. 1; [0023]: the reconstructed landmark points are used to build a full arch tooth model with individual crown models fit to a given set of arch points; Fig. 3; [0030]: reconstruct any arbitrary shape of a full arch that consists of average ACMs with enough constraints), comprising:
one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform operations comprising (Fig. 1; [0021]: a computer processor executes the instructions to implement the method; the instruction memory stores the instructions that are executed by the processor and includes computer readable storage media in the form of volatile and/or nonvolatile memory such as random access memory (RAM), read only memory (ROM), cache, Flash memory, a hard disk, or any other suitable storage component; Fig. 1; [0023]: the reconstructed landmark points are used to build a full arch tooth model with individual crown models fit to a given set of arch points):
the rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to claim 20.
Claims 2-4, 6, and 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Lee (US 20170071706 A1) in view of Wang (US 20200051326 A1), and further in view of Bradley (US 20230104702 A1).
Regarding to claim 2, Lee in view of Wang discloses the computer-implemented method of claim 1, further comprising
Lee in view of Wang fails to explicitly disclose: training the set of neural networks based on one or more losses associated with the set of positions.
In same field of endeavor, Bradley teaches:
training the set of neural networks based on one or more losses associated with the set of positions (Fig. 2; [0034]: canonical shape positions 232 include locations or coordinates of points in canonical shape 220; [0046]: Training engine 122 performs supervised training that jointly optimizes the weights of encoder 204 and decoder 206 and shape token 342 based on one or more losses 208 between decoder output 210 and offsets 228 of the corresponding training shapes 230; [0053]: a trained transformer 200 fits to a new output shape 216 by iteratively optimizing for a corresponding shape code 218 that minimizes a loss between a set of points on a target shape and a corresponding set of positions 226 on the output shape 216; [0085]: generate a second shape code based on a second plurality of offset tokens and a second plurality of position tokens associated with a training shape; the one or more parameters are updated based on a loss between a plurality of ground truth offsets associated with the second plurality of offset tokens and a second plurality of offsets outputted by the decoder neural network based on the second shape code).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee in view of Wang to include training the set of neural networks based on one or more losses associated with the set of positions as taught by Bradley. The motivation for doing so would have been to perform supervised training that jointly optimizes the weights of encoder 204 and decoder 206 and shape token 342 based on one or more losses 208 between decoder output 210 and offsets 228 of the corresponding training shapes 230; to minimize a loss between a set of points on a target shape and a corresponding set of positions 226 on the output shape 216; to improve the quality of fit and expressibility of shape as taught by Bradley in paragraphs [0046] and [0053].
Regarding to claim 3, Lee in view of Wang and Bradley discloses the computer-implemented method of claim 2, wherein the one or more losses (Bradley; [0053]: minimize a loss between a set of points on a target shape and a corresponding set of positions 226 on the output shape 216) comprise a set of differences between the set of positions and a set of ground truth positions of the set of points (Bradley; [0063]: a set of ground truth offsets are applied to the set of positions to produce the training shape; training engine 122 inputs the positions and ground truth offsets into one or more MLPs in the encoder neural network to produce a set of position tokens and a set of offset tokens; [0075]: the one or more parameters are updated based on a loss between a plurality of ground truth offsets associated with the second plurality of offset tokens and a second plurality of offsets outputted by the decoder neural network based on the second shape code).
Same motivation of claim 2 is applied here.
Regarding to claim 4, Lee in view of Wang and Bradley discloses the computer-implemented method of claim 2, wherein the one or more losses (Bradley; [0053]: minimize a loss between a set of points on a target shape and a corresponding set of positions 226 on the output shape 216) comprise an anatomical regularization loss associated with the set of anatomical constraints (Lee; [0005]: enforce anatomical constraints between the consecutive restorative units; Fig. 2; [0024]: the training set of the statistical model, e.g., PCA, is a set of feature points; capture variations and constraints in the overall shape of the arch and relative positioning among teeth in the arch; [0030]: reconstruct any arbitrary shape of a full arch that consists of average ACMs with enough constraints that would make the entire arch look natural; [0035]: compute the loss, cost and difference;
PNG
media_image3.png
70
374
media_image3.png
Greyscale
).
Regarding to claim 6, Lee in view of Wang and Bradley discloses the computer-implemented method of claim 2, wherein the one or more losses (Bradley; [0053]: minimize a loss between a set of points on a target shape and a corresponding set of positions 226 on the output shape 216) comprise a symmetry regularization loss associated with a symmetry of a skeletal structure within the object (Lee; Fig. 3; [0030]: reconstruct any arbitrary shape of a full, i.e., upper or lower, arch that consists of average ACMs with enough constraints that would make the entire arch look natural;
PNG
media_image4.png
252
610
media_image4.png
Greyscale
; a symmetry of skeletal structure is shown in Fig. 3; [0035]: compute the loss, cost and difference;
PNG
media_image3.png
70
374
media_image3.png
Greyscale
).
Regarding to claim 12, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 11,
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to claim 2 is also used to reject claim 12.
Regarding to claim 13, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 12, wherein the one or more losses comprise at least one of: or (one of: or is optional);
The rest claim limitations are similar to claims 3-6, therefore, same rational used to claims 3-6 are also used to reject claim 13.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Lee (US 20170071706 A1) in view of Wang (US 20200051326 A1), in view of Bradley (US 20230104702 A1), and further in view of Beeler (US 20170091529 A1).
Regarding to claim 5, Lee in view of Wang and Bradley discloses the computer-implemented method of claim 2, wherein the one or more losses (Bradley; [0053]: minimize a loss between a set of points on a target shape and a corresponding set of positions 226 on the output shape 216) comprise a thickness regularization loss associated with a thickness of the object (Lee; [0006]: change its overall shape, width, size, curve of Wilson, curve of Spee, etc.; [0007]: overall width, height, depth; width and depth are thickness; [0035]: compute the loss, cost and difference;
PNG
media_image3.png
70
374
media_image3.png
Greyscale
).
Lee in view of Wang and Bradley fails to explicitly disclose a soft tissue thickness.
In same field of endeavor, Beeler teaches a soft tissue thickness ([0060]: tissue thickness of the face; [0062]: introduce subspace skin or other tissue thickness constraints into the model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee in view of Wang and Bradley to include a soft tissue thickness as taught by Beeler. The motivation for doing so would have been to introduce subspace skin or other tissue thickness constraints into the model; to improve the depth reconstruction of the face as taught by Beeler in paragraphs [0066-0062] and [0092].
Claims 7, 9 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Lee (US 20170071706 A1) in view of Wang (US 20200051326 A1), and further in view of Beeler (US 20170091529 A1).
Regarding to claim 7, Lee in view of Wang discloses the computer-implemented method of claim 1,
Lee in view of Wang fails to explicitly disclose:
wherein the set of attributes comprises at least one of a bone point, a bone normal, or a soft tissue thickness.
In same field of endeavor, Beeler teaches:
wherein the set of attributes comprises at least one of a bone point, a bone normal, or a soft tissue thickness (or is optional; [0060]: tissue thickness of the face; Fig. 3A; [0077]: compute the skin thickness for the anatomical subspace; give the skin thickness distance; [0137]: the rigid skull transformation; [0144]: provide an estimate of the underlying skull bone).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee in view of Wang to include wherein the set of attributes comprises at least one of a bone point, a bone normal, or a soft tissue thickness as taught by Beeler. The motivation for doing so would have been to introduce subspace skin or other tissue thickness constraints into the model; to improve the depth reconstruction of the face as taught by Beeler in paragraphs [0066-0062] and [0092].
Regarding to claim 9, Lee in view of Wang discloses the computer-implemented method of claim 8,
Lee in view of Wang fails to explicitly disclose wherein the set of attributes comprises at least one of a jaw bone transformation, a skinning weight, or a residual displacement.
In same field of endeavor, Beeler teaches wherein the set of attributes comprises at least one of a jaw bone transformation, a skinning weight, or a residual displacement (or is optional; [0060]: tissue thickness of the face; Fig. 3A; [0077]: compute the skin thickness for the anatomical subspace; give the skin thickness distance; [0137]: the rigid skull transformation; [0144]: provide an estimate of the underlying skull bone).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee in view of Wang to include wherein the set of attributes comprises at least one of a jaw bone transformation, a skinning weight, or a residual displacement as taught by Beeler. The motivation for doing so would have been to introduce subspace skin or other tissue thickness constraints into the model; to improve the depth reconstruction of the face as taught by Beeler in paragraphs [0066-0062] and [0092].
Regarding to claim 14, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 11, wherein generating the set of attributes (same as rejected in claim 11) comprises
Lee in view of Wang fails to explicitly disclose computing at least one of a bone point, a bone normal, or a soft tissue thickness associated with a point within a baseline shape for the object.
In same field of endeavor, Beeler teaches:
computing at least one of a bone point, a bone normal, or a soft tissue thickness associated with a point within a baseline shape for the object (or is optional; [0060]: tissue thickness of the face; Fig. 3A; [0077]: compute the skin thickness for the anatomical subspace; give the skin thickness distance; [0137]: the rigid skull transformation; [0144]: provide an estimate of the underlying skull bone).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee in view of Wang to include computing at least one of a bone point, a bone normal, or a soft tissue thickness associated with a point within a baseline shape for the object as taught by Beeler. The motivation for doing so would have been to introduce subspace skin or other tissue thickness constraints into the model; to improve the depth reconstruction of the face as taught by Beeler in paragraphs [0066-0062] and [0092].
Regarding to claim 15, Lee in view of Wang discloses the one or more non-transitory computer readable media of claim 11,
Lee in view of Wang fails to explicitly discloses:
wherein generating the set of attributes comprises computing at least one of a jaw bone transformation, a skinning weight, or a corrective displacement associated with a point within a shape included in the plurality of shapes.
In same field of endeavor, Beeler teaches:
wherein generating the set of attributes comprises computing at least one of a jaw bone transformation, a skinning weight, or a corrective displacement associated with a point within a shape included in the plurality of shapes (or is optional; [0060]: tissue thickness of the face; Fig. 3A; [0077]: compute the skin thickness for the anatomical subspace; give the skin thickness distance; [0137]: the rigid skull transformation; [0144]: provide an estimate of the underlying skull bone).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Lee in view of Wang to include wherein generating the set of attributes comprises computing at least one of a jaw bone transformation, a skinning weight, or a corrective displacement associated with a point within a shape included in the plurality of shapes as taught by Beeler. The motivation for doing so would have been to introduce subspace skin or other tissue thickness constraints into the model; to improve the depth reconstruction of the face as taught by Beeler in paragraphs [0066-0062] and [0092].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights