DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/14/2025 has been considered by the examiner.
Status of Claims
Claims 1-20 are currently pending in this application.
Claim Objections
Claims 1, 5, 10, 13, 14, 16, 17, and 20 are objected to because of the following informalities:
Claim 1 lines 9-11 recite “the first or second image data” and “the first or second teeth arrangement”, Examiner suggests amending these both limitations to recite “the first image data or the second image data” and “the first teeth arrangement or the second teeth arrangement”.
Claim 5 line 3 recites “the first or second still images”, Examiner suggests amending the limitation to recite “the first still images or the second still images”.
Claim 10 line 3 recites “the first or second videos”, Examiner suggests amending the limitation to recite “the first videos or the second videos”.
Claim 13 line 5 recites “the first or second image data” Examiner suggests amending the limitation to recite “the first image data or the second image data”.
Claim 14 lines 1-3 recite “the first or second image data” and “the first or second teeth arrangement”, Examiner suggests amending these both limitations to recite “the first image data or the second image data” and “the first teeth arrangement or the second teeth arrangement”.
Claim 16 lines 8-10 recite “the first or second image data” and “the first or second teeth arrangement”, Examiner suggests amending these both limitations to recite “the first image data or the second image data” and “the first teeth arrangement or the second teeth arrangement”.
Claim 20 lines 1-2 recite recites “the first or second image data” Examiner suggests amending the limitation to recite “the first image data or the second image data”. Appropriate correction is required.
Positive Statement regarding 35 U.S.C. 101
Claims 1-20 are determined to be eligible under 35 U.S.C. 101. The claim 1, for example, at lines 12-15 recites "generate, based on one or more of the first or second image data, a novel image data of the teeth in the first or second teeth arrangement, the novel image data representing the teeth in either the first or second teeth arrangement as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data.". It is given the weight of the description in the specification paragraph [0072] that the generated image data allows for the generation of consistent images from the same perspective over time, even when the original acquired images or videos are not captured from the same perspective. Because the claims recite specific, claimed steps and structural elements that produce tangible technical results, they are not directed to an abstract idea. Therefore, the claims are determined to be eligible under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Long et al. (US2022/0189130A1) (hereinafter, “Long”) in view of Li et al. (US 2020/0000551 A1) (hereinafter, “Li”).
Regarding claim 1, Long discloses a system for use in performing an orthodontic treatment (Paragraph [0002] “The present disclosure relates generally to constructing three-dimensional models for use in dental treatment.”; Paragraph [0047] “described herein are systems and methods for generating a three-dimensional (3D) model of a user's dental arch from two-dimensional (2D) images of a dental impression of the user's dental arch. A model generation system receives images of a dental impression of the user's dental arch, and generates a 3D model of the user's dental arch from the images.”), comprising:
a processor (Paragraph [0058] “The model generation engine 204 may include a processing circuit including one or more processors and memory. The memory may store various instructions, routines, or other programs that, when executed by the processor(s), cause the processor(s) to perform various tasks relating to the generation of a 3D model. In some implementations, various subsets of processor(s), memory, instructions, routines, libraries, etc., may form an engine.”); and
memory comprising instructions, that when executed by the processor, cause the system to (Paragraph [0058] “The model generation engine 204 may include a processing circuit including one or more processors and memory. The memory may store various instructions, routines, or other programs that, when executed by the processor(s), cause the processor(s) to perform various tasks relating to the generation of a 3D model. In some implementations, various subsets of processor(s), memory, instructions, routines, libraries, etc., may form an engine.”):
access one or more first image data (previous user image 928 in Paragraph [0107] equates to first image data) of teeth of a patient captured at a first time when the teeth of the patient are in a first teeth arrangement (prior stage of the treatment plan in Paragraph [0107] equates to first teeth arrangement) (Paragraph [0004] “The model generation system is configured to generate a first three-dimensional (3D) model of the dental arch of the user by applying the one or more first images to a machine learning model trained to generate 3D models of dental arches from two-dimensional (2D) images of dental impressions of the dental arches. The model generation system is configured to receive one or more second images. The one or more second images are of one of the first dental impression of the dental arch or a second dental impression of the dental arch.”; Paragraph [0107] “The user interface may include, for example, the 3D model 930 generated based on the user images 928…the user interface may include the 3D model 930 and a prior (and/or subsequent) 3D model. The prior 3D model may be the 3D model from the patient file corresponding to a prior stage of the treatment plan. The prior 3D model may be a 3D model generated from a previous user image 928.”), the first image data having been captured from one or more first camera positions, orientations, or settings (Paragraph [0131] “The plurality of images 928 may each depict or otherwise represent a portion of the dental impression from a different perspective such that the plurality of images 928 together represent the dental impression (e.g., in its entirety).”);
access one or more second image data (images taken at the various stages of treatment in Paragraph [0106] equate to second image data) of the teeth of the patient captured at a second time when the teeth of the patient are in a second teeth arrangement (images taken at different stages of treatment in Paragraph [0106] equates to second teeth arrangement), the second time being different from the first time (Paragraph [0106] “the patient may upload the user images 928 of impressions made following completion of one stage of treatment. For example, the patient may be prompted to upload images 928 at various intervals (such as daily, weekly, every two weeks, following completion of a stage of treatment, every six months or year following treatment via dental aligners, whenever the patient requests to check their progress, etc.). The patient may be prompted to upload images 928 to ensure that the patient's teeth are progressing according to the treatment plan, or to ensure that the patient's teeth have not reverted back to a position prior to treatment via dental aligners. The model generation system 924 may be configured to generate the 3D model 930 based on the user images 928”), the second image data having been captured from one or more second camera positions, orientations, or settings (Paragraph [0131] “The plurality of images 928 may each depict or otherwise represent a portion of the dental impression from a different perspective such that the plurality of images 928 together represent the dental impression (e.g., in its entirety).”); and
generate, based on one or more of the first or second image data, a novel image data (3D model corresponding to the stage of treatment in Paragraph [0107] equates to novel image data) of the teeth in the first or second teeth arrangement, the novel image data representing the teeth in either the first or second teeth arrangement [as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data] (Paragraph [0106] “The patient may be prompted to upload images 928 to ensure that the patient's teeth are progressing according to the treatment plan, or to ensure that the patient's teeth have not reverted back to a position prior to treatment via dental aligners. The model generation system 924 may be configured to generate the 3D model 930 based on the user images 928”; Paragraph [0107] “the user interface may include the 3D model 930 generated based on the user images 928 and an expected 3D model (such as the 3D model from the patient file corresponding to the current stage of treatment). Such embodiment may allow the user to track their progress in comparison to the target 3D model corresponding to the treatment.”).
However, Long fails to teach as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data.
Li teaches as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”; Examiner interprets the rendered image in Paragraph [0175] equates to novel image data).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Li with Long to obtain the invention specified in claim 1.
Regarding claim 2, which claim 1 is incorporated, Long discloses wherein the instructions, when executed by the processor, further cause the system to: receive an orthodontic treatment plan comprising a plurality of treatment stages to move teeth of a patient from a first teeth arrangement towards a second teeth arrangement, each of the plurality of treatment stages including associated with a respective teeth arrangement of the teeth of the patient (Paragraph [0106] “The patient file may include a 3D model for each stage of treatment (e.g., an initial 3D model corresponding to an initial stage of treatment, one or more intermediate 3D models corresponding to intermediate stages of treatment, and a final 3D model corresponding to the final stage of treatment). The 3D model 930 generated by the model generation system 924 may be compared to the 3D models from the patient file. For example, where the 3D model 930 generated from the user images 928 matches (or substantially matches) the 3D model included in the patient file corresponding to the particular stage of the treatment plan, the patient may be determined to be progressing in accordance with the treatment plan, since the patient's teeth are moving according to the progression defined in the treatment plan from their initial position prior to treatment, to one or more intermediate positions, and to a final position following treatment.”).
Regarding claim 3, which claim 1 is incorporated, Long discloses wherein the first and second image data are first still images and second still images (Paragraph [0101] “The model generation system 924 may be or include any device(s), component(s), application(s), element(s), script(s), circuit(s), or other combination of software and/or hardware designed or implemented to generate a 3D model 930 from a set of 2D images 928 received from a user device 926. “).
Regarding claim 4, which claim 3 is incorporated, Long fails to teach wherein the novel image data is a new still image.
Li teaches wherein the novel image data is a new still image (rendered image in Paragraph [0175] equates to new still image) (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include wherein the novel image data is a new still image taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Li with Long to obtain the invention specified in claim 4.
Regarding claim 5, which claim 4 is incorporated, Long discloses wherein the instructions, that when executed by the processor, further cause the system to: display, at the same time, [the new still image] and the one or more of the first or second still images used to generate the new still image (Paragraph [0189] “the user interface includes an image of the patient's smile (e.g., corresponding to the dental impression). The image may be displayed side-by-side with the merged model (and, optionally, the first and second models 2400, 2402). The technician may select the ideal merged model based on the side-by-side photo. Following selection of the merged model for use, the technician may manipulate, modify, etc. the merged model to reposition the patient's teeth, and the technician may export the modified merged model to the manufacturing manager 2114 for manufacturing dental aligners for repositioning the patient's teeth”).
However, Long fails to teach the new still image.
Li teaches the new still image (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include the new still image taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Li with Long to obtain the invention specified in claim 5.
Regarding claim 6, which claim 5 is incorporated, Long discloses wherein [the new still image] is generated as viewed from a position and orientation that matches a position and orientation of an image of the first still images or the second still images (Paragraph [0116] “the model training system 902 calculates the estimated pose of the user in the training image 1300 by performing a camera constant, rotation, and translation (KRT) analysis (or other pose alignment analysis) of the images…The model training system 902 may calculate the estimated pose of a respective training image 1300, to match the pose of the 3D training model with the estimated pose of the user in the training image 1300. For example, the model training system 902 may translate, rotate, or otherwise modify the pose of the 3D training model to match the calculated pose for the training image 1300.”).
However, Long fails to teach the new still image.
Li teaches the new still image (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include the new still image taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Li with Long to obtain the invention specified in claim 6.
Regarding claim 7, which claim 5 is incorporated, Long fails to teach wherein the new still image is generated as viewed from a position and orientation that does not match a position and orientation of the first still images or the second still images.
Li teaches wherein the new still image is generated as viewed from a position and orientation that does not match a position and orientation of the first still images or the second still images (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include wherein the new still image is generated as viewed from a position and orientation that does not match a position and orientation of the first still images or the second still images taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Li with Long to obtain the invention specified in claim 7.
Regarding claim 13, which claim 1 is incorporated, Long discloses wherein the instructions, that when executed by the processor, further cause the system to: access an orthodontic treatment plan comprising a plurality of treatment stages to move teeth of the patient from a first teeth arrangement towards a second teeth arrangement, each of the plurality of treatment stages associated with a respective teeth arrangement of the teeth of the patient (Paragraph [0106] “The patient file may include a 3D model for each stage of treatment (e.g., an initial 3D model corresponding to an initial stage of treatment, one or more intermediate 3D models corresponding to intermediate stages of treatment, and a final 3D model corresponding to the final stage of treatment).”); and
evaluate progress of the orthodontic treatment plan by comparing the novel image data and the one or more of the first or second image data used to generate the novel image data (Paragraph [0106] “The 3D model 930 generated by the model generation system 924 may be compared to the 3D models from the patient file. For example, where the 3D model 930 generated from the user images 928 matches (or substantially matches) the 3D model included in the patient file corresponding to the particular stage of the treatment plan, the patient may be determined to be progressing in accordance with the treatment plan, since the patient's teeth are moving according to the progression defined in the treatment plan from their initial position prior to treatment, to one or more intermediate positions, and to a final position following treatment.”).
Regarding claim 14, which claim 1 is incorporated, Long discloses wherein the generation, based on one or more of the first or second image data, the novel image data of the teeth in the first or second teeth arrangement, the novel image data representing the teeth in either the first or second teeth arrangement [as viewed from a different camera position and orientation than that of the one or more first or second image data] further uses one or more of Neural Radiance Fields, Gaussian Splatting, smart image stitching, gaussian splatting or a fine-tuned diffusion model (Paragraph [103] “The machine learning model 922 may be configured to generate, as an output, a 3D model 930 of the dental arch based on the user images 928. In some embodiments, the machine learning model 922 may be configured to generate a plurality of 3D models of each perspective of the dental arch included in a respective image 928. The machine learning model 922 may be configured to stitch together the plurality of 3D models of each perspective dental arch to establish, generate, or otherwise form the 3D model 930 as an output.”).
However, Long fails to teach as viewed from a different camera position and orientation than that of the one or more first or second image data.
Li teaches as viewed from a different camera position and orientation than that of the one or more first or second image data (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include as viewed from a different camera position and orientation than that of the one or more first or second image data taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Li with Long to obtain the invention specified in claim 14.
Regarding claim 15, which claim 2 is incorporated, Long discloses wherein the first time (prior stage of treatment plan Paragraph [0107] equates to the first time) is prior to implementation of the orthodontic treatment plan, and the second time (completion of a stage of treatment in Paragraph [0107] equates to the second time) is after implementation of at least a portion of the orthodontic treatment plan (Paragraph [0106] “the patient may upload the user images 928 of impressions made following completion of one stage of treatment. For example, the patient may be prompted to upload images 928 at various intervals (such as daily, weekly, every two weeks, following completion of a stage of treatment, every six months or year following treatment via dental aligners”; Paragraph [0107] “The user interface may include, for example, the 3D model 930 generated based on the user images 928…the user interface may include the 3D model 930 and a prior (and/or subsequent) 3D model. The prior 3D model may be the 3D model from the patient file corresponding to a prior stage of the treatment plan. The prior 3D model may be a 3D model generated from a previous user image 928.”).
Regarding claim 16, Long discloses a method for use in performing an orthodontic treatment (Paragraph [0002] “The present disclosure relates generally to constructing three-dimensional models for use in dental treatment.”), comprising:
capturing one or more first image data (previous user image 928 in Paragraph [0107] equates to first image data) of teeth of a patient captured at a first time when the teeth of the patient are in a first teeth arrangement (prior stage of the treatment plan in Paragraph [0107] equates to first teeth arrangement) (Paragraph [0004] “The model generation system is configured to generate a first three-dimensional (3D) model of the dental arch of the user by applying the one or more first images to a machine learning model trained to generate 3D models of dental arches from two-dimensional (2D) images of dental impressions of the dental arches. The model generation system is configured to receive one or more second images. The one or more second images are of one of the first dental impression of the dental arch or a second dental impression of the dental arch.”; Paragraph [0107] “The user interface may include, for example, the 3D model 930 generated based on the user images 928…the user interface may include the 3D model 930 and a prior (and/or subsequent) 3D model. The prior 3D model may be the 3D model from the patient file corresponding to a prior stage of the treatment plan. The prior 3D model may be a 3D model generated from a previous user image 928.”), the first image data having been captured from one or more first camera positions, orientations, or settings (Paragraph [0131] “The plurality of images 928 may each depict or otherwise represent a portion of the dental impression from a different perspective such that the plurality of images 928 together represent the dental impression (e.g., in its entirety).”);
capturing one or more second image data (images taken at the various stages of treatment in Paragraph [0106] equate to second image data) of the teeth of the patient captured at a second time when the teeth of the patient are in a second teeth arrangement (images taken at different stages of treatment in Paragraph [0106] equates to second teeth arrangement) (Paragraph [0106] “the patient may upload the user images 928 of impressions made following completion of one stage of treatment. For example, the patient may be prompted to upload images 928 at various intervals (such as daily, weekly, every two weeks, following completion of a stage of treatment, every six months or year following treatment via dental aligners, whenever the patient requests to check their progress, etc.). The patient may be prompted to upload images 928 to ensure that the patient's teeth are progressing according to the treatment plan, or to ensure that the patient's teeth have not reverted back to a position prior to treatment via dental aligners. The model generation system 924 may be configured to generate the 3D model 930 based on the user images 928”), the second time being different from the first time, the second image data having been captured from one or more second camera positions, orientations, or settings (Paragraph [0131] “The plurality of images 928 may each depict or otherwise represent a portion of the dental impression from a different perspective such that the plurality of images 928 together represent the dental impression (e.g., in its entirety).”);
sending the first and second image data to a remote server device for processing (Paragraph [0131] “The model generation system 924 may receive the images 928 from a user device 926. The user device 926 may transmit, send, or otherwise provide the images 928 to the model generation system 924 (e.g., by uploading the images 928 to a portal associated with the model generation system 924, by transmitting the images 928 to an address associated with the model generation system 924, etc.).”); and
receiving, from the remote server device, a novel image data (3D model corresponding to the stage of treatment in Paragraph [0107] equates to novel image data) of the teeth in the first or second teeth arrangement, wherein the novel image data is generated based on the first and second image data, the novel image data representing the teeth in either the first or second teeth arrangement [as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data] (Paragraph [0106] “The patient may be prompted to upload images 928 to ensure that the patient's teeth are progressing according to the treatment plan, or to ensure that the patient's teeth have not reverted back to a position prior to treatment via dental aligners. The model generation system 924 may be configured to generate the 3D model 930 based on the user images 928”; Paragraph [0107] “the user interface may include the 3D model 930 generated based on the user images 928 and an expected 3D model (such as the 3D model from the patient file corresponding to the current stage of treatment). Such embodiment may allow the user to track their progress in comparison to the target 3D model corresponding to the treatment.”).
However, Long fails to teach as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data.
Li teaches as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”; Examiner interprets the rendered image in Paragraph [0175] equates to novel image data).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include as viewed from a different camera position, orientation, or setting than that of the one or more first or second image data taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Li with Long to obtain the invention specified in claim 16.
Regarding claim 17, which claim 16 is incorporated, Long discloses displaying, at the same time, the novel image data and the and the one or more of the first or second image data used to generate the novel image data (Paragraph [0189] “the user interface includes an image of the patient's smile (e.g., corresponding to the dental impression). The image may be displayed side-by-side with the merged model (and, optionally, the first and second models 2400, 2402). The technician may select the ideal merged model based on the side-by-side photo. Following selection of the merged model for use, the technician may manipulate, modify, etc. the merged model to reposition the patient's teeth, and the technician may export the modified merged model to the manufacturing manager 2114 for manufacturing dental aligners for repositioning the patient's teeth”).
Regarding claim 18, which claim 17 is incorporated, Long discloses wherein the novel image data is generated as viewed from a position and orientation that matches a position and orientation of an image of the one or more first image data or the one or more second image data (Paragraph [0116] “the model training system 902 calculates the estimated pose of the user in the training image 1300 by performing a camera constant, rotation, and translation (KRT) analysis (or other pose alignment analysis) of the images…The model training system 902 may calculate the estimated pose of a respective training image 1300, to match the pose of the 3D training model with the estimated pose of the user in the training image 1300. For example, the model training system 902 may translate, rotate, or otherwise modify the pose of the 3D training model to match the calculated pose for the training image 1300.”).
Regarding claim 19, which claim 16 is incorporated, Long discloses wherein the first time is prior to implementation of an orthodontic treatment plan, and the second time is after implementation of at least a portion of the orthodontic treatment plan (Paragraph [0106] “the patient may upload the user images 928 of impressions made following completion of one stage of treatment. For example, the patient may be prompted to upload images 928 at various intervals (such as daily, weekly, every two weeks, following completion of a stage of treatment, every six months or year following treatment via dental aligners, whenever the patient requests to check their progress, etc.). The patient may be prompted to upload images 928 to ensure that the patient's teeth are progressing according to the treatment plan, or to ensure that the patient's teeth have not reverted back to a position prior to treatment via dental aligners. The model generation system 924 may be configured to generate the 3D model 930 based on the user images 928”; Paragraph [0107] “The user interface may include, for example, the 3D model 930 generated based on the user images 928…the user interface may include the 3D model 930 and a prior (and/or subsequent) 3D model. The prior 3D model may be the 3D model from the patient file corresponding to a prior stage of the treatment plan. The prior 3D model may be a 3D model generated from a previous user image 928.”).
Regarding claim 20, which claim 16 is incorporated, Long discloses wherein the different camera position and orientation correspond to the position and orientation of one of the first or second image data (Paragraph [0136] “the user may capture images from different perspectives, such as those shown in FIG. 3A-FIG. 3C. In some embodiments, the user may capture images of their own teeth. In some embodiments, a different person may capture images of the user's teeth (e.g., from the user's device or from a different device).”; See Figures 3A-3C).
Claims 8-12 are rejected under 35 U.S.C. 103 as being unpatentable over Long et al. (US2022/0189130A1) (hereinafter, “Long”) in view of Li et al. (US 2020/0000551 A1) (hereinafter, “Li”), as applied to applied to claim 1 above, and further in view of Artizzu et al. (US 2023/0260238 A1) (hereinafter, “Artizzu”).
Regarding claim 8, which claim 1 is incorporated, Long and Li both fail to teach wherein the first and second image data are first videos and second videos.
Artizzu teaches wherein the first and second image data are first videos and second videos (different input videos in Paragraph [0022] equate to first video and second video) (Paragraph [0022] “The first phase is generally performed off-line and consists of data preparation. This first part only would have to be performed once for each patient (for example, during the patient's first clinical appointment). The data generated in this phase is used by the second phase, which can be performed on-line if so desired and that can be repeated as many times as desired, providing different input videos (or other types of real-time images) of the patient.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long in view of Li to include wherein the first and second image data are first videos and second videos taught by Artizzu’s reference. The motivation for doing so would have been to allow the patient to see their appearance before or after the medical treatment as suggested by Artizzu (see Artizzu, Paragraph [0019]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Artizzu with Long and Li to obtain the invention specified in claim 8.
Regarding claim 9, which claim 8 is incorporated, Long and Li both fail to teach wherein the novel image data is a new video.
Artizzu teaches wherein the novel image data is a new video (simulation video (8) in Paragraph [0081] equates to new video) (Paragraph [0076] “Inputs are the 2D video (5) and the 3D head model (2). The output is the 3D animated model (6) mimicking the movement observed in the 2D images (5), both in terms of the general head position as well as its facial physiognomy. This is performed in two separate steps, outlined in FIG. 3. First, using facial recognition techniques, the position and movement of the patient in the 2D video (5) is estimated. Then, these movements are transferred to a general 3D head model (9). Several approaches can be used in order to achieve this.”; Paragraph [0079 -0081] “Once the 3D head model (2) has been animated to reflect the action represented in the 2D images (5), the 3D denture model (4) needs to be moved correspondingly, taking into account the movement of the mandible, lips, and maxillary… once the head and denture 3D models (2,4) are in their correct position, the final 3D scene rendering (7) is computed and projected back to 2D, obtaining a simulation video (8).”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long in view of Li to include wherein the novel image data is a new video taught by Artizzu’s reference. The motivation for doing so would have been to allow the patient to see their appearance before or after the medical treatment as suggested by Artizzu (see Artizzu, Paragraph [0019]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Artizzu with Long and Li to obtain the invention specified in claim 9.
Regarding claim 10, which claim 9 is incorporated, Long discloses wherein the instructions, that when executed by the processor, further cause the system to: display, at the same time, [the new video and the one or more of the first or second videos used to generate the new video] (Paragraph [0189] “the user interface includes an image of the patient's smile (e.g., corresponding to the dental impression). The image may be displayed side-by-side with the merged model (and, optionally, the first and second models 2400, 2402). The technician may select the ideal merged model based on the side-by-side photo. Following selection of the merged model for use, the technician may manipulate, modify, etc. the merged model to reposition the patient's teeth, and the technician may export the modified merged model to the manufacturing manager 2114 for manufacturing dental aligners for repositioning the patient's teeth”).
However, Long and Li both fail to teach the new video and the one or more of the first or second videos used to generate the new video
Artizzu teaches the new video and the one or more of the first or second videos used to generate the new video (Paragraph [0022] “The first phase is generally performed off-line and consists of data preparation. This first part only would have to be performed once for each patient (for example, during the patient's first clinical appointment). The data generated in this phase is used by the second phase, which can be performed on-line if so desired and that can be repeated as many times as desired, providing different input videos (or other types of real-time images) of the patient.”; Paragraph [0076] “Inputs are the 2D video (5) and the 3D head model (2). The output is the 3D animated model (6) mimicking the movement observed in the 2D images (5), both in terms of the general head position as well as its facial physiognomy. This is performed in two separate steps, outlined in FIG. 3. First, using facial recognition techniques, the position and movement of the patient in the 2D video (5) is estimated. Then, these movements are transferred to a general 3D head model (9). Several approaches can be used in order to achieve this.”; Paragraph [0079 -0081] “Once the 3D head model (2) has been animated to reflect the action represented in the 2D images (5), the 3D denture model (4) needs to be moved correspondingly, taking into account the movement of the mandible, lips, and maxillary… once the head and denture 3D models (2,4) are in their correct position, the final 3D scene rendering (7) is computed and projected back to 2D, obtaining a simulation video (8).”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long in view of Li to include the new video and the one or more of the first or second videos used to generate the new video taught by Artizzu’s reference. The motivation for doing so would have been to allow the patient to see their appearance before or after the medical treatment as suggested by Artizzu (see Artizzu, Paragraph [0019]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Artizzu with Long and Li to obtain the invention specified in claim 10.
Regarding claim 11, which claim 10 is incorporated, Long discloses wherein the new [video] is generated as viewed from a position and orientation that matches a position and orientation of the first [videos] or the second [videos] (Paragraph [0116] “the model training system 902 calculates the estimated pose of the user in the training image 1300 by performing a camera constant, rotation, and translation (KRT) analysis (or other pose alignment analysis) of the images…The model training system 902 may calculate the estimated pose of a respective training image 1300, to match the pose of the 3D training model with the estimated pose of the user in the training image 1300. For example, the model training system 902 may translate, rotate, or otherwise modify the pose of the 3D training model to match the calculated pose for the training image 1300.”).
However, Long and Li both fail to teach the [new] video and the [first] video or the [second] video.
Artizzu teaches the [new] video and the [first] video or the [second] video (Paragraph [0081] “once the head and denture 3D models (2,4) are in their correct position, the final 3D scene rendering (7) is computed and projected back to 2D, obtaining a simulation video (8). The projection is done in such a way that the final model in the simulation video (8) has a similar size and position compared to the 2D input video (5), so that the simulation is more realistic and aesthetically plausible with respect to the real input.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long in view of Li to include the [new] video and the [first] video or the [second] video taught by Artizzu’s reference. The motivation for doing so would have been to allow the patient to see their appearance before or after the medical treatment as suggested by Artizzu (see Artizzu, Paragraph [0019]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Artizzu with Long and Li to obtain the invention specified in claim 11.
Regarding claim 12, which claim 10 is incorporated, Long fails to teach wherein the new video is generated as viewed from a position and orientation that does not match a position and orientation of the first videos or the second videos.
Li teaches wherein the new [video] is generated as viewed from a position and orientation that does not match a position and orientation of the first [videos] or the second [videos] (Paragraph [0109] “During the rendering process, a 3D model of the patient's teeth is formed based on data describing the teeth and arch of the patient. For example, based on the parametric 3D model formed at block 630, or otherwise describe herein. In some embodiments, the 3D model is directly inserted into the 2D image of the patient. In such embodiments, the actions in block 640 may be omitted or merged with the actions of block 650 such that, for example, the 3D tooth model is rendered in 2D form for insertion into the 2D image.”; Paragraph [0175] “the rendering engine may rendering the visualizations of the 3D models described herein, for example, at block 140 of FIG. 1, at blocks 640, 650, 660 of FIG. 6, process 800, described with reference to FIG. 8, at blocks 910, 920, 930 of FIG. 9, block 1150 of FIG. 10, and block 1150 of FIG. 11…The dental treatment planning system 1230 may be operative to provide for image viewing and manipulation such that rendered images may be scrollable, pivotable, zoomable, and interactive.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long’s reference to include wherein the new [video] is generated as viewed from a position and orientation that does not match a position and orientation of the first [videos] or the second [videos] taught by Li’s reference. The motivation for doing so would have been to obtain an estimated and/or intended outcome of the orthodontic treatment plan as suggested by Li (see Li, Paragraph [0085]).
However, Long and Li both fail to teach the [new] video and the [first] video or the [second] video.
Artizzu teaches the [new] video and the [first] video or the [second] video (Paragraph [0081] “once the head and denture 3D models (2,4) are in their correct position, the final 3D scene rendering (7) is computed and projected back to 2D, obtaining a simulation video (8). The projection is done in such a way that the final model in the simulation video (8) has a similar size and position compared to the 2D input video (5), so that the simulation is more realistic and aesthetically plausible with respect to the real input.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Long in view of Li to include the [new] video and the [first] video or the [second] video taught by Artizzu’s reference. The motivation for doing so would have been to allow the patient to see their appearance before or after the medical treatment as suggested by Artizzu (see Artizzu, Paragraph [0019]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Artizzu with Long and Li to obtain the invention specified in claim 12.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Farkash et al. (US 2021/0244502 A1) discloses a method for monitoring a patient’s teeth during orthodontic treatment by comparing an image to the treatment plan to determine how closely the current position of the patient’s teeth tracks with the expected positions according to the orthodontic treatment plan.
Kearney et al. (US 20210118132 A1) discloses a machine leaning model to determine whether a particular course of treatment is correct or incorrect by using images of a patient’s anatomy indicating the periodontal condition of the patient.
Tsai et al. (US 2019/0117337 A1) discloses a method for modifying a digital picture of a patient based on a tooth model produced during simulated orthodontic treatment after movement of a model tooth.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UROOJ FATIMA whose telephone number is (571)272-2096. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/UROOJ FATIMA/Examiner, Art Unit 2676
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676
Prosecution Insights