DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 9, 10, 12-14 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication US 2025/0217663 A1 (hereinafter “Demlow” (note: PCT application filed June 14, 2023, and U.S. provisional applications filed June 16 and August 2, 2022, all earlier than Application filing date of August 22, 2023) in view of U.S. Patent Application Publication US 2018/0263733 A1 (hereinafter “Pokotilov”).
Regarding claim 1, Demlow discloses a system (system 100 (Fig. 1) includes operating system 116 in processing unit 102) comprising:
- one or more processors (one or more processors 104 (paragraph [0034])) and a non-transitory computer readable medium (memory 106 (paragraph [0034])) containing instructions that when executed by the one or more processors causes the one or more processors to perform operations (processors and memory provide a computer platform for executing the operating system (paragraph [0034])) comprising:
- receiving a three-dimensional (3D) representation of a user’s mouth, wherein the 3D representation is a mesh representation comprising a plurality of surfaces (receiving 3D meshes of one or more teeth, with additional optional data pertaining to the dental procedure (paragraph [0124]));
- mapping the 3D representation of the user’s mouth into a two-dimensional (2D) space (generating one or more 2D raster view of the 3D meshes (paragraph [0124]));
- encoding one or more 3D surface characteristics of the plurality of surfaces (problematic regions of the mesh can be highlighted in patchwork fashion, with different color coding (paragraph [0125]));
- determining an enhanced representation of the user’s mouth in the 2D space by executing a machine learning model on the representation of the user’s mouth in the 2D space (neural network analyzes each aspect of the 2D and/or 3D representations to render a pass/fail determination on the aspects (paragraph [0124]));
- mapping the enhanced representation of the user’s mouth in the 2D space to an enhanced 3D representation of the user’s mouth using the one or more 3D surface characteristics of the plurality of surfaces (if a sufficient number of aspects of the mesh receive a passing accuracy score, then the system can provide the geometry for use in other dental processes (paragraph [0124]), i.e., output a passing digital 3D representation (Fig. 8), which suggests remapping the 2D representation back into the digital 3D representation); and
- outputting the enhanced 3D representation (output: passing digital 3D representation (Fig. 8); results of validation may be displayed using one or more heatmaps, possibly superimposed on a model of the teeth (paragraph [0125])).
Demlow does not expressly disclose encoding one or more 3D surface characteristics of the plurality of surfaces “using one or more channels of a 2D representation.”
Pokotilov discloses a method wherein a 3D model of a patient’s teeth is color coded based on attributes of the 3D model (paragraph [0030]). A 2D rendering of the 3D model is provided, and a color channel of a plurality of pixels of the 2D rendering is coded with attributes of the 3D model (paragraph [0034]). Using a color channel of the 2D representation to obtain a 3D model helps distinguish the surface characteristics from the 2D rendering as they appear on the 3D model. Therefore, it would have been obvious for one of ordinary skill in the art to have provided for encoding one or more 3D surface characteristics using one or more channels of a 2D representation, as taught by Pokotilov, in the method taught by Demlow.
Regarding claim 9, Demlow discloses wherein the enhanced 3D representation is an enhanced mesh representation of the user’s mouth that has been at least one of sculpted, smoothed, filled in, or has had artifacts removed (mesh processing techniques known to one skilled in the art can be applied to labeled mesh elements to remove, modify, and/or extend (e.g., hole-filling, bridging, boundary extension, etc.) areas of the mesh (paragraph [0129])). As for wherein the one or more channels of the 2D representation is at least one of a 3-channel 2D representation, a 4-channel 2D representation, a 5-channel 2D representation, a 10-channel 2D representation, or an n-channel 2D representation, this limitation is taught by Pokotilov (2D rendering of the 3D model is provided, and a color channel of a plurality of pixels of the 2D rendering is coded with attributes of the 3D model (paragraph [0034]), which covers an n-channel 2D representation (“n” can be any number)).
Regarding claim 10, Demlow discloses wherein the instructions executed by the one or more processors cause the one or more processors to perform operations comprising:
- receiving the 2D representation of the user’s mouth (a two-dimensional (2D) representation of a user’s mouth (2D images generated for a patient’s mouth (paragraph [0113])); and
- converting the 2D representation into the 3D representation of the user’s mouth (receiving 3D meshes of one or more teeth, with additional optional data pertaining to the dental procedure (paragraph [0124])).
Regarding claim 12, Demlow discloses a method comprising:
- receiving, by one or more processors (one or more processors 104 (paragraph [0034])), a three-dimensional (3D) representation of a user’s mouth, wherein the 3D representation is a mesh representation comprising a plurality of surfaces (receiving 3D meshes of one or more teeth, with additional optional data pertaining to the dental procedure (paragraph [0124]));
- mapping, by the one or more processors, the 3D representation of the user’s mouth into a two-dimensional (2D) space (generating one or more 2D raster view of the 3D meshes (paragraph [0124]));
- encoding, by the one or more processors, one or more 3D surface characteristics of the plurality of surfaces (problematic regions of the mesh can be highlighted in patchwork fashion, with different color coding (paragraph [0125]));
- applying, by the one or more processors, a machine learning model to the representation of the user’s mouth in the 2D space, the machine learning model trained to enhance the representation of the user’s mouth in the 2D space (neural network analyzes each aspect of the 2D and/or 3D representations to render a pass/fail determination on the aspects (paragraph [0124])); and
- mapping, by the one or more processors, the enhanced representation of the user’s mouth in 2D space to an enhanced representation of the user’s mouth in 3D space using the encoded one or more 3D surface characteristics (if a sufficient number of aspects of the mesh receive a passing accuracy score, then the system can provide the geometry for use in other dental processes (paragraph [0124]), i.e., output a passing digital 3D representation (Fig. 8), which suggests remapping the 2D representation back into the digital 3D representation; results of validation may be displayed using one or more heatmaps, possibly superimposed on a model of the teeth (paragraph [0125])).
Demlow does not expressly disclose encoding one or more 3D surface characteristics of the plurality of surfaces “using one or more channels of a 2D representation.”
As set forth above regarding claim 1, Pokotilov discloses a method wherein a 3D model of a patient’s teeth is color coded based on attributes of the 3D model (paragraph [0030]). A 2D rendering of the 3D model is provided, and a color channel of a plurality of pixels of the 2D rendering is coded with attributes of the 3D model (paragraph [0034]). Using a color channel of the 2D representation to obtain a 3D model helps distinguish the surface characteristics from the 2D rendering as they appear on the 3D model. Therefore, it would have been obvious for one of ordinary skill in the art to have provided for encoding one or more 3D surface characteristics using one or more channels of a 2D representation, as taught by Pokotilov, in the method taught by Demlow.
Regarding claim 13, Pokotilov further comprises:
- generating, by the one or more processors, a treatment plan based on the enhanced representation of the user’s mouth in 3D space (one or more treatment stages are generated based on the digital representation of the teeth (paragraph [0181])).
Regarding claim 14, Pokotilov does not expressly disclose receiving, by the one or more processors, from a treatment planning computing device, validation of the treatment plan. However, Official Notice is taken that it is well known in the art to provide means, either by a physician or machine, for validating a treatment for any medical condition. Such validation, as would be recognized by one of ordinary skill in the art, is necessary as a safeguard for assuring that a determined treatment plan is appropriate for addressing a patient’s medical condition. Therefore, providing such validation would have been an obvious modification of the combined teachings of Demlow and Pokotilov to one of ordinary skill in the art.
Regarding claim 16, Pokotilov further comprises receiving, by the one or more processors, from a user device, an initiation of an order of a product based on the treatment plan (at least one orthodontic appliance is fabricated based on the generated treatment stages (paragraph [0182]); fabrication initiated by a chair side direct fabrication machine at a treating professional’s office (paragraph [0184])).
Regarding claim 17, Pokotilov discloses wherein generating the treatment plan comprises generating, by the one or more processes, a plurality of intermediate 3D representations of the user’s mouth showing a progression of a plurality of teeth from an initial position to a final position, wherein each of the plurality of intermediate 3D representations corresponds to a respective stage of the treatment plan (treatment stages can be incremental repositioning stages of an orthodontic treatment procedure designed to move one or more of the patient’s teeth from an initial tooth arrangement to a target arrangement (paragraph [0181])).
Regarding claim 18, Pokotilov further comprises manufacturing a dental aligner specific to the enhanced representation of the user’s mouth in 3D space (at least one orthodontic appliance is fabricated based on the generated treatment stages (paragraph [0182])).
Regarding claim 19, Demlow discloses a method comprising:
- receiving, by one or more processors (one or more processors 104 (paragraph [0034])), a two-dimensional (2D) representation of a user’s mouth (2D images generated for a patient’s mouth (paragraph [0113]));
- converting, by the one or more processors, the 2D representation into a three-dimensional (3D) representation of a user’s mouth, wherein the 3D representation is a mesh representation comprising a plurality of surfaces (receiving 3D meshes of one or more teeth, with additional optional data pertaining to the dental procedure (paragraph [0124]));
- mapping, by the one or more processors, the 3D representation of the user’s mouth into a two-dimensional (2D) space (generating one or more 2D raster view of the 3D meshes (paragraph [0124]));
- encoding, by the one or more processors, one or more 3D surface characteristics of the plurality of surfaces (problematic regions of the mesh can be highlighted in patchwork fashion, with different color coding (paragraph [0125]));
- applying, by the one or more processors, a machine learning model to the representation of the user’s mouth in the 2D space, the machine learning model trained to enhance the representation of the user’s mouth in the 2D space (neural network analyzes each aspect of the 2D and/or 3D representations to render a pass/fail determination on the aspects (paragraph [0124])); and
- mapping, by the one or more processors, the enhanced representation of the user’s mouth in 2D space to an enhanced representation of the user’s mouth in 3D space using the encoded one or more 3D surface characteristics (if a sufficient number of aspects of the mesh receive a passing accuracy score, then the system can provide the geometry for use in other dental processes (paragraph [0124]), i.e., output a passing digital 3D representation (Fig. 8), which suggests remapping the 2D representation back into the digital 3D representation; results of validation may be displayed using one or more heatmaps, possibly superimposed on a model of the teeth (paragraph [0125])).
Demlow does not expressly disclose encoding one or more 3D surface characteristics of the plurality of surfaces “using one or more channels of a 2D representation.”
As set forth above regarding claims 1 and 12, Pokotilov discloses a method wherein a 3D model of a patient’s teeth is color coded based on attributes of the 3D model (paragraph [0030]). A 2D rendering of the 3D model is provided, and a color channel of a plurality of pixels of the 2D rendering is coded with attributes of the 3D model (paragraph [0034]). Using a color channel of the 2D representation to obtain a 3D model helps distinguish the surface characteristics from the 2D rendering as they appear on the 3D model. Therefore, it would have been obvious for one of ordinary skill in the art to have provided for encoding one or more 3D surface characteristics using one or more channels of a 2D representation, as taught by Pokotilov, in the method taught by Demlow.
7. Claims 2, 3, 5 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Demlow in view of Pokotilov as applied to claims 1 and 19 above, and further in view of U.S. Patent Application Publication 2023/0274492 A1 (hereinafter “Chen”).
Regarding claim 2, neither Demlow nor Pokotilov expressly disclose wherein mapping the 3D representation of the user’s mouth into the 2D space comprises using UV mapping, and wherein the UV mapping comprises mapping at least one vertex of the 3D representation to at least one pixel in the 2D space.
Chen discloses utilizing a network that learns to embed 3D coordinates on a surface on one or more 3D shapes into an aligned 2D texture space, where corresponding parts of different 3D shapes can be mapped to the same location in a texture image. Multi-dimensional (e.g., 3D) coordinates may be embedded on a surface of one or more multi-dimensional shapes into an aligned 2D space, such as a UV space. Texture alignment can be learned in an unsupervised manner, and the learned UV mapping and aligned texture representations can allow for a variety of operations, including texture transfer, texture synthesis, and textured single view reconstruction (paragraph [0027]). As is well known in the art, UV mappings map at least one vertex of a 3D representation to at least one pixel in the 2D space. Such an approach allows for textured 3D shape reconstruction from single images, and have been demonstrated successfully for a variety of types of objects (paragraph [0029]). Therefore, it would have been obvious for one of ordinary skill in the art to have modified the combined teachings of Demlow and Pokotilov by using UV mapping for transforming the 3D representation of the user’s mouth into the 2D space, in a manner such as taught by Chen.
Regarding claim 3, neither Demlow nor Pokotilov expressly disclose assigning a plurality of pixels to each surface of the plurality of surfaces. However, Official Notice is taken that mapping plural pixels to each surface in UV mapping is well known in the art, and one of ordinary skill in the art would have recognized that mapping a plurality of pixels to each surface would enhance the resolution of the image in the 2D space. Therefore, assigning a plurality of pixels to each surface of a plurality of surfaces would have been an obvious modification of the combined teachings of Demlow and Pokotilov to one of ordinary skill in the art.
Regarding claim 5, Chen discloses wherein the machine learning model is trained using a set of training 3D representations and corresponding enhanced 3D representation, the set of training 3D representations and corresponding enhanced 3D representations are mapped into 2D space using UV mapping (neural network can be trained and/or used to embed 3D coordinates on a surface of one or more multi-dimensional shapes into an aligned 2D space, such as a UV space (paragraph [0027])).
Regarding claim 20, Chen discloses wherein mapping, by the one or more processors, the 3D representation of the user’s mouth into the 2D space and mapping, by the one or more processors, the enhanced representation of the user’s mouth in 2D space to the enhanced representation of the user’s mouth in 3D space comprises performing UV mapping, and wherein the UV mapping comprises mapping at least one vertex of the 3D representation to at least one pixel in the 2D space (multi-dimensional (e.g., 3D) coordinates may be embedded on a surface of one or more multi-dimensional shapes into an aligned 2D space, such as a UV space; texture alignment can be learned in an unsupervised manner, and the learned UV mapping and aligned texture representations can allow for a variety of operations, including texture transfer, texture synthesis, and textured single view reconstruction (paragraph [0027]); as is well known in the art, UV mappings map at least one vertex of a 3D representation to at least one pixel in the 2D space). Such an approach allows for textured 3D shape reconstruction from single images, and have been demonstrated successfully for a variety of types of objects (paragraph [0029]). Therefore, it would have been obvious for one of ordinary skill in the art to have modified the combined teachings of Demlow and Pokotilov by using UV mapping for transforming the 3D representation of the user’s mouth into the 2D space, in a manner such as taught by Chen.
8. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Demlow in view of Pokotilov as applied to claim 1 above, and further in view of Chen and U.S. Patent Application Publication US 2019/0208179 A1 (hereinafter “Kasahara”).
Regarding claim 4, neither Demlow nor Pokotilov expressly disclose wherein mapping the enhanced representation of the user’s mouth in 2D space to the enhanced 3D representation of the user’s mouth comprises using UV mapping. As set forth above regarding claim 2, Chen discloses using UV mapping for transforming the 3D representation of the user’s mouth into the 2D space, but Chen does not expressly disclose mapping the enhanced representation of the user’s mouth in 2D space to the enhanced 3D representation of the user’s mouth comprises using UV mapping. However, the concept of providing an inverse UV mapping of a 2D image so as to reconstruct a 3D representation of the image after a UV mapping from an original 3D representation to 2D space is taught by Kasahara (a step of developing the three-dimensional model and perform UV mapping to map the image information projected onto each side surface onto a two-dimensional plane (paragraph [0162]), followed by a step of performing inverse UV mapping to map decoded image information on the two-dimensional plane onto each side surface of the three-dimensional model (paragraph [0166])). As set forth above regarding claim 1, Demlow discloses the mapping of a 3D representation into a 2D space, as well as subsequent mapping of an enhanced representation in 2D space to an enhanced 3D representation. In combination with Chen, which discloses using UV mapping of the image in 2D space to produce a 3D representation, and the above teaching of Kasahara, it would have been obvious for one of ordinary skill in the art to have modified the combined teachings of Demlow, Pokotilov and Chen to utilize an inverse UV mapping so as to provide the predictable result of producing an enhanced 3D representation.
9. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Demlow in view of Pokotilov as applied to claim 10 above, and further in view of U.S. Patent Application Publication US 2022/0313392 A1 (hereinafter “Moray”).
Regarding claim 11, neither Demlow nor Pokotilov expressly disclose representation of the user’s mouth is based on a video stream obtained by a user device associated with the user (Demlow discloses 2D images generated for a patient’s mouth (paragraph [0113]), but does not mention obtaining a video stream from a user device).
Moray discloses a method for creating improved orthodontic aligners. The method includes a step of receiving a digital data set representative of the dentition of a patient, wherein a video of the mouth of the patient is taken by a smartphone or other handheld device (paragraph [0073]). One of ordinary skill in the art would have recognized that when a large number of images of a subject is to be taken, it would be advantageous to use a video camera, which is typically provided in device such as a user’s smartphone, to obtain a video stream of the subject, which saves time over taking multiple individual images. Also, a patient can take the video stream using his/her own device, thereby providing added convenience for the patient. Therefore, providing a video stream obtained by a user device associated with a user would have been an obvious modification of the combined teachings of Demlow and Pokotilov to one of ordinary skill in the art.
Allowable Subject Matter
10. Claims 6-8 and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
11. The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 6, the cited prior art fails to disclose or suggest Applicant’s system of claim 1, wherein the instructions executed by the one or more processors causes the one or more processors to perform operations comprising:
- sampling, from a database, a training 3D representation of a user’s mouth and a corresponding enhanced 3D representation according to a flag associated with at least one of the training 3D representation or the enhanced 3D representation; and
- training the machine learning model using the sampled training 3D representation and the enhanced 3D representation.
Claim 7 depends from claim 6.
Regarding claim 8 and 15, the cited prior art fails to disclose Applicant’s system of claim 1 or method of claim 13, wherein encoding the one or more 3D surface characteristics comprises encoding a normal of each surface of the plurality of surfaces using red green blue (RGB) values associated with orientation information of each surface.
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS D LEE whose telephone number is (571)272-7436. The examiner can normally be reached Mon-Fri 7:30AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi Sarpong can be reached at 571-270-3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS D LEE/Primary Examiner, Art Unit 2681