DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-22, as originally filed, are currently pending and have been considered below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 12 and 19-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Weiss et al., U.S. Publication No. 2021/0059796, hereinafter, “Weiss”, and further in view of Chen et al., Chinese Publication No. CN 113256518 A, hereinafter, “Chen”.
As per claim 1, Weiss discloses an intraoral scanning system, comprising:
an intraoral scanner configured to generate intraoral scan data (Weiss, ¶0116, a user (e.g., a practitioner) may subject a patient to intraoral scanning. In doing so, the user may apply scanner 150 to one or more patient intraoral locations); and
a computing device configured to: receive intraoral scan data comprising one or more intraoral images of a portion of a dental site generated during intraoral scanning of the dental site (Weiss, ¶0116, Via such scanner application, the scanner 150 may provide intraoral scan data 135A-N to computing device 105. The intraoral scan data 135A-N may be provided in the form of intraoral image data sets … separate image data sets are created for the maxillary arch, for the mandibular arch, for a patient bite, and for each preparation tooth. Alternatively, a single large intraoral image data set is generated);
generate a three-dimensional (3D) point cloud of the portion of the dental site using the intraoral scan data (Weiss, ¶0116, Such images may be provided from the scanner to the computing device 105 in the form of one or more points (e.g., one or more pixels and/or groups of pixels). For instance, the scanner 150 may provide such a 3D image as one or more point clouds);
detect one or more regions of the 3D model comprising a bodily fluid (Weiss, ¶0177, a part of a real margin line of a scanned preparation tooth may not be sufficiently clearly defined in the 3D model. For example, during the initial 3D data collection step, for example via scanning, that resulted in the first 3D virtual model being generated, a part of the physical dental surface may have been covered with foreign material, such as for example saliva, blood, or debris. The part of the physical dental surface may also have been obscured by another element such as for example part of the gums, cheek, tongue, dental instruments, artifacts, etc. Alternatively, for example, during the initial 3D data collection step (e.g., via scanning) that resulted in the first virtual 3D model being generated, the region may have been distorted or otherwise defective and may not properly correspond to a physical dental surface (e.g., due to some defect in the actual scanning process). Automatic correction may be performed to remove the representation of the foreign material and show the underlying tooth surface and/or margin line. If automatic correction of the dental surface and/or margin line was performed, then the obscured region may be created, and the obscuring object may be removed in the 3D model);
remove the one or more regions from the 3D model (Weiss, ¶0177, Automatic correction may be performed to remove the representation of the foreign material and show the underlying tooth surface and/or margin line. If automatic correction of the dental surface and/or margin line was performed, then the obscured region may be created, and the obscuring object may be removed in the 3D model; Weiss, ¶0179, Intraoral scan application 115 may then apply eraser logic to delete, erase or otherwise remove the marked portion from the 3D model. All regions other than the marked portion may be locked);
update a 3D surface of the dental site using the 3D point cloud of the portion of the dental site (Weiss, ¶0180, Intraoral scan application 108 may direct a user to generate one or more additional intraoral images of the dental site corresponding to the portion of the 3D model (and corresponding set or sets of intraoral images) that was deleted or removed … The one or more additional intraoral images may be registered with the 3D model (and/or with the intraoral image data sets used to create the 3D model) to provide a composite of the 3D model and the one or more additional intraoral images. In the composite, the part of the 3D model that was previously deleted/removed is at least partially replaced with a corresponding part of the one or more additional intraoral images); and
output the 3D surface of the dental site to a display (Weiss, ¶0022, displaying the updated three-dimensional mode).
Weiss does not explicitly disclose the following limitations as further recited however Chen discloses
receive intraoral scan data comprising one or more intraoral images of a portion of a dental site generated using structured light projection during intraoral scanning of the dental site (Chen, Figure 2; Chen, ¶n0003, structured light cameras have advantages such as high maturity, high precision, fast response speed, and low cost, and are widely used in dental scanners. The principle of a structured light camera is to project specific encoded stripes onto the surface of an object. The camera captures the projected pattern on the object's surface and performs analysis and decoding. Based on the matching relationship between the structured light image and the projected pattern, as well as the camera's calibration parameters, the three-dimensional coordinates of the object's surface are calculated. A point cloud is a collection of all the points on the surface of an object. After steps such as texture stitching and rendering, the three-dimensional model of the object can be reconstructed; Chen, ¶n0006, a structured light image enhancement method for intraoral 3D reconstruction is provided, which enhances the stripe image captured by an oral scanner based on the structured light principle; Chen, ¶n0039, This embodiment provides a method for enhancing structured light images for intraoral 3D reconstruction, which enhances structured light images obtained by scanning with an oral scanner);
detect one or more regions of the 3D point cloud comprising a bodily fluid (Chen, Figure 3; Chen, ¶n0004, due to factors such as the small space inside the oral cavity, the complex environment (the influence of blood, saliva, etc. on imaging), the shaking of the handheld device, the translucent nature of tooth enamel, the reflective properties of tooth surface materials, and the operating temperature of the instrument, the quality of the striped images captured by the structured light camera varies greatly, and the brightness and contrast also differ significantly. This affects the decoding and analysis of the striped images, the accuracy of extracting the stripe center information, and the accuracy of subsequent 3D point cloud and reconstructed dental model; Chen, ¶n0064, The initial pixel grayscale values within the initial window are considered valid data if they fall within the range of 5% to 95%);
remove the one or more regions from the 3D point cloud (Chen, ¶n0007, This invention provides a structured light image enhancement method for intraoral 3D reconstruction, characterized by the following steps: Step S1, performing subtraction on adjacent frame structured light images to extract stripes; Step S2, enhancing the stripes using an adaptive Hessian image enhancement method; Chen, ¶n0033, a structured light image enhancement method for intraoral 3D reconstruction specifically involves first performing subtraction on adjacent frame structured light images and extracting stripes, and then enhancing the stripes using an adaptive Hessian image enhancement method. This achieves the optimal enhancement effect for intraoral 3D reconstructed structured light images ... It achieves a consistent enhancement effect across areas with varying brightness and contrast. It greatly increases the number of effective 3D point clouds and enhances 3D details; Chen, ¶n0076, As shown in Figure 2, in step S3, for the four factors that affect the quality of the striped image encountered in the actual operation of the oral scanner, namely factor 1, the translucent reflective properties of the enamel surface, factor 2, the teeth being covered by reflective objects, factor 3, the teeth being adhered to by saliva, and factor 4, the bright and dark stripes caused by uneven light illumination, the four samples corresponding to the four factors are subjected to adjacent frame subtraction processing, traditional Hessian enhancement processing, and adaptive Hessian enhancement processing proposed in this embodiment, and then the enhancement results of the above methods are compared; Chen, ¶n0080, As shown in Figure 3, the three-dimensional point cloud data obtained before and after stripe image enhancement can be viewed in the MeshLab software visualization. It can be seen from the areas marked by arrows and circles in Figure 3 that after processing the image using the adaptive Hessian enhancement method proposed in this embodiment, the number of three-dimensional points of the teeth is greater, the point cloud is denser, and the accuracy is improved; Chen, ¶n0082, a structured light image enhancement method for intraoral 3D reconstruction specifically involves first performing a subtraction operation on adjacent frame structured light images and extracting the stripes, and then using an adaptive Hessian image enhancement method to enhance the stripes, thereby achieving the optimal enhancement effect for intraoral 3D reconstruction structured light images. In other words, for the 3D reconstructed striped structured light image, the stripe structure similarity function is dynamically and adaptively adjusted to achieve the optimal stripe enhancement effect, making the image clearer and more complete; Chen, ¶n0084, In this embodiment, the range of 5%-95% of the initial pixel grayscale value within the initial window is selected as valid data. This more effectively avoids the reflective properties of the tooth surface, eliminates bright spots appearing in the local area of the image, and helps with subsequent information processing).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Chen with Weiss because they are in the same field of endeavor. One skilled in the art would have been motivated to include the enhancements of the point cloud prior to constructing the model as taught by Chen in the system of Weiss in order to improve the accuracy of the 3D point cloud and the reconstructed model (Chen, ¶n0004).
As per claim 2, Weiss and Chen disclose the intraoral scanning system of claim 1, wherein at least one of the intraoral scanner or the computing device is configured to: generate an alert to perform a corrective action responsive to detection of the one or more regions of the 3D point cloud comprising the bodily fluid (Weiss, ¶0105, A quality of the margin line may be assessed, and a dental practitioner (also referred to herein as a doctor) may be notified that the margin line should be rescanned if the margin line has a low quality score (e.g., if the virtual 3D model has an unclear, inaccurate, indefinite, and/or indeterminate margin line). Additionally, or alternatively, quality scores may be computed for different segments of the margin line, and any segment of the margin line with a low quality score (e.g., a quality score that is below a quality threshold) may be highlighted or otherwise marked in the 3D model).
As per claim 3, Weiss and Chen disclose the intraoral scanning system of claim 1, wherein the computing device is further configured to:
determine an amount of the bodily fluid that has been detected (Weiss, ¶0106, For example, blood, saliva, soft tissue (e.g., a collapsing gum) and/or a retraction material may obscure a margin line in the 3D model);
determine whether the amount of the bodily fluid that has been detected exceeds a threshold (Weiss, ¶0105, quality scores may be computed for different segments of the margin line, and any segment of the margin line with a low quality score (e.g., a quality score that is below a quality threshold) may be highlighted or otherwise marked in the 3D model; Weiss, ¶0106, For example, blood, saliva, soft tissue (e.g., a collapsing gum) and/or a retraction material may obscure a margin line in the 3D model); and
generate an alert to perform a corrective action responsive to determining that the amount of the bodily fluid exceeds the threshold (Weiss, ¶0105,A quality of the margin line may be assessed, and a dental practitioner (also referred to herein as a doctor) may be notified that the margin line should be rescanned if the margin line has a low quality score (e.g., if the virtual 3D model has an unclear, inaccurate, indefinite, and/or indeterminate margin line). Additionally, or alternatively, quality scores may be computed for different segments of the margin line, and any segment of the margin line with a low quality score (e.g., a quality score that is below a quality threshold) may be highlighted or otherwise marked in the 3D model).
As per claim 4, Weiss and Chen disclose the intraoral scanning system of claim 3, wherein the amount of bodily fluid that has been detected comprises a suspected amount of bodily fluid (Weiss, ¶0012, the image comprises a depiction of an interfering surface that obscures the margin line, wherein at least a portion of the depiction of the interfering surface is removed in the modified image, and wherein a portion of the margin line that was obscured in the image is shown in the modified image ... the interfering surface comprises at least one of blood, saliva; Weiss, ¶0106, For example, blood, saliva, soft tissue (e.g., a collapsing gum) and/or a retraction material may obscure a margin line in the 3D model).
As per claim 5, Weiss and Chen disclose the intraoral scanning system of claim 3, wherein the alert comprises a recommendation to remove the bodily fluid from the dental site prior to continuing the intraoral scanning of the dental site (Weiss, ¶0105,The virtual 3D model (or images generated from the virtual 3D model or images used to generate the virtual 3D model) is analyzed to identify the margin line. In some embodiments, images from an intraoral scan and/or images generated by projecting a virtual 3D model onto a 2D surface are analyzed using a trained machine learning model that has been trained to determine margin lines on preparation teeth … A quality of the margin line may be assessed, and a dental practitioner (also referred to herein as a doctor) may be notified that the margin line should be rescanned if the margin line has a low quality score).
As per claim 6, Weiss and Chen disclose the intraoral scanning system of claim 3, wherein the amount of the bodily fluid is determined based on an area covered by the one or more regions (Weiss, ¶0105, quality scores may be computed for different segments of the margin line, and any segment of the margin line with a low quality score (e.g., a quality score that is below a quality threshold) may be highlighted or otherwise marked).
As per claim 7, Weiss and Chen disclose the intraoral scanning system of claim 1, wherein the intraoral scan data further comprises one or more color two-dimensional (2D) images generated during the intraoral scanning of the dental site (Weiss, ¶0115, Intraoral scanner 150 may include a probe (e.g., a hand held probe) for optically capturing three-dimensional structures ... Each intraoral image may be a two-dimensional (2D) or 3D image that includes a height map of a portion of a dental site, and may include x, y and z information. In one embodiment, the intraoral scanner 150 generates numerous discrete (i.e., individual) intraoral images. Sets of discrete intraoral images may be merged; Weiss, ¶0156, color info may help. In one embodiment, color info Is used as an additional 3 layers (e.g., RGB)), and wherein detecting the one or more regions of the 3D point cloud comprising a bodily fluid comprises:
detecting one or more suspected regions in the one or more color 2D images that are suspected to comprise the bodily fluid (Weiss, ¶0106, blood, saliva, soft tissue (e.g., a collapsing gum) and/or a retraction material may obscure a margin line in the 3D model. In some embodiments, images from an intraoral scan and/or images (e.g., height maps) generated by projecting a virtual 3D model onto a 2D surface are analyzed using a trained machine learning model that has been trained to redraw or reshape a surface of a tooth to correct inaccuracies and/or regions lacking clarity);
mapping the one or more suspected regions in the one or more color 2D images to the 3D point cloud to form one or more suspected regions of the 3D point cloud that are suspected to comprise the bodily fluid (Weiss, ¶0106, blood, saliva, soft tissue (e.g., a collapsing gum) and/or a retraction material may obscure a margin line in the 3D model. In some embodiments, images from an intraoral scan and/or images (e.g., height maps) generated by projecting a virtual 3D model onto a 2D surface are analyzed using a trained machine learning model that has been trained to redraw or reshape a surface of a tooth to correct inaccuracies and/or regions lacking clarity. Thus, the trained machine learning model may fabricate a correct margin line in a region of an image of a preparation tooth where no clear margin line was shown, and may output a modified image with the correct margin line. The trained machine learning model may also reshape the surface of the preparation tooth (or other tooth) in the image, and output a modified image with the reshaped surface. The modified image may be used to adjust the virtual 3D model of the tooth); and
determining one or more suspected regions of the 3D point cloud that satisfy one or more criteria, wherein the one or more suspected regions of the 3D point cloud that satisfy the one or more criteria correspond to the one or more regions of the 3D point cloud that comprise the bodily fluid (Weiss, ¶0106, The virtual 3D model (or images generated from the virtual 3D model or images used to generate the virtual 3D model) may additionally or alternatively be analyzed to identify and/or correct inaccurate and/or unclear representations of teeth (e.g., a portion of a tooth that is covered by an interfering surface that obscures the margin line), and to correct the inaccurate and/or unclear representation of the tooth. For example, blood, saliva, soft tissue (e.g., a collapsing gum) and/or a retraction material may obscure a margin line in the 3D model. In some embodiments, images from an intraoral scan and/or images (e.g., height maps) generated by projecting a virtual 3D model onto a 2D surface are analyzed using a trained machine learning model that has been trained to redraw or reshape a surface of a tooth to correct inaccuracies and/or regions lacking clarity. Thus, the trained machine learning model may fabricate a correct margin line in a region of an image of a preparation tooth where no clear margin line was shown, and may output a modified image with the correct margin line. The trained machine learning model may also reshape the surface of the preparation tooth (or other tooth) in the image, and output a modified image with the reshaped surface. The modified image may be used to adjust the virtual 3D model of the tooth; Chen, ¶n0004, due to factors such as the small space inside the oral cavity, the complex environment (the influence of blood, saliva, etc. on imaging), the shaking of the handheld device, the translucent nature of tooth enamel, the reflective properties of tooth surface materials, and the operating temperature of the instrument, the quality of the striped images captured by the structured light camera varies greatly, and the brightness and contrast also differ significantly. This affects the decoding and analysis of the striped images, the accuracy of extracting the stripe center information, and the accuracy of subsequent 3D point cloud and reconstructed dental model; Chen, ¶n076; Chen, Figure 3).
As per claim 8, Weiss and Chen disclose the intraoral scanning system of claim 7, wherein the one or more color 2D images were generated using unstructured white light illumination (Chen, ¶n0062, Steps S2-3: Statistically analyze the grayscale distribution of all initial pixels, sort them, calculate the enhancement parameter c of the center pixel, and obtain the stripe structure similarity function at the center pixel; Chen, ¶n0064, The initial pixel grayscale values within the initial window are considered valid data if they fall within the range of 5% to 95%).
As per claim 12, Weiss and Chen disclose the intraoral scanning system of claim 7, wherein the one or more color 2D images comprise a plurality of color 2D images generated at a same point in time, each color 2D image of the plurality of color 2D images having been generated by a different camera of an intraoral scanner used to perform the intraoral scanning (Weiss, ¶0156, color info may help. In one embodiment, color info Is used as an additional 3 layers (e.g., RGB); Weiss, ¶0157, additional data may include an image generated under specific lighting conditions (e.g., an image generated under ultraviolet or infrared lighting conditions). The additional data may be a 2D or 3D image).
As per claim 19, Weiss and Chen disclose the intraoral scanning system of claim 1, wherein the 3D surface comprises one or more voids corresponding to the one or more regions comprising the bodily fluid, and wherein the computing device is further configured to: highlight the one or more voids in the 3D surface (Weiss, ¶0179, Intraoral scan application 115 may then apply eraser logic to delete, erase or otherwise remove the marked portion from the 3D model. All regions other than the marked portion may be locked; Weiss, ¶0180, Intraoral scan application 108 may direct a user to generate one or more additional intraoral images of the dental site corresponding to the portion of the 3D model (and corresponding set or sets of intraoral images) that was deleted or removed. The user may then use the scanner 150 to generate the one or more additional intraoral images; Chen, Figure 3).
As per claim 20, Weiss and Chen disclose the intraoral scanning system of claim 19, wherein the computing device is further configured to: receive additional intraoral scan data after the bodily fluid has been removed from the dental site; and update the 3D surface based on the additional intraoral scan data, wherein the one or more voids are filled in based on the additional intraoral scan data (Weiss, ¶0180, Intraoral scan application 108 may direct a user to generate one or more additional intraoral images of the dental site corresponding to the portion of the 3D model (and corresponding set or sets of intraoral images) that was deleted or removed … The one or more additional intraoral images may be registered with the 3D model (and/or with the intraoral image data sets used to create the 3D model) to provide a composite of the 3D model and the one or more additional intraoral images. In the composite, the part of the 3D model that was previously deleted/removed is at least partially replaced with a corresponding part of the one or more additional intraoral images).
As per claim 21, Weiss discloses an intraoral scanning system, comprising:
an intraoral scanner configured to generate intraoral scan data (Weiss, ¶0116, a user (e.g., a practitioner) may subject a patient to intraoral scanning. In doing so, the user may apply scanner 150 to one or more patient intraoral locations); and
a first computing device configured to: receive intraoral scan data comprising one or more intraoral images of a portion of a dental site generated during intraoral scanning of the dental site (Weiss, ¶0116, Via such scanner application, the scanner 150 may provide intraoral scan data 135A-N to computing device 105. The intraoral scan data 135A-N may be provided in the form of intraoral image data sets … separate image data sets are created for the maxillary arch, for the mandibular arch, for a patient bite, and for each preparation tooth. Alternatively, a single large intraoral image data set is generated);
generate a three-dimensional (3D) point cloud of the portion of the dental site using the intraoral scan data (Weiss, ¶0116, Such images may be provided from the scanner to the computing device 105 in the form of one or more points (e.g., one or more pixels and/or groups of pixels). For instance, the scanner 150 may provide such a 3D image as one or more point clouds); and
transmit the 3D point cloud to a second computing device (Weiss, ¶0113, System 100 includes a dental office 108 and a dental lab 110. The dental office 108 and the dental lab 110 each include a computing device 105, 106, where the computing devices 105, 106 may be connected to one another via a network 180); and
the second computing device, configured to: detect one or more regions of the 3D model comprising a bodily fluid (Weiss, ¶0177, a part of a real margin line of a scanned preparation tooth may not be sufficiently clearly defined in the 3D model. For example, during the initial 3D data collection step, for example via scanning, that resulted in the first 3D virtual model being generated, a part of the physical dental surface may have been covered with foreign material, such as for example saliva, blood, or debris. The part of the physical dental surface may also have been obscured by another element such as for example part of the gums, cheek, tongue, dental instruments, artifacts, etc. Alternatively, for example, during the initial 3D data collection step (e.g., via scanning) that resulted in the first virtual 3D model being generated, the region may have been distorted or otherwise defective and may not properly correspond to a physical dental surface (e.g., due to some defect in the actual scanning process). Automatic correction may be performed to remove the representation of the foreign material and show the underlying tooth surface and/or margin line. If automatic correction of the dental surface and/or margin line was performed, then the obscured region may be created, and the obscuring object may be removed in the 3D model);
remove the one or more regions from the 3D model (Weiss, ¶0177, Automatic correction may be performed to remove the representation of the foreign material and show the underlying tooth surface and/or margin line. If automatic correction of the dental surface and/or margin line was performed, then the obscured region may be created, and the obscuring object may be removed in the 3D model; Weiss, ¶0179, Intraoral scan application 115 may then apply eraser logic to delete, erase or otherwise remove the marked portion from the 3D model. All regions other than the marked portion may be locked);
update a 3D surface of the dental site using the 3D model of the portion of the dental site (Weiss, ¶0180, Intraoral scan application 108 may direct a user to generate one or more additional intraoral images of the dental site corresponding to the portion of the 3D model (and corresponding set or sets of intraoral images) that was deleted or removed … The one or more additional intraoral images may be registered with the 3D model (and/or with the intraoral image data sets used to create the 3D model) to provide a composite of the 3D model and the one or more additional intraoral images. In the composite, the part of the 3D model that was previously deleted/removed is at least partially replaced with a corresponding part of the one or more additional intraoral images); and
transmit the 3D surface of the dental site to the first computing device (Weiss, ¶0022, displaying the updated three-dimensional mode).
Weiss does not explicitly disclose the following limitations as further recited however Chen discloses
receive intraoral scan data comprising one or more intraoral images of a portion of a dental site generated using structured light projection during intraoral scanning of the dental site (Chen, Figure 2; Chen, ¶n0003, structured light cameras have advantages such as high maturity, high precision, fast response speed, and low cost, and are widely used in dental scanners. The principle of a structured light camera is to project specific encoded stripes onto the surface of an object. The camera captures the projected pattern on the object's surface and performs analysis and decoding. Based on the matching relationship between the structured light image and the projected pattern, as well as the camera's calibration parameters, the three-dimensional coordinates of the object's surface are calculated. A point cloud is a collection of all the points on the surface of an object. After steps such as texture stitching and rendering, the three-dimensional model of the object can be reconstructed; Chen, ¶n0006, a structured light image enhancement method for intraoral 3D reconstruction is provided, which enhances the stripe image captured by an oral scanner based on the structured light principle; Chen, ¶n0039, This embodiment provides a method for enhancing structured light images for intraoral 3D reconstruction, which enhances structured light images obtained by scanning with an oral scanner);
detect one or more regions of the 3D point cloud comprising a bodily fluid (Chen, Figure 3; Chen, ¶n0004, due to factors such as the small space inside the oral cavity, the complex environment (the influence of blood, saliva, etc. on imaging), the shaking of the handheld device, the translucent nature of tooth enamel, the reflective properties of tooth surface materials, and the operating temperature of the instrument, the quality of the striped images captured by the structured light camera varies greatly, and the brightness and contrast also differ significantly. This affects the decoding and analysis of the striped images, the accuracy of extracting the stripe center information, and the accuracy of subsequent 3D point cloud and reconstructed dental model; Chen, ¶n0064, The initial pixel grayscale values within the initial window are considered valid data if they fall within the range of 5% to 95%);
remove the one or more regions from the 3D point cloud (Chen, ¶n0007, This invention provides a structured light image enhancement method for intraoral 3D reconstruction, characterized by the following steps: Step S1, performing subtraction on adjacent frame structured light images to extract stripes; Step S2, enhancing the stripes using an adaptive Hessian image enhancement method; Chen, ¶n0033, a structured light image enhancement method for intraoral 3D reconstruction specifically involves first performing subtraction on adjacent frame structured light images and extracting stripes, and then enhancing the stripes using an adaptive Hessian image enhancement method. This achieves the optimal enhancement effect for intraoral 3D reconstructed structured light images ... It achieves a consistent enhancement effect across areas with varying brightness and contrast. It greatly increases the number of effective 3D point clouds and enhances 3D details; Chen, ¶n0076, As shown in Figure 2, in step S3, for the four factors that affect the quality of the striped image encountered in the actual operation of the oral scanner, namely factor 1, the translucent reflective properties of the enamel surface, factor 2, the teeth being covered by reflective objects, factor 3, the teeth being adhered to by saliva, and factor 4, the bright and dark stripes caused by uneven light illumination, the four samples corresponding to the four factors are subjected to adjacent frame subtraction processing, traditional Hessian enhancement processing, and adaptive Hessian enhancement processing proposed in this embodiment, and then the enhancement results of the above methods are compared; Chen, ¶n0080, As shown in Figure 3, the three-dimensional point cloud data obtained before and after stripe image enhancement can be viewed in the MeshLab software visualization. It can be seen from the areas marked by arrows and circles in Figure 3 that after processing the image using the adaptive Hessian enhancement method proposed in this embodiment, the number of three-dimensional points of the teeth is greater, the point cloud is denser, and the accuracy is improved; Chen, ¶n0082, a structured light image enhancement method for intraoral 3D reconstruction specifically involves first performing a subtraction operation on adjacent frame structured light images and extracting the stripes, and then using an adaptive Hessian image enhancement method to enhance the stripes, thereby achieving the optimal enhancement effect for intraoral 3D reconstruction structured light images. In other words, for the 3D reconstructed striped structured light image, the stripe structure similarity function is dynamically and adaptively adjusted to achieve the optimal stripe enhancement effect, making the image clearer and more complete; Chen, ¶n0084, In this embodiment, the range of 5%-95% of the initial pixel grayscale value within the initial window is selected as valid data. This more effectively avoids the reflective properties of the tooth surface, eliminates bright spots appearing in the local area of the image, and helps with subsequent information processing);
update a 3D surface of the dental site using the 3D point cloud of the portion of the dental site (Chen, Figure 3).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Chen with Weiss because they are in the same field of endeavor. One skilled in the art would have been motivated to include the enhancements of the point cloud prior to constructing the model as taught by Chen in the system of Weiss in order to improve the accuracy of the 3D point cloud and the reconstructed model (Chen, ¶n0004).
As per claim 22, Weiss discloses an intraoral scanner, comprising:
a plurality of cameras to capture a plurality of images of a portion of a dental site illuminated by the cameras (Weiss, ¶0124, To generate the virtual 3D model, intraoral scan application 115 may register and “stitch” or merge together the intraoral images generated from the intraoral scan session. In one embodiment, performing image registration includes capturing 3D data of various points of a surface in multiple images (views from a camera), and registering the images by computing transformations between the images; Weiss, ¶0156, color info may help. In one embodiment, color info Is used as an additional 3 layers (e.g., RGB); Weiss, ¶0157, additional data may include an image generated under specific lighting conditions (e.g., an image generated under ultraviolet or infrared lighting conditions). The additional data may be a 2D or 3D image); and
a processor configured to: generate a three-dimensional (3D) point cloud of the portion of the dental site using at least some of the plurality of images (Weiss, ¶0116, Via such scanner application, the scanner 150 may provide intraoral scan data 135A-N to computing device 105. The intraoral scan data 135A-N may be provided in the form of intraoral image data sets, each of which may include 2D intraoral images and/or 3D intraoral images of particular teeth and/or regions of an intraoral site. In one embodiment, separate image data sets are created for the maxillary arch, for the mandibular arch, for a patient bite, and for each preparation tooth. Alternatively, a single large intraoral image data set is generated (e.g., for a mandibular and/or maxillary arch). Such images may be provided from the scanner to the computing device 105 in the form of one or more points (e.g., one or more pixels and/or groups of pixels). For instance, the scanner 150 may provide such a 3D image as one or more point clouds);
detect one or more regions of the 3D model comprising a bodily fluid (Weiss, ¶0177, a part of a real margin line of a scanned preparation tooth may not be sufficiently clearly defined in the 3D model. For example, during the initial 3D data collection step, for example via scanning, that resulted in the first 3D virtual model being generated, a part of the physical dental surface may have been covered with foreign material, such as for example saliva, blood, or debris. The part of the physical dental surface may also have been obscured by another element such as for example part of the gums, cheek, tongue, dental instruments, artifacts, etc. Alternatively, for example, during the initial 3D data collection step (e.g., via scanning) that resulted in the first virtual 3D model being generated, the region may have been distorted or otherwise defective and may not properly correspond to a physical dental surface (e.g., due to some defect in the actual scanning process). Automatic correction may be performed to remove the representation of the foreign material and show the underlying tooth surface and/or margin line. If automatic correction of the dental surface and/or margin line was performed, then the obscured region may be created, and the obscuring object may be removed in the 3D model);
remove the one or more regions from the 3D model (Weiss, ¶0177, Automatic correction may be performed to remove the representation of the foreign material and show the underlying tooth surface and/or margin line. If automatic correction of the dental surface and/or margin line was performed, then the obscured region may be created, and the obscuring object may be removed in the 3D model; Weiss, ¶0179, Intraoral scan application 115 may then apply eraser logic to delete, erase or otherwise remove the marked portion from the 3D model. All regions other than the marked portion may be locked).
Weiss does not explicitly disclose the following limitations as further recited however Chen discloses
a plurality of structured light projectors to generate a structured light pattern (Chen, Figure 2; Chen, ¶n0003, structured light cameras; Chen, ¶n0006, a structured light image enhancement method for intraoral 3D reconstruction is provided, which enhances the stripe image captured by an oral scanner based on the structured light principle; Chen, ¶n0039, This embodiment provides a method for enhancing structured light images for intraoral 3D reconstruction, which enhances structured light images obtained by scanning with an oral scanner);
detect one or more regions of the 3D point cloud comprising a bodily fluid (Chen, Figure 3; Chen, ¶n0004, due to factors such as the small space inside the oral cavity, the complex environment (the influence of blood, saliva, etc. on imaging), the shaking of the handheld device, the translucent nature of tooth enamel, the reflective properties of tooth surface materials, and the operating temperature of the instrument, the quality of the striped images captured by the structured light camera varies greatly, and the brightness and contrast also differ significantly. This affects the decoding and analysis of the striped images, the accuracy of extracting the stripe center information, and the accuracy of subsequent 3D point cloud and reconstructed dental model; Chen, ¶n0064, The initial pixel grayscale values within the initial window are considered valid data if they fall within the range of 5% to 95%);
remove the one or more regions from the 3D point cloud (Chen, ¶n0007, This invention provides a structured light image enhancement method for intraoral 3D reconstruction, characterized by the following steps: Step S1, performing subtraction on adjacent frame structured light images to extract stripes; Step S2, enhancing the stripes using an adaptive Hessian image enhancement method; Chen, ¶n0033, a structured light image enhancement method for intraoral 3D reconstruction specifically involves first performing subtraction on adjacent frame structured light images and extracting stripes, and then enhancing the stripes using an adaptive Hessian image enhancement method. This achieves the optimal enhancement effect for intraoral 3D reconstructed structured light images ... It achieves a consistent enhancement effect across areas with varying brightness and contrast. It greatly increases the number of effective 3D point clouds and enhances 3D details; Chen, ¶n0076, As shown in Figure 2, in step S3, for the four factors that affect the quality of the striped image encountered in the actual operation of the oral scanner, namely factor 1, the translucent reflective properties of the enamel surface, factor 2, the teeth being covered by reflective objects, factor 3, the teeth being adhered to by saliva, and factor 4, the bright and dark stripes caused by uneven light illumination, the four samples corresponding to the four factors are subjected to adjacent frame subtraction processing, traditional Hessian enhancement processing, and adaptive Hessian enhancement processing proposed in this embodiment, and then the enhancement results of the above methods are compared; Chen, ¶n0080, As shown in Figure 3, the three-dimensional point cloud data obtained before and after stripe image enhancement can be viewed in the MeshLab software visualization. It can be seen from the areas marked by arrows and circles in Figure 3 that after processing the image using the adaptive Hessian enhancement method proposed in this embodiment, the number of three-dimensional points of the teeth is greater, the point cloud is denser, and the accuracy is improved; Chen, a structured light image enhancement method for intraoral 3D reconstruction specifically involves first performing a subtraction operation on adjacent frame structured light images and extracting the stripes, and then using an adaptive Hessian image enhancement method to enhance the stripes, thereby achieving the optimal enhancement effect for intraoral 3D reconstruction structured light images. In other words, for the 3D reconstructed striped structured light image, the stripe structure similarity function is dynamically and adaptively adjusted to achieve the optimal stripe enhancement effect, making the image clearer and more complete; Chen, ¶n0084, In this embodiment, the range of 5%-95% of the initial pixel grayscale value within the initial window is selected as valid data. This more effectively avoids the reflective properties of the tooth surface, eliminates bright spots appearing in the local area of the image, and helps with subsequent information processing); and
transmit the 3D point cloud to a computing device for further processing (Chen, ¶n0004, 3D point cloud and reconstructed dental model; Chen, ¶n0006, a structured light image enhancement method for intraoral 3D reconstruction is provided, which enhances the stripe image captured by an oral scanner based on the structured light principle).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Chen with Weiss because they are in the same field of endeavor. One skilled in the art would have been motivated to include the enhancements of the point cloud prior to constructing the model as taught by Chen in the system of Weiss in order to improve the accuracy of the 3D point cloud and the reconstructed model (Chen, ¶n0004).
Allowable Subject Matter
Claims 9-11 and 13-18 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: Claims 9-11 and 13 18 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims because while the prior art discloses capturing structured light intraoral images and processing the resultant point clouds to remove points that are inconsistent or inaccurate, the prior art does not disclose the limitations, “wherein detecting the one or more suspected regions in the one or more color 2D images comprises performing the following for each color 2D image of the one or more color 2D images: processing the color 2D image using a trained machine learning model, wherein the trained machine learning model outputs a segmentation mask for the color 2D image, and wherein at least one classification of the segmentation mask is a bodily fluid classification” and “wherein the one or more intraoral images of the portion of the dental site generated using the structured light projection comprises a plurality of color 2D images each generated by a different camera of an intraoral scanner at a first point in time, wherein the intraoral scan data further comprises a plurality of color two-dimensional (2D) images each generated by a different camera of the intraoral scanner, each color 2D image of the plurality of color 2D images having been generated by a same camera that generated an image of the plurality of color 2D images, and wherein the computing device is further configured to: process each color 2D image of the plurality of color 2D images using a trained machine learning model, wherein for each color 2D image the trained machine learning model outputs a mask that indicates, for each pixel of the color 2D image, a probability that the pixel depicts the bodily fluid; for each point in the 3D point cloud that maps to pixels in one or more color 2D images of the plurality of color 2D images, combine probabilities of the pixels in the one or more color 2D images depicting the bodily fluid to determine a probability that the point in the 3D point cloud depicts the bodily fluid; and classify points for which the probability that the point depicts the bodily fluid exceeds a threshold as depicting the bodily fluid’ as recited in dependent claims 9 and 13.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Primary Examiner, Art Unit 2668
Prosecution Insights