Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 10-11, 17, 23-24 are rejected under 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) as being anticipated by Tuzoff et al. (US 20200146646 A1), hereinafter Tuzoff.
Regarding claim 1, Tuzoff teaches A method comprising: receiving image data of a current state of a dental site of a patient; (Para. 16 see "The examples and embodiments described in this disclosure provide systems, methods, and data processing device-readable media for processing dental imaging. In particular, the example system and methods described herein apply deep learning techniques to the processing of dental images to provide a platform for computer-aided diagnosis and charting, and in particular to detect and number teeth." Para. 52 see "This data (patient data, including dental images, and pricing data) may be stored locally in a client system 1070, or remotely by a practice management system 1052 and in image data storage 1054" Para. 57 see "The analysis service 1000 implements detection and numbering of present and absent teeth as described above, using the patient, charting, and image data received from the dental support infrastructure 1050." Para. 76 see "Code adapted to provide the systems and methods described above may be provided on many different types of computer-readable media including computer storage mechanisms (e.g., CD-ROM, diskette, RAM, flash memory, computer's hard drive, etc.) that contain instructions for use in execution by one or more processors to perform the operations described herein."). processing the image data using a segmentation pipeline to generate an output comprising segmentation information for one or more teeth in the image data (Para. 17 see "The detection module receives the source dental image and generates data identifying image regions determined to correspond to individual teeth." Para. 20 see "The detection module detects teeth in the original image. Teeth detection may comprise implementation of the Faster R-CNN model" Para. 49 see "Segmentation techniques may be implemented for more accurate localization."). and at least one of identifications or locations of one or more oral conditions observed in the image data, (Para. 57 see "the analysis service 1000 can also facilitate the detection and treatment of conditions: pathological conditions (e.g., missing teeth, caries, apical periodontitis, dental cysts), non-pathological conditions (e.g., restorations, crowns, implants, bridges, endodontic treatments) and post-treatment conditions (e.g., overhanging restorations, endodontic underfillings and overfillings). This automated detection of conditions may either replace or supplement detection and analysis of dental images by dental and medical practitioners. In addition to detecting and numbering teeth from input images, the analysis service 1000 is further trained and identifies and classifies regions of interest within detected teeth, to enable the generation of a symbolic dental chart with conditions provisionally identified or diagnosed for delivery through a computer interface to the practitioner"). wherein each of the one or more oral conditions is associated with a tooth of the one or more teeth; (Para. 57 see "This region of interest information may comprise coordinates defining each portion of the original image (e.g., coordinates identifying absolute pixel positions within the image, or coordinates and offsets defining a rectangular region within the image) for which a tooth and/or a condition was detected, associated with a corresponding tooth number."). generating a visual overlay comprising visualizations for each of the one or more oral conditions; (Para. 61 see "In addition to the patient's PV image displayed in image area 1106, the detected conditions can be indicated by bounding boxes or other visual elements on the displayed image."). outputting the image data to a display; and outputting the visual overlay to the display over the image data. (Para. 11 see "FIG. 5 is an illustration of an example graphical user interface on a client system displaying a panoramic radiograph based on the detection and classification executed by the system of FIG. 1." Para. 12 see "FIG. 6 is an illustration of a graphical user interface on a client system displaying a bitewing radiograph based on the detection and classification executed by the system of FIG. 1." Para. 13 see "FIG. 7 is an illustration of a further example graphical user interface displaying an X-ray image based on output by the system of FIG. 1." Para. 59 see "In one implementation, the updated charting data may be displayed in a graphical user interface (GUI) on a chair-side display of the client system 1070 to practitioners and/or patients to visualise diagnostic findings." Para. 62 see "at least some teeth depicted symbolically in the dental chart 1110 are correlated to the regions of interest identified by the bounding boxes or other visual elements in the displayed image in image area 1106.").
Regarding claim 2, Tuzoff teaches The method of claim 1. wherein the image data comprises a radiograph. (Para. 19 see "In the examples discussed herein, source images are panoramic view (PV) radiographs that depict the upper and lower jaws in one single image.").
Regarding claim 3, Tuzoff teaches The method of claim 2. wherein processing the image data using the segmentation pipeline comprises: processing the image data using one or more first trained machine learning models to generate a first output comprising the segmentation information for the one or more teeth in the image data; (Para. 20 see "The detection module detects teeth in the original image. Teeth detection may comprise implementation of the Faster R-CNN model ... The object detection module uses these proposals for further object localization and classification." Para. 21 see "the object detection module generates the final bounding box coordinates, represented schematically on the original input image as outlines in image 300." Para. 49 see "Segmentation techniques may be implemented for more accurate localization. ... segmentation or localization of objects within previously detected boundaries." Para. 57 see "similar methodology, the analysis service 1000 can also facilitate the detection and treatment of conditions: pathological conditions (e.g., missing teeth, caries, apical periodontitis, dental cysts), non-pathological conditions (e.g., restorations, crowns, implants, bridges, endodontic treatments) and post-treatment conditions (e.g., overhanging restorations, endodontic underfillings and overfillings). ... identifies and classifies regions of interest within detected teeth ... Detection of pathological conditions and radiological findings can be performed using, again, an appropriately trained CNN or other neural network architecture. ... The updated charting data may also comprise an indication of the bounding box or region of interest for the input image as identified during either the tooth detection or condition detection by the analysis service 1000. ... for which a tooth and/or a condition was detected" Examiner Note: Additional models identify and locate oral conditions.). and processing the image data using one or more additional trained machine learning models to generate a second output comprising at least one of the identifications or the locations of the one or more oral conditions. (Para. 57 see "similar methodology, the analysis service 1000 can also facilitate the detection and treatment of conditions: pathological conditions (e.g., missing teeth, caries, apical periodontitis, dental cysts), non-pathological conditions (e.g., restorations, crowns, implants, bridges, endodontic treatments) and post-treatment conditions (e.g., overhanging restorations, endodontic underfillings and overfillings). ... identifies and classifies regions of interest within detected teeth ... Detection of pathological conditions and radiological findings can be performed using, again, an appropriately trained CNN or other neural network architecture. ... The updated charting data may also comprise an indication of the bounding box or region of interest for the input image as identified during either the tooth detection or condition detection by the analysis service 1000. ... for which a tooth and/or a condition was detected" Examiner Note: Additional models identify and locate oral conditions.).
Regarding claim 10, Tuzoff teaches The method of claim 2. herein each instance of one or more oral conditions is provided as a distinct layer of the visual overlay, the method further comprising: generating a dental chart for the patient; populating the dental chart based on data for the one or more oral conditions; and outputting the dental chart to the display. (Para. 11 see "FIG. 5 is an illustration of an example graphical user interface on a client system displaying a panoramic radiograph based on the detection and classification executed by the system of FIG. 1." Para. 57 see "the analysis service 1000 is further trained and identifies and classifies regions of interest within detected teeth, to enable the generation of a symbolic dental chart with conditions provisionally identified or diagnosed" Para. 62 see "Furthermore, a standard symbolic numbered dental chart 1110 is also displayed, comprising a plurality of teeth representing the typical arrangement of a full complement of adult teeth (or primary teeth as appropriate) mapped or correlated to tooth position, number (classification), and detected conditions. The dental chart may also be color coded to signal the location of non-pathological and pathological conditions." Para. 63 see "the list entries are correlated to a corresponding tooth by number and a confidence level for the detected condition" Para. 69 see "This report includes a finalized version of the standard dental chart 1122").
Regarding claim 11, Tuzoff teaches The method of claim 10. wherein each instance of one or more oral conditions is provided as a distinct layer of the visual overlay, the method further comprising: receiving a selection of a tooth based on user interaction with at least one of the tooth in the dental chart or the tooth in the image data; (Para. 62 see "In some implementations of the GUI, a user may select (e.g., using a pointing device or touch interface device, such as a touchscreen) a single tooth in the chart, and the detected conditions may be displayed adjacent to the tooth."). outputting detailed information for instances of each of the one or more oral conditions identified for the selected tooth; (Para. 62 see "a single tooth in the chart, and the detected conditions may be displayed adjacent to the tooth." Para. 63 see "In addition, a listing 1112 of the detected conditions is also included. in the example of FIG. 5, the list entries are correlated to a corresponding tooth by number and a confidence level for the detected condition"). receiving an instruction to remove an instance of an oral condition of the tooth; (Para. 11 see "FIG. 5 is an illustration of an example graphical user interface on a client system displaying a panoramic radiograph based on the detection and classification executed by the system of FIG. 1." Para. 64 see "The listing may include user interface elements, such as checkboxes, for the practitioner to confirm or reject (delete) the findings made by the service 1000."). and marking the tooth as not having the instance of the oral condition. (Para. 11 see "FIG. 5 is an illustration of an example graphical user interface on a client system displaying a panoramic radiograph based on the detection and classification executed by the system of FIG. 1." Para. 64 see "The listing may include user interface elements, such as checkboxes, for the practitioner to confirm or reject (delete) the findings made by the service 1000. Confirmations and rejections, if submitted by the practitioner, are sent to the service 1000 as feedback (again, this may be in the form of charting data (3)) and may be used to provide additional training to the CNN or other neural network." Para. 68 see "As mentioned above, if the practitioner confirms or rejects findings, these responses may be transmitted to the analysis service 1000 for incorporation into training data for future analyses. In addition, the results displayed in the GUI 1100 are updated to remove any rejected entries.").
Regarding claim 17, Tuzoff teaches The method of claim 1. further comprising: receiving a command to generate a report; (Para. 62 see "In some implementations of the GUI, a user may select (e.g., using a pointing device or touch interface device, such as a touchscreen)" Para. 69 see "Once the practitioner has completed their review of the reported findings in the GUI, different forms of reports may be generated" Examiner Note: See Fig. 5 which shows a 'Report' button.). generating the report comprising the image data, the visual overlay, and a dental chart showing, for each tooth of the patient, any oral conditions identified for that tooth; (Para. 69 see "Once the practitioner has completed their review of the reported findings in the GUI, different forms of reports may be generated. ... This report includes a finalized version of the standard dental chart 1122; a listing, by tooth number (American notation is employed in FIG. 19) of teeth that were determined to be present (“Normal appearance”) and missing, together with a listing of the findings as confirmed by the practitioner 1124. ... The report 1120 also includes images cropped from the original PV image for each tooth having a corresponding finding 1126. The report 1120 thus presents the detected conditions (as confirmed or edited by the practitioner) in multiple formats."). formatting the report in a structured data format ingestible by a dental practice management system; (Para. 57 see "The results generated by the analysis service 1000 are provided in the form of updated charting data (4) to the practice management system 1052 (as mentioned above, this may be in accordance with a preferred standard notation)" Para. 62 see "charting and patient data may be maintained in an electronic form and handled in conformance with privacy requirements and established standards such as ANSI/ADA 1067:2013" Para. 69 see "The report 1120 thus presents the detected conditions (as confirmed or edited by the practitioner) in multiple formats."). and adding the report to the dental practice management system. (Para. 52 see "The client system 1070 receives results generated by the analysis service 1000 ... the analysis service 1000 provides its results to the client system 1070 via the practice management system 1052." Para. 57 see "The results generated by the analysis service 1000 are provided in the form of updated charting data (4) to the practice management system 1052 (as mentioned above, this may be in accordance with a preferred standard notation)").
Claim 23 is rejected under the same analysis as claim 1 above.
Claim 24 is rejected under the same analysis as claim 1 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-5, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Tuzoff et al. (US 20200146646 A1), hereinafter Tuzoff, in view of Johnson (US 20210279871 A1), hereinafter Johnson.
Regarding claim 4, Tuzoff teaches The method of claim 3. wherein for an oral condition of the one or more oral conditions an additional trained machine learning model of the one or more additional trained machine learning models outputs a bounding box for an instance of the oral condition, the method further comprising: determining a tooth associated with the bounding box; (Para. 57 see "The updated charting data may also comprise an indication of the bounding box or region of interest for the input image as identified during either the tooth detection or condition detection by the analysis service 1000. This region of interest information may comprise coordinates defining each portion of the original image (e.g., coordinates identifying absolute pixel positions within the image, or coordinates and offsets defining a rectangular region within the image) for which a tooth and/or a condition was detected, associated with a corresponding tooth number. If a condition was determined from the region of interest identified by the coordinates, then the updated charting data also comprises an identifier of the condition associated with the tooth number and the coordinates.").
Tuzoff does not teach determining an intersection of data from the bounding box and a segmentation mask for the tooth from the segmentation information; and determining a pixel-level mask for the instance of the oral condition based at least in part on the intersection of the data from the bounding box and the segmentation mask.
However, Johnson teaches determining an intersection of data from the bounding box and a segmentation mask for the tooth from the segmentation information; (Para. 9 see "Image feature recognition and segmentation is integrated into the neural network itself for tooth identification, and anomaly detection and diagnosis" Para. 19 see "a probability map indicating the likelihood of a pixel or sub-region of pixels belonging to the target structure (teeth)" Claim 15 see "the neural network outputs a bounding box or segmentation map derived from the per-pixel or per-sub-region likelihood of each pixel or sub-region being part of the structure" Examiner Note: The pixels of the segmentation mask are the same region of the bounding box and therefore intersect.). and determining a pixel-level mask for the instance of the oral condition based at least in part on the intersection of the data from the bounding box and the segmentation mask. (Abstract see "The neural network can output a tooth segmentation map, tooth identifiers, a probability map indicating the presence of caries, cavities or other dental anomalies/conditions" Para. 19 see "a probability map indicating the likelihood of a pixel or sub-region of pixels belonging to the target structure (teeth) ... via a standard method of object localization or segmentation utilized in machine vision (i.e. a Mask R-CNN/Bayesian Optimization/Semantic Segmentation)" Claim 15 see "the neural network outputs a bounding box or segmentation map derived from the per-pixel or per-sub-region likelihood of each pixel or sub-region being part of the structure" Examiner Note: Segmentation of per-pixel regions containing an oral condition are provided by the neural network.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tuzoff to incorporate the teachings of Johnson to determine a segmentation mask for the tooth and a pixel-level mask for the instance of an oral condition. Doing so would predictably improve accuracy of visual overlays in a GUI depicting regions in the image that contain an oral condition. This would allow practitioners to quickly identify important regions in the image and save time.
Regarding claim 5, Tuzoff in view of Johnson teaches The method of claim 4.
In addition, Tuzoff teaches wherein the oral condition comprises a caries or a restoration, (Para. 57 see "the analysis service 1000 can also facilitate the detection and treatment of conditions: pathological conditions (e.g., missing teeth, caries, apical periodontitis, dental cysts)"). as a layer of the visual overlay representing the oral condition within the tooth. (Para. 11 see "FIG. 5 is an illustration of an example graphical user interface on a client system displaying a panoramic radiograph based on the detection and classification executed by the system of FIG. 1." Para. 59 see "In one implementation, the updated charting data may be displayed in a graphical user interface (GUI) on a chair-side display of the client system 1070 to practitioners and/or patients to visualise diagnostic findings." Para. 62 see "at least some teeth depicted symbolically in the dental chart 1110 are correlated to the regions of interest identified by the bounding boxes or other visual elements in the displayed image in image area 1106.").
Tuzoff does not teach the method further comprising: subtracting the data from the bounding box that does not intersect with the segmentation mask; wherein the pixel-level mask is provided.
However, Johnson teaches the method further comprising: subtracting the data from the bounding box that does not intersect with the segmentation mask; (Para. 9 see "Image feature recognition and segmentation is integrated into the neural network itself for tooth identification, and anomaly detection and diagnosis" Para. 19 see "a probability map indicating the likelihood of a pixel or sub-region of pixels belonging to the target structure (teeth)" Claim 15 see "the neural network outputs a bounding box or segmentation map derived from the per-pixel or per-sub-region likelihood of each pixel or sub-region being part of the structure"). wherein the pixel-level mask is provided (Para. 19 see "via a standard method of object localization or segmentation utilized in machine vision (i.e. a Mask R-CNN/Bayesian Optimization/Semantic Segmentation)" Para. 22 see "analyze an acquired image and automatically detect and highlight/indicate the presence of abnormalities in dentition (for example but not limited to caries and cavities) ... a probability map indicating the likelihood of a pixel or sub-region of pixels belonging to the target structure (teeth) ... Labels for abnormalities/afflictions can be attached or annotated on the images" Para. 36 see "The term “Mask R-CNN” (mask regional convolutional neural network) refers to a method in the field of machine vision to perform pixel-based image segmentation. Mask R-CNNs can perform both image segmentation and classification.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tuzoff and Johnson to incorporate the teachings of Johnson to subtract data from the bounding box that does not intersect with the segmentation mask and provide a pixel-level mask. Doing so would predictably improve accuracy of visual overlays in a GUI depicting regions in the image that contain caries. This would allow practitioners to quickly identify important regions in the image and save time.
Regarding claim 14, Tuzoff teaches The method of claim 1. wherein the visual overlay comprises the (Para. 11 see "FIG. 5 is an illustration of an example graphical user interface on a client system displaying a panoramic radiograph based on the detection and classification executed by the system of FIG. 1." Para. 59 see "In one implementation, the updated charting data may be displayed in a graphical user interface (GUI) on a chair-side display of the client system 1070 to practitioners and/or patients to visualise diagnostic findings." Para. 62 see "at least some teeth depicted symbolically in the dental chart 1110 are correlated to the regions of interest identified by the bounding boxes or other visual elements in the displayed image in image area 1106.").
Tuzoff does not teach wherein the at least one of the identifications or the locations of the one or more oral conditions comprises a probability map indicating, for each pixel of the image data, a probability of the pixel corresponding to at least one oral condition of the one or more oral conditions, the method further comprising: determining, for the at least one oral condition and for a first tooth, a pixel-level mask indicating pixels having a probability that exceeds a first threshold, pixel-level mask for the first tooth.
However, Johnson teaches wherein the at least one of the identifications or the locations of the one or more oral conditions comprises a probability map indicating, for each pixel of the image data, a probability of the pixel corresponding to at least one oral condition of the one or more oral conditions, the method further comprising: determining, for the at least one oral condition and for a first tooth, a pixel-level mask indicating pixels having a probability that exceeds a first threshold, (Para. 22 see "One embodiment of the dental imaging system can analyze an acquired image and automatically detect and highlight/indicate the presence of abnormalities in dentition (for example but not limited to caries and cavities). ... The output of this embodiment is both a probability map indicating the likelihood of a pixel or sub-region of pixels belonging to the target structure (teeth), as well as locations for the teeth, which can be achieved via a method of object localization or segmentation in machine vision using deep learning (i.e. an R-CNN/Bayesian Optimization/Semantic Segmentation)." Examiner Note: The probability map is used to classify pixels that have an oral condition for a tooth. This inherently uses thresholding.). pixel-level mask for the first tooth. (Para. 19 see "via a standard method of object localization or segmentation utilized in machine vision (i.e. a Mask R-CNN/Bayesian Optimization/Semantic Segmentation)" Para. 22 see "analyze an acquired image and automatically detect and highlight/indicate the presence of abnormalities in dentition (for example but not limited to caries and cavities) ... a probability map indicating the likelihood of a pixel or sub-region of pixels belonging to the target structure (teeth) ... Labels for abnormalities/afflictions can be attached or annotated on the images" Para. 36 see "The term “Mask R-CNN” (mask regional convolutional neural network) refers to a method in the field of machine vision to perform pixel-based image segmentation. Mask R-CNNs can perform both image segmentation and classification.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tuzoff to incorporate the teachings of Johnson to use a probability map to determine which pixels belong to an oral condition. Doing so would predictably improve accuracy of visual overlays in a GUI depicting regions in the image that contain an oral condition. This would allow practitioners to quickly identify important regions in the image and save time.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Tuzoff et al. (US 20200146646 A1), hereinafter Tuzoff, in view of Xue et al. (US 20190180443 A1), hereinafter Xue.
Regarding claim 7, Tuzoff teaches The method of claim 3.
Tuzoff does not teach wherein for an oral condition of the one or more oral conditions an additional trained machine learning model of the one or more additional trained machine learning models outputs a plurality of bounding boxes for an instance of the oral condition, the method further comprising: determining that a first bounding box of the plurality of bounding boxes encapsulates one or more additional bounding boxes of the plurality of bounding boxes; and removing the one or more additional bounding boxes.
However, Xue teaches wherein for an oral condition of the one or more oral conditions an additional trained machine learning model of the one or more additional trained machine learning models outputs a plurality of bounding boxes for an instance of the oral condition, (Para. 12 see "FIG. 7B illustrates a machine learning model that generates a plurality of bounding shapes around different regions of teeth in an image of a mouth" Para. 30 see "the first deep learning model may be trained to generate a first bounding shape around anterior teeth, a second bounding shape around left posterior teeth, a third bounding shape around right posterior teeth, and/or a fourth bounding shape around all teeth."). the method further comprising: determining that a first bounding box of the plurality of bounding boxes encapsulates one or more additional bounding boxes of the plurality of bounding boxes; (Para. 117 see "a single mask may be generated for a union or other combination of multiple bounding boxes. Alternatively, or additionally, multiple masks may be generated, where each mask is for a particular bounding box or combination of bounding boxes."). and removing the one or more additional bounding boxes. (Para. 101 see "processing logic combines multiple bounding boxes (e.g., performs a union of two or more bounding boxes). In one embodiment, processing logic determines for one or more bounding boxes whether those bounding boxes have an associated confidence metric that exceeds a confidence threshold. If a bounding box has a confidence metric that is below the confidence threshold, then that bounding box may not be combined with other bounding boxes. If some or all of the bounding boxes considered have associated confidence metrics that exceed the confidence threshold, then those bounding boxes with confidence metrics that exceed the confidence threshold may be combined." Examiner note: if a bounding box is inside another bounding box, they are combined, effectively removing the additional one.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tuzoff to incorporate the teachings of Xue to generate multiple bounding boxes and remove the additional bounding boxes. Doing so would predictably improve the visibility of visual overlays in a GUI depicting regions in the image by removing clutter. This would allow practitioners to quickly identify important regions in the image and save time.
Regarding claim 8, Tuzoff teaches The method of claim 3. wherein for an oral condition of the one or more oral conditions an additional trained machine learning model of the one or more additional trained machine learning models outputs a bounding box for an instance of the oral condition, the method further comprising: determining a tooth associated with the bounding box; (Para. 57 see "The updated charting data may also comprise an indication of the bounding box or region of interest for the input image as identified during either the tooth detection or condition detection by the analysis service 1000. This region of interest information may comprise coordinates defining each portion of the original image (e.g., coordinates identifying absolute pixel positions within the image, or coordinates and offsets defining a rectangular region within the image) for which a tooth and/or a condition was detected, associated with a corresponding tooth number. If a condition was determined from the region of interest identified by the coordinates, then the updated charting data also comprises an identifier of the condition associated with the tooth number and the coordinates.").
Tuzoff does not teach determining an overlap between the bounding box and a segmentation mask for the tooth from the segmentation information; and determining a location of the oral condition on the tooth based at least in part on the overlap.
However, Xue teaches determining an overlap between the bounding box and a segmentation mask for the tooth from the segmentation information; (Para. 8 see "FIG. 4 illustrates a flow diagram for a method of generating a binary mask based on an image with a bounding box" Para. 28 see "the trained machine learning model may output a mask that defines a single bounding shape around teeth of the input image, wherein the mask indicates, for each pixel of the input image, whether that pixel is inside of a defined bounding shape or is outside of the defined bounding shape. ... for an input image the trained machine learning model may output a mask that defines a union or other combination of multiple different bounding shapes around different teeth in the input image, wherein the mask indicates, for each pixel of the input image, whether that pixel is inside of one of the defined bounding shapes or is outside of the defined bounding shapes. Additionally, for an input image the trained machine learning model may output an indication of a view associated with the image, may output labels associated with one or more bonding shapes (e.g., labeling which teeth are within the bounding box)" Para. 74 see "the ROls may be square or rectangular image patches such as 16×16 pixel patches, 32×32 pixel patches, 16×32 pixel patches, 32×16 pixel patches, 64×64 pixel patches, and so on. Notably, the ROls may have overlapping image data. For example, a first ROI may be based around pixel at column 20, row 20, and may be a 30×30 image patch centered on the pixel at column 20, row 20. A second ROI may be based around pixel at column 22, row 20, and may be a 30×30 image patch centered on the pixel at column 22, row 20. Accordingly, there will be a large overlap of pixels between these two ROls."). and determining a location of the oral condition on the tooth based at least in part on the overlap. (Abstract see "the trained machine learning model is to output a mask that defines a bounding shape around teeth of the input image, wherein the mask indicates, for each pixel of the input image, whether that pixel is inside of a defined bounding shape or is outside of the defined bounding shape." Para. 25 see "The automated analysis may include identifying one or more bounding shapes around teeth in an image ... cropping the image based on the one or more bounding shapes, identifying edge classifications for edges in the cropped image (e.g., edges of teeth, gingival edges, aligner edges, and so on)" Para. 26 see "A rules engine or other processing logic may then make determinations about the teeth or perform other operations based on the classified edges in the cropped image." Para. 30 see "the region of interest is the region that contains the teeth. Accordingly, the first deep leaning model that generates the bounding shape may make a rough determination of the location of teeth, and may define a bounding shape around an area corresponding to that rough determination." Para. 47 see "FIG. 4 illustrates a flow diagram for a method 400 of generating a binary mask based on an image with a bounding box").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tuzoff to incorporate the teachings of Xue to determine a location of an oral condition based at least in part on an overlap between the bounding box and the mask. Doing so would predictably improve the visibility of visual overlays in a GUI depicting regions in the image where oral conditions occur. This would allow practitioners to quickly identify important regions in the image and save time.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Tuzoff et al. (US 20200146646 A1), hereinafter Tuzoff, in view of Golay et al. (US 20240212153 A1), hereinafter Golay.
Regarding claim 12, Tuzoff teaches The method of claim 1. wherein the one or more oral conditions comprise one or more instances of caries, the method further comprising: (Para. 57 see "the analysis service 1000 can also facilitate the detection and treatment of conditions: pathological conditions (e.g., missing teeth, caries, apical periodontitis, dental cysts)"). using a first visualization; (Para. 61 see "non-pathological conditions may be marked on the image in a first color, while pathological conditions are marked on the image using a different color."). using a second visualization. (Para. 61 see "non-pathological conditions may be marked on the image in a first color, while pathological conditions are marked on the image using a different color.").
Tuzoff does not teach for each instance of caries, determining whether the instance of the caries is an enamel caries or a dentin caries; marking instances of caries identified as enamel caries and marking instances of caries identified as dentin caries.
However, Golay teaches for each instance of caries, determining whether the instance of the caries is an enamel caries or a dentin caries; (Para. 82 see "The type of dental conditions/features that can be detected 1402 can include… caries… The methods of image processing applied may include… dentin/enamel."). marking instances of caries identified as enamel caries (Para. 82 see "The type of dental conditions/features that can be detected 1402 can include… caries… The methods of image processing applied may include… dentin/enamel."). and marking instances of caries identified as dentin caries (Para. 82 see "The type of dental conditions/features that can be detected 1402 can include… caries… The methods of image processing applied may include… dentin/enamel.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tuzoff to incorporate the teachings of Golay to detect different types of caries and use unique visualizations for enamel and dentin caries. Doing so would predictably improve the visibility of visual overlays in a GUI depicting regions in the image where oral conditions occur. This would allow practitioners to quickly identify important regions in the image and save time.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Tuzoff et al. (US 20200146646 A1), hereinafter Tuzoff, in view of Han (KR 20210114771 A), hereinafter Han.
Regarding claim 20, Tuzoff teaches The method of claim 1.
Tuzoff does not teach wherein the one or more oral conditions comprise a bone loss value, the method further comprising: determining bone loss values for each of the one or more teeth of the patient; and determining, based on the bone loss values, whether the patient has at least one of horizontal bone loss, vertical bone loss, generalized bone loss, or localized bone loss.
However, Han teaches wherein the one or more oral conditions comprise a bone loss value, the method further comprising: determining bone loss values for each of the one or more teeth of the patient; and determining, based on the bone loss values, whether the patient has at least one of horizontal bone loss, vertical bone loss, generalized bone loss, or localized bone loss. (Para. 29 see "For example, as shown in FIG. 4 , 'dental caries', 'root caries', 'secondary caries, crown fractures, root fractures, crown-root fractures, residual tooth roots', 'vertical bone loss, horizontal bone loss, root Check the checkbox of the reading findings items (select the selected radiographic findings) such as furcation lesion, floating state, calculus observed, PDL space enlargement, 'impeded tooth, apical lesion, erupting inferior permanent tooth, no inferior permanent tooth, normal state' and the selected reading observation item can be recorded in the electronic medical record system as a reading observation." Para. 39 see "there is a readout item of 'vertical bone loss, horizontal bone loss, furcation lesion, floating state, calculus observed, PDL space enlargement', 'vertical bone loss' is 80% probability, 'horizontal bone loss' ' is 10% probability, 'furcation lesion' is 6% probability, 'Floating State' is 3% probability, 'tartar observed' and 'PDL space enlargement' are calculated with 1% probability, respectively, vertical bone loss, Horizontal bone loss, furcation lesion, Floating State, calculus observed, PDL space widening', that is, the arrangement ranking for each item can be determined in the order of highest probability.").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Tuzoff to incorporate the teachings of Han to determine a horizontal bone loss probability value and whether the patient has bone loss. Doing so would predictably save time for practitioners by having the system automatically detect whether a patient has bone loss.
Allowable Subject Matter
Claim(s) 6, 9, 13, 15-16, 18-19, 21-22 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 6, Tuzoff does not teach drawing an ellipse within the bounding box that intersects with the bounding box and the segmentation mask or subtracting the data from the bounding box that intersects with the segmentation mask.
Regarding claim 9, Tuzoff does not teach performing principal component analysis or determining a line between the tooth occlusal surface and a tooth root apex based on the first principal. Nor does it teach determining where the oral condition is based on that line.
Regarding claim 13, Tuzoff does not teach automatically determining dental codes associated with oral conditions and determining a treatment plan and generating an insurance claim for the treatment.
Regarding claim 15, Tuzoff does not teach determining a new pixel-level mask using a second threshold when activating a high sensitivity mode.
Regarding claim 16, Tuzoff does not teach determining a second new pixel-level mask using a second threshold prior to activating a high sensitivity mode when an oral condition was not identified.
Regarding claim 18, Tuzoff does not teach comparing the report to reports of other patients with oral conditions when generating the report or determining severity levels.
Regarding claim 19, Tuzoff does not teach determining a bone loss value or a first and second distance and ratio between the distances using centoenamel junction, periodontal bone line, or root apex.
Regarding claim 21, Tuzoff does not teach determining a bite-wing without a root, a tooth length from previous x-rays, a centoenamel junction, periodontal bone line, bone loss length between CEJ and PBL, or a ratio between bone loss length and tooth length.
Regarding claim 22, Tuzoff does not teach determining a tooth size from a three-dimensional model, registering the image to a three-dimensional model by converting to physical units of measurement, determining a centoenamel junction, a periodontal bone line, or a distance between the CEJ and the PBL by converting pixel distance to physical units of measurement.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Schnabel et al. (US 20210085238 A1) discloses using neural networks to segment patient dental images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER J VAUGHN whose telephone number is (571) 272-5253. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDREW MOYER can be reached on (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER JOSEPH VAUGHN/Examiner, Art Unit 2675
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights