DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 10, 12-13, 15, 17, 19-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kearney et al. (US 20210118132 A1), herein referred to as Kearney, in view of Claessen et al. (US 20210082184 A1), herein referred to as Claessen.
Regarding claim 1, Kearney discloses a method of generating a treatment plan for forming one or more dental appliances (refer to Paragraph [0543], Fig. 44) by determining coordinates of a tooth root apex of a tooth (refer to Paragraph [0466], Fig. 39A), the method (refer to Paragraphs [0466], [0543]) comprising:
obtaining patient data (3910, 3912, 3914, 3916) wherein the patient data includes two-dimensional (2D) panoramic radiograph data of a patient or three-dimensional (3D) intraoral scan data of the patient (refer to Paragraphs [0077], [0468]; the image may be of a of a patient's mouth obtained by means of an X-ray (intra-oral or extra-oral, full mouth series (FMX), panoramic, cephalometric), computed tomography (CT) scan, cone-beam computed tomography (CBCT) scan, intra-oral image capture using an optical camera, magnetic resonance imaging (MRI), or other imaging modality);
obtaining a tooth number (3912) associated with the tooth (refer to Paragraphs [0082], [0468]; each tooth is assigned a tooth number, also referred to as a “mask”);
providing the patient data and the tooth number (3910, 3912) to a trained deep learning network (3900) (refer to Paragraphs [0468], [0473]; the generator (3902) takes an image (3910) with a corresponding tooth number mask as an input; the generator (3902) is trained to generate realistic orthodontic points),
wherein the trained deep learning network (3900) is trained at least in part on imaging tooth data (3910) for a plurality of patients (refer to Paragraphs [0471]-[0473]; each training data entry includes an image (3910) and data describing the features shown in the image; the training algorithm evaluates the realism matrix (3910) which uses labeled orthodontic points for an image of the patient different than the image (3910) used to generate the predicted orthodontic points (3922)), wherein the imaging tooth data (3910) includes segmented data with a label for each tooth number (3912) (refer to Paragraphs [0468], [0471]; the masks included with each image (3910) include tooth number), and tooth root apex coordinates (3924) associated with each tooth number (refer to Paragraphs [0471], [0476], [0483]; the training data entry includes target orthodontic points (3924), wherein the target orthodontic points (3924) include root apex points),
wherein the trained deep learning network (3900) is trained on the tooth root apex coordinates (3924) as ground truth output data (3924) (refer to Paragraph [0471], Fig. 39 A; each training data entry or a set of training data entries includes target orthodontic points 3924) and is optimized to determine tooth root apex coordinates (3922) from the 2D panoramic radiograph data (3910) or the 3D intraoral scan data (3910), and the tooth number (3912) (refer to Paragraphs [0077], [0468], [0471], [0474], [0483], Fig. 39 A; during utilization of the trained algorithm, an image 3910 and its corresponding mask 3912 are processed using the generator 3902 to obtain predicted orthodontic points 3922, such as root apices; the image 3910 is a two- or three-dimensional image according to any of the imaging modalities described, which includes panoramic X-ray and intra-oral image capture, and the mask 3912 includes a tooth number; the training algorithm updates the parameters of the generator based on the difference between the predicted and target (ground truth) root apices (level 1 loss, 3926)); and
determining, via a processor (4502) executing the trained deep learning network (3900), the coordinates of the tooth root apex (3924) based on the patient data and corresponding to the tooth number (3910, 3912) (refer to Paragraphs [0467], [0468], [0483]; the system (3900) generates orthodontic points (3932a-3932h) for each tooth number, based on the provided image (3910) and tooth number mask; the orthodontic points (3932a-3932h) include a lower incisor root apex);
generating or modifying the treatment plan (4418) using the coordinates of the tooth root apex (3924) (refer to Paragraphs [0545], [0547]; an estimated treatment plan (4418) is generated based on the paired orthodontic points (4412) generated by the process of Fig. 39A); and
forming one or more physical dental appliances according to the treatment plan (4418) (refer to Paragraphs [0075], [0547]; the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the flowchart; the flowchart (4400) specifies forming a retainer or appliance).
Kearney does not explicitly teach the patient data (3910) as including two-dimensional (2D) panoramic radiograph data of a patient and three-dimensional (3D) intraoral scan data of the patient as part of the disclosed embodiments disclosed above (3900, 4400).
Kearney further discloses an alternate deep learning network (1100) where a plurality of images of the same patient anatomy according to a plurality of imaging modalities are labeled and processed to form a treatment protocol (refer to Paragraph [0209]), where the plurality of imaging modalities includes X-ray (intra-oral or extra-oral, full mouth series (FMX), panoramic, cephalometric), computed tomography (CT) scan, cone-beam computed tomography (CBCT) scan, intra-oral image capture using an optical camera, magnetic resonance imaging (MRI). Using multiple imaging modalities together enables deciphering dental anatomy more accurately than using a single imaging modality (refer to Paragraph [0126]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the patient imaging data (3910) to include both two-dimensional (2D) panoramic radiograph data of a patient and three-dimensional (3D) intraoral scan data of the patient as taught by the alternative embodiment (1100) to enable an accurate portrayal of dental anatomy (refer to Paragraph [0126]).
Kearney teaches that the training imaging data (3910) input is a two or three-dimensional image according to any of the imaging modalities described (refer to Paragraphs [0468], [0471]) which includes CBCT (refer to Paragraph [0077]), but does not explicitly teach the imaging data (3910) as CBCT tooth data for a plurality of patients.
Claessen discloses a method of training a deep neural network for predicting 3D tooth root shapes in the same field of endeavor (refer to Paragraph [0084], Fig. 2). The training data includes CBCT tooth data for a plurality of patients (refer to Paragraphs [0085], [0087]). As Kearney teaches training the deep learning network (3900) on imaging tooth data (3910) that includes segmented data with a label for each tooth number (3912) and tooth root apex coordinates (3924) (refer to Paragraphs [0468], [0471], [0473], [0476], [0483]), and Claessen teaches training a deep learning network using CBCT tooth data, the combined method of Kearney and Claessen teaches training a deep learning network with segmented and labeled CBCT tooth data. As CBCT scans are widely available in the dental field and easy to create, training a deep learning network with this form of imaging data is beneficial (refer to Paragraphs [0008], [0129]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the imaging data (3910) of Kearney with CBCT tooth data for a plurality of patients as taught by Claessen in order to train a deep learning network based on an imaging modality that is widely used (refer to Paragraphs [0008], [0129]).
Regarding claim 2, Kearney and Claessen disclose the method of claim 1; Kearney further discloses wherein the patient data (3910, 3912, 3914, 3916) includes a first set of data (3912) and a second set of data (3914), wherein the first set of data (3912) is generated from a 2D convolutional neural network (700) using the 2D panoramic radiograph data of the patient (704) (refer to Paragraphs [0077], [0147], [0468] Fig. 7; the tooth labels (706a) or pixel masks for image inputs (704, analogous to 3910) are generated according to a convolutional neural network system (700) accepting an image input (704); the image input (704) is obtained by a panoramic X-ray; Examiner understands 2D convolutional neural networks (CNNs) as networks configured to process 2D images (see Paragraph [0120] of specification); thus, the generator (3902) processing panoramic X-ray images is a 2D CNN).
Regarding claims 3 and 5, Kearney and Claessen disclose the method of claim 2; Kearney teaches wherein the 2D or 3D convolutional neural network (700) is trained based on an image (704) that can be obtained according to any of the imaging modalities described (refer to Paragraphs [0468], [0471]) which includes CBCT (refer to Paragraph [0077]); however, Kearney does not explicitly teach wherein the 2D or 3D convolutional neural network (700) is trained based at least in part on the CBCT tooth data.
As disclosed in the rejection for claim 1 above, Claessen teaches training a deep neural network using CBCT data, as this data is widely available (refer to Paragraphs [0008], [0085], [0087], [0129]). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified training data (704) of Kearney to include CBCT tooth data as taught by Claessen in order to train a deep learning network based on an imaging modality that is widely used (refer to Paragraphs [0008], [0129]).
Regarding claim 4, Kearney and Claessen disclose the method of claim 1; Kearney further discloses wherein the patient data (3910, 3912, 3914, 3916) includes a first set of data (3912) and a second set of data (3914), wherein the first set of data (3912) is generated from a 3D convolutional neural network (700) using the 3D intraoral scan data of the patient (704) (refer to Paragraphs [0077], [0147], [0161], Fig. 7; the tooth labels (706a) or pixel masks for image inputs (704, analogous to 3910) are generated according to a convolutional neural network system (700) accepting an image input (704); the image input (704) is obtained by intra-oral image capture).
Regarding claim 6, Kearney and Claessen disclose the method of claim 1; Kearney does not explicitly teach wherein the tooth number selectively weights an output of the trained deep learning network as part of the embodiments disclosed in claim 1 above (3900, 4400).
Kearney further discloses a generative adversarial network (3400) where non-zero pixels are placed at pixels of the input image (3412) representing the tooth number, such that the masked image (3414+3412) has selectively weighted pixels (output) corresponding to the tooth number (refer to Paragraphs [0413], [0415]; Examiner understands “weights” as enhancing or attenuating, consistent with Paragraph [0138] of the specification). This method allows the patient data, in this instance, tooth number, to be represented with the generated image (refer to Paragraph [0415]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method of Kearney and Claessen with tooth number weighting as taught by the alternative generative adversarial network (3400) of Kearney in order to train a deep learning network based on an imaging modality that is widely used (refer to Paragraphs [0008], [0129]).
Regarding claim 10, Kearney discloses a system (4400), the system (4400) comprising:
a treatment plan generator engine (4402) (refer to Paragraph [0547]; the encoder (4402) of the system (4400) outputs an estimated treatment plan (4418)) configured to:
obtain, from a memory (4504), patient data (3910, 3912, 3914, 3916), wherein the patient data (3910, 3912, 3914, 3916)) includes two-dimensional (2D) panoramic radiograph data of a patient or three-dimensional (3D) intraoral scan data of the patient (refer to Paragraphs [0077], [0468]; images from a plurality of 2D and 3D imaging modalities, including panoramic X-ray and intra-oral image capture may be labeled and processed according to a corresponding system; the computing device memory (4504) is configured to be used with the disclosed systems, and the treatment plan generator (4400) receives images as input, therefore the treatment plan generator (4400) is capable of receiving patient data from a memory);
provide the 2D panoramic radiograph (704) to a first learning model (700) that is trained at least in part on imaging tooth data (704+706b) to generate a first set of data (720) (refer to Paragraphs [0146]-[0147], [0149], Fig. 7; the trained generator (712) produces an output (720) embodied as a tooth label or pixel mask labeling a portion of the input image); and
provide the 3D intraoral scan data (804) to a second learning model (800) that is trained at least in part on the imaging tooth data (804a+804b) to generate a second set of data (820) (refer to Paragraphs [0162]-[0163], [0166], Fig. 8; the anatomical feature label (820) is output as a pixel mask labeling the input image (804) according to a convolutional neural network system (800));
combine the first (720) and second sets of data (820) (refer to Paragraph [0468]; the input image (3910) is concatenated with anatomy and tooth number masks (3912, 3914) generated according to previously described embodiments (700, 800));
obtain, from the memory (4504), a tooth number (3912) associated with a tooth (refer to Paragraphs [0082], [0147], [0468]; the tooth masks (3912, 3914) include a tooth number label); and
provide the combined first and second sets of data and the tooth number (720+820 = 3910+3912+3914) to a trained deep learning network (3900) (refer to Paragraph [0468], Fig. 39A; the resulting concatenated image (3910+3912+3914) is provided to the generator (3902), where the concatenated image includes a tooth number mask),
wherein the trained deep learning network (3900) is trained on segmented data that includes segmented teeth with labels for each of the segmented teeth (3912, 3914, 3916) and tooth root apex coordinates (3924) associated with each of the segmented teeth (refer to Paragraphs [0471], [0473], [0483]; each training data entry or a set of training data entries may include an image 3910 and data describing the features shown in the image, including masks 3912, 3914, 3916, and target orthodontic points 3924, including lower incisor root apices);
wherein the trained deep learning network (3900) is trained on the tooth root apex coordinates (3924) as ground truth output data (3924) (refer to Paragraph [0471], Fig. 39 A; each training data entry or a set of training data entries includes target orthodontic points 3924) and is optimized to determine tooth root apex coordinates (3922) from the 2D panoramic radiograph data (3910) or the 3D intraoral scan data (3910), and the tooth number (3912) (refer to Paragraphs [0077], [0468], [0471], [0474], [0483], Fig. 39 A; during utilization of the trained algorithm, an image 3910 and its corresponding mask 3912 are processed using the generator 3902 to obtain predicted orthodontic points 3922, such as root apices; the image 3910 is a two- or three-dimensional image according to any of the imaging modalities described, which includes panoramic X-ray and intra-oral image capture, and the mask 3912 includes a tooth number; the training algorithm updates the parameters of the generator based on the difference between the predicted and target (ground truth) root apices (level 1 loss, 3926)); and
a processor (4502) configured to determine, via the trained deep learning network (3900), coordinates of a tooth root apex of the tooth (3924) based on the patient data and corresponding to the tooth number (3910, 3912) (refer to Paragraphs [0468], [0475]; the system (3900) generates lower incisor root apex orthodontic points (3924) for each tooth number, based on the provided image (3910) and masks (3912, 3914, 3916)),
wherein the treatment plan generator engine (4400) is configured to use the coordinates of the tooth root apex (3924) to generate or modify a treatment plan (4418) (refer to Paragraphs [0545], [0547], Fig. 44; the orthodontic points (4412) identified by the approach of Fig. 39A are input into the encoder (4402) to generate an estimated treatment plan (4418)).
an appliance fabrication subsystem (data processing apparatus) configured to fabricate one or more physical appliances from the treatment plan (refer to Paragraphs [0075], [0547]; the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the flowchart; the flowchart (4400) specifies forming a retainer or appliance).
Kearney does not explicitly teach the patient data (3910, 3912, 3914, 3916) as including two-dimensional (2D) panoramic radiograph data of a patient and three-dimensional (3D) intraoral scan data of the patient as part of the disclosed embodiments disclosed above (3900, 4400).
Kearney further discloses an alternate deep learning network (1100) where a plurality of images of the same patient anatomy according to a plurality of imaging modalities are labeled and processed to form a treatment protocol (refer to Paragraph [0209]), where the plurality of imaging modalities includes X-ray (intra-oral or extra-oral, full mouth series (FMX), panoramic, cephalometric), computed tomography (CT) scan, cone-beam computed tomography (CBCT) scan, intra-oral image capture using an optical camera, magnetic resonance imaging (MRI). Using multiple imaging modalities together enables deciphering dental anatomy more accurately than using a single imaging modality (refer to Paragraph [0126]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the patient imaging data (3910) to include both two-dimensional (2D) panoramic radiograph data of a patient and three-dimensional (3D) intraoral scan data of the patient as taught by the alternative embodiment (1100) to enable an accurate portrayal of dental anatomy (refer to Paragraph [0126]).
Kearney teaches wherein the first (700) and second (800) learning models are trained based on imaging data (704+706a, 804a+804b), where the imaging modalities include CBCT (refer to Paragraph [0077]); however, Kearney does not explicitly teach wherein the first (700) and second (800) learning models are trained based at least in part on the CBCT tooth data.
Claessen discloses a method of training a deep neural network for predicting 3D tooth root shapes in the same field of endeavor (refer to Paragraph [0084], Fig. 2). The training data includes CBCT tooth data for a plurality of patients (refer to Paragraphs [0085], [0087]). As CBCT scans are widely available in the dental field and easy to create, training a deep learning network with this form of imaging data is beneficial (refer to Paragraphs [0008], [0129]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the imaging data (704+706a, 804a+804b) of Kearney with CBCT tooth data for a plurality of patients as taught by Claessen in order to train a deep learning network based on an imaging modality that is widely used (refer to Paragraphs [0008], [0129]).
Regarding claims 12, 13 and 15 Kearney discloses a non-transitory computer-readable storage medium (4508) comprising instructions that, when executed by one or more processors (4502) of a system (4500), cause the system (4500) to perform operations comprising:
obtaining patient data (3910, 3912, 3914, 3916), wherein the patient data (3910, 3912, 3914, 3916) includes two-dimensional (2D) panoramic radiograph data of a patient or three-dimensional (3D) intraoral scan data of the patient (refer to Paragraphs [0077], [0468]; images from a plurality of 2D and 3D imaging modalities, including panoramic X-ray and intra-oral image capture may be labeled and processed according to a corresponding system);
providing the 2D panoramic radiograph (3910) to a first learning model (3902) that is trained at least in part on imaging tooth data (3910, 3912, 3914, 3916) to generate a first set of data (3922),
wherein the first learning model (3902) is a 2D convolutional neural network configured to determine the first set of data (3922) from the 2D panoramic radiograph data of the patient (3910) (refer to Paragraphs [0077], [0467], [0468], Fig. 39A; the generator (3902) takes 2D or 3D images according to any of the described imaging modalities, which includes panoramic X-ray; as demonstrated in Fig. 39A, the generator (3902) is configured with convolutional kernels; Examiner understands 2D convolutional neural networks (CNNs) as networks configured to process 2D images (see Paragraph [0120] of specification); thus, the generator (3902) processing panoramic X-ray images is a 2D CNN),
wherein the first set of data (3922) corresponds to a first set of tooth root apex coordinates (3922) (refer to Paragraphs [0471]-[0473], Fig. 39A; each training data entry includes an image (3910) and data describing the features shown in the image to generate a predicted set of orthodontic points (3922)); and
generating a second set of data (3928), wherein the second set of data (3928) corresponds to a second set of tooth root apex coordinates (3928)
combining the first (3922) and second sets of data (3928) (refer to Paragraph [0472]; the input to the discriminator is a combination of the real orthodontic points (3928) and the predicted orthodontic points (3922));
obtaining a tooth number (3912) associated with a tooth (refer to Paragraph [0468]; the tooth masks (3912, 3914, 3916) include a tooth number label); and
providing the combined first and second sets of data and the tooth number (3922+3928+3912) to a trained deep learning network (3900) (refer to Paragraphs [0468], [0472]-[0473], Fig. 39A; the resulting concatenated image (3910+3912+3914) is provided to the generator (3902), where the concatenated image includes a tooth number mask; the combined real orthodontic points (3928) and the predicted orthodontic points (3922) are provided to the discriminator (3908)),
wherein the trained deep learning network (3900) is trained on segmented data that includes segmented teeth with labels for each of the segmented teeth and tooth root apex coordinates (3924) associated with each of the segmented teeth (refer to Paragraphs [0471], [0473], [0483]; each training data entry or a set of training data entries may include an image 3910 and data describing the features shown in the image, including masks 3912, 3914, 3916, and target orthodontic points 3924, including lower incisor root apices);
wherein the trained deep learning network (3900) is trained on the tooth root apex coordinates (3924) as ground truth output data (3924) (refer to Paragraph [0471], Fig. 39 A; each training data entry or a set of training data entries includes target orthodontic points 3924) and is optimized to determine tooth root apex coordinates (3922) from the 2D panoramic radiograph data (3910) or the 3D intraoral scan data (3910), and the tooth number (3912) (refer to Paragraphs [0077], [0468], [0471], [0474], [0483], Fig. 39 A; during utilization of the trained algorithm, an image 3910 and its corresponding mask 3912 are processed using the generator 3902 to obtain predicted orthodontic points 3922, such as root apices; the image 3910 is a two- or three-dimensional image according to any of the imaging modalities described, which includes panoramic X-ray and intra-oral image capture, and the mask 3912 includes a tooth number; the training algorithm updates the parameters of the generator based on the difference between the predicted and target (ground truth) root apices (level 1 loss, 3926)); and
determining, via a processor (4502), executing the trained deep learning network (3900), coordinates of a tooth root apex of the tooth (3924) based on the combined first and second sets of data (3922+3928) and the tooth number (3912) (refer to Paragraphs [0468], [0472]-[0473], [0475]; the system (3900) generates lower incisor root apex orthodontic points (3924), based on the provided image (3910), masks (3912, 3914, 3916), which include tooth number, and the realism matrix (3930) output, which is based on the combined input of the real orthodontic points (3928) and predicted orthodontic points (3922)),
generating or modifying a treatment plan (4418) using the coordinates of the tooth root apex (3924) (refer to Paragraphs [0545], [0547], Fig. 44; the orthodontic points (4412) identified by the approach of Fig. 39A are input into the encoder (4402) to generate an estimated treatment plan (4418)).
forming one or more physical appliances according to the treatment plan (refer to Paragraphs [0075], [0547]; the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the flowchart; the flowchart (4400) specifies forming a retainer or appliance).
Kearney does not explicitly teach the patient data as including two-dimensional (2D) panoramic radiograph data of a patient and three-dimensional (3D) intraoral scan data of the patient as part of the disclosed embodiments disclosed above (3900, 4400).
Kearney further discloses an alternate deep learning network (1100) where a plurality of images of the same patient anatomy according to a plurality of imaging modalities are labeled and processed to form a treatment protocol (refer to Paragraph [0209]), where the plurality of imaging modalities includes X-ray (intra-oral or extra-oral, full mouth series (FMX), panoramic, cephalometric), computed tomography (CT) scan, cone-beam computed tomography (CBCT) scan, intra-oral image capture using an optical camera, magnetic resonance imaging (MRI). Using multiple imaging modalities together enables deciphering dental anatomy more accurately than using a single imaging modality (refer to Paragraph [0126]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the patient imaging data (3910) to include both two-dimensional (2D) panoramic radiograph data of a patient and three-dimensional (3D) intraoral scan data of the patient as taught by the alternative embodiment (1100) to enable an accurate portrayal of dental anatomy (refer to Paragraph [0126]).
Kearney teaches wherein the first learning model (3902) is trained based on imaging tooth data (3910, 3912, 3914, 3916) (refer to Paragraphs [0471]-[0473]), where the imaging modalities include CBCT (refer to Paragraphs [0077], [0468]); however, Kearney does not explicitly teach:
wherein the first (3902) learning model is trained based at least in part on CBCT tooth data,
providing the 3D intraoral scan data to a second learning model that is trained at least in part on CBCT tooth data to generate the second set of tooth data (3928),
wherein the second learning model is a 3D convolutional neural network configured to determine the second set of data from the 3D intraoral scan data of the patient.
Kearney can be modified to meet this/these limitation(s) by Claessen, which discloses a method of training a deep neural network for predicting 3D tooth root shapes in the same field of endeavor (refer to Paragraph [0084], Fig. 2), as follows:
Modifying the imaging tooth data (3910, 3912, 3914, 3916) of the first model (3902) of Kearney in the same fashion as the training data of Claessen, to include CBCT tooth data for a plurality of patients (refer to Paragraphs [0085], [0087] of Claessen).
A person of ordinary skill in the art prior to the effective filing date of the claimed invention would have been motivated to make the above modification(s) because:
Claessen teaches that CBCT scans are widely available in the dental field and easy to create, so training a deep learning network with this form of imaging data is beneficial (refer to Paragraphs [0008], [0129]).
Generating the second set of tooth data (3928) of Kearney in the same manner as Claessen, which teaches providing the 3D intraoral scan data (806) to a learning model (816) that is trained at least in part on CBCT tooth data (804) to generate a set of tooth data (refer to Paragraphs [0130], [0132]-[0133], Fig. 8; the 3D deep learning neural network receives 3D CBCT training data to output collections of voxel data, wherein each collection may represent a distinct part e.g. teeth or jaw bone of the 3D image data). This learning model (816) is a 3D convolutional neural network configured to determine the set of data from the 3D intraoral scan data of the patient (806) (refer to Paragraphs [0130], [0132]-[0133], Fig. 8; Examiner understands 3D convolutional neural networks (CNNs) as networks configured to process 3D images (see Paragraph [0121] of specification); thus, the deep neural network processing 3D optical scanning data is a 3D CNN).
A person of ordinary skill in the art prior to the effective filing date of the claimed invention would have been motivated to make the above modification(s) because:
Claessen teaches accurately classifying voxels for determining root shape using a neural network that relies on both 3D intraoral scan data and CBCT tooth data (refer to Paragraphs [0129], [0133]).
Regarding claim 17, Kearney and Claessen disclose the non-transitory computer-readable storage medium of claim 12; Kearney does not explicitly teach wherein the tooth number (3912) selectively weights an output of the trained deep learning network as part of the embodiments disclosed in claim 1 above (3900, 4400).
Kearney further discloses a generative adversarial network (3400) where non-zero pixels are placed at pixels of the input image (3412) representing the tooth number, such that the masked image (3414+3412) has selectively weighted pixels (output) corresponding to the tooth number (refer to Paragraphs [0413], [0415]; Examiner understands “weights” as enhancing or attenuating, consistent with Paragraph [0138] of the specification). This method allows the patient data, in this instance, tooth number, to be represented with the generated image (refer to Paragraph [0415]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method of Kearney and Claessen with tooth number weighting as taught by the alternative generative adversarial network (3400) of Kearney in order to train a deep learning network based on an imaging modality that is widely used (refer to Paragraphs [0008], [0129]).
Regarding claim 21, Kearney and Claessen disclose the method of claim 1;
Kearney further discloses wherein the trained deep learning network (3900) is trained by;
providing the 2D panoramic radiograph (704) to a first learning model (700) that is trained at least in part on imaging tooth data (704+706b) to generate a first set of data (720) (refer to Paragraphs [0146]-[0147], [0149], Fig. 7; the trained generator (712) produces an output (720) embodied as a tooth label or pixel mask labeling a portion of the input image); and
providing the 3D intraoral scan data (804) to a second learning model (800) that is trained at least in part on the imaging tooth data (804a+804b) to generate a second set of data (820) (refer to Paragraphs [0162]-[0163], [0166], Fig. 8; the anatomical feature label (820) is output as a pixel mask labeling the input image (804) according to a convolutional neural network system (800));
combining the first (720) and second sets of data (820) (refer to Paragraph [0468]; the input image (3910) is concatenated with anatomy and tooth number masks (3912, 3914) generated according to previously described embodiments (700, 800));
wherein providing the patient data and the tooth number (3910, 3912, 3914, 3916) to the trained deep learning network (3900) comprises providing the combined first and second sets of data and the tooth number (720+820 = 3910+3912+3914) to a trained deep learning network (3900) (refer to Paragraph [0468], Fig. 39A; the resulting concatenated image (3910+3912+3914) is provided to the generator (3902), where the concatenated image includes a tooth number mask),
Kearney teaches wherein the first (700) and second (800) learning models are trained based on imaging data (704+706a, 804a+804b), where the imaging modalities include CBCT (refer to Paragraph [0077]); however, Kearney does not explicitly teach wherein the first (700) and second (800) learning models are trained based at least in part on the CBCT tooth data.
Claessen discloses a method of training a deep neural network for predicting 3D tooth root shapes in the same field of endeavor (refer to Paragraph [0084], Fig. 2). The training data includes CBCT tooth data for a plurality of patients (refer to Paragraphs [0085], [0087]). As CBCT scans are widely available in the dental field and easy to create, training a deep learning network with this form of imaging data is beneficial (refer to Paragraphs [0008], [0129]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the imaging data (704+706a, 804a+804b) of Kearney with CBCT tooth data for a plurality of patients as taught by Claessen in order to train a deep learning network based on an imaging modality that is widely used (refer to Paragraphs [0008], [0129]).
Regarding claim 24, Kearney and Claessen disclose the non-transitory computer-readable storage medium of claim 12; Kearney further discloses wherein the first learning model (3902) minimizes a loss function (3926) between a target tooth apex (3924) and a predicted tooth apex based on the 2D panoramic radiograph data (3922) (refer to Paragraph [0471], Fig. 39A; the training algorithm updates parameters of the generator (3902) according to the loss (3926); by definition (https://www.ibm.com/think/topics/loss-function), a loss function refers specifically to where minimization is the objective of a machine learning model).
Kearney does not teach the target tooth apex (3924) as CBCT data.
Claessen further teaches the deep learning model (816) as minimizing a loss function which represents the deviation of the output of the deep neural network (predicted data) to the target data, where the target data is represented by classified voxel data from the CBCT image data (804) (refer to Paragraph [0133], Fig. 8). The use of CBCT data as target is beneficial, as CBCT scans are widely available in the dental field and easy to create (refer to Paragraphs [0008], [0129]), and further, the deep learning model (816) is already trained on this CBCT data (refer to Paragraph [0132]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have further modified the target tooth apex data as taught by Kearney and Claessen to be CBCT data as taught by Claessen, as this data is widely available and used for training the model (refer to Paragraphs [0008], [0129], [0132]).
Regarding claim 25, Kearney and Claessen disclose the method of claim 1; Kearney is silent to wherein the CBCT tooth data includes segmented 3D voxel data.
Claessen further discloses wherein the CBCT tooth data includes segmented 3D voxel data (refer to Paragraph [0087]; the pre-processing of CBCT data includes defining a voxel representation of a tooth (206), crown (208) and root (207)). This segmented data is part of the normalization of data to enable the deep learning network to accurately train relevant features needed for root prediction (refer to Paragraph [0087]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have further modified the imaging data as taught by Kearney and Claessen with segmented 3D voxel data as taught by Claessen to enable the deep learning network to predict roots (refer to Paragraph [0087]).
Regarding claim 25, Kearney and Claessen disclose the method of claim 1; Kearney further discloses wherein the trained deep learning network (3900) is trained on minimizing a cost function associated with determined tooth root apices (3922) of the patient and the ground truth output data (3924) (refer to Paragraphs [0471]; the predicted orthodontic points 3922 obtained for a training data entry are compared to the target orthodontic points 3924 for the training data entry to obtain a level 1 (L1) loss 3926; the calculated loss is zero if the target point 3924 lies on a center of the dilated point 3922 and increases with distance of the target point 3924 from the center of the dilated point 3922; thus, the function is minimized the closer to the target or ground truth that the predicted points are).
Claim 26-Cancelled
Claim(s) 19-20, 22-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kearney et al. (US 20210118132 A1), herein referred to as Kearney, in view of Claessen et al. (US 20210082184 A1), herein referred to as Claessen as applied to claim 12 above, and further in view of Sabina et al. (US 20190231492 A1), herein referred to as Sabina.
Regarding claims 19-20, Kearney and Claessen disclose the non-transitory computer-readable storage medium of claim 12; Kearney does not disclose wherein the trained deep learning network is trained based at least in part on the cone beam computed tomography (CBCT) tooth data and 2D panoramic radiograph data corresponding to the CBCT tooth data, and further, wherein the deep learning network is trained based at least in part on the CBCT tooth data and 3D intraoral scan data corresponding to the CBCT tooth data.
Claessen teaches wherein the deep learning network is trained based at least in part on the cone beam computed tomography (CBCT) tooth data (804) and corresponding 3D intraoral scan data (806) (refer to Paragraph [0132]), as disclosed in the rejection for claim 12 above.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the deep learning network of Kearney with the cone beam computed tomography (CBCT) tooth data (804) and corresponding 3D intraoral scan data (806) as taught by Claessen in order to accurately classify voxels for determining root shape using a neural network that relies on both 3D intraoral scan data and CBCT tooth data (refer to Paragraphs [0129], [0133]).
Claessen does not teach training the deep learning network using corresponding 2D panoramic radiograph data.
Sabina discloses a method of automatic characterization of dental features in the same field of endeavor (refer to Paragraph [0201]). Sabina further teaches that images of a patient’s teeth from different imaging modalities are used as part of a trained machine learning model to detect the one or more dental features, where the different imaging modalities include CBCT and panoramic x-rays (refer to Paragraphs [0202]-[0203]). By using multiple imaging modalities, the actionable dental features can be cross-referenced for confirmation (refer to Paragraphs [0057], [0206]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of training the deep learning network as taught by Kearney and Claessen with corresponding CBCT and panoramic x-rays as taught by Sabina in order to cross reference the identified dental features (refer to Paragraphs [0057], [0206]).
Regarding claim 22, Kearney and Claessen disclose the non-transitory computer-readable storage medium of claim 12; Kearney does not teach wherein the operations further comprise comparing CBCT and 2D panoramic data to ensure that both data sets share a common coordinate system and that the data sets match each other within a tolerance amount as part of the original embodiments (3900, 4400).
Kearney teaches an alternative embodiment of a deep learning algorithm (3200C), which compares two imaging data sets (3204a-3212a, 3204b-3212b), where one of the input images (3204b) is obtained by transformation (3204b) (refer to Paragraphs [0394]-[0395], Fig. 32C), and where transforming means converting the original image to a different imaging modality (refer to Paragraph [0079]), such as the transforming between CBCT images and 2D panoramic imaging (refer to Paragraph [0125]). Thus, Kearney teaches comparing two imaging data sets, where one is CBCT tooth data, and the other is 2D panoramic radiograph data. This comparison ensures the images are a match for the particular patient within a tolerance amount (refer to Paragraphs [0408]- [0409]; the vectors for a first image are compared to one or more vectors for a second image to obtain one or more distance metric; the distance metrics are used as a cutoff criterion or tolerance amount to determine if the two images are sufficiently similar).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have further modified the operations of Kearney and Claessen with comparing CBCT and 2D panoramic data as taught by the alternative embodiment of a deep learning algorithm (3200C) of Kearney to ensure the images from two different modalities are a match for the particular patient (refer to Paragraph [0408]).
Neither Kearney or Claessen disclose that both data sets share a common coordinate system.
Sabina discloses a method of automatic characterization of dental features in the same field of endeavor (refer to Paragraph [0201]). Sabina further teaches registration between all the records, where the records include different imaging modalities, such as panoramic x-ray and CBCT (refer to Paragraphs [0202], [0209]). By definition (Wikipedia), image registration is the process of transforming different sets of data into one coordinate system. The image registrations allows the user to identify actionable dental features in both records are real (refer to Paragraph [0210]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have modified the operations of Kearney and Claessen with image registration of CBCT and 2D panoramic data as taught by Sabina, to allow the user to identify actionable dental features in both records are real (refer to Paragraph [0210]).
Regarding claim 23, Kearney and Claessen disclose the non-transitory computer-readable storage medium of claim 12; Kearney does not teach wherein the operations further comprise comparing the 3D intraoral scan data to CBCT tooth data to ensure that both data sets share a common coordinate system and that the data sets match each other within a tolerance amount as part of the original embodiments (3900, 4400).
Kearney teaches an alternative embodiment of a deep learning algorithm (3200C), which compares two imaging data sets (3204a-3212a, 3204b-3212b), where one of the input images (3204b) is obtained by transformation (3204b) (refer to Paragraphs [0394]-[0395], Fig. 32C), and where transforming means converting the original image to a different imaging modality (refer to Paragraph [0079]), such as the transforming between 3D optical images and CBCT (refer to Paragraphs [0077], [0125]). Thus, Kearney teaches comparing two imaging data sets, where one is CBCT tooth data, and the other is 3D intraoral scan data. This comparison ensures the images are a match for the particular patient within a tolerance amount (refer to Paragraphs [0408]- [0409]; the vectors for a first image are compared to one or more vectors for a second image to obtain one or more distance metric; the distance metrics are used as a cutoff criterion or tolerance amount to determine if the two images are sufficiently similar).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have further modified the operations of Kearney and Claessen with comparing CBCT data and 3D intraoral scan data as taught by the alternative embodiment of a deep learning algorithm (3200C) of Kearney to ensure the images from two different modalities are a match for the particular patient (refer to Paragraph [0408]).
Kearney does not teach wherein both data sets share a common coordinate system.
Claessen teaches comparing the 3D intraoral scan data (808) to CBCT tooth data (804) to ensure that both data sets share a common coordinate system (refer to Paragraph [0132], Fig. 8; an alignment function 810 may be employed which is configured to align the 3D surface meshes to the 3D CBCT image data to use the same spatial coordinate system). Positional features (812) and voxel data (814) can be determined from the aligned 3D intraoral scan data (808) and CBCT tooth data (804) and input into the deep neural network (816) to classify tooth structures, and ultimately predict root shape (refer to Paragraphs [0129]-[0130], [0132]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to have further modified the operations of Kearney and Claessen with comparing the 3D intraoral scan data (808) to CBCT tooth data (804) as taught by Claessen in order to classify tooth structures, and ultimately predict root shape (refer to Paragraphs [0129]-[0130], [0132]).
Response to Arguments
Applicant's arguments filed 09/23/2026 have been fully considered but they are not persuasive.
In response to Applicant’s arguments that amended claims 1, 10 and 12 recite limitations that are not obvious in view of Kearney and Claessen, Examiner points to the above rejection of these claims.
In response to the argument that Kearney does not disclose using CBCT images to identify orthodontic points because Kearney states that multiple imaging modalities can be used for the patient data, which includes CBCT data, Examiner notes that Kearney still anticipates the use of CBCT data (refer to Paragraphs [0077], [0468]), as CBCT data is explicitly disclosed as a patient imaging input option.
In response to the argument that Kearney does not disclose using both CBCT and 2D panoramic data, Examiner points to the obviousness argument based on an alternative embodiment of a deep learning network (Kearney) that uses multiple imaging modalities as input (refer to Paragraphs [0126], [0209]). It is this combined teaching which is relied upon for this limitation.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). In this case, the argument that Claessen does not disclose model optimization or a trained deep learning network trained on tooth root apex coordinates, it is Kearney that is specifically relied upon for these teachings. Further, the test for obviousness is not that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Adriena J Webb Lyttle whose telephone number is (571)270-7639. The examiner can normally be reached Mon - Fri 10:00-7:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Edelmira Bosques can be reached at (571) 270-5614. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADRIENA J WEBB LYTTLE/Examiner, Art Unit 3772
/EDELMIRA BOSQUES/Supervisory Patent Examiner, Art Unit 3772