Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-2, 4-14 and 16-30 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claims contain subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention.
Applicant claims that the machine learning models perform image level classification and output a probability of the image containing a dental condition without performing pixel-level classification or image segmentation. However, applicant’s specification [0068] describes an embodiment of an image being segmented into upper dental arch segment, a lower dental arch segment and a patient bite segment. Additionally, applicant’s specification does not describe in detail a machine learning model is able to classify an image without pixel-level classification or image segmentation. The specification describes GAN, U-net, and Grad-CAM among others, but does not describe how these models do not use pixel-level classification or image segmentation. See MPEP 2173.05(i). For further prosecution, it will be assumed that outputting a probability of the image containing a dental condition does not use pixel-level classification or image segmentation.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-2, 4-14 and 16-30 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Applicant claims that the machine learning models perform image level classification and output a probability of the image containing a dental condition without performing pixel-level classification or image segmentation. It is ambiguous in the claim whether the machine learning models perform image level classification without pixel-level classification or image segmentation or if there is an output of a probability of the image containing a dental condition without pixel-level classification or image segmentation. For further prosecution, it will be assumed that outputting a probability of the image containing a dental condition does not use pixel-level classification or image segmentation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6, 16-17, 22-26, 29 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Mason (US 20160135925) in view of Zhang (US 20240379230) and further in view of Vannahme (CN 113811916) and further in view of Carter (US 20240087725).
Regarding claim 1, Mason teaches a non-transitory computer readable medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
Receiving an image of dental site of a patient (Paragraph 85, images depicting the positions and orientations of the patient's teeth, gingiva, etc.);
Separately estimating, for each dental condition of a plurality of dental conditions, a presence of the dental condition in the image (Paragraph 185, used to identify existing conditions based on digital data of the intraoral cavity);
While Mason fails to disclose the following, Zhang teaches:
Processing the image using one or more trained machine learning models that perform image level classification and output a probability of the image containing the dental condition without performing pixel-level classification or image segmentation (Paragraph 40, a first neural network model is used to perform disease prediction on an individual image from the original images in order to obtain image features and a classification probability for the disease categories of the original images, where the classification probability represents the probability of the individual image belonging to different disease categories. In the embodiment, the first neural network model, for example, is a convolutional neural network (abbreviated as CNN) model);
Mason and Zhang are both considered to be analogous to the claimed invention because they are in the same field of disease detection in images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mason by using Zhang and processing the image using a machine learning model and outputting a probability of the image containing a dental condition without performing pixel-level classification or image segmentation. Doing so would allow for analyzing multiple disease categories and determining a specific diagnosis (Zhang, Paragraph 49).
While the combination of Mason and Zhang fails to disclose the following, Vannahme teaches:
Separately estimating, for each dental condition of the plurality of dental conditions, a severity level of the dental condition at one or more locations in the image (Page 17, paragraph 2, the identification and severity value of the tooth condition is determined based on the specific dental object (such as tooth));
Generating a plurality of severity maps for the image based on the estimated presence of the plurality of dental conditions and the estimated severity level of each of the plurality of dental conditions at the one or more locations in the image (Page 19, paragraph 6, a thermal map or any other graphical representation generally superimposed on the patient tooth three-dimensional model, indicating the area specific severity value or severity level or other attribute, such as score value, development of rate or tooth condition); and
Projecting each of the plurality of severity maps onto a model of the dental site to generate a plurality of projected severity maps, each associated with a different one or the plurality of dental conditions (Page 19, paragraph 6, a thermal map or any other graphical representation generally superimposed on the patient tooth three-dimensional model, indicating the area specific severity value or severity level or other attribute, such as score value, development of rate or tooth condition).
Vannahme and the combination of Mason and Zhang are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason and Zhang by using Vannahme and estimating the severity level of a dental condition in an image, generating a severity map of the condition, and projecting the severity map onto a model of the dental site. In turn, because the above steps are repeated as claimed has properties predicted by the prior art, it would have been obvious to estimate the severity of a condition, generate a severity map, and project the severity map for a plurality of conditions. Doing so would allow dentists to efficiently visualize and identify many dental conditions at once more easily from patient images. In addition, it has been judicially determined that duplication of parts is not patentable (In re Harza, 274 F.2d 669, 124 USPQ 378 (CCPA 1960)). Therefore, it would have been obvious to an artisan before the effective filing of the current application to have duplicated the steps. Further, Carter from the same field of endeavor, discloses detecting multiple conditions, each detected condition may indicate one or more regions or locations in the image and an identifier or label of the specific associated condition (such as a specific dental pathology, restoration, anatomy or anomaly, paragraphs 37, 40, and 42, etc.). Based on this teaching, it would have been obvious to a person of ordinary skill in the art before the effective filing date to have modified the combination of Mason and Vannahme and use Carter to identify a plurality of dental conditions in an image. Doing so would allow dentists to efficiently visualize and identify many dental conditions at once more easily from patient images.
Apparatus claim 29 and method claim 30 correspond to CRM claim 1. Therefore, claims 29 and 30 are rejected for the same reasons as used above.
Regarding claim 2, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination as presented previously fails to disclose the following, Vannahme further teaches:
Wherein the model is a 3D model and the plurality of projected severity maps are 3D severity maps, the operations further comprising: presenting the 3D model of the dental site together with the 3D severity maps of the plurality of dental conditions (Page 19, paragraph 6, a thermal map or any other graphical representation generally superimposed on the patient tooth three-dimensional model, indicating the area specific severity value or severity level or other attribute, such as score value, development of rate or tooth condition) in a graphical user interface (GUI) (Vannahme, Page 27, paragraph 1, the user interface configured to implement the disclosed method).
Vannahme and the combination of Mason, Zhang, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, and Carter by using Vannahme and presenting the model of the dental site with the 3D severity map. In turn, because the above steps are repeated as claimed has properties predicted by the prior art, it would have been obvious to present the model of the dental site with the 3D severity map. Doing so would allow dentists to efficiently visualize and identify many dental conditions at once more easily from patient images.
Regarding claim 6, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1, wherein the plurality of dental conditions are selected from a group consisting of caries, gum recession, gum inflammation, tooth wear, malocclusion, tooth crowding, tooth spacing (Mason, Paragraph 67, condition comprises one or more of: malocclusion, tooth decay, loss of one or more teeth, root resorption, periodontal disease, gingival recession… an amount of tooth spacing, an amount of tooth crowding, an amount of tooth wear, an amount of gum recession), plaque, tooth stains, and tooth cracks. While plaque, tooth stains, and tooth cracks are not explicitly disclosed, they are other dental conditions that can be identified in the same ways as above.
Regarding claim 16, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination as presented previously fails to disclose the following, Carter further teaches:
Generating a combined projected severity map based on the plurality of projected severity maps, wherein for each location of the one or more locations a combined severity level is determined based on severity levels of each of the plurality of projected severity maps at the location (Paragraph 33, In such embodiments in which two or more machine learning models may be trained to detect the same or overlapping sets of potential pathologies, the medical image analysis system 120 may be configured to apply a voting methodology or other resolution process to determine an ultimate classification result based on collective output of the models; Paragraph 57, Labels within a label category may be similar in morphology and/or root cause (for example, three different degrees of severity of caries)).
Note: Carter teaches identifying conditions by combining the results of different machine learning models, which are trained to identify the conditions based on level of severity.
Carter and the combination of Mason, Zhang, and Vannahme are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, and Vannahme by using Carter and combining severity maps to determine severity of conditions at locations in an image. Doing so would allow for more accurately predicting the severity of a dental condition by validating the severity with multiple machine learning models.
Regarding claim 17, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 16. While the combination as presented previously fails to disclose the following, Vannahme further teaches:
Presenting the combined projected severity map overlaid on the model in a graphical user interface (GUI) (Page 27, paragraph 1, the user interface configured to implement the disclosed method);
Receiving a selection of a location of the one or more locations (Page 16, Paragraph 4, The selection of more than one dentistry object can be automatically achieved by manually selecting the area or by selecting a dental object after dividing the digital 3 D tooth model); and
Presenting separate data for one or more of the plurality of projected severity maps at the selected location (Page 27, Paragraph 2, such segmentation allows comparing the same dental object in the first digital 3 D model and the second digital 3 D model or area in the same dental object). Note: Vannahme teaches comparing the severity of a condition in the same location in different models. This can be used to present separate data from a plurality of severity maps at a selected location.
Vannahme and the combination of Mason, Zhang, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, and Carter by using Vannahme and presenting the combined severity map in the GUI, receiving a selection of a location, and presenting separate data from the plurality of severity maps at the selection location. Doing so would allow for dentists to have more information regarding the various levels of severity predicted at a specific location before deciding a treatment plan.
Regarding claim 22, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1, wherein the image comprises a first time stamp (Mason, Paragraph 56, receiving first digital data representative of the intraoral cavity at a first time point), the operations further comprising:
Receiving a second image that comprises a second time stamp that predates the first timestamp (Mason, Paragraph 56, receiving second digital data representative of the intraoral cavity at a second time point different from the first time point); A second time point that is different from the first time point can be used to compare the two images, regardless of if the second came before the first.
Determine differences therebetween (Mason, Paragraph 38, A predictive method can involve comparing surface scan data and/or sub-surface data of the teeth at a plurality of time points in order to determine changes to the position and/or shape of the teeth over time);
Determining rates of change in severity levels for one or more of the plurality of dental conditions based on a result of the comparing (Mason, Paragraph 117, As depicted in FIGS. 7A and 7B, the tooth 702 has an initial height 704a at the first time point and an increased height 704b at the second time point. The difference 705 between heights 704a, 704b can be used to calculate a height change rate for the tooth).
While the combination as presented previously fails to disclose the following, Vannahme further teaches:
Comparing the projected severity map to a second projected severity map generated for the second image (Page 19, paragraph 6, a thermal map or any other graphical representation generally superimposed on the patient tooth three-dimensional model, indicating the area specific severity value or severity level or other attribute, such as score value, development of rate or tooth condition);
Vannahme and the combination of Mason, Zhang, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, and Carter by using Vannahme and generating a second severity map for a second image. Doing so would allow dentists to obtain more accurate severity information by comparing the first and second severity maps.
Regarding claim 23, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 22, the operations further comprising:
Identifying one or more locations at which the severity level exceeds a severity threshold (Mason, Paragraph 153, correction can be recommended if the severity exceeds a threshold value); and
Flagging the one or more locations (Mason, Paragraph 143, predicted digital representation is displayed to a user).
Regarding claim 25, claim 23 encompasses all of the limitations of claim 25. Therefore, claim 25 is rejected for the same reasons as above.
Regarding claim 24, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 22, the operations further comprising:
Determining a recommended frequency for the patient to visit a dentist based on at least one of the severity level or the rate of change of the severity level for the one or more dental conditions (Mason, Paragraph 153, recommended treatment products and/or procedures can be generated based on the severity of the condition or predicted condition; Paragraph 155, Predicted outcomes for different treatment products and/or procedures can be compared to each other in order to facilitate selection of an optimal course of treatment (e.g., with respect to treatment effectiveness, duration). Recommended frequency for visiting a dentist is part of a course of treatment including duration.
Regarding claim 26, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1, the operations further comprising:
Generating, based on the estimated severity level of the one or more dental conditions at the one or more locations in the image, a recommendation for a dentist to assess the dental site (Mason, Paragraph 153, the severity of the patient's condition or predicted condition can be displayed to the medical professional, with or without the corresponding threshold values, in order to allow the medical professional to determine whether correction is needed).
Claims 18 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Carter ‘425 (US 20210074425).
Regarding claim 18, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Carter ‘425 teaches:
Presenting the projected model of the dental site (Paragraph 50, pathologies identified in the displayed image);
Receiving a selection of a dental condition of interest (Paragraph 50, a number of pathologies 520 are listed and selectable by the user); and
Presenting a projected severity map of the plurality of projected severity maps that is associated with the selected dental condition of interest (Paragraph 51, labels may only be displayed as overlay content within the image for one or more particular pathologies or other conditions selected by the user from the lists; Vannahme, Page 19, paragraph 6, a thermal map or any other graphical representation generally superimposed on the patient tooth three-dimensional model, indicating the area specific severity value or severity level or other attribute, such as score value, development of rate or tooth condition).
Carter ‘425 and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Carter ‘425 and presenting the projected model of the dental site, receiving a selection of a dental condition of interest, and projecting the severity map associated with the selected dental condition of interest. Doing so would allow for dentists to distinguish severity of different conditions and easily visualize individual conditions.
Regarding claim 27, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination as presented previously fails to disclose the following, Vannahme further teaches:
Generating a plurality of additional projected severity maps of the one or more dental conditions (Page 19, paragraph 6, a thermal map or any other graphical representation generally superimposed on the patient tooth three-dimensional model, indicating the area specific severity value or severity level or other attribute, such as score value, development of rate or tooth condition) from a plurality of additional images (Mason, Paragraph 91, one or more two-dimensional images);
Vannahme and the combination of Mason, Zhang, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, and Carter by using Vannahme and generating additional severity maps. In turn, because the above steps are repeated as claimed has properties predicted by the prior art, it would have been obvious to generate an additional severity map. Doing so would allow for more accurate severity information by comparing the severity maps.
While the combination fails to disclose the following, Carter ‘425 teaches:
Resolving differences between the projected severity map and the plurality of additional projected severity maps using a voting algorithm (Paragraph 33, two or more machine learning models may be trained to detect the same or overlapping sets of potential pathologies, the medical image analysis system 120 may be configured to apply a voting methodology or other resolution process to determine an ultimate classification result).
Carter ‘425 and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. and the Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Carter ‘425 and using a voting algorithm to determine differences between severity maps. Doing so would allow for an efficient way to resolve discrepancies between the maps.
Claims 4 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Brown (US 20210074061).
Regarding claim 4, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination as presented previously fails to disclose the following, Vannahme further teaches:
Wherein the model is a 3D model of the dental site (Page 19, paragraph 6, patient tooth three-dimensional model).
Vannahme and the combination of Mason, Zhang, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, and Carter by using Vannahme using a 3D model of a dental site. Doing so would allow for dentists to more easily visualize the dental site and conditions.
While the combination fails to disclose the following, Brown teaches:
Generating the 3D model of the dental site using the plurality of intraoral scans (Paragraph 74, process 2D images captured at the scanner/camera 1904, generate 3D dental models using the 2D images).
Brown and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Vannahme, and Carter by using Brown and generating the 3D model from 2D scan images. Doing so would allow an efficient way of creating a 3D model using 2D images rather than using a 3D model system such as casts or molds.
Regarding claim 7, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Brown teaches:
Wherein the image comprises one of a two-dimensional (2D) color image generated by an intraoral scanner (Paragraph 71, the scanner/camera 1904 captures data about color inputs), a 2D near infrared image generated by the intraoral scanner, or a 2D color image generated by an image sensor of a device other than an intraoral scanner.
Brown and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Brown and using a color image generated by an intraoral scanner. Doing so would allow for more refined data and detail when generating the 3D model.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Minchenkov (US 20200349698).
Regarding claim 5, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1, the operations further comprising:
Receiving an intraoral scan of the dental site that is associated with the image (Mason, Paragraph 85, images depicting the positions and orientations of the patient's teeth, gingiva, etc.);
While the combination fails to disclose the following, Minchenkov teaches:
Wherein the presence of the plurality of dental conditions is estimated by inputting the image and the intraoral scan into a trained machine learning model that outputs a probability of the dental site containing one or more of the plurality of dental conditions (Paragraph 5, wherein the trained machine learning model outputs a probability map comprising, for each pixel in the intraoral image, a first probability that the pixel belongs to a first dental class and a second probability that the pixel belongs to a second dental class).
Minchenkov and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Minchenkov and using machine learning to predict the probability that dental conditions are present in a dental site. Doing so would allow dentists to quickly and more accurately identify areas of various dental conditions in a dental site.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Luengo Hendriks (US 20200134814).
Regarding claim 8, the combination of Mason, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Luengo Hendriks teaches:
Wherein estimating the severity level of each of the plurality of dental conditions at one or more locations in the image comprises: Taking a derivative of the estimation with respect to input pixel intensities of the image (Paragraph 6, calculating an intensity value for each pixel in the digital image, then calculating a second order derivative value for each pixel… determining a patient status for the patient associated with the tissue sample based on the detected dot(s)), wherein severity is a function of the derivative (Paragraph 15, patient status includes diagnosis of disease state, disease severity).
Luengo Hendriks and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of medical condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Luengo Hendriks and using severity as a function of a derivative of pixel intensity. Doing so would utilize an efficient algorithm to determine severity in medical images.
Claims 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Sun (An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection, August 2015, Journal of Latex Class Files, Vol. 14, No. 8).
Regarding claim 9, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Sun teaches:
Wherein estimating the severity level of each of the plurality of dental conditions at one or more locations in the image comprises:
Erasing a region of the image (Page 4, B. Implementation, mapping that isolates and removes the lesion);
Generating a modified image by processing the image with the erased region by a machine learning model trained to generate images of healthy dental sites, wherein the machine learning model fills in data for the erased region of the image (Page 2, Paragraph 1, develop an “abnormal-to-normal translation generative adversarial network” (ANT-GAN) model to predict what a lesion-free image should look like that corresponds to an input image; Page 7, Paragraph 1, corresponding generated lesion-free MRI slice GA2N(x) from an already-trained ANT-GAN);
Estimating the presence of one or more of the plurality of dental conditions in the modified image (Page 7, Column 2, Paragraph 1, if the input image contains a lesion or not); and
Determining a change in the estimation of the presence of the one or more of the plurality of dental conditions between the modified image and the image (Page 6, C. Practical Applications, Paragraph 2, Since ANTGAN is trained to isolate abnormal tumor regions and fix those regions only, the difference between an input image x and output image GA2N(x) can be used to segment MRI for areas of potential concern).
Sun and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of medical condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Vannahme, and Carter by using Sun and erasing a region of the image, generating a modified image using a machine learning model trained on healthy dental sites, estimating the presence of a dental condition in the generated image, and comparing to the original image. Doing so would provide a “normal” counterpart to a medical image which can provide useful side information for medical imaging tasks (Sun, Abstract).
Regarding claim 10, the combination of Mason, Zhang, Vannahme, Carter, and Sun teaches the non-transitory computer readable medium of claim 9. While the combination as presented previously fails to disclose the following, Sun further teaches:
Wherein the machine learning model is a generative adversarial network (GAN) (Sun, Page 2, Paragraph 1, develop an “abnormal-to-normal translation generative adversarial network” (ANT-GAN) model).
Sun and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of medical condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Sun and using a GAN. Doing so would utilize a well-known machine learning model that will effectively generate the desired image modifications.
Regarding claim 11, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination as presented previously fails to disclose the following, Vannahme further teaches:
Wherein estimating the severity level of each of the plurality of dental conditions at one or more locations in the image comprises:
Generating the severity map based on the differences between the image and the reconstructed image (Page 19, paragraph 6, a thermal map or any other graphical representation generally superimposed on the patient tooth three-dimensional model, indicating the area specific severity value or severity level or other attribute, such as score value, development of rate or tooth condition; Mason, Paragraph 38, A predictive method can involve comparing surface scan data and/or sub-surface data of the teeth at a plurality of time points in order to determine changes to the position and/or shape of the teeth over time).
Vannahme and the combination of Mason, Zhang, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, and Carter by using Vannahme and generating additional severity maps. In turn, because the above steps are repeated as claimed has properties predicted by the prior art, it would have been obvious to generate an additional severity map. Doing so would allow for more accurate severity information by comparing the severity maps.
While the combination fails to disclose the following, Sun teaches:
Inputting the image into a trained machine learning model that generates a feature vector representing the image and reconstructs the image from the feature vector (Page 2 Column 2, Paragraph 1, The data distribution of brain MRI of healthy subjects are learned using an auto-encoder with unsupervised learning where the constraint that real lesion-containing medical data and its corresponding underlying lesion-free counterpart lie closely in latent space is also imposed), the trained machine learning model having been trained on images lacking the plurality of dental conditions (Page 2, Paragraph 1, develop an “abnormal-to-normal translation generative adversarial network” (ANT-GAN) model to predict what a lesion-free image should look like that corresponds to an input image; Page 7, Paragraph 1, corresponding generated lesion-free MRI slice GA2N(x) from an already-trained ANT-GAN); (Note: the broadest reasonable interpretation of the feature vector can encompass an image, which can be defined as a multi-dimensional vector, with each pixel being defined by one or more dimensions (pixel values), and two other dimensions providing the x and y locations of the pixel in the image.)
Determining differences between the image and the reconstructed image (Page 6, C. Practical Applications, Paragraph 2, Since ANTGAN is trained to isolate abnormal tumor regions and fix those regions only, the difference between an input image x and output image GA2N(x) can be used to segment MRI for areas of potential concern);
Sun and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of medical condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Sun and using a machine learning model trained on images lacking a dental condition and comparing the output of that model to the original input image with the dental condition. Doing so would allow a dentist to understand the difference between what a healthy dental site should look like and the current image.
Regarding claim 12, the combination of Mason, Zhang, Vannahme, Carter, and Sun teaches the non-transitory computer readable medium of claim 11. While the combination as presented previously fails to disclose the following, Sun further teaches:
Wherein for each location of the image a degree of difference between the image and the reconstructed image at the location provides the severity level of one or more of the plurality of dental conditions at the location (Page 6, C. Practical Applications, Paragraph 2, Since ANTGAN is trained to isolate abnormal tumor regions and fix those regions only, the difference between an input image x and output image GA2N(x) can be used to segment MRI for areas of potential concern; Page 7, Paragraph 2, since the only difference between a synthesized normal-looking image and its real abnormal counterpart is region with the lesion. To illustrate this, we calculate the absolute difference between x and GA2N(x) and show the segmentation after binary thresholding at 0.1 in Figure 9.).
Sun and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of medical condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Sun and determine severity level based on the difference between the image and reconstructed image. Doing so would allow for an efficient and low cost method of calculating severity.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Jellinggaard (US 20210264600).
Regarding claim 13, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Jellinggaard teaches:
Wherein the image is a near infrared image and the plurality of dental conditions comprises caries (Paragraph 8, the inside of a tooth, i.e. approximal caries lesions in enamel, can be imaged using infrared (IR) light).
Jellinggaard and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Jellinggaard and using an infrared image of caries. Doing so would allow for detecting caries inside the tooth and Light in the IR domain is suitable for providing information about caries inside the teeth (Jellinggaard, Paragraph 57).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Li (US 20240324868).
Regarding claim 14, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Li teaches:
Wherein the image is a color image and the plurality of dental conditions comprises gum inflammation (Paragraph 20, The intraoral cameras may allow users to capture (e.g., efficiently capture) color images and/or video of the oral cavity of the user; Paragraph 28, Camera 104 may provide qualitative information about tissue health of an oral cavity. For example, inflammation in tissue (such as gum tissue) may be captured).
Li and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Li and using a color image of gum inflammation. Doing so would allow for capturing gum inflammation as it is external and color helps determine the severity of inflammation (Li, Paragraph 39, using color ranges to perform gum image thresholding).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Sano (US 20220245919).
Regarding claim 19, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Sano teaches:
Wherein estimating the severity level of each of the plurality dental conditions (Paragraph 27, oral cavity cancer (lingual cancer, gum cancer, floor of the mouth cancer) or the like, a disease name and the like of the target can be identified based on an image of the target being the diagnostic target) at one or more locations in the image is performed using a gradient-weighted class activation mapping (Grad-CAM) algorithm (Paragraph 123, for example, gradient-weighted class activation mapping (Grad-CAM), guided grad-CAM, or the like).
Sano and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Sano and using Grad-CAM to estimate the severity of dental conditions. Doing so would allow for efficient and transparent reasoning when classifying parts of the images.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Subhash (US 20220240786).
Regarding claim 20, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Subhash teaches:
Wherein the severity map comprises a heat map (Paragraph 62, modify the image in a heat-map or other color coding map to indicate score or severity of a condition or tissue health for ease of visual understanding by the user).
Subhash and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Subhash and using a heat map for the severity map. Doing so would allow for ease of visual understanding by the user (Subhash, Paragraph 62).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Yip (JP 7548932) and further in view of Subhash (US 20220240786).
Regarding claim 21, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Yip teaches:
Wherein estimating the severity level of each of the plurality of dental conditions at one or more locations in the image comprises:
Dividing the image into a plurality of overlapping patches, wherein each pixel of the image contributes to more than one of the plurality of overlapping patches (Page 37, Paragraph 6, In one example, two adjacent small square tiles share an edge and each is at the center of a medium square tile. The two medium square tiles overlap. Of the 466*466 small pixels in each medium square tile, the two medium square tiles share all but 32*466 pixels); While Yip teaches the example that there are pixels excluded from the overlap, it can easily be modified to include all pixels in the image in the overlapping patches while achieving the same result.
For each patch of the plurality of overlapping patches, processing the patch using a model that outputs a probability of that patch containing one or more of the plurality of dental conditions (Page 37, Paragraph 6, analyzes both medium square areas simultaneously, such that the algorithm produces two vectors of values, one for each of the two small square tiles; Page 37, Paragraph 7, The value vector contains a probability value for each tissue class label, indicating the likelihood that the small square tile represents that tissue class);
Yip and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of medical condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Yip and dividing the image into a plurality of overlapping patches and outputting a probability that the patch contains a dental condition. Doing so would allow for more efficiency when analyzing the patches (Page 37, Paragraph 8, By analyzing both medium square tiles simultaneously, the algorithm increases efficiency).
While the combination fails to disclose the following, Subhash teaches:
For each pixel of the image, determining a severity level of the one or more dental conditions at the pixel based on a combination of probabilities of patches that include the pixel containing the one or more of the plurality of dental conditions (Paragraph 62, Classifications may be pixel-by-pixel classifications… Classifications may be classifications based on sets of binned pixels… indicate score or severity of a condition). Classification may be determined by binned pixels (patches), which can identify the probability of a condition present at the pixel.
Subhash and the combination of Mason, Zhang, Vannahme, Carter, and Yip are both considered to be analogous to the claimed invention because they are in the same field of medical condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, Carter, and Yip by using Subhash and determining a severity level for each pixel based on a combination of probabilities of patches. Doing so would allow for an efficient way to resolve discrepancies between the patches and determine a probability through multiple trials.
Claim 28 is rejected under 35 U.S.C. 103 as being unpatentable over Mason in view of Zhang and further in view of Vannahme and further in view of Carter as applied to claims 1-2, 6, 16-17, 22-26, 29 and 30 above, and further in view of Ashida (US 20240285171).
Regarding claim 28, the combination of Mason, Zhang, Vannahme, and Carter teaches the non-transitory computer readable medium of claim 1. While the combination fails to disclose the following, Ashida teaches:
Wherein the model comprises a panoramic image of the dental site (Paragraph 71, makes or aids determination as to whether there is any disease such as caries, demineralization, adjacent surface caries, microcracks, microvoids, calculus, or dental plaque, on each tooth, on the basis of an image (including an 3D image or a panoramic image).
Ashida and the combination of Mason, Zhang, Vannahme, and Carter are both considered to be analogous to the claimed invention because they are in the same field of dental condition imaging. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Mason, Zhang, Vannahme, and Carter by using Ashida and using a panoramic image of the dental site. Doing so would allow capturing of a larger and more detailed surface area in one image.
Response to Amendment
Applicant’s arguments with respect to claims 1, 29 and 30 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Zhang teaches the new limitation in the independent claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SNIGDHA SINHA whose telephone number is (571)272-6618. The examiner can normally be reached Mon-Fri. 12pm-8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SNIGDHA SINHA/Examiner, Art Unit 2619
/JASON CHAN/Supervisory Patent Examiner, Art Unit 2619