DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 is objected to because of the following informalities: the claim recites “performing preprocessing on at least one image.” However, “at least one image” has already been introduced with “capturing at least one image of a face of a user.” It appears to be a typographical error. Appropriate correction is required.
The Examiner recommends amending the limitation to be “performing preprocessing on the at least one image.”
Claim 1 is objected to because of the following informalities: the claim recites “applying the at least one image and the selection to one or more machine learning algorithms.” It is unclear whether “the at least one image” refers to the at least one image before or after the claimed performed preprocessing, although Applicant’s intention appears to recite “applying the at least one preprocessed image.” Otherwise, it would be unclear about the purpose or relevance of the preprocessing within the claimed invention. Appropriate correction/clarification is required.
The Examiner recommends amending the limitation to be “applying the at least one preprocessed image and the selection to one or more machine learning algorithms.”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites “generating a prop.” However, the specification does not appear to provide any context/description about the feature; the claimed feature does not appear to be related to the body of the claim in any fashion; and it is difficult to discern whether to give patent weight to the limitation.
The specification never uses the term “prop,” and the only occurrence of term is in the claim.
The specification does recite “Once the mock-up is generated, the patient and dental professionals can preview the proposed design.” Spec. p. 8. Here, do we have “prop” for “proposed design”? There is no sufficient level of confidence in the understanding.
According to Webster dictionary, prop means “something that props or sustains: support.” There is no clear connection between “generating a prop” and the claim body, which appears to focus on generating and outputting “one or more procedures required to transform the smile.”
Further, MPEP states, “If the claim preamble, when read in the context of the entire claim, recites limitations of the claim, or, if the claim preamble is ‘necessary to give life, meaning, and vitality’ to the claim, then the claim preamble should be construed as if in the balance of the claim." MPEP 2111.02. Due to the insufficient level of confidence and the disconnect between the preamble and the body of the claim, it is difficult to discern whether “the claim preamble is ‘necessary to give life, meaning, and vitality’ to the claim, then the claim preamble should be construed as if in the balance of the claim."
Therefore, the claim is indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
For the purposes of art rejection, the Examiner is taking the position that if the requirements in claim body are satisfied, and requirement of the preamble is also satisfied.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Scahdeva et al. (US 20200066391 A1) in view of AWS (“What is machine learning?”).
Regarding Claim 1, Scahdeva teaches A method of generating a prop (
[BRI on the record] With respect to “generating a prop,” the Examiner has found the limitation indefinite. Detailed explanation has been provided in the Examiner’s 112(b) rejection.
For the purposes of art rejection, the Examiner is taking the position that if the requirements in the claim body are satisfied, requirement of the preamble is also satisfied.), comprising:
capturing at least one image of a face of a user (
Scahdeva Fig. 14a 1401a:
PNG
media_image1.png
224
358
media_image1.png
Greyscale
“The input capturing device may include such as a scanner, an image capturing device like a camera, an input capture mechanism available on a mobile device, and other similar technologies available for capturing multiple 2D images of the user. In an example, the image capturing technology may include capturing patient's facial morphology using computer vision technology, such as using free open source tools for facial recognition and facial orientation detection. In some example embodiments, computer vision may be used in combination with the image capturing technology to ensure that the user takes pictures of the all appropriate orientations (facing front, right-side, left-side) required by a system, such as the orthodontic care management platform 102a.” Scahdeva ¶ 183.);
performing preprocessing on at least one image (
Scahdeva Fig. 14a 1402a, 1043a. “Once patient data is successfully captured, the method 1400 may proceed to, at step 1402, to analyze patient morphology.” Scahdeva ¶ 183. “In some example embodiments, the modification to the patient data may include morphological analysis and 2D-to-3D conversion, such as by building 3D models using image conversion tools known in the art. For example, OpenCV may be used to perform morphological analysis of the patient's face by extracting pertinent features from the 3D model and showing the user how their face is likely to change with the proposed orthodontic treatment. Further, after conversion, the image may be subjected to deep learning using tools known in the art, such as Tensor flow (www.tensorflow.org), to provide the machine learning capabilities required for such analysis and projection. Convolutional neural networks (CNN), well-known to those skilled in the art, are one of the examples of several deep learning algorithms that can be applied to this task. The machine learning techniques such as CNN may include analysis of various factors, such as a patient's view of their own attractiveness, which may be used to determine their attractiveness preference (affective sense) prior to the start of treatment, so that the treatment can be patterned to meet their objectives.” Scahdeva ¶ 184.
Here, the preprocessing could be mapped to morphological analysis and/or 2D-3D conversion.
This mapping is consistent with the specification, which states, “ The dental imaging system operates to create smile mockups. The system uses facial and dental features from captured images generated by a user (or prospective patient), 3D image processing technology, and a robust library of desirable smiles to create an enhanced smile mockup.” Spec. ¶ 11.);
receiving a selection of one smile template by the user (
PNG
media_image2.png
346
752
media_image2.png
Greyscale
Scahdeva teaches determining a patient’s attractiveness preference by selecting smile template, stating “FIG. 23 illustrates an exemplary user interface for smile selection, in accordance with an embodiment of the present invention.” Scahdeva ¶ 97. “The smile customization may be performed based on automatic and interactive smile recommendations provided to the patient based on the collected patient data. In some example embodiments, the patient may be able to specify a smile type, such as a smile similar to that of a celebrity, based patient data.” Scahdeva ¶ 120.);
applying the at least one image and the selection to one or more
In summary, the above collected patient data includes captured patient image (at least one image), patient image after morphology and/or 3D analysis (at least one image), and the selection of patient preference (selection).
“The data collection service may include such as patient data collection. The patient data may include patient's demographic data, patient image captured, . . ..” Scahdeva ¶ 119.
“In some example embodiments, the modification to the patient data may include morphological analysis and 2D-to-3D conversion, such as by building 3D models using image conversion tools known in the art.” Scahdeva ¶ 184.
“In some example embodiments, the user interface for patient data collection may also provide options for smile customization for the patient. . . . In some example embodiments, the patient may be able to specify a smile type, such as a smile similar to that of a celebrity, based patient data.” Scahdeva ¶ 119.
Further, the collected patient data are used by computer algorithms to propose orthodontic treatment plans, stating “In some example embodiments, the user interface for patient data collection may also provide a display of recommended care plan for the patient based on the collected patient data.” “Once patient's preferences are factored in, the method 1400a may include, at step 1404a, generating personalized care plan and precision therapy for the patient. This may be achieved such as by using built in algorithms, such as within a processing module in the orthodontic care management platform 102a, wherein the built in algorithms may be configured to provide an augmented reality toolkit, a plan generator and other built in functions suitable for orthodontic care processing. The personalized care plan generated in this manner may be presented to the patient, such as using the UI unit 102a-7 of the orthodontic care management platform 102a..” Scahdeva ¶ 185.
Finally, Scahdeva teaches visualization the proposed orthodontic treatment plans, stating “For example, OpenCV may be used to perform morphological analysis of the patient's face by extracting pertinent features from the 3D model and showing the user how their face is likely to change with the proposed orthodontic treatment. . . . The machine learning techniques such as CNN may include analysis of various factors, such as a patient's view of their own attractiveness, which may be used to determine their attractiveness preference (affective sense) prior to the start of treatment, so that the treatment can be patterned to meet their objectives.” Scahdeva ¶ 184.),
wherein the one or used to generate one or more modified images that transform a smile based on the selection of one smile template (
Scahdeva teaches generating and displaying modified images to visualize proposed orthodontic treatment based on a patient’s attractive preference, stating “OpenCV may be used to perform morphological analysis of the patient's face by extracting pertinent features from the 3D model and showing the user how their face is likely to change with the proposed orthodontic treatment. Further, after conversion, the image may be subjected to deep learning using tools known in the art, such as Tensor flow (www.tensorflow.org), to provide the machine learning capabilities required for such analysis and projection. Convolutional neural networks (CNN), well-known to those skilled in the art, are one of the examples of several deep learning algorithms that can be applied to this task. The machine learning techniques such as CNN may include analysis of various factors, such as a patient's view of their own attractiveness, which may be used to determine their attractiveness preference (affective sense) prior to the start of treatment, so that the treatment can be patterned to meet their objectives.” Scahdeva ¶ 184.
Scahdeva further teaches that the patient’s attractive preference could be based on selection of one smile template, stating “In some example embodiments, the user interface for patient data collection may also provide options for smile customization for the patient. . . . In some example embodiments, the patient may be able to specify a smile type, such as a smile similar to that of a celebrity, based patient data.” Scahdeva ¶ 119. ) and
used to output one or more procedures required to transform the smile (
“In some example embodiments, the user interface for patient data collection may also provide a display of recommended care plan for the patient based on the collected patient data.” “Once patient's preferences are factored in, the method 1400a may include, at step 1404a, generating personalized care plan and precision therapy for the patient. This may be achieved such as by using built in algorithms, . . ..” Scahdeva ¶ 185. “. . . which may be used to determine their attractiveness preference (affective sense) prior to the start of treatment, so that the treatment can be patterned to meet their objectives.” Scahdeva ¶ 184. “In some example embodiments, the user interface for patient data collection may also provide options for smile customization for the patient. . . . In some example embodiments, the patient may be able to specify a smile type, such as a smile similar to that of a celebrity, based patient data.” Scahdeva ¶ 119.
Scahdeva also suggests the use of machine learning model, stating “Further, the module SC1 1801 may provide a plurality of care management options. The care management options may include screening the patient's ability to self-manage their own care based upon factors that include but are not limited to the patient's desires, severity of malocclusions, and cost. Furthermore, SC1 1801 may also provide the patient with other care management approaches that may include: a hybrid approach involving limited professional supervision at identified points of care, or comprehensive professional management through the entire care cycle, involving regular doctor visits. The determination of the appropriate care management path can also be accomplished by machine learning techniques. For example, a database of dento-facial images may be compiled. Each image may then be labelled based upon the appropriate care management approaches including but not limited to self-care management, hybrid or total professional management. Further, a neural network may be trained using labelled data to classify the dento-facial images based on a care management path. When the patient's dento-facial image is presented to such a trained network, it may be able to determine the recommended care management path, with a reasonably high accuracy.” Scahdeva ¶ 222.); and
transmitting the one or more modified images and the one or more procedures to the user (
“OpenCV may be used to perform morphological analysis of the patient's face by extracting pertinent features from the 3D model and showing the user how their face is likely to change with the proposed orthodontic treatment.” Scahdeva ¶ 184.
“In some example embodiments, the user interface for patient data collection may also provide a display of recommended care plan for the patient based on the collected patient data. For example, the user interface may provide recommendation for an orthodontic care plan based on matching of patient' facial features against similar proportioned treated patient.” Scahdeva ¶ 120.).
Scahdeva teaches the use of algorithms based on claimed inputs to produce claimed outputs. Scahdeva also teaches the use of machine learning models for various features. However, Scahdeva is not sufficiently specific/clear regarding the inputs to and outputs from its machine learning models.
Therefore, Scahdeva does not explicitly disclose the disclosed algorithms are implemented by machine learningalgorithms based on claimed inputs and outputs, and the machine learningalgorithms are trained to be used.
AWS teaches a machine learningalgorithms could be based on claimed inputs and outputs and trained to be used (
AWS states, “The central idea behind machine learning is an existing mathematical relationship between any input and output data combination.” AWS p. 2.
PNG
media_image3.png
118
480
media_image3.png
Greyscale
AWS p. 3.
PNG
media_image4.png
222
476
media_image4.png
Greyscale
AWS p. 3 (teaching learning/training).
Since Scahdeva teaches the inputs and outputs of recommendation algorithms, after the combination of Scahdeva and AWS, machine learnings algorithms may be used to generate the specific output based on the specific inputs as taught by Scahdeva. In particular, AWS states, “While this is basic understanding, machine learning focusses on the principle that all complex data points can be mathematically linked by computer systems . . ..” AWS p. 3.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine AWS machine learning solutions with Scahdeva’s orthodontic treatment recommendation system. One of ordinary skill in the art would be motivated to make the system more intelligent, easier to maintain, and cheaper to create. “Machine learning is the science of developing algorithms and statistical models that computer systems use to perform tasks without explicit instructions, relying on patterns and inference instead. Computer systems use machine learning algorithms to process large quantities of historical data and identify data patterns. This allows them to predict outcomes more accurately from a given input data set.” AWS p. 1.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cofar et al. (US-20210393380-A1):
PNG
media_image5.png
776
524
media_image5.png
Greyscale
“It is an object of embodiments of the present invention to show a photo-realistic image of how the smile of the patient will look like after dental treatment, and for adjusting this image in a fast and efficient manner.” Cofar ¶ 9.
Chekh et al. (US-20190350680-A1):
PNG
media_image6.png
464
750
media_image6.png
Greyscale
“In order to design an aesthetically pleasing smile, the distance between the target position of the inferior incisal edge point of the upper central incisors 1960 and the superior border of the lower lip intersecting the facial midline 2010 may be less than or equal to 1 millimeter. In some embodiments, this distance may be less than or equal to 2 mm, for example, when the patient's lower lip is a dynamic or V-shaped lip in a social smile expression. In the treatment planning process, the teeth may be moved such that in their final positions, they are these threshold distances.” Chekh ¶ 289.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHENGXI LIU whose telephone number is (571)270-7509. The examiner can normally be reached M-F 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHENGXI LIU/Primary Examiner, Art Unit 2611