Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
A human machine interface unit in claim 15 See page 9 “HMI system may comprise a visual display or screen for displaying visual images or video.”
A human machine interface unit in claim 22 See page 9 “HMI system may comprise a visual display or screen for displaying visual images or video.”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 14-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over “3D intraoral scanning system using fixed pattern mask and tunable-focus lens” Joo Beom Eom et al 2020 Meas. Sci. Technol. 31 015401 in view of Kopelman US 2018/0168780.
Re claim 1
Joo Beom Eom discloses
A computer-implemented method for visualizing a dental condition, comprising (section 4 paragraph 5 note that a processor was used to perform the method):
projecting a pattern using an intraoral scanner on to the dental condition (see abstract “The scanner uses a tunable-focus lens for depth scanning and a pattern mask for projecting structured light onto the sample”);
providing via the intraoral scanner, a plurality of projection images of a reflected pattern from the dental condition; (see abstract “By fixing the pattern mask, a camera installed at the same focal point as the sample focus acquired an image at each focal plane, resulting in an improvement in the scanning speed. The proposed method performs optical sectioning on sequentially acquired images while varying the focal planes by estimating pattern modulation in the spatial domain; it then reconstructs 3D point clouds by stacking up the optically sectioned image”)
rendering, color information to the one or more anatomical features in the said plurality of projection images to generate an at least partially artificially colorized projection images (see abstract “The fixed pattern mask, however, caused a color loss at the pixels where the patterns are projected. The proposed method solves this problem by applying a low-pass filter and a polynomial interpolation to fill the loss.” Note the missing color information is “artificially” filed using interpolation to determine the lost color values), wherein the pattern is removed through the rendering (see section 4 third paragraph “However, as the proposed scanner system uses a fixed pattern mask, the generated color map contains pattern information; thus, that color information is lost, as shown in figure 7. Figure 7(a) is a region where we tested, to verify the color distortion. Figure 7(b) shows intensity profiles on the line segment (red line in figure 7(b)) perpendicular to the grid pattern in the image for the region (figure 7(a)). The image included the repeated sinusoidal pattern. To remove the pattern, the proposed system used the mean filter, which is one of the low-pass filters, on each line segment as follows” note that the median filter is used to remove the grid pattern).
Does not expressly disclose
detecting one or more anatomical features in each of the projection images;
rendering via a machine learning method, color information to the one or more anatomical features in the said plurality of projection images to generate an at least partially artificially colorized projection images, wherein the pattern is removed through the rendering.
In a similar filed of endeavor of dental imaging, Kopelman discloses
detecting one or more anatomical features in each of the images; (see paragraph 65 note that areas of interest may be found from the images of the oral cavity)
rendering via a machine learning method, (see paragraph 184 note that machine learning is used to analyze the dental conditions see also paragraph 65 “The analysis may involve direct analysis (e.g., pixel-based and/or other point-based analysis), the application of machine learning, the application of image registration, and/or the application of image recognition. The AOI identifying modules 115 may identify areas of interest directly from the image data 135 received from the image capture device 160 or based on a comparison of the received image data 135 and reference data 138 or previous patient data 140. For example, an AOI identifying module 115 may use one or more algorithms or detection rules to analyze the shape of a tooth, color of a tooth, position of a tooth, or other characteristics of a tooth to determine if there is any AOI that should be highlighted for a dental practitioner” note machine learning is used to identify areas of interest) color information to the one or more anatomical features in the said plurality of images to generate an at least partially artificially colorized images (see paragraph 148 note that areas of interest may be highlighted with different color i.e to create artificially colorized images.) The motivation to combine is to “highlight AOIs or other elements on an AR display” (see paragraph 148). One or ordinary skill in the art could have easily used the method/system of Kopelman to highlight features in the method/system of Joo Beom Eom. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Re claim 14 Joo Beom Eom further discloses building, using at least some of the said plurality of projection images, a visualized 3D model of the dental condition (see section 3 and figure 3 note that a 3d point cloud is rendered using the images), - rendering, color information to one or more anatomical features in the visualized 3D model to generate an at least partially artificially colorized 3D model (see section 4 figure 10 note that a colorized version of the dental image is generated), wherein the pattern information is removed through the rendering (see section 4 and figure 6 note that pattern information is removed).
Joo Beom Eom does not expressly disclose rendering, via a machine learning method, color information to one or more anatomical features. Kopelman discloses rendering, via a machine learning method, (see paragraph 184 note that machine learning is used to analyze the dental conditions see also paragraph 65 “The analysis may involve direct analysis (e.g., pixel-based and/or other point-based analysis), the application of machine learning, the application of image registration, and/or the application of image recognition. The AOI identifying modules 115 may identify areas of interest directly from the image data 135 received from the image capture device 160 or based on a comparison of the received image data 135 and reference data 138 or previous patient data 140. For example, an AOI identifying module 115 may use one or more algorithms or detection rules to analyze the shape of a tooth, color of a tooth, position of a tooth, or other characteristics of a tooth to determine if there is any AOI that should be highlighted for a dental practitioner” note machine learning is used to identify areas of interest) color information to one or more anatomical features ( see paragraph 148 note that areas of interest may be highlighted with different color i.e. to create artificially colorized images.) The motivation to combine is to “highlight AOIs or other elements on an AR display” (see paragraph 148). One or ordinary skill in the art could have easily used the method/system of Kopelman to highlight features in the method/system of Joo Beom Eom. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Re claim 15 Joo Beom Eom further discloses displaying, at least some of the partially artificially colorized projection images and/or the at least partially artificially colorized 3D model (see figure 10 and figure 6 note that artificially colorized images are displayed in the figure). While Joo Beom Eom almost certainly intends the image to be displayed via a human machine interface, Joo Beom Eom does not expressly disclose displaying via a human machine interface. Kopelman discloses displaying via a human machine interface (see paragraph 148 “In the examples, particular colors or types of indicators may be used to highlight AOIs or other elements on an AR display.”) The motivation to combine is to “highlight AOIs or other elements on an AR display” (see paragraph 148). One or ordinary skill in the art could have easily used the method/system of Kopelman to display images in the method/system of Joo Beom Eom. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Re claim 16 Joo Beom Eom further discloses projection images and a 3d model wherein the pattern information is removed through the rendering (see section 4 third paragraph “However, as the proposed scanner system uses a fixed pattern mask, the generated color map contains pattern information; thus, that color information is lost, as shown in figure 7. Figure 7(a) is a region where we tested, to verify the color distortion. Figure 7(b) shows intensity profiles on the line segment (red line in figure 7(b)) perpendicular to the grid pattern in the image for the region (figure 7(a)). The image included the repeated sinusoidal pattern. To remove the pattern, the proposed system used the mean filter, which is one of the low-pass filters, on each line segment as follows” note that the median filter is used to remove the grid pattern. see also figure 10 note that a colorized 3d model is built). Joo Beom Eom does not disclose incrementally building the visualized 3D model further in response to further projection images captured via the intraoral scanner, incrementally rendering color information to correspondingly incrementally built visualized 3D model to further build the at least partially artificially colorized 3D model.
Kopelman further discloses incrementally building the visualized 3D model further in response to further projection images captured via the intraoral scanner (see paragraph 242-245 and figure 23 steps 2330-2360 note that new images from the intraoral scanner are continuously processed to update 3d model and results are displayed in the VR display ), incrementally rendering color information to correspondingly incrementally built visualized 3D model to further build the at least partially artificially colorized 3D model (see paragraph 242-245 and figure 23 steps 2330-2360 note that new images from the intraoral scanner are continuously processed to update 3d model and results are displayed in the VR display). The motivation to combine is “he virtual 3-D model is then updated to incorporate the new image data. Accordingly, the virtual 3-D model may grow and become more complete as the patient's dental arch is scanned.” see paragraph 235. One of ordinary skill in the art could have easily modified the teachings of Joo Beom Eom with the teachings of Kopelman to generate more complete renderings. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Re claim 17 Joo Beom Eom wherein the at least partially artificially colorized 3D model is displayed (see figure 10 and figure 12 note the 3d model is display in the figure), wherein the rendering of color information in the visualized 3D model is performed dependent upon a perspective view (see section 4 first paragraph “Figure 6 shows a picture of the dental cast (figure 6(a)), and 3D point clouds (figures 6(b)–(e)) acquired from different points of view. The field of view (FOV) of the proposed system was 10.7 mm × 10.7 mm, where the pixel size of the camera is 7.6 µm × 7.6 µm and the size of images acquired by the camera is 1408 × 1408 pixels. The depth FOV was 10 mm, and 50 images were acquired by varying the focal planes at a uniform interval.” Section 5 “the full arch of the dental cast was scanned by merging partially overlapped point clouds that are scanned from different points of view” note that rendering is based on capturing images from different points of view.
Re claim 18 Joo Beom Eom discloses wherein rendering of the color information in the projection images and/or the visualized 3D model involves at least one image correction operation (see section 4 third paragraph “However, as the proposed scanner system uses a fixed pattern mask, the generated color map contains pattern information; thus, that color information is lost, as shown in figure 7. Figure 7(a) is a region where we tested, to verify the color distortion. Figure 7(b) shows intensity profiles on the line segment (red line in figure 7(b)) perpendicular to the grid pattern in the image for the region (figure 7(a)). The image included the repeated sinusoidal pattern. To remove the pattern, the proposed system used the mean filter, which is one of the low-pass filters, on each line segment as follows” note that the median filter is used to remove the grid pattern which is a correction).
Re claim 19 Joo Beom Eom discloses method uses a sequence of projection images for performing the rendering of color information (see abstract “The proposed method performs optical sectioning on sequentially acquired images while varying the focal planes by estimating pattern modulation in the spatial domain; it then reconstructs 3D point clouds by stacking up the optically sectioned image” note that a sequence of images are used). Joo Beom Eom does not express rendering via machine learning method. Kopelman discloses rendering via machine learning method (see paragraph 184 note that machine learning is used to analyze the dental conditions see also paragraph 65 “The analysis may involve direct analysis (e.g., pixel-based and/or other point-based analysis). The motivation to combine is to “highlight AOIs or other elements on an AR display” (see paragraph 148). One or ordinary skill in the art could have easily used the method/system of Kopelman to highlight features in the method/system of Joo Beom Eom. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Re claim 20 Joo Beom Eom discloses anon-transitory computer-readable storage medium storing the program, comprising instructions which when executed by one or more computing units cause any of the computing units to (see section 4 5th paragraph “we implemented a self-developed multithreaded C+ + software built and compiled in Microsoft Visual Studio 2015. To efficiently perform intraoral scanning, the software was run on an Intel Core i7-7800k CPU @ 3.50 GHz with 16 GB memory.”):
project a pattern using an intraoral scanner on to the dental condition (see abstract “The scanner uses a tunable-focus lens for depth scanning and a pattern mask for projecting structured light onto the sample”);
provide via the intraoral scanner, a plurality of projection images of a reflected pattern from the dental condition; (see abstract “By fixing the pattern mask, a camera installed at the same focal point as the sample focus acquired an image at each focal plane, resulting in an improvement in the scanning speed. The proposed method performs optical sectioning on sequentially acquired images while varying the focal planes by estimating pattern modulation in the spatial domain; it then reconstructs 3D point clouds by stacking up the optically sectioned image”)
render, color information to the one or more anatomical features in the said plurality of projection images to generate an at least partially artificially colorized projection images (see abstract “The fixed pattern mask, however, caused a color loss at the pixels where the patterns are projected. The proposed method solves this problem by applying a low-pass filter and a polynomial interpolation to fill the loss.” Note that the missing color information is “artificially” filed using interpolation to determine the lost color values), wherein the pattern is removed through the rendering (see section 4 third paragraph “However, as the proposed scanner system uses a fixed pattern mask, the generated color map contains pattern information; thus, that color information is lost, as shown in figure 7. Figure 7(a) is a region where we tested, to verify the color distortion. Figure 7(b) shows intensity profiles on the line segment (red line in figure 7(b)) perpendicular to the grid pattern in the image for the region (figure 7(a)). The image included the repeated sinusoidal pattern. To remove the pattern, the proposed system used the mean filter, which is one of the low-pass filters, on each line segment as follows” note that the median filter is used to remove the grid pattern).
Joo Beom Eom does not expressly disclose
detect one or more anatomical features in each of the projection images;
render via a machine learning method, color information to the one or more anatomical features in the said plurality of projection images to generate an at least partially artificially colorized projection images, wherein the pattern is removed through the rendering.
In a similar filed of endeavor of dental imaging, Kopelman discloses
detecting one or more anatomical features in each of the images; (see paragraph 65 note that areas of interest may be found from the images of the oral cavity)
rendering via a machine learning method, (see paragraph 184 note that machine learning is used to analyze the dental conditions see also paragraph 65 “The analysis may involve direct analysis (e.g., pixel-based and/or other point-based analysis), the application of machine learning, the application of image registration, and/or the application of image recognition. The AOI identifying modules 115 may identify areas of interest directly from the image data 135 received from the image capture device 160 or based on a comparison of the received image data 135 and reference data 138 or previous patient data 140. For example, an AOI identifying module 115 may use one or more algorithms or detection rules to analyze the shape of a tooth, color of a tooth, position of a tooth, or other characteristics of a tooth to determine if there is any AOI that should be highlighted for a dental practitioner” note machine learning is used to identify areas of interest) color information to the one or more anatomical features in the said plurality of images to generate an at least partially artificially colorized images ( see paragraph 148 note that areas of interest may be highlighted with different color i.e to create artificially colorized images.) The motivation to combine is to “highlight AOIs or other elements on an AR display” (see paragraph 148). One or ordinary skill in the art could have easily used the method/system of Kopelman to highlight features in the method/system of Joo Beom Eom. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Re claim 21 Joo Beom Eom discloses A system comprising a processor configured to: project a pattern using an intraoral scanner on to the dental condition (see section 4 5th paragraph “we implemented a self-developed multithreaded C+ + software built and compiled in Microsoft Visual Studio 2015. To efficiently perform intraoral scanning, the software was run on an Intel Core i7-7800k CPU @ 3.50 GHz with 16 GB memory.”):
project a pattern using an intraoral scanner on to the dental condition (see abstract “The scanner uses a tunable-focus lens for depth scanning and a pattern mask for projecting structured light onto the sample”);
provide via the intraoral scanner, a plurality of projection images of a reflected pattern from the dental condition; (see abstract “By fixing the pattern mask, a camera installed at the same focal point as the sample focus acquired an image at each focal plane, resulting in an improvement in the scanning speed. The proposed method performs optical sectioning on sequentially acquired images while varying the focal planes by estimating pattern modulation in the spatial domain; it then reconstructs 3D point clouds by stacking up the optically sectioned image”)
render, color information to the one or more anatomical features in the said plurality of projection images to generate an at least partially artificially colorized projection images (see abstract “The fixed pattern mask, however, caused a color loss at the pixels where the patterns are projected. The proposed method solves this problem by applying a low-pass filter and a polynomial interpolation to fill the loss.” Note that the missing color information is “artificially” filed using interpolation to determine the lost color values), wherein the pattern is removed through the rendering (see section 4 third paragraph “However, as the proposed scanner system uses a fixed pattern mask, the generated color map contains pattern information; thus, that color information is lost, as shown in figure 7. Figure 7(a) is a region where we tested, to verify the color distortion. Figure 7(b) shows intensity profiles on the line segment (red line in figure 7(b)) perpendicular to the grid pattern in the image for the region (figure 7(a)). The image included the repeated sinusoidal pattern. To remove the pattern, the proposed system used the mean filter, which is one of the low-pass filters, on each line segment as follows” note that the median filter is used to remove the grid pattern).
Joo Beom Eom does not expressly disclose
detect one or more anatomical features in each of the projection images;
render via a machine learning method, color information to the one or more anatomical features in the said plurality of projection images to generate an at least partially artificially colorized projection images, wherein the pattern is removed through the rendering.
In a similar filed of endeavor of dental imaging, Kopelman discloses
detecting one or more anatomical features in each of the images; (see paragraph 65 note that areas of interest may be found from the images of the oral cavity)
rendering via a machine learning method, (see paragraph 184 note that machine learning is used to analyze the dental conditions see also paragraph 65 “The analysis may involve direct analysis (e.g., pixel-based and/or other point-based analysis), the application of machine learning, the application of image registration, and/or the application of image recognition. The AOI identifying modules 115 may identify areas of interest directly from the image data 135 received from the image capture device 160 or based on a comparison of the received image data 135 and reference data 138 or previous patient data 140. For example, an AOI identifying module 115 may use one or more algorithms or detection rules to analyze the shape of a tooth, color of a tooth, position of a tooth, or other characteristics of a tooth to determine if there is any AOI that should be highlighted for a dental practitioner” note machine learning is used to identify areas of interest) color information to the one or more anatomical features in the said plurality of images to generate an at least partially artificially colorized images ( see paragraph 148 note that areas of interest may be highlighted with different color i.e to create artificially colorized images.) The motivation to combine is to “highlight AOIs or other elements on an AR display” (see paragraph 148). One or ordinary skill in the art could have easily used the method/system of Kopelman to highlight features in the method/system of Joo Beom Eom. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Re claim 22 Joo Beom Eom discloses An intraoral scanning system comprising: - a structured light source configured to projecting structured light on a dental condition(see abstract “The scanner uses a tunable-focus lens for depth scanning and a pattern mask for projecting structured light onto the sample”);, and generating a plurality of projection images of the dental condition;; - a sensor unit configured to receive reflection of the structured light from the intraoral surfaces (see abstract “By fixing the pattern mask, a camera installed at the same focal point as the sample focus acquired an image at each focal plane, resulting in an improvement in the scanning speed. The proposed method performs optical sectioning on sequentially acquired images while varying the focal planes by estimating pattern modulation in the spatial domain; it then reconstructs 3D point clouds by stacking up the optically sectioned image”)
and one or more computing units, any of which computing units are is configured to: (see section 4 5th paragraph “we implemented a self-developed multithreaded C+ + software built and compiled in Microsoft Visual Studio 2015. To efficiently perform intraoral scanning, the software was run on an Intel Core i7-7800k CPU @ 3.50 GHz with 16 GB memory.” ) -render, color information to one or more anatomical features in the projection images to generate an at least partially artificially colorized plurality of projection images, (see abstract “The fixed pattern mask, however, caused a color loss at the pixels where the patterns are projected. The proposed method solves this problem by applying a low-pass filter and a polynomial interpolation to fill the loss.” Note that the missing color information is “artificially” filed using interpolation to determine the lost color values), wherein the pattern information is removed through the rendering (see section 4 third paragraph “However, as the proposed scanner system uses a fixed pattern mask, the generated color map contains pattern information; thus, that color information is lost, as shown in figure 7. Figure 7(a) is a region where we tested, to verify the color distortion. Figure 7(b) shows intensity profiles on the line segment (red line in figure 7(b)) perpendicular to the grid pattern in the image for the region (figure 7(a)). The image included the repeated sinusoidal pattern. To remove the pattern, the proposed system used the mean filter, which is one of the low-pass filters, on each line segment as follows” note that the median filter is used to remove the grid pattern).
Joo Beom Eom does not expressly disclose detect, in the projection images, one or more anatomical features; and render, via a machine learning, color information to one or more anatomical features in the images to generate an at least partially artificially colorized plurality of projection images, a human machine interface unit configured to display at least some of the artificially colorized images.
In a similar filed of endeavor of dental imaging, Kopelman discloses detect, in the projection images, one or more anatomical features; (see paragraph 65 note that areas of interest may be found from the images of the oral cavity) and render, via a machine learning, (see paragraph 184 note that machine learning is used to analyze the dental conditions see also paragraph 65 “The analysis may involve direct analysis (e.g., pixel-based and/or other point-based analysis), the application of machine learning, the application of image registration, and/or the application of image recognition. The AOI identifying modules 115 may identify areas of interest directly from the image data 135 received from the image capture device 160 or based on a comparison of the received image data 135 and reference data 138 or previous patient data 140. For example, an AOI identifying module 115 may use one or more algorithms or detection rules to analyze the shape of a tooth, color of a tooth, position of a tooth, or other characteristics of a tooth to determine if there is any AOI that should be highlighted for a dental practitioner” note machine learning is used to identify areas of interest) color information to one or more anatomical features in the images to generate an at least partially artificially colorized plurality of projection images see paragraph 148 note that areas of interest may be highlighted with different color i.e. to create artificially colorized images.), a human machine interface unit configured to display at least some of the artificially colorized images. (see paragraph 148 “In the examples, particular colors or types of indicators may be used to highlight AOIs or other elements on an AR display.”) The motivation to combine is to “highlight AOIs or other elements on an AR display” (see paragraph 148). One or ordinary skill in the art could have easily used the method/system of Kopelman to display images in the method/system of Joo Beom Eom. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom and Kopelman to reach the aforementioned advantage.
Claim(s) 2 and 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Joo Beom Eom et al “3D intraoral scanning system using fixed pattern mask and tunable-focus lens” 2020 Meas. Sci. Technol. 31 015401 in view of Kopelman US 2018/0168780 in further view of Chang et al “Toward Universal Stripe Removal via Wavelet-Based Deep Convolutional Neural Network” IEEE Transactions on Geoscience and Remote Sensing Volume: 58, Issue: 4, April 2020.
Re claim 2 Joo Beom Eom discloses generating images with gridlines via projections (see abstract note that grid lines are projection to create images with grid lines) Joo Beom Eom and Kopelman does not expressly disclose wherein the machine learning method is trained on pattern images as input, and produces pattern free images as output. Chang solving a similar problem of removing stripe patters discloses wherein the machine learning method is trained on pattern images as input (see section 3 part d note that images with stripes are used to train a neural network), and produces pattern free images as output (see abstract and section C note that the a neural network is used to produce images with stripes removed). The motivation to combines is “It is shown that the TSWEU has completely removed the stripe and consistently achieved a visually pleasing quality for all cases,” (see section IV part C first paragraph). One of ordinary skill in the art could have easily applied a neural network method of removing stripes as in Chang to replace the stripe removal of Joo Beom Eom to achieve stripe removal. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom, Kopelman and Chang to reach the aforementioned advantage.
Re claim 3 Joo Beom Eom, and Kopelman further do not expressly disclose implementing the machine learning method is implemented through a convolutional neural network following an encoder-decoder architecture. Chang further discloses further discloses implementing the machine learning method is implemented through a convolutional neural network following an encoder-decoder architecture (See abstract and figure 3 note that the feature extraction module corresponds to the encoder and the reconstruction module corresponds to the decoder. The motivation to combines is “It is shown that the TSWEU has completely removed the stripe and consistently achieved a visually pleasing quality for all cases,” (see section IV part C first paragraph). One of ordinary skill in the art could have easily applied a neural network method of removing stripes as in Chang to replace the stripe removal of Joo Beom Eom to achieve stripe removal. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom, Kopelman and Chang to reach the aforementioned advantage.
Claim(s) 12 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over “3D intraoral scanning system using fixed pattern mask and tunable-focus lens” Joo Beom Eom et al 2020 Meas. Sci. Technol. 31 015401 in view of Kopelman US 2018/0168780 in further view of Pesach US 2019/0254529.
Re claim 12 Joo Beom Eom and Kopelman discloses all of the features of claim 1. They do not expressly disclose wherein the pattern information distorts more than 50 % of the a pixel- wise color information of the one or more anatomical features in the projection images initially provided. In a similar field of endeavor of oral scanning Pesach discloses wherein the pattern information distorts more than 50 % of the a pixel- wise color information of the one or more anatomical features in the projection images initially provided (see paragraph 7 “projecting onto the intra-oral scene a color-coded pattern comprising an arrangement of entities having edges between them; each entity comprising a different narrow band of wavelengths; and detecting the projected pattern as a plurality of pixels in an acquired image of the scene using at least two narrowband filters, wherein for each pixel of at least 95% of the pixels of an entity of interest comprising a first band of wavelengths” note that 95 percent of the pixel may be affected by projected light). The motivation to combine is “An aspect of some embodiments of the present invention relates to choosing spectral bands that can produce accurate images on a dental surface.”. One of ordinary skill in the art could have easily modified Joo Beom Eom and Kopelman to use the pattern of Pesach to reach the aforementioned advantage. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom, Kopelman and Pesach to reach the aforementioned advantage.
Re claim 13 Joo Beom Eom and Kopelman discloses all of the features of claim 1. They do not expressly disclose wherein the pattern information reduces an intensity more than 80 % of the pixel-wise color information of the one or more anatomical features in the projection images initially provided. In a similar field of endeavor of oral scanning Pesach discloses wherein the pattern information reduces an intensity more than 80 % of the pixel-wise color information of the one or more anatomical features in the projection images initially provided (see paragraph 7 “projecting onto the intra-oral scene a color-coded pattern comprising an arrangement of entities having edges between them; each entity comprising a different narrow band of wavelengths; and detecting the projected pattern as a plurality of pixels in an acquired image of the scene using at least two narrowband filters, wherein for each pixel of at least 95% of the pixels of an entity of interest comprising a first band of wavelengths” note that 95 percent of the pixel may be affected by projected light see figure 2 not that the pattern may darken pixels). The motivation to combine is “An aspect of some embodiments of the present invention relates to choosing spectral bands that can produce accurate images on a dental surface.”. One of ordinary skill in the art could have easily modified Joo Beom Eom and Kopelman to use the pattern of Pesach to reach the aforementioned advantage. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Joo Beom Eom, Kopelman and Pesach to reach the aforementioned advantage.
Allowable Subject Matter
Claim 4-11 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Re claim 4 while Chang discloses training a neural network (see rejection to claim 2) The prior art of record does not expressly disclose “generating training pairs for input and output of the- machine learning method are generated by photorealistic rendering of a 3D-model comprising geometry and surface color information”
Re claim 5 While Joo Beom Eom discloses capturing projection images (see abstract); The prior art of record does not disclose providing the input and output pairs of the machine learning method are provided by a movable camera following an optical configuration that captures projection images and white light exposure images, at different points in time.
Claims 6-11 depend from claim 5.
Cited Art
The following is a recitation of prior art considered relevant but not applied in a rejection above:
SCHMIDT-KRULIG US 20240144600 A1 discloses A method is provided for generating a texture for a three-dimensional (3D) model of an oral structure. The method includes providing the 3D model of the oral structure in the form of a polygon mesh, identifying a set of points located on the polygon mesh, and determining, for each respective point in the set of points, a respective texture value. Each respective texture value is determined by identifying a set of frames, filtering the set of frames to identify a subset of frames, determining a set of candidate texture values for the respective texture value, computing, for each respective candidate texture value in the set of candidate texture values, a quality factor, and computing the respective texture value for the respective point by combining, based on their respective quality factors, candidate texture values selected from the set of candidate texture values. (see abstract).
SARKAR US 20230210642 discloses An intraoral scanning system is described herein. The intraoral scanning system includes a scanning device and a processing device. The scanning device is configured to illuminate an object of interest with a burst light pattern at one or more intervals and receive a set of reflected images reflected off the object of interest. The processing device is configured to receive the set of reflected images from the scanning device and convert the set of reflected images into a three-dimensional model of the object of interest. The burst light pattern comprises a first image and a second image in succession. One of the first image and the second image is a structured light image and the other of the first image and the second image is an unstructured light image. (See abstract)
Phipps US 20210291358 A1 discloses: In an embodiment, said method further includes: providing a 3D digital representation of an object; and superimposing with the processor the first colorized 3D image onto the 3D representation to obtain a first colorized 3D representation. In an embodiment, the method further includes: measuring 3D coordinates of a first point on the object with the AACMM or with the tracker; placing the 3D representation within the tracker frame of reference based at least in part on the measured 3D coordinates; and superimposing with the processor the measured 3D coordinates of the first point onto the first colorized 3D representation. (see paragraph 233)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN T MOTSINGER whose telephone number is (571)270-1237. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN T MOTSINGER/Primary Examiner, Art Unit 2673
Prosecution Insights