DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to Applicant’s remarks, filed on 6/5/2026. The amendments to claim(s) 1, 3-11 and 14-20 have been entered. Claim(s) 2 and 12-13 is/are cancelled by Applicant and therefore withdrawn from further consideration pursuant to 37 CFR 1.142(b). No new claim(s) have been entered. Accordingly, claim(s) 1, 3-11 and 14-20 remain pending for examination on the merits.
Response to Arguments
Applicant’s arguments, see p. 10-23, with respect to the rejections of claim(s) 1, 3-11 and 14-20 have been fully considered.
Upon review of the Applicant’s remarks and amendments regarding the objections to claim(s) 2, 7 and 13, Examiner respectfully agrees with Applicant and the prior objections to the claims have been withdrawn. Regarding the prior rejections under 35 USC § 101, after review of the Applicant’s remarks and amendments to the claim(s), Examiner respectfully agrees with the Applicant and the prior 35 USC § 101 rejections have been withdrawn.
Regarding the rejection(s) under 35 U.S.C. § 112, Examiner respectfully disagrees with the remarks and does not find Applicant’s arguments persuasive. New 35 U.S.C. § 112 rejections are issued in light of the amendments to the claim(s).
Regarding the rejection(s) under 35 U.S.C. § 102, Examiner respectfully disagrees with the remarks and does not find Applicant’s arguments persuasive. New grounds of rejection are made in view of the following: new amendments provided by Applicant and attached remarks; updated search and review of pertinent, eligible prior art; and/or different interpretation of the previously applied references. Regarding the rejections of claim(s) 1, 3-11 and 14-20, Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Examiner respectfully notes that Applicant’s arguments only address the independent claim(s), and no remarks regarding the subject matter of the dependent claim(s) have been presented. Accordingly, the rejections to dependent claims are modified to address Applicant’s amendments and the new rejection to the independent claim(s) and are sustained. The rejections of claim(s) 1, 3-11 and 14-20 under 35 U.S.C. § 102 and 35 U.S.C. § 103 are maintained.
Claim Rejections - 35 USC § 112
35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim(s) 8 and 19 is/are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 8 recites “identifying the type of the object as the specific class based on a percentage of frames identified as the specific class among the reference number of frames being greater than or equal to a second reference value”; claim 19 recites “a percentage of frames identified as the specific class among the reference number of frames being greater than or equal to a second reference value”. which does not comply with the written description requirement. There is no support for a ‘second reference value’ in either the written description or the drawings that is used in combination with a ‘first reference value’ as recited in claim 7. Similar consideration is applied to claim 19. Appropriate correction is required.
35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1, 3-11 and 14-20 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim(s) 3-9 and 14-20 are rejected at least by virtue of dependency upon a rejected base claim.
Claim 1 recites the limitation "generating, by the data processing apparatus, 3D scan data of the object based on the plurality of frames according to the activated or deactivated state of the scan filtering mode, wherein, when the scan filtering mode is deactivated, the 3D scan data is generated without performing a data filtering operation on the plurality of frames, and wherein, when the scan filtering mode is activated, the 3D scan data is generated as filtered 3D scan data by performing the data filtering operation on the plurality of frames" which renders the claim indefinite. Additionally the limitations “generating, by a data processing apparatus, 3D scan data of the object based on the plurality of frames […] when the scan filtering mode is activated, the 3D scan data is generated as filtered 3D scan data by performing the data filtering operation on the plurality of frames” in claim 10; and “generate 3D scan data of the object based on the plurality of frames […] when the scan filtering mode is activated, the 3D scan data is generated as filtered 3D scan data by performing the data filtering operation on the plurality of frames” in claim 11; renders the claims indefinite.
The claim language does not clearly describe how the 3D scan data is being generated. It is not clear whether the ‘3D scan data’ describes the entire scanned object or data elements withing a 3D dataset. In an interpretation, the 3D scan data is generated either by using the data filtering operation or without using the data filtering operation. In another, distinct interpretation the 3D scan data is generated using a combination of filtered and unfiltered data. The claim does not clearly define the logical flow of data and how each step of the method transforms the acquired data; it is suggested to amend the claim to clearly define the composition of the 3D scan data and when/how the data filtering operation is applied to the plurality of frames. Mirrored limitations in independent claims 10 and 11 are similarly rejected under the logic applied to claim 1 above. For the purposes of examination the broadest reasonable interpretations of the 3D scan data, including those discussed above, are any combination of filtered or unfiltered data.
Claim 8 recites the limitation “wherein the identifying of the type of the object comprises identifying the type of the object as the specific class based on a percentage of frames identified as the specific class among the reference number of frames being greater than or equal to a second reference value”; claim 19 recites the limitation “to identify the type of the object as the specific class based on a percentage of frames identified as the specific class among the reference number of frames being greater than or equal to a second reference value”; which renders the claims indefinite. It unclear how the identification of the type of object is performed based on the ‘calculating’ and ‘identifying’ steps provided in claim 7 (upon which claim 8 depends). For example, it is not clear how the ‘identification’ steps are used together, nor is it clear if the ‘object type’ is identified multiple times. Similarly, it is not clear if the ‘second reference value’ and ‘first reference value’ are equally weighted when performing the identification of the type of object. The same consideration is applied to claim 19. It is suggested to amend the claims to clarify how the identification step is performed.
Claim 10 recites the limitation "receiving, from the 3D scanner, a plurality of frames obtained by scanning an object;". There is insufficient antecedent basis for this limitation in the claim. It is not clear what “the 3D scanner” is specifically referring to because there is no prior recitation of ‘a 3D scanner’ in the claim. It is suggested to amend the claim to recite –a 3D scanner– if that is Applicant’s intended interpretation.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3-5, 10-11 and 14-16 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Hawkins et al. (US20200265653A1, 2020-08-20) as provided by Applicant.
Regarding claim 1, Hawkins teaches a data processing method performed by a data processing apparatus connected to a 3D scanner (“A method implemented in association with an intra-oral scanning device and an associated computing system executing software, the software configured to extract a three-dimensional (3D) point cloud from data captured by the scanning device,” [clm 1]; “The frames used to capture the data for the 3D model are partially-illuminated frames (such as shown in FIG. 2, wherein the LCOS paints a series of lines in a pattern)” [0024]; “A 3D intra-oral scanner is able to obtain 3D surface information as well as capture monochrome or color images that can then be superimposed on the 3D model.” [0035]; [0023-0049], [fig. 1, 4-7, 9, 11-12]), the data processing method comprising:
receiving, from the 3D scanner, a plurality of frames obtained by scanning an object (“the software configured to extract a three-dimensional (3D) point cloud from data captured by the scanning device, […] the 3D model representing a scanned object and configured as a collection of discrete, 3D samples that approximate an actual continuous surface of the scanned object,” [clm 1]; “The frames used to capture the data for the 3D model are partially-illuminated frames […] To facilitate the operation of the device and provide live video as feedback to the operator (as well as the 3D-computed data), a sequence of patterns throughout which full illumination frames are selectively interspersed. A full illumination frame involves all or substantially all lines being turned on, […] The partially-illustrated frames provide the data from which the 3D coordinates of the surface are determined.” [0024]; “A 3D intra-oral scanner is able to obtain 3D surface information as well as capture monochrome or color images that can then be superimposed on the 3D model.” [0035]; [0023-0049], [fig. 1, 4-7, 9, 11-12]);
identifying, based on the plurality of frames, a type of the object (“associating a weight with each discrete 3D sample in at least a portion of the 3D model, the weight representing a density of data associated with that sample available to the software for rendering the 3D model;” [clm 1]; “during the scanning process the user […] may accidentally scan items they wish to be deleted. For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue” [0036]; “the software routines (including those that are used to display the 3D model) are enhanced to provide an automated “active” delete function/feature. This feature may be activated by a default configuration setting, […] When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model” [0037]; [0023-0049], [fig. 1, 4-7, 9, 11-12]);
determining whether to activate or deactivate a scan filtering mode based on the identified type of the object (“actively deleting one or more 3D samples based at least in part on a sample's weight or adjusted weight,” [clm 1]; “An intra-oral scanning device and system are augmented to provide an active delete (or filter) tool” [0006]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen.” [0037]; “The scanned object is represented in software as a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] Each point may carry with it a “weight” representing the data density at that location. […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface” [0039]; 3D samples are weighted based on certainty that the data represents a desired surface [0023-0049], [fig. 1, 4-7, 9, 11-12]);
based on the identified type of the object being a physical model representing at least a part of an oral cavity or a dental structure determining to deactivate the scan filtering mode (“A method implemented in association with an intra-oral scanning device and an associated computing system executing software, […] and render a 3D model to an associated display screen, the 3D model representing a scanned object and configured as a collection of discrete, 3D samples” [clm 1]; “a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; “the output of the active delete functionality […] is to used to update a rendering of the 3D model that is available to the user. Thus, the deletions themselves (if the 3D samples representing the portions identified for deletion are already on the screen) are updated in real-time on the display rendering,” [0047]; The desired surface (e.g., tooth surface) is not filtered out from the 3D model by the algorithm during scanning [0023-0049], [fig. 1, 4-7, 9, 11-12]);
based on the identified type of the object not being the physical model, determining to activate the scan filtering mode (“actively deleting one or more 3D samples based at least in part on a sample's weight or adjusted weight, wherein the one or more deleted 3D samples represent an unwanted item in the rendered 3D model.” [clm 1]; “For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue.” [0036]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen” [0037]; Unwanted objects and artifacts are automatically filtered/deleted from the 3D model by the algorithm based on the weighted data [0023-0049], [fig. 1, 4-7, 9, 11-12]); and
generating, by the data processing apparatus, 3D scan data of the object based on the plurality of frames according to the activated or deactivated state of the scan filtering mode (“updating the 3D model in real-time to render on the associated display screen without the unwanted item” [clm 3]; “the full illumination frames are used for texturing the 3D model generated by the partially-illuminated frame data. In one sequence, a first set (e.g., six) pattern frames are used, interspersed with a second set (e.g., three) illumination frames, for a sequence total of nine total CCD frames. A software traffic shaper is then used to separate captured frames in two streams, namely, a live preview stream, and a data processing stream from which the 3D model is generated.” [0024]; [0023-0049], [fig. 1, 4-7, 9, 11-12]),
wherein, when the scan filtering mode is deactivated, the 3D scan data is generated without performing a data filtering operation on the plurality of frames (“a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; [0023-0049], [fig. 1, 4-7, 9, 11-12]), and
wherein, when the scan filtering mode is activated, the 3D scan data is generated as filtered 3D scan data by performing the data filtering operation on the plurality of frames (“actively deleting one or more 3D samples based at least in part on a sample's weight or adjusted weight, wherein the one or more deleted 3D samples represent an unwanted item in the rendered 3D model.” [clm 1]; “For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue.” [0036]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen” [0037]; “the active delete approach herein leverages two (2) separate mechanisms to delete unwanted items. The first approach uses “line of sight” deletion, whereby the algorithm iteratively decreases the weight of 3D samples that it (the algorithm) determines are likely to be erroneous. […] The second approach—referred to as “immediate deletion”—immediately deletes 3D samples that the algorithm is confident are erroneous” [0040]; [0023-0049], [fig. 1, 4-7, 9, 11-12]).
Regarding claim 3, Hawkins teaches the data processing method of claim 1, Hawkins further teaching further comprising:
identifying a current state of the scan filtering mode before generating the 3D scan data (“The traffic shaper separates the data into live preview and 3D processing input frames. The 3D processing input frames contain projected patterns. On the GPU these frames are first run through a centroid detector implemented as a recursive sub-pixel edge detector, a correspondence block, and finally a point cloud generation block. This output is then run on the CPU side through a bilateral filter for data smoothing, and through an alignment block to stitch scans together” [0031]; “extracts a 3D point cloud from the captured data, aligns the 3D point cloud to previously captured data, and renders to a display screen (or other output)” [0039]; The input data is parallel processed and the filtering is confirmed by the algorithm [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]); and
in response to determining that the current state of the scan filtering mode does not correspond to a state determined based on the identified type of the object, changing the current state of the scan filtering mode before generating the 3D scan data (“These 3D samples may be organized into a voxel grid, or may be simply a list of 3D points. Each point may carry with it a “weight” representing the data density at that location. […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; “the active delete approach herein leverages two (2) separate mechanisms to delete unwanted items. The first approach uses “line of sight” deletion, whereby the algorithm iteratively decreases the weight of 3D samples that it (the algorithm) determines are likely to be erroneous. Once the weight is below a threshold, the weight is set equal to zero, thereby causing the corresponding sample to be deleted” [0040]; Line of sight deletion filters data iteratively to decrease weight and compare against deletion threshold [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]).
Regarding claim 4, Hawkins teaches the data processing method of claim 3, Hawkins further teaching further comprising;
displaying, on a display of the data processing apparatus, an interactive user interface screen for changing the current state of the scan filtering mode (“align the 3D point cloud to previously-captured data, and render a 3D model to an associated display screen,” [clm 1]; “updating the 3D model in real-time to render on the associated display screen without the unwanted item.” [clm 3]; “A representative display interface is used to display the 3D model, on the one hand, and the live video preview window, on the other. […] These views may be juxtaposed in any convenient display format” [0033]; “the software routines (including those that are used to display the 3D model) are enhanced to provide an automated “active” delete function/feature. This feature may be activated by a default configuration setting, or it may be selected by the user (e.g., the technician performing the scan)” [0037]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection])
wherein the interactive user interface screen comprises at least one of the identified type of the object, a message guiding a change of the current state of the scan filtering mode based on the identified type of the object, and a message guiding a change of a scan operation (“updating the 3D model in real-time to render on the associated display screen without the unwanted item” [clm 3]; “the active delete function is activated by selecting an icon on the display interface” [0050]; “When running the fine-connected components algorithm, e.g., color may be masked so that the user can see that the identified points will be deleted. More generally, color may also be used to emphasize points that are more likely to be deleted.” [0052]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]).
Regarding claim 5, Hawkins teaches the data processing method of claim 4, Hawkins further teaching wherein the changing of the current state of the scan filtering mode comprises:
receiving, through a user input unit of the data processing apparatus, a user input signal for changing the current state of the scan filtering mode in response to the displaying of the interactive user interface screen; and changing the current state of the scan filtering mode based on the user input signal (“updating the 3D model in real-time to render on the associated display screen without the unwanted item” [clm 3]; “the active delete function is activated by selecting an icon on the display interface” [0050]; The user may selectively activate the delete function using display interface in response to seeing the 3D model [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 4 rejection]).
Regarding claim 10, Hawkins teaches a non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute a data processing method (“A computer program product in a non-transitory computer readable medium for use in a data processing system in association with an intra-oral scanning device,” [clm 17]; “The frames used to capture the data for the 3D model are partially-illuminated frames (such as shown in FIG. 2, wherein the LCOS paints a series of lines in a pattern)” [0024]; “A 3D intra-oral scanner is able to obtain 3D surface information as well as capture monochrome or color images that can then be superimposed on the 3D model.” [0035]; [0023-0049], [fig. 1, 4-7, 9, 11-12]) comprising:
receiving, from the 3D scanner, a plurality of frames obtained by scanning an object (“in association with an intra-oral scanning device, […] instructions that, when executed by the data processing system during a scanning operation of an object having a true surface geometry, are operative to: assign weights to sampled data according to a density of data collection; and” [clm 17]; “The frames used to capture the data for the 3D model are partially-illuminated frames […] To facilitate the operation of the device and provide live video as feedback to the operator (as well as the 3D-computed data), a sequence of patterns throughout which full illumination frames are selectively interspersed. A full illumination frame involves all or substantially all lines being turned on, […] The partially-illustrated frames provide the data from which the 3D coordinates of the surface are determined.” [0024]; “A 3D intra-oral scanner is able to obtain 3D surface information as well as capture monochrome or color images that can then be superimposed on the 3D model.” [0035]; [0023-0049], [fig. 1, 4-7, 9, 11-12]);
identifying, based on the plurality of frames, a type of the object (“assign weights to sampled data according to a density of data collection; and” [clm 17]; “during the scanning process the user […] may accidentally scan items they wish to be deleted. For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue” [0036]; “the software routines (including those that are used to display the 3D model) are enhanced to provide an automated “active” delete function/feature. This feature may be activated by a default configuration setting, […] When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model” [0037]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]);
determining, whether to activate or deactivate a scan filtering mode based on the identified type of the object (“adjusting the weights according to an algorithm that ensures that erroneous sampled data close to the true surface geometry is gradually deleted as more information is collected about the true surface geometry” [clm 17]; “An intra-oral scanning device and system are augmented to provide an active delete (or filter) tool” [0006]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen.” [0037]; “The scanned object is represented in software as a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] Each point may carry with it a “weight” representing the data density at that location. […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface” [0039]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]);
based on the identified type of the object being a physical model representing at least a part of an oral cavity or a dental structure, determining to deactivate the scan filtering mode (“adjusting the weights according to an algorithm that ensures that erroneous sampled data close to the true surface geometry is gradually deleted as more information is collected about the true surface geometry” [clm 17]; “a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; “the output of the active delete functionality […] is to used to update a rendering of the 3D model that is available to the user. Thus, the deletions themselves (if the 3D samples representing the portions identified for deletion are already on the screen) are updated in real-time on the display rendering,” [0047]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]);
based on the identified type of the object not being the physical model, determining to activate the scan filtering mode (“adjusting the weights according to an algorithm that ensures that erroneous sampled data close to the true surface geometry is gradually deleted as more information is collected about the true surface geometry” [clm 17]; “For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue.” [0036]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen” [0037]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]); and
generating, by a data processing apparatus, 3D scan data of the object based on the plurality of frames according to the activated or deactivated state of the scan filtering mode (“the full illumination frames are used for texturing the 3D model generated by the partially-illuminated frame data. In one sequence, a first set (e.g., six) pattern frames are used, interspersed with a second set (e.g., three) illumination frames, for a sequence total of nine total CCD frames. A software traffic shaper is then used to separate captured frames in two streams, namely, a live preview stream, and a data processing stream from which the 3D model is generated.” [0024]; “the output of the active delete functionality (or individual operations thereof, such as the immediate deletion) is to used to update a rendering of the 3D model that is available to the user. Thus, the deletions themselves (if the 3D samples representing the portions identified for deletion are already on the screen) are updated in real-time on the display rendering” [0047]; [0023-0049], [fig. 1, 4-7, 9, 11-12]),
wherein, when the scan filtering mode is deactivated, the 3D scan data is generated without performing a data filtering operation on the plurality of frames (“a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]), and
wherein, when the scan filtering mode is activated, the 3D scan data is generated as filtered 3D scan data by performing the data filtering operation on the plurality of frames (“adjusting the weights according to an algorithm that ensures that erroneous sampled data close to the true surface geometry is gradually deleted as more information is collected about the true surface geometry” [clm 17]; “For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue.” [0036]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen” [0037]; “the active delete approach herein leverages two (2) separate mechanisms to delete unwanted items. The first approach uses “line of sight” deletion, whereby the algorithm iteratively decreases the weight of 3D samples that it (the algorithm) determines are likely to be erroneous. […] The second approach—referred to as “immediate deletion”—immediately deletes 3D samples that the algorithm is confident are erroneous” [0040]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]).
Regarding claim 11, Hawkins teaches a data processing apparatus connected to a 3D scanner (“Apparatus configured in association with an intra-oral scanning device,” [clm 9]; “The frames used to capture the data for the 3D model are partially-illuminated frames (such as shown in FIG. 2, wherein the LCOS paints a series of lines in a pattern)” [0024]; “A 3D intra-oral scanner is able to obtain 3D surface information as well as capture monochrome or color images that can then be superimposed on the 3D model.” [0035]; [0023-0049], [fig. 1, 4-7, 9, 11-12]), the data processing apparatus comprising;
a communication interface configured to receive, from the 3D scanner, a plurality of frames obtained by scanning an object (“computer memory holding computer program code executed by the processor and configured to extract a three-dimensional (3D) point cloud from data captured by the scanning device, […] the 3D model representing a scanned object and configured as a collection of discrete, 3D samples that approximate an actual continuous surface of the scanned object,” [clm 9]; “The frames used to capture the data for the 3D model are partially-illuminated frames […] To facilitate the operation of the device and provide live video as feedback to the operator (as well as the 3D-computed data), a sequence of patterns throughout which full illumination frames are selectively interspersed. A full illumination frame involves all or substantially all lines being turned on, […] The partially-illustrated frames provide the data from which the 3D coordinates of the surface are determined.” [0024]; “A 3D intra-oral scanner is able to obtain 3D surface information as well as capture monochrome or color images that can then be superimposed on the 3D model.” [0035]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]);
at least one processor configured to execute at least one instruction (“a processor; and computer memory holding computer program code executed by the processor” [clm 9]; “the system architecture comprises a tightly-integrated IP FPGA core […] The FPGA is assisted by an ARM processor,” [0030]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]), wherein the at least one processor is further configured to execute the at least one instruction to:
identify, based on the plurality of frames, a type of the object (“associate a weight with each discrete 3D sample in at least a portion of the 3D model, the weight representing a density of data associated with that sample available to the software for rendering the 3D model;” [clm 9]; “during the scanning process the user […] may accidentally scan items they wish to be deleted. For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue” [0036]; “the software routines (including those that are used to display the 3D model) are enhanced to provide an automated “active” delete function/feature. This feature may be activated by a default configuration setting, […] When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model” [0037]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]);
determine whether to activate or deactivate a scan filtering mode based on the identified type of the object (“actively deleting one or more 3D samples based at least in part on a sample's weight or adjusted weight, wherein the one or more deleted 3D samples represent an unwanted item in the rendered 3D model.” [clm 9]; “An intra-oral scanning device and system are augmented to provide an active delete (or filter) tool” [0006]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen.” [0037]; “The scanned object is represented in software as a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] Each point may carry with it a “weight” representing the data density at that location. […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface” [0039]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]);
based on the identified type of the object being a physical model representing at least a part of an oral cavity or a dental structure, determine to deactivate the scan filtering mode (“and render a 3D model to an associated display screen, the 3D model representing a scanned object and configured as a collection of discrete, 3D samples that approximate an actual continuous surface of the scanned object,” [clm 9]; “a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; “the output of the active delete functionality […] is to used to update a rendering of the 3D model that is available to the user. Thus, the deletions themselves (if the 3D samples representing the portions identified for deletion are already on the screen) are updated in real-time on the display rendering,” [0047]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]);
based on the identified type of the object not being the physical model, determine to activate the scan filtering mode (“actively deleting one or more 3D samples based at least in part on a sample's weight or adjusted weight, wherein the one or more deleted 3D samples represent an unwanted item in the rendered 3D model” [clm 9]; “For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue.” [0036]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen” [0037]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]); and
generate 3D scan data of the object based on the plurality of frames according to the activated or deactivated state of the scan filtering mode (“wherein the program code is further configured to cause updating of the 3D model in real-time to render on the associated display screen without the unwanted item” [clm 11]; “the full illumination frames are used for texturing the 3D model generated by the partially-illuminated frame data. In one sequence, a first set (e.g., six) pattern frames are used, interspersed with a second set (e.g., three) illumination frames, for a sequence total of nine total CCD frames. A software traffic shaper is then used to separate captured frames in two streams, namely, a live preview stream, and a data processing stream from which the 3D model is generated.” [0024]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]),
wherein, when the scan filtering mode is deactivated, the 3D scan data is generated without performing a data filtering operation on the plurality of frames (“a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; [0023-0049], [fig. 1, 4-7, 9, 11-12]), and
wherein, when the scan filtering mode is activated, the 3D scan data is generated as filtered 3D scan data by performing the data filtering operation on the plurality of frames (“actively deleting one or more 3D samples based at least in part on a sample's weight or adjusted weight, wherein the one or more deleted 3D samples represent an unwanted item in the rendered 3D model.” [clm 9]; “For the dental user, these items include, for example, fingers, lips, cheek, dental instruments, and tongue.” [0036]; “When this feature is activated and used (e.g., with the 3D model displayed on the screen), unwanted/undesirable items, objects and other artifacts are automatically “deleted” (filtered) from the model that is being rendered on the display screen” [0037]; “the active delete approach herein leverages two (2) separate mechanisms to delete unwanted items. The first approach uses “line of sight” deletion, whereby the algorithm iteratively decreases the weight of 3D samples that it (the algorithm) determines are likely to be erroneous. […] The second approach—referred to as “immediate deletion”—immediately deletes 3D samples that the algorithm is confident are erroneous” [0040]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]).
Regarding claim 14, Hawkins teaches the data processing apparatus of claim 11, Hawkins further teaching wherein the at least one processor is further configured to execute the at least one instruction to:
identify a current state of the scan filtering mode before generating the 3D scan data (“The traffic shaper separates the data into live preview and 3D processing input frames. The 3D processing input frames contain projected patterns. On the GPU these frames are first run through a centroid detector implemented as a recursive sub-pixel edge detector, a correspondence block, and finally a point cloud generation block. This output is then run on the CPU side through a bilateral filter for data smoothing, and through an alignment block to stitch scans together” [0031]; “extracts a 3D point cloud from the captured data, aligns the 3D point cloud to previously captured data, and renders to a display screen (or other output)” [0039]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 3 rejection]); and
in response to determining that the current state of the scan filtering mode does not correspond to a state determined based on the identified type of the object, change the current state of the scan filtering mode before generating the 3D scan data (“These 3D samples may be organized into a voxel grid, or may be simply a list of 3D points. Each point may carry with it a “weight” representing the data density at that location. […] The weight also may be understood as the algorithm's certainty that the 3D samples are a good representation of the true surface;” [0039]; “the active delete approach herein leverages two (2) separate mechanisms to delete unwanted items. The first approach uses “line of sight” deletion, whereby the algorithm iteratively decreases the weight of 3D samples that it (the algorithm) determines are likely to be erroneous. Once the weight is below a threshold, the weight is set equal to zero, thereby causing the corresponding sample to be deleted” [0040]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 3 rejection]).
Regarding claim 15, Hawkins teaches the data processing apparatus of claim 14,
Hawkins further teaching further comprising a display, wherein the at least one processor is further configured to execute the at least one instruction to display, on the display, an interactive user interface screen for changing the current state of the scan filtering mode (“render a 3D model to an associated display screen,” [clm 9]; “updating the 3D model in real-time to render on the associated display screen without the unwanted item.” [clm 3]; “A representative display interface is used to display the 3D model, on the one hand, and the live video preview window, on the other. […] These views may be juxtaposed in any convenient display format” [0033]; “the software routines (including those that are used to display the 3D model) are enhanced to provide an automated “active” delete function/feature. This feature may be activated by a default configuration setting, or it may be selected by the user (e.g., the technician performing the scan)” [0037]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 4 rejection]),
wherein the interactive user interface screen comprises at least one of the identified type of the object, a message guiding a change of the current state of the scan filtering mode based on the identified type of the object, and a message guiding a change of a scan operation (“wherein the program code is further configured to cause updating of the 3D model in real-time to render on the associated display screen without the unwanted item” [clm 11]; “the active delete function is activated by selecting an icon on the display interface” [0050]; “When running the fine-connected components algorithm, e.g., color may be masked so that the user can see that the identified points will be deleted. More generally, color may also be used to emphasize points that are more likely to be deleted.” [0052]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 4 rejection]).
Regarding claim 16, Hawkins teaches the data processing apparatus of claim 15,
Hawkins further teaching further comprising a user input unit, wherein the at least one processor is further configured to execute the at least one instruction to: receive, through the user input unit, a user input signal for changing the current state of the scan filtering mode in response to the displaying of the interactive user interface screen; and change the current state of the scan filtering mode based on the user input signal (“wherein the program code is further configured to cause updating of the 3D model in real-time to render on the associated display screen without the unwanted item” [clm 11]; “the active delete function is activated by selecting an icon on the display interface” [0050]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 5 rejection]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 6-8 and 17-19 is/are rejected under 35 U.S.C. 103 as being obvious over Hawkins as applied to claim 1 above, in view of Kumar et al. (US20210275028A1, 2021-09-09).
Regarding claim 6, Hawkins teaches the data processing method of claim 1,
but Hawkins fails to teach the trained neural network model.
However, in the same field of endeavor, Kumar teaches a data processing method performed by a data processing apparatus connected to a scanner (“method of identifying existence of oral diseases, the method comprising: […] operating said light source to generate a first light with a desired wavelength band to illuminate a target area in a mouth of a subject; and” [clm 1]; “Device 100 is shown containing lighting system 110, filter mechanism 120, camera 130, processor 140 and control board 150. Device 100 may contain other blocks/components (such as those for wired or wireless communication) […] Object 105 represents specific regions in a patient's mouth whose details are to be captured in the form of one or more images by device 100 and analyzed for presence of oral disease” [0022]; [0022-0113], [fig. 1-7]);
Kumar further teaching wherein the identifying of the type of the object comprises: inputting at least one frame among the plurality of frames to a trained neural network model (“passing said normalized and contrasted image to a pre-trained deep neural net algorithm” [clm 8]; “the normalized and contrasted image obtained from step 640 is passed to a pre-trained deep neural net algorithm to classify whether the image indicates presence of disease. The deep neural net algorithm uses a supervised CNN (convolutional neural network) that has been trained using images from healthy and unhealthy patients.” [0058]; [0052-0113], [fig. 4, 6-7]);
obtaining, from the trained neural network model, pixel-based class information indicating classes of pixels included in the at least one frame (“if said pre-trained deep neural net algorithm indicates presence of disease, then passing said normalized and contrasted image through a thresholding algorithm to create a pixelated image that highlights the areas that have high contrast values;” [clm 8]; “In step 655, device 100 determines if the operation of step 650 classified the image as indicative of disease.” [0059]; “the normalized grayscale image (obtained in step 650) is passed through a thresholding algorithm to create a pixelated image that highlights the areas that have high contrast values.” [0060]; The neural network identifies images as indicative of disease, and identified images are passed through thresholding algorithm to create pixelated images having high contrast values [0052-0113], [fig. 4, 6-7]); and
identifying the type of the object based on the pixel-based class information (“passing said pixelated image to a contour recognition algorithms to identify and highlight diseased areas in said pixelated image.” [clm 8]; “In step 670, the pixelated image obtained from step 660 is passed to a contour recognition algorithm which identifies the diseased areas in the image and highlights them.” [0061]; Diseased areas are identified and highlighted from pixelated images [0052-0113], [fig. 4, 6-7]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the data processing method performed by a data processing apparatus connected to a 3D scanner taught by Hawkins by applying a trained neural network model as taught by Kumar. While existing scanner devices provide satisfactory results, during the scanning process the user may accidentally scan items they wish to be deleted. Depending on the nature of the artifact or its location in the scan, known deletion processes may be time-consuming and frustrating for the user and it may not possible to delete without adversely impacting the integrity of the scan (Hawkins [0005]). The weighting technique may provide significant advantages. For line-of-sight deletion in particular, the weight-based approach ensures that erroneous data close to the true surface will be gradually deleted as more information is collected about the true surface geometry, thus preserving surface accuracy (Hawkins [0051]). Additionally, an off-the-shelf device (e.g., a smart phone with an operating system) can install and execute applications for analysis of captured images and identification of oral diseases (Kumar [0030]). Using categorized images from a neural net algorithm, a contour recognition algorithm may identify diseased areas within the image and highlight them. A clinician can interpret the image and advise the patient as to the next steps (Kumar [0058-0061]).
Regarding claim 7, Hawkins and Kumar teach the data processing method of claim 6,
Hawkins further teaching wherein the identifying of the type of the object based on the pixel-based class information comprises:
calculating, based on the pixel-based class information, a percentage of pixels identified as a specific class among all pixels included in the at least one frame (“adjusting the weights occurs as scanning proceeds and includes: decreasing a weight of a 3D sample that is determined to be within a line-of-sight to a point within the 3D point cloud;” [clm 4]; “The scanned object is represented in software as a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) These 3D samples may be organized into a voxel grid, or may be simply a list of 3D points.” [0039]; “the normalized grayscale image (obtained in step 650) is passed through a thresholding algorithm to create a pixelated image that highlights the areas that have high contrast values.” [0060]; 3D samples representing pixels are classified as true surfaces using algorithmic weighting [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]); and
identifying the type of the object as the specific class based on the calculated percentage being greater than or equal to a first reference value (“responsive to decreasing the weight, determining whether a resulting adjusted weight of the 3D sample determined to be within the line-of-sight falls below a threshold; responsive to determining that the resulting adjusted weight of the 3D sample has fallen below the threshold, marking that 3D sample for deletion.” [clm 4]; “For any of these points that are closer to the camera than the scan point at the corresponding pixel, its weight is decreased, say by a factor S. Then, if the weight of any point in the point cloud falls below a threshold, that weight is set equal to zero. […] In addition, the threshold itself may be configurable.” [0043]; Threshold(s) are applied to the identified regions to determine if weight adjustment and deletion are indicated [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]).
Regarding claim 8, Hawkins and Kumar teach the data processing method of claim 7,
Hawkins further teaching wherein the at least one frame comprises a reference number of frames obtained after a scan operation starts (“In one sequence, a first set (e.g., six) pattern frames are used, interspersed with a second set (e.g., three) illumination frames, for a sequence total of nine total CCD frames.” [0024]; “good results were obtained with a live preview window of approximately 20 frames per second, coupled with approximately 15 frames per second for the 3D data” [0032]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]), and
wherein the identifying of the type of the object comprises identifying the type of the object as the specific class based on a percentage of frames identified as the specific class among the reference number of frames being greater than or equal to a second reference value (“In one sequence, a first set (e.g., six) pattern frames are used, interspersed with a second set (e.g., three) illumination frames, for a sequence total of nine total CCD frames. A software traffic shaper is then used to separate captured frames in two streams, namely, a live preview stream, and a data processing stream from which the 3D model is generated.” [0024]; “The traffic shaper separates the data into live preview and 3D processing input frames. The 3D processing input frames contain projected patterns.” [0031]; “For any of these points that are closer to the camera than the scan point at the corresponding pixel, its weight is decreased, say by a factor S. Then, if the weight of any point in the point cloud falls below a threshold, that weight is set equal to zero. This is equivalent to deleting the point. The weighting factor S may vary depending on the geometry. For example, if the point cloud point is very close to the corresponding scan point, a lower value of S may be used, in effect deleting the point cloud point more slowly. […] In addition, the threshold itself may be configurable.” [0043]; A plurality of frames representing the 3D model surface are evaluated by the algorithm to render true surfaces using configurable thresholds [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 1 rejection]).
Regarding claim 17, Hawkins teaches the data processing apparatus of claim 11,
but Hawkins fails to teach the trained neural network model.
However, in the same field of endeavor, Kumar teaches a data processing apparatus connected to a 3D scanner (“A non-transitory machine readable medium storing one or more sequences of instructions for enabling a user to identify existence of oral diseases using a system, […] operating said light source to generate a first light with a desired wavelength band to illuminate a target area in a mouth of a subject; and” [clm 19]; “Device 100 is shown containing lighting system 110, filter mechanism 120, camera 130, processor 140 and control board 150. Device 100 may contain other blocks/components (such as those for wired or wireless communication) […] Object 105 represents specific regions in a patient's mouth whose details are to be captured in the form of one or more images by device 100 and analyzed for presence of oral disease” [0022]; [0022-0113], [fig. 1-7]);
wherein the at least one processor is further configured to execute the at least one instruction to: input at least one frame among the plurality of frames to a trained neural network model (“the normalized and contrasted image obtained from step 640 is passed to a pre-trained deep neural net algorithm to classify whether the image indicates presence of disease. The deep neural net algorithm uses a supervised CNN (convolutional neural network) that has been trained using images from healthy and unhealthy patients.” [0058]; [0052-0113], [fig. 4, 6-7], [see claim 6 rejection]);
obtaining, from the trained neural network model, pixel-based class information indicating classes of pixels included in the at least one frame (“In step 655, device 100 determines if the operation of step 650 classified the image as indicative of disease.” [0059]; “the normalized grayscale image (obtained in step 650) is passed through a thresholding algorithm to create a pixelated image that highlights the areas that have high contrast values.” [0060]; [0052-0113], [fig. 4, 6-7], [see claim 6 rejection]); and
identify the type of the object based on the pixel-based class information (“In step 670, the pixelated image obtained from step 660 is passed to a contour recognition algorithm which identifies the diseased areas in the image and highlights them.” [0061]; [0052-0113], [fig. 4, 6-7], [see claim 6 rejection]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the data processing apparatus taught by Hawkins by applying a trained neural network model as taught by Kumar. While existing scanner devices provide satisfactory results, during the scanning process the user may accidentally scan items they wish to be deleted. Depending on the nature of the artifact or its location in the scan, known deletion processes may be time-consuming and frustrating for the user and it may not possible to delete without adversely impacting the integrity of the scan (Hawkins [0005]). The weighting technique may provide significant advantages. For line-of-sight deletion in particular, the weight-based approach ensures that erroneous data close to the true surface will be gradually deleted as more information is collected about the true surface geometry, thus preserving surface accuracy (Hawkins [0051]). Additionally, an off-the-shelf device (e.g., a smart phone with an operating system) can install and execute applications for analysis of captured images and identification of oral diseases (Kumar [0030]). Using categorized images from a neural net algorithm, a contour recognition algorithm may identify diseased areas within the image and highlight them. A clinician can interpret the image and advise the patient as to the next steps (Kumar [0058-0061]).
Regarding claim 18, Hawkins and Kumar teach the data processing apparatus of claim 17,
Hawkins further teaching wherein the at least one processor is further configured to execute the at least one instruction to: calculate, based on the pixel-based class information, a percentage of pixels identified as a specific class among all pixels included in the at least one frame (“wherein the program code configured to adjust the weights operates as scanning proceeds and includes program code further configured to: decrease a weight of a 3D sample that is determined to be within a line-of-sight to a point within the 3D point cloud;” [clm 12]; “The scanned object is represented in software as a collection of discrete, 3D samples that approximate the real, continuous surface (such as the surface of a tooth.) These 3D samples may be organized into a voxel grid, or may be simply a list of 3D points.” [0039]; “the normalized grayscale image (obtained in step 650) is passed through a thresholding algorithm to create a pixelated image that highlights the areas that have high contrast values.” [0060]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 7 rejection]); and
identify the type of the object as the specific class based on the calculated percentage being greater than or equal to a first reference value (“responsive to determining that the resulting adjusted weight of the 3D sample has fallen below the threshold, mark that 3D sample for deletion.” [clm 12]; “For any of these points that are closer to the camera than the scan point at the corresponding pixel, its weight is decreased, say by a factor S. Then, if the weight of any point in the point cloud falls below a threshold, that weight is set equal to zero. […] In addition, the threshold itself may be configurable.” [0043]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 7 rejection]).
Regarding claim 19, Hawkins and Kumar teach the data processing apparatus of claim 18,
Hawkins further teaching wherein the at least one frame comprises a reference number of frames obtained after a scan operation starts (“In one sequence, a first set (e.g., six) pattern frames are used, interspersed with a second set (e.g., three) illumination frames, for a sequence total of nine total CCD frames.” [0024]; “good results were obtained with a live preview window of approximately 20 frames per second, coupled with approximately 15 frames per second for the 3D data” [0032]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 8 rejection]), and
wherein the at least one processor is further configured to execute the at least one instruction to identify the type of the object as the specific class based on a percentage of frames identified as the specific class among the reference number of frames being greater than or equal to a second reference value (“In one sequence, a first set (e.g., six) pattern frames are used, interspersed with a second set (e.g., three) illumination frames, for a sequence total of nine total CCD frames. A software traffic shaper is then used to separate captured frames in two streams, namely, a live preview stream, and a data processing stream from which the 3D model is generated.” [0024]; “The traffic shaper separates the data into live preview and 3D processing input frames. The 3D processing input frames contain projected patterns.” [0031]; “For any of these points that are closer to the camera than the scan point at the corresponding pixel, its weight is decreased, say by a factor S. Then, if the weight of any point in the point cloud falls below a threshold, that weight is set equal to zero. This is equivalent to deleting the point. The weighting factor S may vary depending on the geometry. For example, if the point cloud point is very close to the corresponding scan point, a lower value of S may be used, in effect deleting the point cloud point more slowly. […] In addition, the threshold itself may be configurable.” [0043]; [0023-0049], [fig. 1, 4-7, 9, 11-12], [see claim 8 rejection]).
Claim(s) 9 and 20 is/are rejected under 35 U.S.C. 103 as being obvious over Hawkins as applied to claim 1 above, in view of Minchenkov et al. (US20200349698A1, 2020-11-05).
Regarding claim 9, Hawkins teaches the data processing method of claim 1,
but Hawkins may fail to explicitly teach a plurality of activation modes corresponding to respective different classes to be filtered.
However, in the same field of endeavor, Minchenkov teaches a data processing method performed by a data processing apparatus connected to a 3D scanner (“A method includes processing an input comprising data from an intraoral image using a trained machine learning model that has been trained to classify regions of dental sites” [abst]; “A method comprising: […] processing a plurality of inputs using a trained machine learning model that has been trained to classify regions of dental sites,” [clm 19]; “the scanner 150 may provide intraoral scan data 135A-N […] each of which may be referred to as intraoral images.” [0044]; [0040-0052, 0083-0094], [fig. 1-3C, 5A-6]);
Minchenkov further teaching wherein the scan filtering mode comprises a plurality of activation modes respectively corresponding to different classes to be filtered (“machine learning is used to segment images (e.g., 2D images comprising height maps) into their relevant dental classes. The relevant dental classes may include teeth, gums, and excess material (e.g., material other than teeth or gums). The relevant dental classes may additionally or alternatively include other classes as well, such as an upper palate, a gingival line, a scan body, a finger, or a preparation tooth.” [0038]; “the machine learning model may be trained to generate a probability map, where each point in the probability map corresponds to a pixel of an input image and indicates one or more of a first probability that the pixel represents a first dental class, a second probability that the pixel represents a second dental class, a third probability that the pixel represents a third dental class, a fourth probability that the pixels represents a fourth dental class, a fifth probability that the pixel represents a fifth dental class, and so on.” [0080]; A trained machine learning model generates a probability map (i.e., activation mode) representing respective dental classes (e.g., teeth, gums, excess material) during image segmentation [0040-0052, 0083-0094], [fig. 1-3C, 5A-6]),
wherein, when the scan filtering mode is activated, the generating of the 3D scan data as the filtered 3D scan data comprises generating the filtered 3D scan data by removing, from the plurality of frames, pixels corresponding to one or more classes to be filtered according to one activation mode among the plurality of activation modes (“The trained machine learning model 255 outputs a probability map 260, where each point in the probability map corresponds to a pixel in the image and indicates probabilities that the pixel represents one or more dental classes. In the case of teeth/gums/excess material segmentation, three valued labels are generated for each pixel” [0105]; “In a first technique for applying the probability map, processing logic determines whether the probability of a pixel being excess material is larger than the probabilities of the pixel being anything else (e.g., larger than the probability of the pixel being gums or teeth. A pixel is then determined to be in the excess material class” [0111]; The trained machine model assigns probabilities to the pixels, and then may apply a filter to remove excess material based on the determination that one probability is greater than other probabilities [0040-0052, 0083-0094], [fig. 1-3C, 5A-6]), and
wherein the one activation mode among the plurality of activation modes comprises at least one of an activation mode selected by a user, an activation mode selected by default, and an activation mode used in a previous scan operation (“given a probability map that includes probabilities of pixels belonging to various dental classes, where at least one of those dental classes is an excess material dental class, the system can filter out pixels that correspond to excess material.” [0110]; [0040-0052, 0083-0094], [fig. 1-3C, 5A-6]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the data processing method performed by a data processing apparatus connected to a 3D scanner taught by Hawkins by applying a plurality of activation modes as taught by Minchenkov. Inclusion of excess material in intraoral scans, and ultimately in the 3D model of the dental arch generated from the 3D scans, can be problematic. The excess material often occludes clinically significant regions, such as teeth and gums (Minchenkov [0003]). The weighting technique may provide significant advantages. For line-of-sight deletion in particular, the weight-based approach ensures that erroneous data close to the true surface will be gradually deleted as more information is collected about the true surface geometry, thus preserving surface accuracy (Hawkins [0051]). The techniques may allow a system to significantly improve scanning quality of a dental site and user experience of a user of an intraoral scanner and may reduce the cost of dental prosthetics or orthodontia manufactured based on intraoral scans by increasing the accuracy and quality of those scans (Minchenkov [0035]).
Regarding claim 20, Hawkins teaches the data processing apparatus of claim 11, but Hawkins may fail to explicitly teach a plurality of activation modes corresponding to respective different classes to be filtered.
However, in the same field of endeavor, Minchenkov teaches a data processing apparatus connected to a 3D scanner (“A method includes processing an input comprising data from an intraoral image using a trained machine learning model that has been trained to classify regions of dental sites” [abst]; “a system 100 for performing intraoral scanning and/or generating a virtual three-dimensional model of an intraoral site” [0040]; “the scanner 150 may provide intraoral scan data 135A-N […] each of which may be referred to as intraoral images.” [0044]; [0040-0052, 0083-0094], [fig. 1-3C, 5A-6], [see claim 9 rejection]);
Minchenkov further teaching wherein the scan filtering mode comprises a plurality of activation modes respectively corresponding to different classes to be filtered (“machine learning is used to segment images (e.g., 2D images comprising height maps) into their relevant dental classes. The relevant dental classes may include teeth, gums, and excess material (e.g., material other than teeth or gums). The relevant dental classes may additionally or alternatively include other classes as well, such as an upper palate, a gingival line, a scan body, a finger, or a preparation tooth.” [0038]; “the machine learning model may be trained to generate a probability map, where each point in the probability map corresponds to a pixel of an input image and indicates one or more of a first probability that the pixel represents a first dental class, a second probability that the pixel represents a second dental class, a third probability that the pixel represents a third dental class, a fourth probability that the pixels represents a fourth dental class, a fifth probability that the pixel represents a fifth dental class, and so on.” [0080]; [0040-0052, 0083-0094], [fig. 1-3C, 5A-6], [see claim 9 rejection]),
wherein, when the scan filtering mode is activated, the at least one processor is further configured to execute the at least one instruction to generate the 3D scan data as the filtered 3D scan data by removing, from the plurality of frames, pixels corresponding to one or more classes to be filtered according to one activation mode among the plurality of activation modes (“The trained machine learning model 255 outputs a probability map 260, where each point in the probability map corresponds to a pixel in the image and indicates probabilities that the pixel represents one or more dental classes. In the case of teeth/gums/excess material segmentation, three valued labels are generated for each pixel” [0105]; “In a first technique for applying the probability map, processing logic determines whether the probability of a pixel being excess material is larger than the probabilities of the pixel being anything else (e.g., larger than the probability of the pixel being gums or teeth. A pixel is then determined to be in the excess material class” [0111]; [0040-0052, 0083-0094], [fig. 1-3C, 5A-6], [see claim 9 rejection]), and
wherein the one activation mode among the plurality of activation modes comprises at least one of an activation mode selected by a user, an activation mode selected by default, and an activation mode used in a previous scan operation (“given a probability map that includes probabilities of pixels belonging to various dental classes, where at least one of those dental classes is an excess material dental class, the system can filter out pixels that correspond to excess material.” [0110]; [0040-0052, 0083-0094], [fig. 1-3C, 5A-6]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the data processing apparatus taught by Hawkins by applying a plurality of activation modes as taught by Minchenkov. Inclusion of excess material in intraoral scans, and ultimately in the 3D model of the dental arch generated from the 3D scans, can be problematic. The excess material often occludes clinically significant regions, such as teeth and gums (Minchenkov [0003]). The weighting technique may provide significant advantages. For line-of-sight deletion in particular, the weight-based approach ensures that erroneous data close to the true surface will be gradually deleted as more information is collected about the true surface geometry, thus preserving surface accuracy (Hawkins [0051]). The techniques may allow a system to significantly improve scanning quality of a dental site and user experience of a user of an intraoral scanner and may reduce the cost of dental prosthetics or orthodontia manufactured based on intraoral scans by increasing the accuracy and quality of those scans (Minchenkov [0035]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James F. McDonald III whose telephone number is (571)272-7296. The examiner can normally be reached M-F; 8AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Koharski can be reached at 5712727230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JAMES FRANKLIN MCDONALD III
Examiner
Art Unit 3797
/CHRISTOPHER KOHARSKI/Supervisory Patent Examiner, Art Unit 3797
