DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 11-14, and 17-18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Inglese (US 20190046276 A1).
Regarding Claim 11, Inglese teaches An electronic device, comprising (As can be appreciated by those skilled in the image processing arts, a computer program of an embodiment of the present disclosure can be utilized by a suitable, general-purpose computer system, such as a personal computer or workstation):
A processor (Paragraph [0157]: “As can be appreciated by those skilled in the image processing arts, a computer program of an embodiment of the present disclosure can be utilized by a suitable, general-purpose computer system, such as a personal computer or workstation that acts as an image processor, when provided with a suitable software program so that the processor operates to acquire, process, and display data as described herein. Many other types of computer systems architectures can be used to execute the computer program of the present disclosure, including an arrangement of networked processors, for example”); and
A memory configured to store an instruction that is executable by the processor, wherein the processor is configured to read the executable instruction from the memory, and execute the executable instruction to (Paragraph [0157]: “Consistent with one embodiment, the present disclosure utilizes a computer program with stored instructions that control system functions for image acquisition and image data processing for image data that is stored and accessed from an electronic memory. As can be appreciated by those skilled in the image processing arts, a computer program of an embodiment of the present disclosure can be utilized by a suitable, general-purpose computer system, such as a personal computer or workstation that acts as an image processor, when provided with a suitable software program so that the processor operates to acquire, process, and display data as described herein. Many other types of computer systems architectures can be used to execute the computer program of the present disclosure, including an arrangement of networked processors, for example”; Paragraph [0159]: “It is noted that the term “memory”, equivalent to “computer-accessible memory” in the context of the present disclosure, can refer to any type of temporary or more enduring data storage workspace used for storing and operating upon image data and accessible to a computer system, including a database. The memory could be non-volatile, using, for example, a long-term storage medium such as magnetic or optical storage”):
in response to an acquired current point cloud data frame of a tooth, generate and display a current tooth model according to a preset tracking registering algorithm and the current point cloud data frame (Paragraph [0077]: “By projecting and capturing images that show structured light patterns that duplicate the arrangement shown in FIG. 3 multiple times, the image of the contour line on the camera simultaneously locates a number of surface points of the imaged object. This speeds the process of gathering many sample points, while the plane of light (and usually also the receiving camera) is laterally moved in order to “paint” some or all of the exterior surface of the object with the plane of light”; Paragraph [0078]: “FIG. 4 shows surface imaging using a pattern with multiple lines of light. Incremental shifting of the line pattern and other techniques help to compensate for inaccuracies and confusion that can result from abrupt transitions along the surface, whereby it can be difficult to positively identify the segments that correspond to each projected line. In FIG. 4, for example, it can be difficult over portions of the surface to determine whether line segment 16 is from the same line of illumination as line segment 18 or adjacent line segment 19”; Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three-dimensional surface of an object”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D,demonstrating the application to a tooth/teeth. Notes: a preset tracking registering algorithm, in its broadest reasonable interpretation, is an algorithm that tracks scans/points of an object or objects and registers them with one another to create a unified model or representation);
store the current point cloud data frame into a preset database, and determine whether a preset model updating condition is met according to all point cloud data frames comprised in the preset database (Paragraph [0153]: “The processing carried out in steps S2320 and S2330 of FIG. 23 is executed for each B-scan obtained by the OCT imaging apparatus. A decision step S2350 then determines whether or not all B-scans in the set have been processed. Once processing is complete for the B-scans, the combined B-scans form a surface point cloud for the teeth. A mesh generation and rendering step S2380 then generates and renders a 3D mesh from the surface point cloud. The rendered OCT surface data can be displayed, stored, or transmitted”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D. Notes: A database, in its broadest reasonable interpretation, is a collection of stored data. Processing data inherently requires the data to be stored. The broadest reasonable interpretation of an updating condition is any condition that results in an update, which is satisfied by the overlap condition in Paragraph [0086]); and
in a case that the preset model updating condition is met, determine a reference tooth model according to all the point cloud data frames, and updating the current tooth model according to the reference tooth model (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D. Notes: a reference tooth model, in its broadest reasonable interpretation, is any model that is used for the update of the current tooth model).
Claim 1, being similar in scope to Claim 11, is rejected under the same rationale.
Claim 12, being similar in scope to Claim 11, is rejected under the same rationale.
Regarding Claim 17, the electronic device as claimed in Claim 11 is rejected over Inglese.
Inglese teaches the electronic device as claimed in Claim 11, the processor further configured to:
in a case that acquisition of the point cloud data frame of the tooth is completed, determine that the displayed current tooth model is a target tooth model (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object”; Paragraph [0156]: “A reconstruction step S2560 then reconstructs a 3D image of the tooth according to the depth-resolved response signal and the adjusted tooth surface structure information. A rendering step S2570 then renders the volume image content for display, transmission, or storage”. Notes: the broadest reasonable interpretation of a target tooth is a tooth of interest, which is displayed in Inglese, Paragraph [0156])
Claim 2, being similar in scope to Claim 17, is rejected under the same rationale.
Regarding Claim 18, the electronic device as claimed in Claim 11 is rejected over Inglese.
Inglese teaches the electronic device as claimed in Claim 11, wherein the processor is further configured to:
in a case that the current point cloud data frame is the first point cloud data frame acquired, construct and displaying the current tooth model according to the current point cloud data frame (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”);
in a case that the current point cloud data frame is not the first point cloud data frame acquired, acquire a historical tooth model recently displayed (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D for the display of a previous tooth model with an acquired tooth model; Paragraph [0156]: “A reconstruction step S2560 then reconstructs a 3D image of the tooth according to the depth-resolved response signal and the adjusted tooth surface structure information. A rendering step S2570 then renders the volume image content for display, transmission, or storage”); and
register the current point cloud data frame on the historical tooth model to acquire and display the current tooth model (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”).
Claim 3, being similar in scope to Claim 18, is rejected under the same rationale.
Regarding Claim 13, the method as claimed in Claim 1 is rejected over Inglese.
Inglese teaches the method as claimed in Claim 1, wherein the current point cloud data frame comprises:
A combination of three-dimensional data points of a plurality of scanning points on a teeth scanned at the same time (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”).
Regarding Claim 14, the method as claimed in Claim 1 is rejected over Inglese.
Inglese teaches the method as claimed in Claim 1, the generating and displaying the current tooth model according to the preset tracking algorithm and the current point cloud data frame comprises:
matching the point cloud data frame in the current point cloud data frame and point cloud data frame on a target tooth model in a previous acquisition period, to determine an overlapping area of the current point cloud data frame and the point cloud data frame on the target tooth model in the previous acquisition period (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”);
or matching the point cloud data frame in the current point cloud data frame and the point cloud data frame in the previous acquisition period, to determine an overlapping area of the current point cloud data frame and the point cloud data frame on the target tooth model in the previous acquisition period (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”);
positioning a registering position of the current point cloud data frame based on the overlapping area (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D. Notes: The broadest reasonable interpretation of registering the position of point cloud data is aligning point cloud data with other point cloud data); and
registering the current point cloud data frame on the target tooth model of the previous acquisition period based on the registering position, to obtain the current tooth model of the current period (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D. Notes: The broadest reasonable interpretation of registering the position of point cloud data is aligning point cloud data with other point cloud data).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 4, 8, 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Inglese (US 20190046276 A1), in view of Lim (US 20180341836 A1) and Gao (An approach to combine progressively captured point clouds for BIM update, 2015).
Regarding Claim 19, the electronic device as claimed in Claim 11 is rejected over Inglese.
Inglese teaches determining a preset model updating condition (Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”).
Inglese does not teach a case where a preset number threshold is used for determining that the preset model updating condition is met.
However, Gao teaches a case where a preset number threshold is used for determining that the preset model updating condition is met (Section 4.2: “The model iteratively combines the remaining point clouds to the baseline point cloud till either all the point clouds have been registered together or the maximum dissimilarity ratio is smaller than a threshold, whichever occurs earlier. Users determine the value of the threshold. If the dissimilarity ratio of a point cloud is smaller than the threshold, it is not worth adding this point cloud to the baseline point cloud set since the combination will increase the file size without bringing enough additional geometric information for the model update/construction. The output of the content improvement module is a point cloud that is composed of the selected point clouds”; Figure 8 illustrates the process of combining multiple point clouds of an object based on a preset number threshold; Figure 5 also illustrates the general process of combining point clouds from different points in time).
Inglese and Gao are considered analogous in the art with respect to obtaining point clouds over time for an object(s) of interest. A common motivation in the art is to utilize a numeric threshold to check whether a point cloud model should be updated based on a current point cloud model, as is evident in Gao.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the preset model updating condition for displaying a point cloud tooth model of Inglese with the use of a numeric threshold for updating a point cloud model of Gao; Doing so would yield the predictable result of an update condition that checks whether it is needed to add a current point cloud to an existing point cloud model.
Inglese as modified does not teach counting the number of current frames of all the point cloud data frames comprised in the preset database, nor does it teach a case where the number of current frames is the preset number threshold used to determine that the preset model updating condition is met.
However, Lim teaches counting the number of current frames of all the point cloud data frames comprised in the preset database, where there is a preset number threshold (Paragraph [0054]: “The training phase may not be completed until at least a threshold number of training point clouds are used to generate super-resolved point clouds that the discriminator mistakenly perceives to depict actual, real objects, and not computer-generated objects. For example, the GAN system may be trained using ten or more training point clouds”).
Furthermore, while Inglese as modified does not explicitly teach a number of current frames of point cloud data as a threshold, it does teach that not all point cloud data frames are required to update a preset model (Gao, Section 4.2: “The model iteratively combines the remaining point clouds to the baseline point cloud till either all the point clouds have been registered together or the maximum dissimilarity ratio is smaller than a threshold, whichever occurs earlier. Users determine the value of the threshold. If the dissimilarity ratio of a point cloud is smaller than the threshold, it is not worth adding this point cloud to the baseline point cloud set since the combination will increase the file size without bringing enough additional geometric information for the model update/construction. The output of the content improvement module is a point cloud that is composed of the selected point clouds”).
Inglese as modified and Lim are considered analogous in the art with respect to the use of multiple point clouds for visualization purposes. Counting the number of point clouds as a completion metric (threshold) is common in the art, as is demonstrated by Lim. Additionally, as noted by Gao, a point cloud in a set does not need to be utilized for an update of a combined point cloud model if the additional geometric information for the model update is not worth adding. Gao teaches combining the point clouds together until a dissimilarity threshold is reached, which can result in excluded point clouds. A person having ordinary skill in the art would find it obvious that instead of a dissimilarity threshold, a more basic threshold based on a number of point cloud frames utilized for the combined point cloud model may be implemented. A motivation for doing so would be to reduce the computations needed for calculating a dissimilarity threshold in favor of a more general, efficient method requiring only the counting of point cloud frames used.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the updating condition involving a numeric threshold for displaying a tooth model of Inglese as modified with the counting and threshold of a number of point cloud frames of Lim; Doing so would yield the predictable result of an updating condition for a tooth model that efficiently updates the model based on a simple point cloud frame counting threshold.
Claim 4, being similar in scope to Claim 19, is rejected under the same rationale.
Regarding Claim 8, the method as claimed in Claim 1 is rejected over Inglese.
Inglese teaches the method as claimed in Claim 1, wherein the determining the reference tooth model according to all the point cloud data frames comprises:
in a case that a previous reference tooth model determined by previous optimization is acquired, generating an initial reference tooth model according to all the point cloud data frames comprised in the preset database and the first model position of each point cloud data frame in the previous reference tooth model (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object.”; Paragraph [0080]: “The surface data for surface contour characterization, also referred to as a surface data set, is obtained by a process that derives individual points from the structured images, typically in the form of a point cloud, wherein the individual points represent points along the surface of the imaged tooth or other feature. A close approximation of the surface object can be generated from a point cloud by connecting adjacent points and forming polygons, each of which closely approximates the contour of a small portion of the surface”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D. Notes: the model position of the point clouds are known by virtue of being composed of points with known coordinates. An existing mesh point cloud can be augmented with additional point clouds, as is evident in Figure 6C and Figure 6D); and
updating the initial reference tooth model according to the remaining point cloud data frames (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”; Refer to Figure 6C and Figure 6D).
Inglese does not teach the number of remaining point cloud data frames being less than a preset number threshold for updating a reference tooth model.
However, Gao teaches a numerical value related to dissimilarity between point clouds being less than a preset number threshold in relation to updating a point cloud model with point clouds (Section 4.2: “The model iteratively combines the remaining point clouds to the baseline point cloud till either all the point clouds have been registered together or the maximum dissimilarity ratio is smaller than a threshold, whichever occurs earlier. Users determine the value of the threshold. If the dissimilarity ratio of a point cloud is smaller than the threshold, it is not worth adding this point cloud to the baseline point cloud set since the combination will increase the file size without bringing enough additional geometric information for the model update/construction. The output of the content improvement module is a point cloud that is composed of the selected point clouds”; Figure 8 illustrates the process of combining multiple point clouds of an object based on a preset number threshold; Figure 5 also illustrates the general process of combining point clouds from different points in time).
Inglese and Gao are considered analogous in the art with respect to obtaining point clouds over time for an object(s) of interest. A common motivation in the art is to utilize a numeric threshold to check whether a point cloud model should be updated based on a current point cloud model, as is evident in Gao.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the preset model updating condition for displaying a point cloud tooth model of Inglese with the use of a numeric threshold for updating a point cloud model of Gao; Doing so would yield the predictable result of an update condition that checks whether it is needed to add a current point cloud to an existing point cloud model.
Inglese as modified does not teach determining a number of remaining point cloud data frames in the preset database is less than the preset number threshold.
However, Lim teaches determining the number of current frames of all the point cloud data frames comprised in the preset database, where there is a preset number threshold (Paragraph [0054]: “The training phase may not be completed until at least a threshold number of training point clouds are used to generate super-resolved point clouds that the discriminator mistakenly perceives to depict actual, real objects, and not computer-generated objects. For example, the GAN system may be trained using ten or more training point clouds”).
Inglese as modified and Lim are considered analogous in the art with respect to the use of multiple point clouds for visualization purposes. Counting the number of point clouds as a completion metric (threshold) is common in the art, as is demonstrated by Lim.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the updating condition involving a numeric threshold for displaying a tooth model of Inglese as modified with the counting and threshold of a number of point cloud frames of Lim; Doing so would yield the predictable result of an updating condition for a tooth model that efficiently updates the model based on a simple point cloud frame counting threshold.
Inglese as modified does not teach in a case that the number of remaining point cloud data frames is not less than the preset number threshold, updating the initial reference tooth model according to the remaining point cloud data frames, until the number of remaining point cloud data frames in the preset database is less than the preset number threshold; and in a case that the number of remaining point cloud data frames in the preset database is less than the preset number threshold, interrupting acquisition of the point cloud data frame of the tooth, and updating the initial reference tooth model according to the remaining point cloud data frames, so as to obtain the reference tooth model.
However, Inglese as modified teaches that not all point cloud data frames are required to update a preset model (Gao, Section 4.2: “The model iteratively combines the remaining point clouds to the baseline point cloud till either all the point clouds have been registered together or the maximum dissimilarity ratio is smaller than a threshold, whichever occurs earlier. Users determine the value of the threshold. If the dissimilarity ratio of a point cloud is smaller than the threshold, it is not worth adding this point cloud to the baseline point cloud set since the combination will increase the file size without bringing enough additional geometric information for the model update/construction. The output of the content improvement module is a point cloud that is composed of the selected point clouds”). Note that Inglese teaches acquiring point cloud data over time (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object”).
Additionally, as noted by Gao, a point cloud in a set does not need to be utilized for an update of a combined point cloud model if the additional geometric information for the model update is not worth adding. Gao teaches combining the point clouds together until a dissimilarity threshold is reached, which can result in excluded point clouds. A person having ordinary skill in the art would find it obvious that instead of a dissimilarity threshold, a more basic threshold based on a number of point cloud frames utilized for the combined point cloud model may be implemented. A motivation for doing so would be to reduce the computations needed for calculating a dissimilarity threshold in favor of a more general, efficient method requiring only the counting of point cloud frames used. Hence, in the context of determining whether the number of remaining point cloud data frames in the preset database is less than the preset number threshold, in a case that the number of remaining point cloud data frames is not less than the preset number threshold, updating the initial reference tooth model according to the remaining point cloud data frames, until the number of remaining point cloud data frames in the preset database is less than the preset number threshold would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Likewise, in a case that the number of remaining point cloud data frames in the preset database is less than the preset number threshold, interrupting acquisition of the point cloud data frame of the tooth, and updating the initial reference tooth model according to the remaining point cloud data frames, so as to obtain the reference tooth model would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention. Doing so would result in the predictable result of a simpler and more generic implementation of determining whether a tooth model should continue to utilize additional acquired point cloud data.
Regarding Claim 9, the method as claimed in Claim 8 is rejected over Inglese as modified.
Inglese as modified teaches determining whether the acquisition of the point cloud data frame of the tooth is completed (Inglese, Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object”; Refer to Inglese, Figure 6C and Inglese, Figure 6D. Notes: a determination of whether the acquisition of the point cloud data frame of the tooth is completed is inherent, considering obtaining portions of the tooth model for combination is performed separately, as is evident in Inglese, Figure 6C and Inglese, Figure 6D, which necessitates a determination of the completion of the acquisition of the point cloud data associated with that tooth portion), and
In a case that the acquisition of the point cloud data frame of the tooth is not completed, continuously acquiring the current point cloud data frame of the tooth (Inglese, Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object”; Refer to Inglese, Figure 6C and Inglese, Figure 6D).
Claims 5 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Inglese (US 20190046276 A1), in view of Urbach (DPDist: Comparing Point Clouds Using Deep Point Cloud Distance, 2020) and Metal Industries Research and Development Centre (CN106725966A).
Regarding Claim 20, the electronic device as claimed in Claim 11 is rejected over Inglese.
Inglese teaches the electronic device as claimed in Claim 11, wherein the processor is further configured to:
according to the current all point cloud data frames comprised in the preset database, determine a first model position of each point cloud data frame (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three dimensional surface of an object”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”. Notes: The broadest reasonable interpretation of a model position is coordinates of the model points).
Inglese does not teach determining a second model position of each point cloud data frame in the displayed current tooth model, does not teach calculating a position error of each point cloud data frame according to the first model position and the second model position.
However, Urbach teaches determining a second model position of each point cloud data frame in the displayed current tooth model (Figure 1: “Distance to surface vs. distance to sampled point cloud. Every figure contains two samples of the same object (a table). The samples can be identical (left) or different (right). The distance of every point from one point cloud (disks) to the surface estimated from the other point cloud (+) is given as a colored code in the top two figure.”), and
calculating a position error of each point cloud data frame according to the first model position and the second model position (Section 2.1, Point Cloud Distance: “Consider two point clouds
S
A
,
S
B
i
n
R
3
, with
N
A
,
N
B
points in each cloud, respectively. An early method for comparing point sets is the Hausdorff distance (
D
H
()), which builds on the minimum distance from a point to a set
d
x
,
y
=
x
-
y
2
,
D
x
,
S
=
m
i
n
y
∈
S
d
(
x
,
y
)
and calculates a symmetric max min distance
D
H
S
A
,
S
B
=
m
a
x
{
m
a
x
a
∈
S
a
D
a
,
S
B
,
m
a
x
b
∈
S
b
D
b
,
S
A
”).
Inglese and Urbach are considered analogous in the art with respect to the use of multiple point clouds for visualization purposes. A common motivation in the art is to derive multiple point clouds for validation purposes, such as checking whether point clouds of an object are similar to one another, as is evident in Urbach.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the determination of a first model position of a point cloud data frame with the determination of a first and second model position of a point cloud data frame and subsequent position error calculation between the two point clouds of Urbach; Doing so would yield the predictable result of an error quantity between the two point clouds of a particular object.
Inglese as modified does not teach determining whether the preset model updating condition is met in accordance with the position error.
However, Metal Industries Research and Development Centre (going forward abbreviated to MIRDC) teaches determining whether the preset model updating condition is met in accordance with position error (Abstract: “The method comprises steps of scanning a dental body, and acquiring first point cloud data and second point cloud data of three-dimensional data of the dental body; calculating a three-dimensional space range where the data of the three-dimensional data is located; calculating initial rotation and displacement quantity of the second point cloud data so as to perform rigid body conversion; calculating multiple corresponding point pairs corresponding to the first and second point cloud data, and filtering at least one outlier pair in the multiple corresponding point pairs; and according to the corresponding point pairs of the first and second cloud point data of filtered outlier points, estimating estimated rotation and displacement quantity of the second point cloud data, carrying out another rigid body conversion, and calculating whether an error value of overlapping attachment of the first point cloud data and the second point cloud data is lower than a threshold value, thereby obtaining a dental body model corresponding to the dental body)
Inglese as modified and MIDRC are considered analogous in the art with respect to utilizing point clouds to model teeth. A common motivation for updating a model is to utilize a thresholding of a difference between the point clouds such as position error, as is evident in MIRDC.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the determination of a first point cloud position and second point cloud position and subsequent position error calculation of Inglese as modified with the use of position error as a threshold; Doing so would yield the predictable result of an updating condition based on position error for point clouds reaching a threshold.
Claim 5, being similar in scope to Claim 20, is rejected under the same rationale.
Claims 6, 7 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Inglese (US 20190046276 A1), in view of Urbach (DPDist: Comparing Point Clouds Using Deep Point Cloud Distance, 2020) and Metal Industries Research and Development Centre (CN106725966A), in further view of Fuse (Development of a Change Detection Method with Low-Performance Point Cloud Data for Updating Three-Dimensional Road Maps, 2017).
Regarding Claim 21, the electronic device as claimed in Claim 11 is rejected over Inglese as modified.
Inglese as modified teaches the electronic device as claimed in Claim 11, wherein the processor is further configured to::
Calculate an error mean of all the position errors (Urbach, Section 2.1, Point Cloud Distance: “Consider two point clouds
S
A
,
S
B
i
n
R
3
, with
N
A
,
N
B
points in each cloud, respectively. An early method for comparing point sets is the Hausdorff distance (
D
H
()), which builds on the minimum distance from a point to a set
d
x
,
y
=
x
-
y
2
,
D
x
,
S
=
m
i
n
y
∈
S
d
(
x
,
y
)
and calculates a symmetric max min distance
D
H
S
A
,
S
B
=
m
a
x
{
m
a
x
a
∈
S
a
D
a
,
S
B
,
m
a
x
b
∈
S
b
D
b
,
S
A
”; Urbach, Section 2.1, Point Cloud Distance: “The Chamfer distance … and partial Hausdorff … distances are two of its variants, based on averaging (instead of taking the maximum)”); and
determining whether the error mean is satisfies a preset average error threshold, and in a case that the error mean satisfies the preset average error threshold, determining that the preset model updating condition is met (MIDRC, Abstract: “The method comprises steps of scanning a dental body, and acquiring first point cloud data and second point cloud data of three-dimensional data of the dental body; calculating a three-dimensional space range where the data of the three-dimensional data is located; calculating initial rotation and displacement quantity of the second point cloud data so as to perform rigid body conversion; calculating multiple corresponding point pairs corresponding to the first and second point cloud data, and filtering at least one outlier pair in the multiple corresponding point pairs; and according to the corresponding point pairs of the first and second cloud point data of filtered outlier points, estimating estimated rotation and displacement quantity of the second point cloud data, carrying out another rigid body conversion, and calculating whether an error value of overlapping attachment of the first point cloud data and the second point cloud data is lower than a threshold value, thereby obtaining a dental body model corresponding to the dental body).
Inglese as modified does not teach that the error mean is greater than a preset average error threshold.
However, Fuse teaches that the error is greater than a preset threshold when comparing between point clouds (Section 1.2, Related Work: “Other than a few exceptions [25], most of the change detection methods only use image or point cloud data. In this study, we only work with point cloud data. To detect changes between two point clouds, a basic method is to look for differences based on the distance between points. If the distances between pairs of points are larger than a threshold, then this area is considered to have changed. However, this approach is not robust against noise or differences in point density. The Hausdorff distance is known to be robust against data damages”).
Inglese as modified and Fuse are considered analogous in the art with respect to the comparison of position and derivation of position error between two point clouds using Hausdorff distance. One would be motivated to determine an update condition based on an observable difference between point clouds through position error being greater than a threshold, as is evident in Fuse.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the determination of an error mean for two point clouds and a determination of whether the error mean satisfies a threshold of Inglese as modified with the determination that the error between two point clouds is greater than a threshold of Fuse; Doing so would yield the predictable result of determining whether two point clouds are different enough for an updating condition to be fulfilled.
Claim 6, being similar in scope to Claim 21, is rejected under the same rationale.
Regarding Claim 7, the method as claimed in Claim 5 is rejected over Inglese as modified.
Inglese as modified teaches the method as claimed in Claim 5, wherein the determining whether the preset model updating condition is met according to the position error comprises:
determining a maximum position error among all the position errors (Urbach, Section 2.1, Point Cloud Distance: “Consider two point clouds
S
A
,
S
B
i
n
R
3
, with
N
A
,
N
B
points in each cloud, respectively. An early method for comparing point sets is the Hausdorff distance (
D
H
()), which builds on the minimum distance from a point to a set
d
x
,
y
=
x
-
y
2
,
D
x
,
S
=
m
i
n
y
∈
S
d
(
x
,
y
)
and calculates a symmetric max min distance
D
H
S
A
,
S
B
=
m
a
x
{
m
a
x
a
∈
S
a
D
a
,
S
B
,
m
a
x
b
∈
S
b
D
b
,
S
A
”); and
determining whether the maximum position error is satisfies a preset maximum error threshold, and in a case that the maximum position error satisfies the maximum threshold, determining that the preset model updating condition is met (MIDRC, Abstract: “The method comprises steps of scanning a dental body, and acquiring first point cloud data and second point cloud data of three-dimensional data of the dental body; calculating a three-dimensional space range where the data of the three-dimensional data is located; calculating initial rotation and displacement quantity of the second point cloud data so as to perform rigid body conversion; calculating multiple corresponding point pairs corresponding to the first and second point cloud data, and filtering at least one outlier pair in the multiple corresponding point pairs; and according to the corresponding point pairs of the first and second cloud point data of filtered outlier points, estimating estimated rotation and displacement quantity of the second point cloud data, carrying out another rigid body conversion, and calculating whether an error value of overlapping attachment of the first point cloud data and the second point cloud data is lower than a threshold value, thereby obtaining a dental body model corresponding to the dental body).
Inglese as modified does not teach that the maximum position error is greater than a preset maximum error threshold.
However, Fuse teaches that the error is greater than a preset threshold when comparing between point clouds (Section 1.2, Related Work: “Other than a few exceptions [25], most of the change detection methods only use image or point cloud data. In this study, we only work with point cloud data. To detect changes between two point clouds, a basic method is to look for differences based on the distance between points. If the distances between pairs of points are larger than a threshold, then this area is considered to have changed. However, this approach is not robust against noise or differences in point density. The Hausdorff distance is known to be robust against data damages”).
Inglese as modified and Fuse are considered analogous in the art with respect to the comparison of position and derivation of position error between two point clouds using Hausdorff distance. One would be motivated to determine an update condition based on an observable difference between point clouds through position error being greater than a threshold, as is evident in Fuse.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the determination of a maximum position error for two point clouds and a determination of whether the maximum position error satisfies a threshold of Inglese as modified with the determination that the error between two point clouds is greater than a threshold of Fuse; Doing so would yield the predictable result of determining whether two point clouds are different enough for an updating condition to be fulfilled.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Inglese (US 20190046276 A1), in view of Stack Overflow (Separation of GUI and logic in different threads in a Windows Form application, 2009).
Regarding Claim 15, the method as claimed in Claim 1 is rejected over Inglese.
Inglese teaches determining the current tooth model of the current acquisition period according to the point cloud data frame, and displaying the current tooth model on the preset display interface (Paragraph [0079]: “By knowing the instantaneous position of the scanner and the instantaneous position of the line of light within a object-relative coordinate system when the image was acquired, a computer equipped with appropriate software can use triangulation methods to compute the coordinates of numerous illuminated surface points. As the plane is moved to intersect eventually with some or all of the surface of the object, the coordinates of an increasing number of points are accumulated. As a result of this image acquisition, a point cloud of vertex points or vertices can be identified and used to characterize the surface contour. FIG. 5 shows a portion of a point cloud, with connected vertices 138 to form a mesh 140. The points or vertices 138 in the point cloud then represent actual, measured points on the three-dimensional surface of an object”; Paragraph [0156]: “A reconstruction step S2560 then reconstructs a 3D image of the tooth according to the depth-resolved response signal and the adjusted tooth surface structure information. A rendering step S2570 then renders the volume image content for display, transmission, or storage”; Refer to Figure 6C and Figure 6D); and
storing the current point cloud data frame to the preset database, and determine whether all the point cloud data frames of the teeth in the preset database meet the preset model updating condition (Paragraph [0153]: “The processing carried out in steps S2320 and S2330 of FIG. 23 is executed for each B-scan obtained by the OCT imaging apparatus. A decision step S2350 then determines whether or not all B-scans in the set have been processed. Once processing is complete for the B-scans, the combined B-scans form a surface point cloud for the teeth. A mesh generation and rendering step S2380 then generates and renders a 3D mesh from the surface point cloud. The rendered OCT surface data can be displayed, stored, or transmitted”; Paragraph [0086]: “In certain exemplary embodiments, the existing mesh 140 can be updated when a newly acquired 3-D image (e.g., newly acquired 3-D image 142) partly overlaps with 3-D surface of the existing mesh 140 by augmenting the existing mesh 140 with a portion of the newly acquired 3-D image that does not overlap with the existing mesh 140”. Notes: A database, in its broadest reasonable interpretation, is a collection of stored data. Processing data inherently requires the data to be stored. The broadest reasonable interpretation of an updating condition is any condition that results in an update, which is satisfied by the overlap condition in Paragraph [0086])
Inglese does not explicitly teach constructing two parallel threads in advance, wherein a first thread in the two parallel threads is configured to determine the current tooth model of the current acquisition period according to the point cloud data frame, and display the current tooth model on the preset display interface; and a second thread in the two parallel threads is configured to store the current point cloud data frame to the preset database, and determine whether all the point cloud data frames of the teeth in the preset database are meet the preset model updating condition.
However, Stack Overflow teaches constructing two parallel threads in advance (title reads “Separation of GUI and logic in different threads”; post by yeyeyerman: “In a Windows Forms application I want to separate the GUI from the logic. The user requests are complex and involve communications so I don't want it to depend on the GUI thread”), wherein a first thread in the two parallel threads is configured to perform display related tasks (answer by Martin Harris: “Traditionally work like this is done using a BackgroundWorker… Basically it is a simple class that gives you the ability to perform a function on a worker thread”); and
A second thread in the two parallel threads is configured to perform logic operations (answer by Martin Harris: “Traditionally work like this is done using a BackgroundWorker… Basically it is a simple class that gives you the ability to perform a function on a worker thread and then will automatically invoke back to the UI thread after that function is complete”).
Inglese and Stack Overflow are considered analogous in the art with respect to the use of displays of virtual content. It is well known in the art that display related threads are often separate from worker/logic related threads, as is evident in Stack Overflow. One would be motivated to do so to improve the stability of a system.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the point cloud processing and display of Inglese with the separation of tasks into two threads for display and logic operations of Stack Overflow; Doing so would yield the predictable result of improving the stability of a system performing a method for generating a tooth model with point clouds.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Inglese (US 20190046276 A1), in view of Urbach (DPDist: Comparing Point Clouds Using Deep Point Cloud Distance, 2020) and Metal Industries Research and Development Centre (CN106725966A), and in further view of Diskin (Dense 3D point-cloud model using optical flow for a monocular reconstruction system, 2013).
Regarding Claim 16, the method as claimed in Claim 5 is rejected over Inglese as modified.
Inglese as modified teaches the method as claimed in Claim 5, the calculating the position error of each point cloud data frame according to the first model position and the second model position comprises:
aligning the first model position and the second model position according to an alignment algorithm (Figure 1: “Distance to surface vs. distance to sampled point cloud. Every figure contains two samples of the same object (a table). The samples can be identical (left) or different (right). The distance of every point from one point cloud (disks) to the surface estimated from the other point cloud (+) is given as a colored code in the top two figure.”; Refer to Figure 1; Section 2.4, Registration: “Registration between point clouds is a fundamental task. A popular, classic (non-learnable) algorithm is Iterative Closest Point”).
Inglese as modified does not teach calculating optical flow information between the first model position and the second model position and determined the position error based on the optical flow information.
However, Diskin teaches calculating optical flow information for a model, and using the optical flow information to construct a point cloud (Section 2, Methodology: “We have selected to use a technique in which a point cloud is generated using both global and local information. Specifically, we generate optical flow disparities using the Horn-Schunck optical flow estimation technique [9], [10] in addition to using the SURF keypoint detection method”; Section 2, Methodology: “The initial 3D model appears as a single point cloud constructed from a pair of frames. It is a collection of the (x,y,z) points computed from the consecutive steps presented”).
Inglese as modified and Diskin are considered analogous in the art with respect to deriving point clouds. One would be motivated to construct a point cloud through optical flow information to better capture the position of objects identified through images/videos captured through a camera, as is evident in Diskin.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the alignment of model positions of Inglese as modified with the derivation of point clouds through calculating optical flow information of Diskin; Doing so would yield the predictable result of obtaining accurate point clouds from images captured from a camera.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights