DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 10 objected to because of the following informalities:
Claim 10, line 6-9, the limitation of “generate, based on reducing one or more data points of the point cloud, a decimated point cloud associated with the environment; generate, based on reducing one or more data points of the point cloud, a decimated point cloud associated with the environment ” is duplicate.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6, 10 and 12-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Benhimane et al. (US 20140293016 A1), hereinafter as Benhimane, in view of NPL Liu et al. (“A Point Clouds Filtering Algorithm Based on Grid Partition and Moving Least Squares”), hereinafter as Liu.
Regarding claim 1, Benhimane teaches A method (paragraph [0015] “a method for estimating a camera motion and for determining a three-dimensional model of an environment”) comprising:
receiving, by a computing device, field of view imaging data associated with an environment from one or more imaging devices (Benhimane teaches two cameras as the imaging devices, a digital computer performing the program as a computing device, paragraph [0081] “At least two cameras, recording intensities with known relative pose and ideally known intrinsic parameters, can capture images at approximately the same time or, when not moving, at different times. Correspondences can be found in both images and the relative pose and intrinsic of the cameras can be used to calculate the correspondences depth in either image coordinate system.”, paragraph [0035] “Another aspect of the invention is also related to a computer program product which is adapted to be loaded into the internal memory of a digital computer and comprises software code sections by means of which the method according to the invention is performed when said product is running on said computer.”);
generating, based on the imaging data, a point cloud associated with the environment (Benhimane paragraph [0057] “FIG. 2a shows the creation of the initial three-dimensional reference model from the reference depth samples associated to the at least one reference intensity image. In S1, the reference three-dimensional point cloud is computed using the depth samples associated to the reference intensity image.”);
…… and generating, based on the refined point cloud, a mesh representation associated with the environment (Benhimane paragraph [0060] “In the next step S2, a reference mesh is determined from the three-dimensional point cloud, for example, by defining triangles within the three-dimensional point cloud.”).
Benhimane is not relied on for the below claim language …… generating, based on reducing one or more data points of the point cloud, a decimated point cloud associated with the environment; generating, based on applying a data refinement process to the decimated point cloud, a refined point cloud associated with the environment …… the refined point cloud Liu teaches…… generating, based on reducing one or more data points of the point cloud, a decimated point cloud associated with the environment (Liu teaches a lidar point cloud filtering process on Page 3, Figure 1, Liu teaches rarefy points to generate decimated point cloud in step 3, Page 2, Fourth paragraph, “(3) If the data in the grid is dense, rarefy the points by retaining the points with lowest elevation in every grid.”); generating, based on applying a data refinement process to the decimated point cloud, a refined point cloud associated with the environment …… the refined point cloud (Liu teaches using moving lease square method as the data refinement process to filter ground point cloud as the refined point cloud in steps 4 and 5, Page 2, Fourth paragraph, “(4) Generate a surface by moving least squares and take it as the reference surface for Digital Elevation Model (DEM). (5) Calculate the distance to the established DEM reference surface for the points within the grids. If the distance for a point above the surface exceeds the threshold value, it will be grouped into non-ground points while others ground ones and stored for later application.”).
Benhimane and Liu are in the same field of endeavor, namely 3D reconstruction based on images. Liu teaches a point cloud filtering method to improve efficiency and accuracy (Liu Page 7, First paragraph, “a novel automatic point clouds filtering algorithm that combines dynamic quad trees grid partition and moving least squares reference surface fitting is proposed. By fitting the DEM reference surface using moving least squares technology, the algorithm improves the precision and reliability of result. It has good generality to the point clouds filtering. It is able to deal with the complex and massive point clouds efficiently with precision assured.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu with the method of Benhimane to improve efficiency and accuracy.
Regarding claim 2, Benhimane in view of Liu teach The method of claim 1, and further teach wherein the one or more imaging devices comprise one or more RGB camera devices (Benhimane paragraph [0024-0025] “Intensity images can encode intensity in one (e.g. greyscale) or more than one channels (e.g. RGB--red-green-blue) in different bit resolutions (e.g. 8 bit or high dynamic range)……. the method may use a set of reference intensity image including at least one reference intensity image, captured by a camera at a first camera pose”), wherein each imaging device of the one or more imaging devices comprises one or more of a gimbal, a GPS sensor, a laser range finder, an accelerometer, or an inertial measurement unit (Benhimane paragraph [0086] “at least one camera takes images at different points of time from different positions. Ideally some additional sensor measurements are available (for example GPS positions).”).
Regarding claim 3, Benhimane in view of Liu teach The method of claim 1, and further teach wherein the field of view imaging data comprises one or more pixel-based digital images of the environment, GPS data associated with the one or more imaging devices, laser range finder (LRF) data, real-time kinematics (RTK) data associated with the one or more imaging devices, distance data associated with the one or more imaging devices, orientation data associated with the one or more imaging device, pose metadata, or one or more combinations thereof (Benhimane teaches the pixel-based digital images, paragraph [0102] “An intensity-based image registration method may be based on an iterative minimization process, wherein a first set of pixels from the set of reference two-dimensional imaged points is compared with a computed set of pixels from the set of current two-dimensional imaged points and the computed set of pixels from the set of current two-dimensional imaged points used for the comparison varies at each iteration.”).
Regarding claim 4, Benhimane in view of Liu teach The method of claim 1, and further teach wherein the environment comprises one or more physical objects (Benhimane paragraph [0011] “FIG. 8 shows a scene consisting of two sets of dolls S1 and S2 (each set comprising a tall and a small doll), and a capturing device CD”).
Regarding claim 6, Benhimane in view of Liu teach The method of claim 1, and further teach wherein the data refinement process comprises one or more of a weighted moving least square process or a screened Poisson reconstruction process (Liu Page 4, Last Section 2.4. “Fitting DEM reference surface with Moving Least Squares”, and “In the moving least squares approximation, the interpolant function of the unknown function f (x) is defined as …… The coefficients can be derived by performing a weighted least squares fit”).
Benhimane and Liu are in the same field of endeavor, namely 3D reconstruction based on images. Liu teaches a point cloud filtering method to improve efficiency and accuracy (Liu Page 7, First paragraph, “a novel automatic point clouds filtering algorithm that combines dynamic quad trees grid partition and moving least squares reference surface fitting is proposed. By fitting the DEM reference surface using moving least squares technology, the algorithm improves the precision and reliability of result. It has good generality to the point clouds filtering. It is able to deal with the complex and massive point clouds efficiently with precision assured.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu with the method of Benhimane to improve efficiency and accuracy.
Regarding claim 10, it recites similar limitations of claim 1 but in a non-transitory computer-readable media form. The rationale of claim 1 rejection is applied to reject claim 10. In addition, Benhimane teaches One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to (Benhimane claim 21 “A non-transitory computer readable medium comprising software code sections which are adapted to perform a method”, paragraph [0035] “Another aspect of the invention is also related to a computer program product which is adapted to be loaded into the internal memory of a digital computer and comprises software code sections by means of which the method according to the invention is performed when said product is running on said computer.”).
Regarding claim 12, Benhimane teaches A method (paragraph [0015] “a method for estimating a camera motion and for determining a three-dimensional model of an environment”) comprising:
receiving, by a computing device, field of view imaging data associated with an environment from one or more imaging devices (Benhimane teaches two cameras as the imaging devices, a digital computer performing the program as a computing device, paragraph [0081] “At least two cameras, recording intensities with known relative pose and ideally known intrinsic parameters, can capture images at approximately the same time or, when not moving, at different times. Correspondences can be found in both images and the relative pose and intrinsic of the cameras can be used to calculate the correspondences depth in either image coordinate system.”, paragraph [0035] “Another aspect of the invention is also related to a computer program product which is adapted to be loaded into the internal memory of a digital computer and comprises software code sections by means of which the method according to the invention is performed when said product is running on said computer.”);
determining, based on a first two images of the imaging data, a scaling process (Benhimane paragraph [0172] “in a first step, two intensity images I1 and I2 are captured by a camera or loaded which may then optionally be subject to pre-processing. After a scale space or a set of discrete scales has been defined, features are detected in that scale space in I1 and/or I2 and correspondences are determined …… scale could also be defined globally to 10 mm or 1000 mm for certain depths or using a general formula dependent from the depth d, which applies to all features in the application”);
generating, based on an image of the imaging data, a depth image associated with the environment (Benhimane paragraph [0172] “A detected features correspondence C(F1(x,y), F2(x,y)) with Fi designating the two corresponding features and x, y designating a 2-dimensional position of the respective feature Fi is considered to describe projections of the same point in 3D space into I1 and I2 and thus, the depth, i.e. the position in 3D space of this point can be computed for example by triangulation”);
generating, based on the determined scaling process and the depth image, a point cloud associated with the environment (Benhimane Figure 2a and Figure 2b, paragraph [0061-0066] “a current intensity image and current depth samples associated to that current intensity image are provided. ……The updating of the three-dimensional model of the environment (S8a) may comprise concatenating two three-dimensional point clouds which may be achieved by transforming the three-dimensional points with the inverse of the camera pose associated to the current intensity image”) and paragraph [0168] “according to the invention the depth of the feature provided by a method provided in step S71 is incorporated. In this case, the depth forms a part of the descriptor and is multiplied with the feature scale and optionally the focal length, as explained above. Eventually, the described features are used in an application in step S78. In possible implementations of the invention, features are extracted in order to provide depth “);
…… and generating, based on the refined point cloud, a mesh representation associated with the environment (Benhimane paragraph [0060] “In the next step S2, a reference mesh is determined from the three-dimensional point cloud, for example, by defining triangles within the three-dimensional point cloud.”).
Benhimane is not relied on for the below claim language …… generating, based on applying a data refinement process to the point cloud, a refined point cloud associated with the environment …… the refined point cloud. Liu teaches…… generating, based on applying a data refinement process to the point cloud, a refined point cloud associated with the environment …… the refined point cloud (Liu teaches a lidar point cloud filtering process on Page 3, Figure 1, Liu teaches rarefy points to generate decimated point cloud in step 3, further teaches using moving lease square method as the data refinement process to filter ground point cloud as the refined point cloud in steps 4 and 5, Page 2, Fourth paragraph, “(3) If the data in the grid is dense, rarefy the points by retaining the points with lowest elevation in every grid. (4) Generate a surface by moving least squares and take it as the reference surface for Digital Elevation Model (DEM). (5) Calculate the distance to the established DEM reference surface for the points within the grids. If the distance for a point above the surface exceeds the threshold value, it will be grouped into non-ground points while others ground ones and stored for later application.”).
Benhimane and Liu are in the same field of endeavor, namely 3D reconstruction based on images. Liu teaches a point cloud filtering method to improve efficiency and accuracy (Liu Page 7, First paragraph, “a novel automatic point clouds filtering algorithm that combines dynamic quad trees grid partition and moving least squares reference surface fitting is proposed. By fitting the DEM reference surface using moving least squares technology, the algorithm improves the precision and reliability of result. It has good generality to the point clouds filtering. It is able to deal with the complex and massive point clouds efficiently with precision assured.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liu with the method of Benhimane to improve efficiency and accuracy.
Regarding claim 13, Benhimane in view of Liu teach The method of claim 12, and further teach wherein the one or more imaging devices comprise one or more RGB camera devices (Benhimane paragraph [0024-0025] “Intensity images can encode intensity in one (e.g. greyscale) or more than one channels (e.g. RGB--red-green-blue) in different bit resolutions (e.g. 8 bit or high dynamic range)……. the method may use a set of reference intensity image including at least one reference intensity image, captured by a camera at a first camera pose”), wherein each imaging device of the one or more imaging devices comprises one or more of a gimbal, a GPS sensor, an accelerometer, or an inertial measurement unit (Benhimane paragraph [0086] “at least one camera takes images at different points of time from different positions. Ideally some additional sensor measurements are available (for example GPS positions).”).
Regarding claim 14, Benhimane in view of Liu teach The method of claim 12, and further teach wherein the field of view imaging data comprises one or more pixel-based images of the environment, GPS data associated with the one or more imaging devices, real-time kinematics (RTK) data associated with the one or more imaging devices, distance data associated with the one or more imaging devices, orientation data associated with the one or more imaging devices, pose metadata, or one or more combinations thereof (Benhimane paragraph [0102] “An intensity-based image registration method may be based on an iterative minimization process, wherein a first set of pixels from the set of reference two-dimensional imaged points is compared with a computed set of pixels from the set of current two-dimensional imaged points and the computed set of pixels from the set of current two-dimensional imaged points used for the comparison varies at each iteration.”).
Regarding claim 15, Benhimane in view of Liu teach The method of claim 12, and further teach wherein the environment comprises one or more physical objects (Benhimane paragraph [0011] “FIG. 8 shows a scene consisting of two sets of dolls S1 and S2 (each set comprising a tall and a small doll), and a capturing device CD”).
Regarding claim 16, Benhimane in view of Liu teach The method of claim 12, and further teach wherein the determined scaling process comprises triangulating features associated with the first two images according to pose data associated with the one or more imaging devices or optimizing for scale correction to match an expected scene depth according to the pose data (Benhimane teaches using camera pose data to generate real scale factor for features, further teaches defining triangles within point cloud, based on the triangulating features, paragraph [0176] “an initial pose of the capturing device while capturing I1 is estimated from rough sensor measurements such as GPS position and orientation sensor information. …… The environment model is loaded using the assumed camera pose, i.e. the environment model is rendered from the camera viewpoint of intensity image I1. Depth information is retrieved from the environment model and used in a next step for calculating the real scale descriptors of detected features. In other words, using the depth information registered with the image I1, real scale features are extracted at a fixed scale of, for example 1 m.”, paragraph [0060] “by defining triangles within the three-dimensional point cloud”).
Regarding claim 17, Benhimane in view of Liu teach The method of claim 12, and further teach wherein generating, based on the determined scaling process and the depth image, the point cloud associated with the environment comprises:
generating, based on the depth image, an initial point cloud associated with the environment (Benhimane Figure 2a and paragraph [0057] “In S1, the reference three-dimensional point cloud is computed using the depth samples associated to the reference intensity image.”);
iteratively projecting, for each subsequent image from the image, based on pose data associated with the one or more imaging devices, a corresponding previously scaled point cloud onto the corresponding subsequent image (Benhimane teaches an iterative process in Figure 2b, paragraph [0061-0064] “Then a current three-dimensional point cloud, i.e. a current three-dimensional model, is determined using the current depth samples associated to the current intensity image and the intrinsic parameters (S3). Optionally, the current three-dimensional point cloud may also be meshed in S3 …… Using the determined current camera pose, in S6 a similarity measure between the reference three-dimensional model of the environment and the current three-dimensional model is determined in a common coordinate system.”);
iteratively generating, based on the corresponding projection, a subsequent depth image (Benhimane Figure 2b and paragraph [0064] “this may be achieved by using the determined camera pose for rendering the three-dimensional model into the coordinate system of the current view of the camera (i.e.t into the coordinate system of the current three-dimensional model) and thus determining a depth map of the reference three-dimensional model in the coordinate system of the current view of the camera (S5).”);
iteratively adding, based on each corresponding subsequent depth image, data to the initial point cloud (Benhimane Figure 2b, paragraph [0066] “In the following step S7, it is thus determined depending on the similarity measure whether to update the estimate of the three-dimensional model of the environment using at least one point of the three-dimensional point cloud determined from the current depth samples associated to the current intensity image (S8a)”),
wherein each iteration of the initial point cloud is scaled according to the determined scaling process (Benhimane teaches deciding a scale factor for features, and further teaches using features to generate point cloud, paragraph [0172] “in a first step, two intensity images I1 and I2 are captured by a camera or loaded which may then optionally be subject to pre-processing. After a scale space or a set of discrete scales has been defined, features are detected in that scale space in I1 and/or I2 and correspondences are determined …… scale could also be defined globally to 10 mm or 1000 mm for certain depths or using a general formula dependent from the depth d, which applies to all features in the application”);
and generating, based on iteratively adding data to the initial point cloud, the point cloud (Benhimane paragraph [0066] “If the similarity measure does exceed the threshold, step 8 is processed and steps S3 to S7 are then repeated based on the updated reference three-dimensional model of the environment and the updated set of reference images including at least the reference image and the current intensity image of the previous cycle as additional reference intensity image. The updating of the three-dimensional model of the environment (S8a) may comprise concatenating two three-dimensional point clouds which may be achieved by transforming the three-dimensional points with the inverse of the camera pose associated to the current intensity image.”).
Claim(s) 5 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Benhimane et al. (US 20140293016 A1), hereinafter as Benhimane, in view of NPL Liu et al. (“A Point Clouds Filtering Algorithm Based on Grid Partition and Moving Least Squares”), hereinafter as Liu, further in view of Jiang et al. (CN 117933128 A), hereinafter as Jiang. The original and a machine translation of Jiang are provided by the examiner. The paragraph of the machine translation of Jiang starts with both number and letter ‘n’.
Regarding claim 5, Benhimane in view of Liu teach The method of claim 1, but are not relied on for the below claim language wherein reducing the one or more data points of the point cloud comprises reducing, based on applying voxel filtering to the point cloud, a point density associated with the point cloud. Jiang teaches wherein reducing the one or more data points of the point cloud comprises reducing, based on applying voxel filtering to the point cloud, a point density associated with the point cloud (Jiang paragraph [n0191] “The voxel grid filtering algorithm is used to divide the point cloud space into voxel squares. The center of a point in each square is calculated to represent it, thereby achieving down sampling. Reduce point cloud density and improve processing efficiency.”).
Benhimane, Liu and Jiang are in the same field of endeavor, namely 3D reconstruction based on images. Jiang teaches using drone and laser scanner to generate a digital twin of pump station, further teaches using a voxel filtering method to reduce point cloud density to improve processing efficiency. Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Jiang with the method of Benhimane and Liu to improve processing efficiency.
Regarding claim 11, claim 11 has similar limitations as claim 5, therefore it is rejected under the same rationale as claim 5.
Claim(s) 7-9, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Benhimane et al. (US 20140293016 A1), hereinafter as Benhimane, in view of NPL Liu et al. (“A Point Clouds Filtering Algorithm Based on Grid Partition and Moving Least Squares”), hereinafter as Liu, further in view of Su et al. (US 20230290090 A1), hereinafter as Su.
Regarding claim 7, Benhimane in view of Liu teach The method of claim 1, but are not relied on for the below claim language further comprising: receiving, via a user interface of the computing device, one or more user interactions with the mesh representation; and determining, based on the one or more user interactions with the mesh representation, measurement data associated with the environment.
Su teaches further comprising: receiving, via a user interface of the computing device, one or more user interactions with the mesh representation; and determining, based on the one or more user interactions with the mesh representation, measurement data associated with the environment (Su paragraph [0089-0090] “FIG. 10 depicts an example interface 1000 that allows for taking measurements within the context of a 3D mesh 1002 and layout …… Two measurements marks 1006 and 1008 are visible in the mesh 1002, with corresponding measurement marks 1010 and 1012, respectively, visible in the camera view 1004. The measurement marks may be placed by a user to take specific measurements of structures within the virtual reconstruction, which will correspond to the physical dimensions of the captured space”).
Benhimane, Liu and Su are in the same field of endeavor, namely 3D reconstruction based on images. Su teaches a user interface with manipulation of 3D representation to improve accuracy and interaction (Su paragraph [0027-0031] “allow integration of depth and motion data to enable a device to both measure its environment and know its position within the environment, these array of sensors allow for the relatively accurate capture and/or calculation of depth information for a captured scene……it may be desirable for a user interface to display the locations of different objects in a user-manipulable manner. For example, the scan itself may be of a 3D space, but it may be desired for the display of the scan to be a two-dimensional (2D) or 3D representation (e.g., an image) of the 3D space. Further, it may be desirable for the locations of various objects with the 3D space to be displayed in either the 2D or 3D image.“). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Su with the method of Benhimane and Liu to improve accuracy and interaction.
Regarding claim 8, Benhimane in view of Liu and Su teach The method of claim 7, and further teach wherein the measurement data comprises one or more of feature locations associated with the environment, distance measurements associated with the environment, or geospatial coordinates associated with the environment (Su paragraph [0057] “Measurements that may include lengths of edges, areas of surfaces, volumes of spaces, distances between objects, and/or any other measurements that could be obtained or approximated from actually measuring the physical 3D space. Thus, the virtual construction approximates or matches the physical size of the captured 3D space. Measurements taken in the virtual construction, which is of an identical scale to the captured 3D space”, paragraph [0090] “The measurement marks may be placed by a user to take specific measurements of structures within the virtual reconstruction, which will correspond to the physical dimensions of the captured space”).
Benhimane, Liu and Su are in the same field of endeavor, namely 3D reconstruction based on images. Su teaches a user interface with manipulation of 3D representation to improve accuracy and interaction (Su paragraph [0027-0031] “allow integration of depth and motion data to enable a device to both measure its environment and know its position within the environment, these array of sensors allow for the relatively accurate capture and/or calculation of depth information for a captured scene……it may be desirable for a user interface to display the locations of different objects in a user-manipulable manner. For example, the scan itself may be of a 3D space, but it may be desired for the display of the scan to be a two-dimensional (2D) or 3D representation (e.g., an image) of the 3D space. Further, it may be desirable for the locations of various objects with the 3D space to be displayed in either the 2D or 3D image.“). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Su with the method of Benhimane and Liu to improve accuracy and interaction.
Regarding claim 9, Benhimane in view of Liu and Su teach The method of claim 7, and further teach wherein the measurement data is generated based on mapping 2D image pixels of the imaging data to 3D points of the mesh representation and mapping 3D points of the mesh representation back onto 2D images of the imaging data (Su paragraph [0090-0091] “The measurement marks 1006 and 1010 are synchronized, as are measurement marks 1008 and 1012. Adjustment of the marks in one view will automatically be reflected in the other view…… The synchronization between the two views, the mesh 1002 and the camera view 1004, which includes synchronization of the various measurement marks 1006 and 1008 with corresponding marks 1010 and 1012, can allow a user to better and more accurately ascertain what is being measured within the virtual reconstruction, by allowing the virtual reconstruction to be simultaneously visualized in two different fashions, with the measurement marks overlaid in each visualization and remaining in synchronization.”).
Benhimane, Liu and Su are in the same field of endeavor, namely 3D reconstruction based on images. Su teaches a user interface with manipulation of 3D representation to improve accuracy and interaction (Su paragraph [0027-0031] “allow integration of depth and motion data to enable a device to both measure its environment and know its position within the environment, these array of sensors allow for the relatively accurate capture and/or calculation of depth information for a captured scene……it may be desirable for a user interface to display the locations of different objects in a user-manipulable manner. For example, the scan itself may be of a 3D space, but it may be desired for the display of the scan to be a two-dimensional (2D) or 3D representation (e.g., an image) of the 3D space. Further, it may be desirable for the locations of various objects with the 3D space to be displayed in either the 2D or 3D image.“). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Su with the method of Benhimane and Liu to improve accuracy and interaction.
Regarding claim 18, claim 18 has similar limitations as claim 7, therefore it is rejected under the same rationale as claim 7.
Regarding claim 19, claim 19 has similar limitations as claim 8, therefore it is rejected under the same rationale as claim 8.
Regarding claim 20, claim 20 has similar limitations as claim 9, therefore it is rejected under the same rationale as claim 9.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kim et al. (US 20180096463 A1) teaches a method to remove noise and outliers in point cloud data (paragraph [0042] “operations involved in 214 can be performed by a point filtering component the same as or substantially similar to the point filtering component 114 described and illustrated herein.”, paragraph [0024] “The point filtering component 114 can be configured to determine, for each p, whether p should be kept based on its geometric and photometric consistency as determined by the geometric consistency component 108 and point photo-consistency determination component 110”);
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMING WEI whose telephone number is (571)272-3831. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIAOMING WEI/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights