Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. The United States Patent & Trademark Office appreciates the application that is by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on 12/14/2023. The submission is in compliance. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
4. The drawings are objected to because in operational block 212 of FIG. 2, there is an incorrect spelling in the form of “wrt”. This should be corrected from “wrt” to “with”. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 102
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
6. Claims 10, 12-13, and 16-18 are rejected under 35 U.S.C. 102(a)(2) as being unpatentable over Xiong et al (U.S. Patent Pub. No. 10902664, hereafter referred to as Xiong).
7. Regarding Claim 10, Xiong teaches a method for generating a pose estimation for an object from a 2D borescope inspection video, the method comprising: generating a model 3D point cloud of a predetermined area of an object using a Computer Aided Design (CAD) assembly model of the object (page 4, lines 23-32, Xiong teaches the generation of a refined 3D model by improving the accuracy of the 3D structure model using a CAD model. The refining of the original structure of the model is done by matching the two embodiments, just like 3D point cloud generation. The Examiner interprets the 3D structure model made using a CAD model as the CAD assembly model and the refined 3D model as the model 3D point cloud being generated.); extracting a video frame sequence from a 2D borescope inspection video of the object (col 4 lines 16-22, Xiong teaches how the images can be transmitted to a processing system for a plurality of techniques. The Examiner interprets the transmission of the images, known as video frame sequences as a means to use the frames for initial stages of the pose estimation generation, which requires the technique of extracting video and image frame sequences.); generating an estimated 3D point cloud by processing the video frame sequence using a Structure-from-Motion (SfM) algorithm (col 4 lines 18-24 and col 4 lines 35-45, Xiong teaches using the images from the camera, which is the borescope to perform the SFM technique to generate a three-dimensional model of which the accuracy will be improved at a later time. The Examiner interprets this as a coarse generation or a first draft of a model, a point cloud using the images from the captured borescope frames.); identifying a common coordinate system with respect to the model 3D point cloud and the estimated 3D point cloud (col 4 lines 44-59, Xiong teaches a starting point in the form of identifying and representing the coordinates of the two representations, initial model and estimation, as homogenous coordinates. The Examiner interprets this as creating a level of commonality with respect to the model 3D point cloud and the estimate 3D point cloud.); and computing a rough alignment by processing the common coordinate system, the model 3D point cloud and the estimated 3D point cloud using a global registration algorithm (col 4 lines 44-59, Xiong teaches using an algorithm such as Gauss-Newton or Levenberg-Marquardt to transform one model to match closely with the other, which is an example of using a global registration system, an image alignment method that applied a single continuous geometric transformation to the entire image to align it with a reference image. The Examiner interprets this as the development of a rough alignment using a global registration algorithm to transform one model to closely match another.); and generating a fine registration pose estimation by processing the rough alignment of the object (col 4, lines 60-66, Xiong teaches a precise calculation of an object’s position and pose, as well as intensity, which influences pose, to a reference point that was previously achieved through using the global registration. The precise calculation produces a finer registration by way of an augmented three-dimensional model from the existing three-dimensional model. The registration includes fiducial marks and computing a homograph. The Examiner interprets this as fine registration pose estimation by processing a rough alignment from an earlier part of the process.).
8. In regards to Claim 12, Xiong Srivastava teaches wherein extracting a video frame sequence includes generating the 2D borescope inspection video of the object using a borescope (col 4 lines 16-20, Xiong teaches a borescope performing the inspection of the object by way of obtaining 2D images and videos. The Examiner interprets this as the borescope, which operates two dimensionally to generate images and videos that are 2D.).
9. In regards to Claim 13, Xiong teaches wherein extracting a video frame sequence includes generating a 2D borescope inspection video of a predetermined area of the object (col 6 lines 1-6, Xiong teaches the possibility of previously knowing the location and pose of the intended area of recording on the object before generating the 2D borescope inspection video of a predetermined area of the object. From there the image or video frames can be taken and then mapped with location interpolation. The Examiner interprets this obtaining image or video frames by knowing the location and pose of the intended area of recording as generating the 2D borescope inspection video of a predetermined area.).
10. In regards to Claim 16, Xiong teaches wherein identifying a common coordinate system includes comparing the model 3D point cloud and the estimated 3D point cloud (col 4 lines 44-59, Xiong teaches a starting point in the form of identifying and representing the coordinates of the two representations, initial model and estimation, as homogenous coordinates. The Examiner interprets this as creating a level of commonality with respect to the model 3D point cloud and the estimate 3D point cloud).
11. In regards to Claim 17, Xiong teaches wherein computing a rough alignment includes processing the model 3D point cloud and the estimated 3D point cloud using a global registration algorithm to identify common areas of curvature (col 4 lines 44-59, Xiong teaches Gauss-Newton or Levenberg- Marquardt algorithms being used to transform match closely the first model, the 3D point cloud structure with the second model, the estimation. This is an example of using a global registration system, an image alignment method that applied a single continuous geometric transformation to the entire image to align it with a reference image. This registration produces refined values of 15 parameters relating to the commonality between the two structures in the form of translation, rotation, stretching, and shearing, which are factors in understanding similarities and differences in curvature. The Examiner interprets the commonalities between the two structures as means to identify common areas of curvature.).
12. In regards to Claim 18, Xiong teaches wherein generating a fine registration pose estimation of the object includes processing the rough alignment using an Iterative Close Points (ICP) algorithm (col 4, lines 60-66, Xiong teaches a precise calculation of an object’s position and pose, as well as intensity, which influences pose, to a reference point that was previously achieved through using the global registration. The precise calculation produces a finer registration by way of an augmented three-dimensional model from the existing three-dimensional model. The registration includes fiducial marks and computing a homograph. The Examiner interprets this as fine registration pose estimation by processing a rough alignment from an earlier part of the process.).
Claim Rejections - 35 USC § 103
13. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
14. Claims 1-9, 11, 14-15, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Xiong et al (U.S. Patent Pub. No. 10902664, hereafter referred to as Xiong) in view of Srivastava et al (U.S. Patent Pub. No. 11199831, hereafter referred to as Srivastava).
15. Regarding Claim 1, Xiong teaches a method for generating a pose estimation for an object from a 2D borescope inspection video, the method comprising: generating a Computer Aided Design (CAD) assembly model of an object (col 3 lines 64–col 4 line 1, Xiong teaches an as-designed, as-built CAD model of an object can be used to assist the multiple parts of the method of generating a pose estimation. The Examiner interprets an as-designed, as-built CAD model as the generation of a CAD model based on information relating to an object.);
generating a model 3D point cloud of the object using the CAD assembly model (page 4, lines 23-32, Xiong teaches the generation of a refined 3D model by improving the accuracy of the 3D structure model using a CAD model. The refining of the original structure of the model is done by matching the two embodiments, just like 3D point cloud generation. The Examiner interprets the 3D structure model made using a CAD model as the CAD assembly model and the refined 3D model as the model 3D point cloud being generated.);
generating a 2D borescope inspection video of the object (col 3 lines 19-20 and col 4 lines 16-19, Xiong teaches a typically manual inspection using a borescope to obtain a sequence of 2D images and videos of an object. The Examiner interprets this as generating a 2D inspection video of an object using a borescope.), wherein the 2D borescope inspection video includes a plurality of video frames (col 4 lines 16-20, Xiong teaches a system and method that includes obtaining a sequence of images or videos that are obtained using a borescope. The Examiner is interpreting the sequence of images and videos as 2D.); extracting a video frame sequence from the plurality of video frames (col 4 lines 16-22, Xiong teaches how the images can be transmitted to a processing system for a plurality of techniques. The Examiner interprets the transmission of the images, known as video frame sequences as a means to use the frames for initial stages of the pose estimation generation, which requires the technique of extracting video and image frame sequences.); generating a coarse estimated point cloud video frame sequence by applying a Structure-from-Motion (SfM) algorithm to the video frame sequence (col 4 lines 18-24 and col 4 lines 35-45, Xiong teaches using the images from the camera, which is the borescope to perform the SFM technique to generate a three-dimensional model of which the accuracy will be improved at a later time. The Examiner interprets this as a coarse generation or a first draft of a model, a point cloud using the images from the captured borescope frames.); identifying a common coordinate system with respect to the model 3D point cloud and the estimated 3D point cloud (col 4 lines 44-59, Xiong teaches a starting point in the form of identifying and representing the coordinates of the two representations, initial model and estimation, as homogenous coordinates. The Examiner interprets this as creating a level of commonality with respect to the model 3D point cloud and the estimate 3D point cloud.); computing a rough alignment using a global registration algorithm (col 4 lines 44-59, Xiong teaches using an algorithm such as Gauss-Newton or Levenberg-Marquardt to transform one model to match closely with the other, which is an example of using a global registration system, an image alignment method that applied a single continuous geometric transformation to the entire image to align it with a reference image. The Examiner interprets this as the development of a rough alignment using a global registration algorithm to transform one model to closely match another.); and generating a fine registration pose estimation of the object by processing the rough alignment (col 4, lines 60-66, Xiong teaches a precise calculation of an object’s position and pose, as well as intensity, which influences pose, to a reference point that was previously achieved through using the global registration. The precise calculation produces a finer registration by way of an augmented three-dimensional model from the existing three-dimensional model. The registration includes fiducial marks and computing a homograph. The Examiner interprets this as fine registration pose estimation by processing a rough alignment from an earlier part of the process.).
Xiong does not teach generating an estimated 3D point cloud by applying a filtering algorithm to the coarse estimated point cloud video frame sequence to filter out outliers.
Srivastava is in the same field of art of 3D model generation using 2D borescope camera poses. Further Srivastava teaches generating an estimated 3D point cloud by applying a filtering algorithm to the coarse estimated point cloud video frame sequence to filter out outliers (col 7 lines 17-35, Srivastava teaches filtering out non-edges from the model that was produced using 3D scan data that came from image frames, in turn generating a more accurate 3D point cloud by using algorithms that extract edge features and calculations of point normals such as perpendicular vectors for given 3D points. The Examiner interprets the image frames as 2D frame sequences, 3D scan data as information extracted from the image and video frame sequences for the pose estimation generation, and filtering out of non-edges using algorithms such as extract edge features and calculations of point normals as filtering algorithms used to filter out outliers.).
Therefore it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Xiong by adding the generation of an estimated 3D point cloud by applying a filtering algorithm to the coarse estimated point cloud video frame sequence to filter out outliers that is taught by Srivastava to make the invention more efficient in generating more accurate 3D point clouds by using a filtering algorithms that removes deformities in the form of outliers; thus one of ordinary skilled in the art would be motivated to combine the references since they are both in the field of 3D model generation using 2D borescope camera poses (col 7 lines 17-35, Srivastava).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
16. In regards to Claim 2, Xiong in view of Srivastava teaches wherein generating a CAD assembly model includes generating a CAD assembly model of predetermined area of the object (col 1 lines 61-65, Srivastava teaches the operation of the disclosed CAD and CAM systems on created CAD models of part designs. The Examiner interprets part designs as designed on predetermined objects and locations that would be determined prior to the design and the eventual CAD model which is interpreted as CAD assembly.).
17. In regards to Claim 3, Xiong in view of Srivastava teaches wherein generating a 2D borescope inspection video of the object includes generating the 2D borescope inspection video using a borescope (col 4 lines 16-20, Xiong teaches a borescope performing the inspection of the object by way of obtaining 2D images and videos. The Examiner interprets this as the borescope, which operates two dimensionally to generate images and videos that are 2D.).
18. In regards to Claim 4, Xiong in view of Srivastava teaches wherein generating a 2D borescope inspection video of the object includes generating the 2D borescope inspection video of a predetermined area of the object (col 6 lines 1-6, Xiong teaches the possibility of previously knowing the location and pose of the intended area of recording on the object before generating the 2D borescope inspection video of a predetermined area of the object. From there the image or video frames can be taken and then mapped with location interpolation. The Examiner interprets this obtaining image or video frames by knowing the location and pose of the intended area of recording as generating the 2D borescope inspection video of a predetermined area.).
19. In regards to Claim 5, Xiong in view of Srivastava teaches wherein generating a coarse estimated point cloud video frame sequence includes processing the video frame sequence using COLMAP (col 6 lines 17-22, Srivastava teaches a set of data points from original camera poses in a 3D space used to generate a point cloud as continuing example of a form of 3D scan data. The Examiner interprets this as using COLMAP, which is use of creating 3D models by using camera poses and 3D point from overlapping photos, to generate the coarse estimate point cloud from the set of data points.).
20. In regards to Claim 6, Xiong in view of Srivastava teaches wherein generating an estimated 3D point cloud includes processing the coarse estimated point cloud using a filtering algorithm to statistically filter out outlier data points (col 5 lines 35- 57 and col 6 lines 15-22, Srivastava teaches identifying defect locations and delineating parts features statically on many part designs for their removal on models in the generation of manufactured geometry from 3D scan data such as a 3D point cloud. The Examiner interprets parts designs as any product of coarse estimate data from initial image and video data and the removal of defects and delineating parts as filtering out outlier data points.).
21. In regards to Claim 7, Xiong in view of Srivastava teaches wherein identifying a common coordinate system includes comparing the model 3D point cloud and the estimated 3D point cloud (col 4 lines 44-59, Xiong teaches a starting point in the form of identifying and representing the coordinates of the two representations, initial model and estimation, as homogenous coordinates. The Examiner interprets this as creating a level of commonality with respect to the model 3D point cloud and the estimate 3D point cloud.).
22. In regards to Claim 8, Xiong in view of Srivastava teaches wherein computing a rough alignment includes processing the model 3D point cloud and the estimated 3D point cloud using a global registration algorithm to identify common areas of curvature (col 4 lines 44-59, Xiong teaches Gauss-Newton or Levenberg- Marquardt algorithms being used to transform match closely the first model, the 3D point cloud structure with the second model, the estimation. This is an example of using a global registration system, an image alignment method that applied a single continuous geometric transformation to the entire image to align it with a reference image. This registration produces refined values of 15 parameters relating to the commonality between the two structures in the form of translation, rotation, stretching, and shearing, which are factors in understanding similarities and differences in curvature. The Examiner interprets the commonalities between the two structures as means to identify common areas of curvature.).
23. In regards to Claim 9, Xiong in view of Srivastava teaches wherein generating a fine registration pose estimation of the object includes processing the rough alignment using an Iterative Close Points (ICP) algorithm (col 5 lines 25-39, Xiong teaches the mapping of the two-dimensional temperature image to the three-dimensional model, by executing a transformation. This transformation includes parameters relating to translation, rotation, stretching, squeezing, shearing, and an eventual three-dimensional-to-two-dimensional projection, where pairs of three-dimensional vertices and two-dimensional pixel coordinates are identified to be corresponding to each other from the image and 3D model. The Examiner interprets this mapping and eventual three-dimensional-to-two-dimensional projection to be the execution of an ICP algorithm which is known to align two datasets by iteratively minimizing the distance between corresponding points by way of transformations such as rotation and translation.).
24. Regarding Claim 11, Xiong in view of Srivastava teaches the pose estimation generation method of Claim 10 that utilizes a 2D borescope inspection video.
Xiong does not teach comprising generating a CAD assembly model of the object.
Srivastava is in the same field of art of 3D model generation using 2D borescope camera poses. Further Srivastava teaches comprising generating a CAD assembly model of the object (col 1 lines 61-65, Srivastava teaches the operation of the disclosed CAD and CAM systems on created CAD models of part designs to generate CAD assemblies. The Examiner interprets part designs as designed on predetermined objects and locations that would be determined prior to the design and the eventual CAD model which is interpreted as CAD assemblies.).
Therefore it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Xiong by adding the generating a CAD assembly model of the object that is taught by Srivastava to make the invention more efficient in utilizing the borescope inspection images in 3D model generation; thus one of ordinary skilled in the art would be motivated to combine the references since they are both in the field of 3D model generation using 2D borescope camera poses (col 1 lines 61-65, Srivastava).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
25. Regarding Claim 14, Xiong in view of Srivastava teaches the pose estimation generation method of Claim 10 that utilizes a 2D borescope inspection video.
Xiong does not teach wherein generating an estimated 3D point cloud includes generating a coarse estimated point cloud by processing the video frame sequence using COLMAP.
Srivastava is in the same field of art of 3D model generation using 2D borescope camera poses. Further Srivastava teaches wherein generating an estimated 3D point cloud includes generating a coarse estimated point cloud by processing the video frame sequence using COLMAP (col 6 lines 17-22, Srivastava teaches a set of data points from original camera poses in a 3D space used to generate a point cloud as continuing example of a form of 3D scan data. The Examiner interprets this as using COLMAP, which is use of creating 3D models by using camera poses and 3D point from overlapping photos, to generate the coarse estimate point cloud from the set of data points.).
Therefore it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Xiong by adding the generation of an estimated 3D point cloud that includes generating a coarse estimated point cloud by processing the video frame sequence using COLMAP that is taught by Srivastava to make the invention more efficient in generating the coarse estimated point cloud and processing the video frame sequence; thus one of ordinary skilled in the art would be motivated to combine the references since they are both in the field of 3D model generation using 2D borescope camera poses (col 6 lines 17-22, Srivastava).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
26. Regarding Claim 15, Xiong in view of Srivastava teaches the pose estimation generation method of Claim 14 that utilizes a 2D borescope inspection video.
Xiong does not teach wherein generating an estimated 3D point cloud includes processing the coarse estimated point cloud to statically filter out outlier data points.
Srivastava is in the same field of art of 3D model generation using 2D borescope camera poses. Further Srivastava teaches wherein generating an estimated 3D point cloud includes processing the coarse estimated point cloud to statistically filter out outlier data points (col 5 lines 35- 57 and col 6 lines 15-22, Srivastava teaches identifying defect locations and delineating parts features statically on many part designs for their removal on models in the generation of manufactured geometry from 3D scan data such as a 3D point cloud. The Examiner interprets parts designs as any product of coarse estimate data from initial image and video data and the removal of defects and delineating parts as filtering out outlier data points.).
Therefore it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Xiong by adding the generation of an estimated 3D point cloud that includes processing the coarse estimated point cloud to statistically filter out outlier data points that is taught by Srivastava to make the invention more efficient in generating pose estimations and 3D model from 2D images; thus one of ordinary skilled in the art would be motivated to combine the references since they are both in the field of 3D model generation using 2D borescope camera poses (col 5 lines 35- 57 and col 6 lines 15-22, Srivastava).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
27. Regarding Claim 19, Xiong teaches a computer-implemented method for generating a pose estimation for an object from a 2D borescope inspection video, comprising: generating a Computer Aided Design (CAD) assembly model of an object (col 3 lines 64–col 4 line 1, Xiong teaches an as-designed, as-built CAD model of an object can be used to assist the multiple parts of the method of generating a pose estimation. The Examiner interprets an as-designed, as-built CAD model as the generation of a CAD model based on information relating to an object.); generating a model 3D point cloud of the object using the CAD assembly model (page 4, lines 23-32, Xiong teaches the generation of a refined 3D model by improving the accuracy of the 3D structure model using a CAD model. The refining of the original structure of the model is done by matching the two embodiments, just like 3D point cloud generation. The Examiner interprets the 3D structure model made using a CAD model as the CAD assembly model and the refined 3D model as the model 3D point cloud being generated.); generating a 2D borescope inspection video of the object (col 3 lines 19-20 and col 4 lines 16-19, Xiong teaches a typically manual inspection using a borescope to obtain a sequence of 2D images and videos of an object. The Examiner interprets this as generating a 2D inspection video of an object using a borescope.), wherein the 2D borescope inspection video includes a plurality of video frames (col 4 lines 16-20, Xiong teaches a system and method that includes obtaining a sequence of images or videos that are obtained using a borescope. The Examiner is interpreting the sequence of images and videos as 2D.);
extracting a video frame sequence from the plurality of video frames (col 4 lines 16-22, Xiong teaches how the images can be transmitted to a processing system for a plurality of techniques. The Examiner interprets the transmission of the images, known as video frame sequences as a means to use the frames for initial stages of the pose estimation generation, which requires the technique of extracting video and image frame sequences.); generating a coarse estimated point cloud video frame sequence by applying a Structure-from-Motion (SfM) algorithm to the video frame sequence (col 4 lines 18-24 and col 4 lines 35-45, Xiong teaches using the images from the camera, which is the borescope to perform the SFM technique to generate a three-dimensional model of which the accuracy will be improved at a later time. The Examiner interprets this as a coarse generation or a first draft of a model, a point cloud using the images from the captured borescope frames.); identifying a common coordinate system with respect to the model 3D point cloud and the estimated 3D point cloud (col 4 lines 44-59, Xiong teaches a starting point in the form of identifying and representing the coordinates of the two representations, initial model and estimation, as homogenous coordinates. The Examiner interprets this as creating a level of commonality with respect to the model 3D point cloud and the estimate 3D point cloud.); computing a rough alignment using a global registration algorithm (col 4 lines 44-59, Xiong teaches using an algorithm such as Gauss-Newton or Levenberg-Marquardt to transform one model to match closely with the other, which is an example of using a global registration system, an image alignment method that applied a single continuous geometric transformation to the entire image to align it with a reference image. The Examiner interprets this as the development of a rough alignment using a global registration algorithm to transform one model to closely match another.); and generating a fine registration pose estimation of the object by processing the rough alignment (col 4, lines 60-66, Xiong teaches a precise calculation of an object’s position and pose, as well as intensity, which influences pose, to a reference point that was previously achieved through using the global registration. The precise calculation produces a finer registration by way of an augmented three-dimensional model from the existing three-dimensional model. The registration includes fiducial marks and computing a homograph. The Examiner interprets this as fine registration pose estimation by processing a rough alignment from an earlier part of the process.).
Xiong does not teach generating an estimated 3D point cloud by applying a filtering algorithm to the coarse estimated point cloud video frame sequence to filter out outliers.
Srivastava is in the same field of art of 3D model generation using 2D borescope camera poses. Further Srivastava teaches generating an estimated 3D point cloud by applying a filtering algorithm to the coarse estimated point cloud video frame sequence to filter out outliers (col 7 lines 17-35, Srivastava teaches filtering out non-edges from the model that was produced using 3D scan data that came from image frames, in turn generating a more accurate 3D point cloud by using algorithms that extract edge features and calculations of point normals such as perpendicular vectors for given 3D points. The Examiner interprets the image frames as 2D frame sequences, 3D scan data as information extracted from the image and video frame sequences for the pose estimation generation, and filtering out of non-edges using algorithms such as extract edge features and calculations of point normals as filtering algorithms used to filter out outliers.);
Therefore it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Xiong by adding the generation of an estimated 3D point cloud by applying a filtering algorithm to the coarse estimated point cloud video frame sequence to filter out outliers that is taught by Srivastava to make the invention more efficient in generating more accurate 3D point clouds by using a filtering algorithms that removes deformities in the form of outliers; thus one of ordinary skilled in the art would be motivated to combine the references since they are both in the field of 3D model generation using 2D borescope camera poses (col 7 lines 17-35, Srivastava).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
28. In regards to Claim 20, Xiong in view of Srivastava teaches wherein extracting a video frame sequence includes generating the 2D borescope inspection video of the object using a borescope (col 4 lines 16-20, Xiong teaches a borescope performing the inspection of the object by way of obtaining 2D images and videos. The Examiner interprets this as the borescope, which operates two dimensionally to generate images and videos that are 2D.).
Conclusion
29. Any inquiry concerning this communication or earlier communications from the examiner should be directed to LOUIS NWUHA whose telephone number is (571)272-0219. The examiner can normally be reached Monday to Friday 8 am to 5 pm.
30. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
31. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at 3134464912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
32. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LOUIS NWUHA/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674