3Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 03/23/26 have been entered. Applicant’s amendments to the claims have overcome the 35 USC § 103 rejections previously set forth in the non-final office action mailed 12/22/2025, however new rejections have been issued as necessitated by amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5-11, 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Noh (US 8,599,199 B2) in view of Côté (US 10,755,483 B1), Totty (US 2020/0302686 A1), Kim (US 2021/0358211 A1), Ceylan (US 10,679,408 B2) and O. Cooper, N. Campbell and D. Gibson, "Automatic Augmentation and Meshing of Sparse 3D Scene Structure," 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1, Breckenridge, CO, USA, 2005, pp. 287-293, doi: 10.1109/ACVMOT.2005.28. (hereinafter “Cooper”).
Regarding claims 1, 11 Noh teaches A method for generating three dimensional (3D) spatial information, comprising:
detecting feature points in an image sequence (Column 4, Lines 65-67);
creating a mesh (Column 2, Lines 20-23);
modifying the mesh (Figure 2 #S230, and Column 7, Lines 16-23);
and performing texture mapping on the modified mesh (Col 7, Lines 56-60 Column 2, Lines 41-44).
Noh describes generating a mesh and correcting that mesh based upon detected error regions. Noh describes creating a projection map based on a texture map and a visibility mask. This projection map is used to create the final stereo image. This process is analogous to performing texture mapping on a modified mesh.
Noh fails to teach obtaining a sparse point cloud generated in a process of predicting camera information based on the feature points;
creating a mesh based on the sparse point cloud;
detecting a line of an object in the image sequence using a deep-learning model that is trained for straight-line object-edge detection; and modifying the mesh applying the detected line of the object in the image sequence into 3D space using the predicted camera information and placing positions of points of an object edge area of the mesh directly onto a corresponding 3D line position, thereby refining the edge area of the mesh into a straight-line form corresponding to the detected line;
wherein the mesh is created from the sparse point cloud without generating a dense point cloud, and
wherein the line is detected from the image sequence independently of the mesh and then applied to the mesh.
However Côté teaches obtaining a sparse point cloud generated in a process of predicting camera information based on the feature points (Figure 6 #620 & #630, Column 6 lines 66-67 & Column 7 line 1);
creating a mesh based on the sparse point cloud (Column 7 lines 16-18);
Côté describes creating a sparse point cloud using “keypoints” of images. Côté also describes estimating camera pose information based on image features. This is analogous to creating a sparse point cloud by predicting camera information based on feature points. Côté further describes constructing a mesh based on an extracted dense point cloud. This dense point cloud is produced by intensifying the sparse point cloud. In other words, the generation of the mesh is ultimately based on the sparse point cloud.
Côté is considered analogous to the claimed invention as it is in the same field of three-dimensional modeling. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Côté with Noh to improve the mesh construction of Noh.
Côté fails to teach detecting a line of an object in the image sequence using a deep-learning model that is trained for straight-line object-edge detection; and modifying the mesh by placing positions of points of an object edge area of the mesh on the detected line, thereby refining the edge area of the mesh into a straight-line form corresponding to the detected line;
wherein the mesh is created from the sparse point cloud without generating a dense point cloud, and
wherein the line is detected from the image sequence independently of the mesh and then applied to the mesh.
However, Totty teaches detecting a line of an object in the image sequence using a deep-learning model (Figure 8 #S410, paragraph [0097]) that is trained for straight-line object-edge detection (paragraph [0097] – “a perceptual edge detector, such as BDCN; a line segment detector, such as LCNN; a multiscale classical line detector, such as MCMLSD, and/or any other suitable detector” is); and modifying the mesh (Paragraph [0121], Figures 3 & 7); wherein the line is detected from the image sequence independently of the mesh (Totty Figure 8 #S410, #S230 paragraph [0097])
Totty is considered analogous to the claimed invention as it is in the field of three-dimensional information generation. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to utilize the line-based refinement methods of Totty to improve the mesh refinement of Noh in view of Côté.
Noh in view of Côté and Totty fails to teach applying the detected line of the object in the image sequence into 3D space using the predicted camera information and placing positions of points of an object edge area of the mesh directly onto a corresponding 3D line position, thereby refining the edge area of the mesh into a straight-line form corresponding to the detected line.
wherein the mesh is created from the sparse point cloud without generating a dense point cloud.
However, Kim teaches applying the detected line of the object in the image sequence into 3D space using the predicted camera information and placing positions of points of an object edge area of the mesh on the detected line, directly onto a corresponding 3D line position, thereby refining the edge area of the mesh to the detected line (Fig. 4, paragraphs [0023] & [0042]). Kim teaches modification of the mesh geometry based on estimated shading normals. The collection of shading normals form a line, as can be seen in Figure 4. The mesh edge is placed on this line and is updated to match by moving the mesh vertices to align with the line. The mesh edge is analogous to the object edge are of the mesh since the mesh in Kim is representative of an object.
Kim is considered analogous to the claimed invention as it is in the same field of three-dimensional computer graphics and spatial information generation. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Kim with Totty, Côté and Noh to improve the acquisition of three-dimensional geometry for three-dimensional objects.
Noh in view of Côté, Totty and Kim fails to teach applying the detected line of the object in the image sequence into 3D space using the predicted camera information; refining the edge area into a straight-line form
However, Ceylan teaches applying the detected line of the object in the image sequence into 3D space using the camera information (Col. 3 lines 3-17, line 54-55); refining the edge area into a straight-line form (Figs. 5A-5D, 10, Col. 4 lines 1-10, Col 19 lines 44-50, Claim 9). Ceylan describes a method for generating a three-dimensional model from a scanned object. More specifically Ceylan describes replacing “curved or bending lines” with “straight lines” on the edges of the edge of a mesh/ this is analogous to refining the edge area into straight-line form. Additionally, Ceylan additionally describes scanning an object using a camera and creating a 3D edge map based on this scan. While this is not predicted camera information, it is applying a detected line of an object in the image sequence into 3D space. Furthermore, it would be obvious to one of ordinary skill in the art to substitute the predicted camera information of Côté with the scanned camera information of Ceylan. Ceylan is considered analogous to the claimed invention as it is in the same field of three-dimensional computer graphics. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ceylan with the teachings of Noh in view of Côté, Totty and Kim to allow for more user-friendly and more cost effective generation of three-dimensional models (Col. 1 lines 52-65)
Noh in view of Côté, Totty, Kim and Ceylan fails to teach wherein the mesh is created from the sparse point cloud without generating a dense point cloud.
However, Cooper teaches wherein the mesh is created from the sparse point cloud without generating a dense point cloud (introduction, conclusion). Cooper describes generating a mesh based on only a sparse point cloud. Cooper is considered analogous to the claimed invention as it is in the same field of 3D graphics processing. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Cooper with Noh in view of Côté, Totty, Kim and Ceylan to specify the mesh construction is done without generating a dense point cloud because this dense point cloud generation can lead to over complex meshes that contain inaccurate corners and edges (background paragraph 2).
Regarding claims 5, 15 the combination Noh, Côté, Totty, Kim and Ceylan teach the method of claim 1 as stated above. Totty further teaches wherein the deep-learning model includes a Lookup-based Convolutional Neural Network (LCNN) (Fig 8 #S410, paragraph [0097]). Totty describes the use of an LCNN to find lines.
Regarding claims 6, 16 the combination of Noh, Côté, Totty, Kim and Ceylan teach the method of claim 1. Noh further teaches wherein the camera information includes at least one of a camera position, or a camera parameter, or a combination thereof (Column 6, Lines 24-44). Noh describes the detection of feature points using camera movement and the camera positions needed to determine camera movement.
Regarding claims 7, 17, the combination of Noh, Côté, Totty, Kim and Ceylan teach the method of claim 1. Noh further teaches wherein the feature points are detected by applying a Scale Invariant Feature Transform (SIFT) algorithm to the image sequence (Column 2, Lines 41-51). Noh describes the use of a SIFT algorithm to extract feature points.
Regarding claims 8, 18, the combination of Noh, Côté, Totty, Kim and Ceylan teach the method of claim 1. Kim further teaches predicting camera information from feature points using a Structure-from-Motion (SfM) algorithm (Paragraph [0069]). Kim acquires “camera extrinsic parameters and a rough base geometry”, analogous to camera information and feature points, through the use of an SfM algorithm .
Kim is considered analogous to the claimed invention as it is in the same field of three-dimensional computer graphics and spatial information generation. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Kim with Noh, Côté, Totty, and Ceylan to improve the acquisition of three-dimensional geometry for three-dimensional objects and their associated camera information.
Regarding claims 9, 19, the combination of Noh, Côté, Totty, Kim and Ceylan teach the method of claim 1. Kim further teaches creating the mesh from the sparse point cloud using a Poisson surface reconstruction algorithm (Paragraph [0023]). Kim describes acquiring a mesh from a 3D point cloud using a using Poisson surface reconstruction.
Kim is considered analogous to the claimed invention as it is in the same field of three-dimensional computer graphics and spatial information generation. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Kim with Noh, Côté, Totty, and Ceylan to implement a specific mesh reconstruction algorithm.
Regarding claims 10, 20, the combination of Noh, Côté, Totty, Kim and Ceylan teach the method of claim 1. Kim further teaches wherein the image sequence includes multi-view images (Paragraph [0083] first sentence). Kim describes using multi-view stereo from captured images, which requires multi-view images.
Kim is considered analogous to the claimed invention as it is in the same field of three-dimensional computer graphics and spatial information generation. Therefore it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Kim with Noh, Côté, Totty, and Ceylan to specify a source of data to perform reconstruction.
Response to Arguments
Applicant's arguments filed 03/23/2026 have been fully considered but they are not persuasive. Applicant argues that t "modifying the mesh by applying the detected line of the object in the image sequence into 3D space using the predicted camera information and placing positions of points of an object edge area of the mesh directly onto a corresponding 3D line position, thereby refining the edge area of the mesh into a straight-line form corresponding to the detected line," as recited in claim 1. Applicant goes on to describe the citation of paragraphs [0023] and [0042] of Kim. Applicant fails to respond to the example of fig. 4 present in the previous office action. Figure 4 of Kim clearly shows modifying a mesh to correspond to a line. Applicant is correct in their assertion that Kim does not teach modifying a mesh into straight line form. However, Kim is not relied upon to teach this limitation. Ceylan, which describes replacing “curved or bending lines” with “straight lines” on the edges of the edge of a mesh, is relied upon to teach this. Examiner maintains that it would have been obvious to one of ordinary skill in the art to combine the straight-line edge refinement of Ceylan with the above methods of Kim. Furthermore applicant does not make an argument against the figure 4 reference of Kim or the citation of Ceylan with respect to the above limitation. Examiner does not find this argument to be persuasive.
Applicant’s remaining arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
A. Boulch, P. -A. Langlois, G. Puy and R. Marlet, "NeeDrop: Self-supervised Shape Representation from Sparse Point Clouds using Needle Dropping," 2021 International Conference on 3D Vision (3DV), London, United Kingdom, 2021, pp. 940-950, doi: 10.1109/3DV53792.2021.00102.
R. Daroya, R. Atienza and R. Cajote, "REIN: Flexible Mesh Generation from Point Clouds," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 2020, pp. 1444-1453, doi: 10.1109/CVPRW50498.2020.00184.
M. Niehaus, L. Esch and G. Schuller, "Parametric Mesh Reconstruction Pipeline from 3D Point Clouds," ISWCS 2013; The Tenth International Symposium on Wireless Communication Systems, Ilmenau, Germany, 2013, pp. 1-5.
M. Niehaus, L. Esch and G. Schuller, "Parametric Mesh Reconstruction Pipeline from 3D Point Clouds," ISWCS 2013; The Tenth International Symposium on Wireless Communication Systems, Ilmenau, Germany, 2013, pp. 1-5.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aidan W McCoy whose telephone number is (571)272-5935. The examiner can normally be reached 8:00 AM-5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AIDAN W MCCOY/Examiner, Art Unit 2611
/TAMMY PAIGE GODDARD/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights