DETAILED ACTION
Response to Arguments
Applicant's arguments filed 03/09/2026 regarding the 35 USC 103 rejections with respect to the amended limitations of claims 1-10, 12-21 have been considered but are moot in view of the new ground(s) of rejection necessitated by the amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application that indicate obviousness or nonobviousness.
Claims 1-2, 8, 17-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al (US 20190370989 A1), in view of Li et al (US 20180255283 A1), and further in view of Dotsenko et al (US 20230098187 A1) and Lucas et al (US 20190325638 A1).
RE claim 1, Zhao teaches An apparatus, comprising: one or more memories configured to store a plurality of two-dimensional (2D) images of a scene including one or more objects; and one or more processors, coupled to the one or more memories (Abstract, Figs 1, 5, [0009]), configured to cause the apparatus to:
obtain a plurality of voxels of a three-dimensional (3D) voxel grid that represent the scene (Fig 2, [0025], [0035]);
identify a subset of voxels from the plurality of voxels, wherein for each voxel in the subset of voxels: the voxel is within a threshold distance of one or more surfaces of the one or more objects based on respective depth information, for the voxel, associated with the plurality of 2D images (Figs 1-2, [0027]-[0030], [0039]-[0041]); and
generate a point cloud comprising a set of point data that correspond to the subset of voxels (Fig 1, [0022], [0029], [0042], [0056]).
Zhao is silent RE: And a respective confidence score associated with the respective depth information for the voxel satisfies a confidence threshold;
However Li teaches a respective confidence score associated with the respective depth information for the voxel in Abstract, [0111]-[0112], [0095] providing an adaptive weighting for uncertainty parameters of a TSDF. He further teaches a confidence threshold for a pixel in [0062] to select reliable reference points according to the threshold. This is readily available or can equally applied in Zhao to select the voxel based on the voxel satisfies a confidence threshold associated with the respective depth information, wherein Zhao teaches the voxel depth has a weight in [0041].
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Zhao a system and method a respective confidence score associated with the respective depth information for the voxel satisfies a confidence threshold, as set forth above applying Li, in order to further tune the voxel data filtering noise and uncertain data improving the result of the three-dimensional reconstruction and thereby increasing system effectiveness and user experience.
Zhao as modified by Li is silent RE: point data structures and process the point cloud to reconstruct a 3D representation of the scene.
However Dotsenko teaches typical point data structures to represent a point cloud in Fig 5 and [0048]-[0049]. In addition Lucas teaches process the point cloud to reconstruct a 3D representation of the scene in abstract, [0211] to display the 3D model.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Zhao as modified by Li a system and method to generate point data structures process and the point cloud to reconstruct a 3D representation of the scene, as suggested by Dotsenko and Lucas, in order to effectively represent the point cloud data and display the 3D model from the point cloud data and thereby increasing system effectiveness and user experience.
RE claim 2, Zhao as modified by Li, Dotsenko and Lucas teaches wherein cause the apparatus to identify the subset of voxels, the one or more processors are configured to cause the apparatus to: for each voxel of the subset of voxels, include the voxel in the subset of voxels based on a respective difference between a respective voxel distance from a viewpoint based on the 3D voxel grid and the respective depth information associated with the voxel based on the plurality of 2D images being less than the threshold distance (Zhao Figs 1-2, [0027]-[0030], [0039]-[0041]).
RE claim 8, Zhao as modified by Li, Dotsenko and Lucas teaches wherein to cause the apparatus to generate the point cloud, the one or more processors are configured to cause the apparatus to, for each voxel in the subset of voxels: create a respective point data structure in the set of point data structures; and for respective point data structure: a that correspond 3D grid coordinate location of the associated respective voxel as a 3D position for the respective point data structure and aggregated feature vectors associated with the associated respective voxel as point cloud feature vectors of the respective point data structure (Zhao Fig 1, [0036]-[0038], [0042], [0049]- [0050], [0056] wherein the weighted sum of the depth and color information are aggregated feature vectors that can be equally stored in the point data structures).
Claims 17-18 recite limitations similar in scope with limitations of claims 1-2 as method and therefore rejected under the same rationale.
Claim 20 recites limitations similar in scope with limitations of claim 1 and therefore rejected under the same rationale. In addition Zhao teaches A non-transitory computer-readable medium comprising instructions ([0010]).
Claims 3-7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao as modified by Li, Dotsenko and Lucas, and further in view of Li et al (US 20190325638 A1, hereinafter Li638).
RE claim 3, Zhao as modified by Li, Dotsenko and Lucas is silent RE: wherein to cause the apparatus to obtain the plurality of voxels of the 3D voxel grid, the one or more processors are configured to cause the apparatus to: generate, by an encoder, a plurality of encoded feature representations associated with the plurality of 2D images; and back-project the plurality of encoded feature representations into the plurality of voxels of the 3D voxel grid.
However Li638 teaches in Figs 1-4, 9, [0060], [0065], [0070]-[0071], [0105].
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Zhao as modified by Li, Dotsenko and Lucas a system and method wherein to cause the apparatus to obtain the plurality of voxels of the 3D voxel grid, the one or more processors are configured to cause the apparatus to generate, by an encoder, a plurality of encoded feature representations associated with the plurality of 2D images; and back-project the plurality of encoded feature representations into the plurality of voxels of the 3D voxel grid that represent the scene, as suggested by Li638, in order to generate the voxel grid utilizing a neural network and thereby increasing system effectiveness and user experience.
RE claim 4, Zhao as modified by Li, Dotsenko, Lucas and Li638 teaches, wherein to cause the apparatus to back-project the plurality of encoded feature representations, the one or more processors are configured to cause the apparatus to:
generate a 3D voxel position along a viewpoint ray that extend between an origin point associated with an image capture device and an image pixel of a respective 2D image, the image pixel that correspond to a surface of the one or more surfaces of the one or more objects; and assign one or more encoded feature representations of the plurality of encoded feature representations associated with the image pixel to a voxel of the plurality of voxels based on the 3D voxel position that correspond to a depth value of the voxel (Li638 Figs 1-4, [0060], [0065], [0070]-[0071]).
RE claim 5, Zhao as modified by Li, Dotsenko, Lucas and Li638 teaches, wherein the respective depth information, for each voxel in the subset of voxels, includes per-pixel depth values for a plurality of pixels in the plurality of 2D images (Zhao [0032], Li638 [0103]).
RE claim 6, Zhao as modified by Li, Dotsenko, Lucas and Li638 teaches, wherein: the plurality of encoded feature representations comprise a plurality of feature vectors associated with pixels of the plurality of 2D images; and to back-project the plurality of encoded feature representations comprises to back-project the plurality of feature vectors into the plurality of voxels, guided by the per-pixel depth values (Li638 Figs 1-4, [0060], [0065], [0070]-[0071]).
RE claim 7, Zhao as modified by Li, Dotsenko, Lucas and Li638 teaches, wherein to cause the apparatus to generate the plurality of encoded feature representations, the one or more processors are configured to cause the apparatus to: input the plurality of 2D images into a convolutional neural network encoder to generate at least one feature vector for each pixel of a plurality of pixels of the plurality of 2D images (Li638 Figs 1-4, [0060], [0065], [0070]-[0071]).
Claim 19 recites limitations similar in scope with limitations of claim 3 and therefore rejected under the same rationale.
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao as modified by Li, Dotsenko and Lucas, and further in view of Fan et al (US 20240161395 A1).
RE claim 9, Zhao as modified by Li, Dotsenko and Lucas is silent RE: wherein to cause the apparatus to process the point cloud, the one or more processors are configured to cause the apparatus to, for each subset of point data structures of one or more subsets of the set of point data structures:voxelize respective local neighborhood point data structures of the subset of point data structures;perform 3D convolutions on the voxelized respective local neighborhood point data structures; andde-voxelize respective outputs of the 3D convolutions to obtain respective aggregated feature vectors for the subset of point data structures.
However Fan teaches in Figs 1-3, abstract, [0028], [0031]-[0033], to permit the extraction of both global and local geometric features, increasing the accuracy of 3D modeling of objects utilizing a neural network, wherein the processing can be equally applied to the set of point data structures, as readily recognized by one of ordinary skill in the art.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Zhao as modified by Li, Dotsenko and Lucas a system and method of wherein to cause the apparatus to process the point cloud, the one or more processors are configured to cause the apparatus to, for each subset of point data structures of one or more subsets of the set of point data structures: voxelize respective local neighborhood point data structures of the subset of point data structures; perform 3D convolutions on the voxelized respective local neighborhood point data structures; and de-voxelize respective outputs of the 3D convolutions to obtain respective aggregated feature vectors for the subset of point data structures, as set forth above applying Fan, in order to increase the accuracy of 3D modeling of objects utilizing a neural network and thereby increasing system effectiveness and user experience.
RE claim 10, Zhao as modified by Li, Dotsenko, Lucas and Fan teaches, wherein to cause the apparatus to process the point cloud, the one or more processors are configured to cause the apparatus to, for each point data structure of each subset of point data structures: predict a respective Truncated Signed Distance Function (TSDF) value based on the respective aggregated feature vectors for the subset of point data structures (Fan [0035]).
Claims 12-15 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao as modified by Li, Dotsenko and Lucas, and further in view of Tremblay et al (US 12444126 B1).
RE claim 12, Zhao as modified by Li, Dotsenko and Lucas is silent RE: wherein the one or more processors are configured to: further comprising: receiving input that indicates a specified object of the one or more objects; and identify pixels that represent the specified object in the plurality of 2D images, wherein the subset of voxels does not include voxels that correspond to the pixels that represent the specified object.
However Tremblay teaches in Figs 4, col 9 lines 12-25, col 9 lines 54-62 to generate the 3D representation excluding selected objects.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Zhao as modified by Li, Dotsenko and Lucas a system and method wherein the one or more processors are configured to: further comprising: receiving input that indicates a specified object of the one or more objects; and identify pixels that represent the specified object in the plurality of 2D images, wherein the subset of voxels does not include voxels that correspond to the pixels that represent the specified object, as suggested by Tremblay, in order to generate the 3D representation excluding selected objects and thereby increasing system effectiveness and user experience.
RE claim 13, Zhao as modified by Li, Dotsenko, Lucas and Tremblay teaches, wherein the 3D representation of the scene, reconstructed from the point cloud, excludes the specified object (Tremblay Figs 4, col 9 lines 12-25).
RE claim 14, Zhao as modified by Li, Dotsenko and Lucas is silent RE: wherein the one or more processors are configured to: receive input that indicate at least one object of the one or more objects; and identify pixels that represent at least one object in the plurality of 2D images, and identify the subset of voxels comprises to include voxels that correspond to the identified pixels that represent the at least one object.
However Tremblay teaches in Figs 4, col 9 lines 12-25, col 9 lines 54-62 to generate the 3D representation only selected objects.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Zhao as modified by Li, Dotsenko and Lucas a system and method wherein the one or more processors are configured to: receive input that indicate at least one object of the one or more objects; and identify pixels that represent at least one object in the plurality of 2D images, and identify the subset of voxels comprises to include voxels that correspond to the identified pixels that represent the at least one object, as suggested by Tremblay, in order to generate the 3D representation only selected objects and thereby increasing system effectiveness and user experience.
RE claim 15, Zhao as modified by Li, Dotsenko, Lucas and Tremblay teaches, wherein the 3D representation of the scene, reconstructed from the point cloud, includes the at least one object (Tremblay Figs 4, col 9 lines 12-25).
Claim 21 recites limitations similar in scope with limitations of claim 12 as method and therefore rejected under the same rationale.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Zhao as modified by Li, Dotsenko and Lucas, and further in view of Halder (US 20210263152 A1).
RE claim 16, Zhao as modified by Li, Dotsenko and Lucas is silent RE: wherein to cause the apparatus to process the point cloud to reconstruct the 3D representation of the scene, the one or more processors are configured to cause the apparatus to: utilize a semantic label to reconstruct surfaces, of the one or more surfaces, that correspond to an object of the one or more objects with known geometric properties.
However Halder teaches in [0076], [0081] to generate the 3D representation using a machine learning (ML) model.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Zhao as modified by Li, Dotsenko and Lucas a system and method wherein to cause the apparatus to process the point cloud to reconstruct the 3D representation of the scene, the one or more processors are configured to cause the apparatus to: utilize a semantic label to reconstruct surfaces, of the one or more surfaces, that correspond to an object of the one or more objects with known geometric properties, as suggested by Halder, in order to generate the 3D representation using a machine learning (ML) model and thereby increasing system effectiveness and user experience.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached 892.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SULTANA MARCIA ZALALEE whose telephone number is (571)270-1411. The examiner can normally be reached Monday- Friday 8:00am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Sultana M Zalalee/ Primary Examiner, Art Unit 2614
Prosecution Insights