DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In response to applicant’s amendment received on 5/7/26, all requested changes to the claims have been entered. Claims 1-20 were previously pending. Claims 5 and 17 have been cancelled. Claims 1-4, 6-16 and 18-20 are currently pending. The amendments have resolved the pending 101 rejection of claim 20 which is herein withdrawn.
Allowable Subject Matter
The indicated allowability of previously pending claims 5 and 17, now incorporated into independent claims 1, 13 and 20, is withdrawn in view of the newly discovered reference(s) to US 2021/0279943 to Murez. Rejections based on the newly cited reference(s) follow.
As a result the current rejection is again being given non-final status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 6, 7, 9-13, 15, 16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over US 2017/0169603 to Chen et al. (“Chen”) in view of US 2021/0279943 to Murez.
Regarding claim 1, Chen discloses an apparatus for image processing, comprising:
at least one memory (Fig. 1, element 130); and
at least one processor coupled to the at least one memory and, based at least in part on information stored in the at least one memory, the at least one processor, individually or in any combination, is configured to (Fig. 1, element 140 and paragraph 22):
receive, from a camera, a stream of posed images (Fig. 2; Figs. 5A, 5B, element 505; paragraphs 28, 38, wherein a stream depth images (i.e. pose images) are received, each representing a different pose of the object/scene as the camera moves around it);
update, based on each posed image in the stream of posed images, a feature volume recursively (Figs. 5A, 5B, elements 510, 515; paragraphs 38-39, wherein each depth image is sequentially/recursively converted into a 3D point cloud with estimated pose and (e.g. translation and rotation), corresponding to a feature volume, updated over a previous pose and point cloud);
update, based on the updated feature volume, a truncated signed distance function (TSDF) volume (Figs. 5A, 5B, elements 520, 525; paragraph 39, wherein each 3D point cloud and estimate posed are used to update the TSDF volume)
output an indication of the updated TSDF volume (paragraphs 26, 28, wherein an output 3D representation (i.e. updated TSDF volume) is displayed).
Chen does not disclose expressly updating the TSDF volume using a 3D CNN with the updated feature volume as an input.
Murez discloses image processing that includes receiving a stream of posed images, update, based on each posed image in the stream of posed images, a feature volume and update, based on the updated feature volume, a truncated signed distance function (TSDF) volume (Fig. 3; paragraphs 58-66).
Chen & Murez are combinable because they are from the same art of image processing, specifically updating a TSDF volume.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of updating, based on each posed image in the stream of posed images, a feature volume and updating, based on the updated feature volume, a truncated signed distance function (TSDF) volume, as taught by Murez, into the apparatus for updating a TSDF volume disclosed by Chen.
The suggestion/motivation for doing so would have been to provide accurate and full 3D models, that only requires running a large 3D CNN once at the end of a sequence (Murez, paragraphs 55, 58-60).
Therefore, it would have been obvious to combine Murez with Chen to obtain the invention as specified in claim 1.
Regarding claim 2, the combination of Chen and Murez discloses the apparatus of claim 1, wherein to receive the stream of posed images, the at least one processor, individually or in any combination, is configured to: receive each posed image in the stream of posed images consecutively in time (Chen, Fig. 2; paragraphs 28-29, 38, wherein the depth images are captured consecutively over time (e.g. every second)).
Regarding claim 6, the combination of Chen and Murez discloses the apparatus of claim 1, wherein the feature volume is a portion of a global feature volume (Chen, Fig. 2; Figs. 5A, 5B, element 505; paragraphs 28-29, 38, wherein each 3D point cloud with estimated pose and (e.g. translation and rotation) is a portion of a global feature volume represented by all the point clouds), wherein the TSDF volume is a portion of a global TSDF volume that has a one-to-one mapping to the global feature volume (Chen, Fig. 2; Figs. 5A, 5B, elements 520, 525; paragraph 39, wherein each 3D point cloud and estimate posed are fused into a TSDF volume (520), in a one-to-one mapping, and is fused into a global TSDF volume (525)).
Regarding claim 7, the combination of Chen and Murez discloses the apparatus of claim 6, wherein the at least one processor, individually or in any combination, is further configured to: determine the portion of the global feature volume to be updated based on a previous update (Chen, Fig. 2; Figs. 5A, 5B, element 505; paragraphs 28-29, 38, wherein the pose (i.e. translation and rotation) from a previous 3D point cloud (i.e. previous update) are used to determine the portion of the global feature volume associated with current update).
Regarding claim 9, the combination of Chen and Murez discloses the apparatus of claim 1, wherein to output the indication of the updated TSDF volume, the at least one processor, individually or in any combination, is configured to:
perform a three-dimensional (3D) scene reconstruction based on the updated TSDF volume (Chen, paragraphs 01, 26, 28, wherein the overall objective of generating an updated TSDF volume is to provide/reconstruct a 3D model of the object/scene).
Regarding claim 10, the combination of Chen and Murez discloses the apparatus of claim 1, wherein each posed image in the stream of posed images corresponds to an image taken by the camera and pose information of the camera associated with the image (Chen, Fig. 2; paragraphs 28-29, 38, wherein the depth images (i.e. posed images) are captured sequentially at different poses associated with the object/scene and therefore each contains information related to the pose of the camera when captured).
Regarding claim 11, the combination of Chen and Murez discloses the apparatus of claim 1, wherein the at least one processor, individually or in any combination, is further configured to:
receive a stream of depth maps associated with the stream of posed images, where the TSDF volume is updated further based on the stream of depth maps (Chen, Figs. 5A, 5B, elements 510, 515; paragraphs 38-39, wherein each depth image of the stream is sequentially/recursively converted into a 3D point cloud with estimated pose and (e.g. translation and rotation), corresponding to a feature volume or depth map used to update the TSDF volume).
Regarding claim 12, the combination of Chen and Murez discloses the apparatus of claim 1, wherein to output the indication of the updated TSDF volume the at least one processor, individually or in any combination, is configured to:
transmit the indication of the updated TSDF volume (Chen, paragraphs 26, 28, wherein an output 3D representation (i.e. updated TSDF volume) is transmitted to the display);
OR
store the indication of the updated TSDF volume (Chen, paragraph 33).
Regarding claims 13, 15, 16, 18 and 19, please refer to the rejections of claims 1, 6-9, respectively, above.
Regarding claim 20, Chen discloses a non-transitory computer-readable medium storing computer executable code (Fig. 1, element 130 and paragraph 22), the code when executed by at least one processor causes the at least one processor to (Fig. 1, element 140 and paragraph 22):
receive, from a camera, a stream of posed images (Fig. 2; Figs. 5A, 5B, element 505; paragraphs 28, 38, wherein a stream depth images (i.e. pose images) are received, each representing a different pose of the object/scene as the camera moves around it);
update, based on each posed image in the stream of posed images, a feature volume recursively (Figs. 5A, 5B, elements 510, 515; paragraphs 38-39, wherein each depth image is sequentially/recursively converted into a 3D point cloud with estimated pose and (e.g. translation and rotation), corresponding to a feature volume, updated over a previous pose and point cloud);
update, based on the updated feature volume, a truncated signed distance function (TSDF) volume (Figs. 5A, 5B, elements 520, 525; paragraph 39, wherein each 3D point cloud and estimate posed are used to update the TSDF volume),
output an indication of the updated TSDF volume (paragraphs 26, 28, wherein an output 3D representation (i.e. updated TSDF volume) is displayed).
Chen does not disclose expressly updating the TSDF volume using a 3D CNN with the updated feature volume as an input.
Murez discloses image processing that includes receiving a stream of posed images, update, based on each posed image in the stream of posed images, a feature volume and update, based on the updated feature volume, a truncated signed distance function (TSDF) volume (Fig. 3; paragraphs 56-66).
Chen & Murez are combinable because they are from the same art of image processing, specifically updating a TSDF volume.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of updating, based on each posed image in the stream of posed images, a feature volume and updating, based on the updated feature volume, a truncated signed distance function (TSDF) volume, as taught by Murez, into the apparatus for updating a TSDF volume disclosed by Chen.
The suggestion/motivation for doing so would have been to provide accurate and full 3D models, that only requires running a large 3D CNN once at the end of a sequence (Murez, paragraphs 55, 58-60).
Therefore, it would have been obvious to combine Murez with Chen to obtain the invention as specified in claim 20.
Allowable Subject Matter
Claims 3, 4, 8, and 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON W CARTER whose telephone number is (571)272-7445. The examiner can normally be reached 8am - 5pm (Mon - Fri).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON W CARTER/Primary Examiner, Art Unit 2661
Prosecution Insights