DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1 and 11 objected to because of the following informalities: Claim 1 and Claim 11, each recite, “based on and n depth maps for the n multi-view images;”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 11-12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Jang et al (US 20230103967 A1) in view of Lu et al (US 20250054238 A1) and further view of Kirchmay et al (US 20230334754 A1).
Regarding claim 1, Jang discloses a method for reconstructing a three-dimensional model (Jang [0054], “an example method”; [0070], “reconstruct a 2D object corresponding to the improved depth map into a 3D object … further generate and/or obtain a 3D model of the object”), the method comprising:
obtaining n (n>1) multi-view images, and a first two-dimensional feature map and a first three-dimensional feature map, based on and n depth maps for the n multi-view images (Jang [0035], “capture a plurality of frames of multiple views”; [0087], “input images 401 of different views (n multi-view images)”; Jang fig. 4; [0095], “generate the 2D feature map 440 of the set view (a first 2D feature map, based on a view) … generated 3D feature map 440 (3D feature map) … of all different views represented in the multi-view depth maps 410 (based on n depth maps for the n multi-view images, 401)”;
Jang does not disclose
estimating a mesh through an occupancy prediction based on the first two-dimensional feature map and the first three-dimensional feature map; and
applying a texture estimated based on a second two-dimensional feature map and a second three-dimensional feature map obtained based on the n multi-view images and the n depth maps to the estimated mesh to obtain a texture-applied final model.
However, Lu discloses
estimating a mesh through an occupancy prediction based on the first two-dimensional feature map and the first three-dimensional feature map (Lu fig. 4; [0080], “Step S31 … obtain first feature information (2D feature map) of a target object, and a three-dimensional point cloud of the target scene … to obtain second feature information (3D feature map) of the target object”; [0085], “Step S34 a target occupancy mesh of the target object is predicted by using an occupancy mesh prediction algorithm (estimating a mesh through an occupancy prediction at step S34 based on the 2D and 3D feature maps obtained at step S31)”); and
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Jang with Lu to utilize an occupancy mesh prediction algorithm to estimate a mesh based on 2D and 3D feature information. This would have been done to improve the quality of the reconstructed model. See for example, Lu [0098], “a semantic instance reconstruction result is outputted and represented in the form of reconstructed mesh”; [0099], “improve the quality of semantic instance reconstruction by utilizing two-dimensional semantic information and three-dimensional geometric information”.
Jang in view of Lu does not disclose
applying a texture estimated based on a second two-dimensional feature map and a second three-dimensional feature map obtained based on the n multi-view images and the n depth maps to the estimated mesh to obtain a texture-applied final model.
However, Kirchmay discloses
applying a texture estimated based on a second two-dimensional feature map and a second three-dimensional feature map obtained based on the n multi-view images and the n depth maps to the estimated mesh to obtain a texture-applied final model (Kirchmay [0073], “2D feature maps may be obtained”; [0087], “perform texture completion (obtain a texture-applied final model)”; [0095], “a volumetric data channel, comprising full-featured 3D data (a 3D feature map) … , such as a mesh representation, … depth map representation”; [0096], “… texture data to be applied on to the volumetric data contained in the volumetric data channel (texture data applied on to volumetric data, comprising a 3D feature map)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Jang further with Kirchmay to apply texture based on 2D and 3D feature data to a mesh and obtain a completed model with applied texture. This would have been taken to all for better 3D reconstruction of the model. See for example, Kirchmay [0079], “better 3D reconstruction of parts”.
Regarding claim 2, Jang in view of Lu and further view of Kirchmay discloses the method of Claim 1, wherein:
the first two-dimensional feature map corresponds to n first improved two-dimensional feature maps based on n first pixel-aligned two-dimensional feature maps extracted based on the n multi-view images and the n depth maps (Jang fig. 7; [0104], “The computing apparatus 700 may apply the improved depth map 750 to the reconstruction network 760 and reconstruct a 2D object (interpreted as reading on an refined/improved 2D feature map with pixel to pixel alignment, see for example, fig. 7)”).
Regarding claim 10, Jang in view of Lu and further view of Kirchmay discloses the method of Claim 1, wherein:
the n depth maps are estimated based on:
the n multi-view images (Jang [0026], “generate the multi-view depth maps using a depth map generation network respectively provided each of a plurality of frames”); and
a mesh estimated based on a first image among the n multi-view images (Lu [0067], “an original image of a target scene is processed by using a first target detection network to obtain first feature information of a target object”).
Claim 11 recites a method which is substantially similar to the function performed by the method of claim 1. As such, the mapping and rejection of claim 1 above is considered applicable to the method of claim 11.
Claim 12 recites a method which is substantially similar to the function performed by the method of claim 2. As such, the mapping and rejection of claim 2 above is considered applicable to the method of claim 12.
Claim 19 recites a device which is substantially similar to the function performed by the method of claim 1. As such, the mapping and rejection of claim 1 above is considered applicable to the device of claim 19.
Additionally, Jang discloses
A device for reconstructing a three-dimensional model, the device comprising: at least one processor; and at least one memory operably connected to the at least one processor, and storing an instruction to make the device perform an operation when executed by the at least one processor (Jang [0123]).
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Jang in view of Lu and further view of Kirchmay and further view of Hwang (US 11514323 B1).
Regarding claim 3, Jang in view of Lu and further view of Kirchmay discloses the method of Claim 2, wherein:
But does not disclose
the n first improved two-dimensional feature maps is obtained through a feature-improved multi-layer perceptron for a i-th view, for each i (1≤i≤n)-th first pixel-aligned two-dimensional feature map of the n first pixel-aligned two-dimensional feature maps.
However, Hwang discloses
the n first improved two-dimensional feature maps is obtained through a feature-improved multi-layer perceptron for a i-th view, for each i (1≤i≤n)-th first pixel-aligned two-dimensional feature map of the n first pixel-aligned two-dimensional feature maps (Hwang fig. 2 - Multiview images with pixel aligned feature maps; col. 4, l. 28, “inputs the (1_1)-st pooled feature map and the (1_2)-nd pooled feature map to a shared multi-layer perceptron (MLP) … generate the 1-st channel refined feature map to the d-th channel refined feature map. (n first improved two-dimensional feature maps is obtained through a refined/feature-improved multi-layer perceptron)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Jang further with Hwang to generate improved feature maps using a multi-layer perceptron. This would have been done to improv multi-view image processing. See for example, Hwang col. 33, l. 35, “The present disclosure has still another effect of improving the accuracy of the multi-view aggregation”.
Claim 13 recites a method which is substantially similar to the function performed by the method of claim 3. As such, the mapping and rejection of claim 3 above is considered applicable to the method of claim 13.
Allowable Subject Matter
Claims 4-9 and 14-18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 4, none of the prior art of record, alone or in combination, disclose the claim as recited in its entirety.
Regarding claim 5, none of the prior art of record, alone or in combination, disclose the claim as recited in its entirety.
Claims 6-7 are allowable for depending from claim 5.
Regarding claim 8, none of the prior art of record, alone or in combination, disclose the claim as recited in its entirety.
Regarding claim 9, none of the prior art of record, alone or in combination, disclose the claim as recited in its entirety.
Regarding claim 14, none of the prior art of record, alone or in combination, disclose the claim as recited in its entirety.
Regarding claim 15, none of the prior art of record, alone or in combination, disclose the claim as recited in its entirety.
Regarding claim 16, none of the prior art of record, alone or in combination, disclose the claim as recited in its entirety.
Claims 17-18 are allowable for depending from claim 15.
Conclusion
See the notice of references cited (PTO-892) for prior art made of record, including art that is not relied upon but considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JITESH PATEL whose telephone number is (571)270-3313. The examiner can normally be reached 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A. Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JITESH PATEL/Primary Examiner, Art Unit 2612
Prosecution Insights