DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page 6, filed 04/27/2026, with respect to claim 12 have been fully considered and are persuasive. The minor objection of 01/28/2026 has been withdrawn.
Applicant’s arguments, see page 6, filed 04/27/2026, with respect to claims 1-14 have been fully considered and are persuasive. The 35 USC 112(b) rejection of 01/28/2026 has been withdrawn.
Applicant’s arguments, see pages 7-10, filed 04/27/2026, with respect to the rejection(s) of claim(s) 1-14 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of the amendment of 04/27/2026, which altered the scope of the claims.
Response to Amendment
The amendment of 04/27/2026 has been accepted.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 10-16 are rejected under 35 U.S.C. 103 as being unpatentable over Mora (US 2015/0178988 A1), in further view of Feng (‘Bottom and Concave Surface Rendering in Image-based Visual Hull’).
Consider claims 1, 13, and 14, Mora discloses an image processing apparatus/method of generating three-dimensional shape data corresponding to a foreground object for which synchronous image capturing is performed by a plurality of imaging apparatuses, the image processing apparatus comprising (FIGs. 2, 24; ¶98-99; “local and peripheral cameras are synchronized using a common trigger source”; ¶100; foreground segmentation; ¶108; Volumetric Shape from Silhouette, ¶125; Local High Accuracy Mesh Generation):
a non-transitory computer readable storage medium storing a program for causing a computer to perform a control method of an apparatus generating three-dimensional shape data corresponding to a foreground object for which synchronous image capturing is performed by a plurality of imaging apparatuses, the control method (FIGs. 1, 2, 24; ¶98-99; “local and peripheral cameras are synchronized using a common trigger source”; ¶100-107; foreground segmentation; ¶108; Volumetric Shape from Silhouette, ¶125; Local High Accuracy Mesh Generation) comprising the steps of:
one or more hardware processors (¶169-170); and
one or more memories storing one or more programs configured to be executed by the one or more hardware processors, the one or more programs including instructions for (¶169-170):
obtaining captured images from the plurality of imaging apparatuses and image capturing parameters of the plurality of imaging apparatuses (¶97-99, 109; “The proposed system requires a capture room equipped with a set of cameras Surrounding the person to be scanned. These cameras must be previously calibrated in a common reference frame. This implies to retrieve their intrinsic parameters (focal distance, principal point, lens distortion), which model each camera sensor and lens properties, as well as their extrinsic parameters (projection center and rotation matrix), which indicate the geometrical position of each camera in an external reference frame. These parameters are required by the system to reconstruct the 3D geometry of the observed scene… a common requirement for all the embodiments is that local and peripheral cameras are synchronized using a common trigger source.”);
obtaining a position of a virtual imaging apparatus (¶93; “View-dependent texturing uses a subset of cameras that are closest to the virtual camera as textured images, with a weight defined according to the cameras relative distance to the virtual viewpoint.”)(¶109-120; Visual hull method);
generating the three-dimensional shape data corresponding to the foreground object based on the obtained image capturing parameters of the plurality of imaging apparatuses, the obtained captured images from the plurality of imaging apparatuses, (¶98-101, 109-120; Visual hull method; ¶126-131; Mesh generation; 171-172; “foreground segmentation… The obtained subset of global masks is used to extract the visual hull of the RHM. A 3D scalar field expressed in voxels is obtained. Then a global 3D polygonal mesh is obtained by applying the marching cubes algorithm to this volume.”).
Mora fails to specifically disclose obtaining a position of a virtual imaging apparatus
obtaining a mask image that is data for deleting the partial space, wherein the mask image corresponds to an image captured by the virtual imaging apparatus; and
generating the three-dimensional shape data corresponding to the foreground object based on the obtained image capturing parameters of the plurality of imaging apparatuses, the obtained captured images from the plurality of imaging apparatuses, the obtained position of the virtual imaging apparatus, the image capturing parameters of the virtual imaging apparatus, the obtained mask image, and the virtual space.
In related art, Feng discloses obtaining captured images from the plurality of imaging apparatuses and image capturing parameters of the plurality of imaging apparatuses (Feng 2.1 Image-based Visual Hull; “the input is a group of images taken around the object with calibrated cameras”; 3 New Reference Image Acquiring Platform, FIG. 3; object and virtual object);
obtaining a position of a virtual imaging apparatus (Feng 4.1 Virtual Camera and Virtual Image; given one top image, “let p be the position of its corresponding camera C, and p’ be its symmetrical point about the mirror plane… the virtual image at position p’, and its C’ the virtual camera”) and image capturing parameters of the virtual imaging apparatus (Feng 4.2 The Calibration of Virtual Cameras; “the parameters of a virtual camera C’ could be conveniently deduced from that of its corresponding true camera C.”), wherein the virtual imaging apparatus is used to delete a partial space from a virtual space that is a generation target of the three-dimensional shape data (Feng 5.1 Concave Silhouette Cone; “projecting the silhouette curves of I onto the image plane of Cv, a new virtual silhouette image I’ could be synthesized… I’ can be considered as “captured” by Cv. Its silhouette, along with the parameters of Cv, can be used to generate a virtual “negative” silhouette cone, which is an approximation of the concave surface. Subtracting this negative cone from the intersection of other silhouette cones will result in a concave approximation of the object.”);
obtaining a mask image that is data for deleting the partial space, wherein the mask image corresponds to an image captured by the virtual imaging apparatus (Feng 5.1 Concave Silhouette Cone; “I’ can be considered as “captured” by Cv. Its silhouette, along with the parameters of Cv, can be used to generate a virtual “negative” silhouette cone, which is an approximation of the concave surface. Subtracting this negative cone from the intersection of other silhouette cones will result in a concave approximation of the object.”); and
generating the three-dimensional shape data corresponding to the foreground object based on the obtained image capturing parameters of the plurality of imaging apparatuses, the obtained captured images from the plurality of imaging apparatuses, the obtained position of the virtual imaging apparatus, the image capturing parameters of the virtual imaging apparatus, the obtained mask image, and the virtual space (Feng 4.1 Virtual Camera and Virtual Image; 4.2 The Calibration of Virtual Cameras; 5.2 Concave Surface Rendering; FIG. 7, concave surface rendering).
Mora states, “VH can provide a full reconstruction of the user volume except for its concavities [0126].” Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the virtual camera, virtual image, and concave surface rendering techniques of Feng into the shape generation method of Mora to provide a full reconstruction of the volume. Further, Feng states, “we can reconstruct an approximation of such concave surface, and the rendering result can be significantly improved (Feng Introduction).”
Consider claim 10, Mora, as modified by Feng, discloses the claimed invention wherein the one or more programs further include an instruction for:
obtaining distance information indicating a distance between each of a plurality of points on a boundary surface between an actually existing space corresponding to a generation space other than the partial space in the virtual space and an actually existing space corresponding to the partial space, and the predetermined position (Feng 2.1, 4, 5; FIGs. 3, 7), and wherein
the partial space is identified by using the distance information, in addition to the mask image and the image capturing parameters of the virtual imaging apparatus (Feng 2.1, 4, 5; FIGs. 3, 5, 7).
Consider claim 11, Mora, as modified by Feng, discloses the claimed invention wherein the three-dimensional shape data is generated by a visual hull method by using silhouette images indicating a foreground area, which is an image area in which the foreground object is captured in the obtained captured images, the silhouette images being generated based on the obtained captured images (Mora ¶101-124; Feng 2.1, 4, 5; FIGs. 3, 5, 7).
Consider claim 12, Mora, as modified by Feng, discloses the claimed invention wherein the one or more programs further include an instruction for:
generating a virtual viewpoint image based on the generated three-dimensional shape data, the obtained captured images, and virtual viewpoint information including information indicating a position of a virtual viewpoint and a direction of a line-of-sight from the virtual viewpoint (Mora ¶93; Feng 5, FIGs. 3, 5, 7).
Consider claim 15, Mora, as modified by Feng, discloses the claimed invention wherein the image capturing parameters of the virtual imaging apparatus and the mask image corresponding to the virtual imaging apparatus are stored in advance in a storage device and are obtained by reading the image capturing parameters and the mask image from the storage device (Mora ¶97, 103-107; Feng 3, 4).
Consider claim 16, Mora, as modified by Feng, discloses the claimed invention wherein the one or more programs further include an instruction for storing voxel data indicating a generation space after deletion, and wherein the stored voxel data is reused when generating the three-dimensional shape data for subsequent frames (Feng 2.1; 3-5; FIGs. 4, 5, 7, 8).
Relevant Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
JP5295044B2 discloses a method and program for extracting a mask image with high accuracy, including an unnecessary portion removal process capable of sequentially removing unnecessary portions in the vicinity of the floor surface and on the silhouette outline, and the mask.
JP6914734B2 discloses a method for silhouette extraction.
JP2020187528A, JP2020126393A (previous action).
US 2013/0315471 A1 discloses a method for concave surface modeling.
Guan, ‘Visual Hull Construction in the Presence of Partial Occlusion.’
Furukawa, ‘Carved visual hulls for image-based modeling.’
Kleinkort, ‘Visual Hull Method for Realistic 3D Particle Shape Reconstruction Based on High-
Resolution Photographs of Snowflakes in Free Fall from Multiple Views.’
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASHLEY HYTREK whose telephone number is (703)756-4562. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Steve Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASHLEY HYTREK/ Examiner, Art Unit 2665
/Stephen R Koziol/ Supervisory Patent Examiner, Art Unit 2665
Prosecution Insights