DETAILED ACTION
1. This Office Action is sent in response to Applicant’s communication received on 07/08/2025 for application number 19/262,969. The Office herby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, and claims.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
3. Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed.
Information Disclosure Statement
4. The information disclosure statement (IDS) submitted on 07/08/2025 is in accordance with provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Double Patenting
5. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
6. A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
7. The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
8. Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-19 of U.S. Patent No. 12,401,776. Although the claims at issue are not identical, they are not patentably distinct from each other: Table 1 shows comparison between the instant claims and the U.S. Patent 12,401,776 B1 claims.
Remaining claims are analyzed similarly.
Application: 19/262,969
1. A method performed by a data processing system, the method comprising: receiving, from a single camera, a series of images of a scene captured as the single camera is moved along a path relative to the scene;
storing first motion path metadata about the path;
storing distance metadata that provides an estimate of a distance between one or more objects in the scene and the single camera;
and/or dynamic range metadata that indicates a dynamic range in each image in a set of images in the series of images, the dynamic range for each image indicating a luminance range; associating the series of images with the first motion path metadata;
and associating a metadata label with the series of images, the metadata label indicating that the series of images represent a volumetric image of the scene.
Patent: 12,401,776
1. A method performed by a data processing system, the method comprising: receiving a series of images with associated motion path metadata and a volumetric metadata label indicating the series of images represent a volumetric image of a scene;
determining a desired viewpoint of the volumetric image;
determining from the desired viewpoint a selected image based on the series of images and upon a comparison of the desired viewpoint to the motion path metadata; displaying the selected image.
2. The method as in claim 1, wherein a volumetric image comprises a set of images of the scene from different camera positions or viewpoints.
2. The method as in claim 1, wherein a volumetric image comprises a set of images of the scene from different camera positions or viewpoints.
3. The method as in claim 1, wherein the capturing is performed at a rate based upon movement of the camera along the path.
3. The method as in claim 1, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a rate based upon movement of the camera along the path.
4. The method as in claim 1, wherein the capturing is performed at a predetermined frame rate used for displaying video.
4. The method as in claim 1, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a predetermined rate used for displaying video.
5. The method as in claim 1, wherein the series of images, the associated first motion path metadata and the metadata label are assembled into a package, and wherein the series of images are compressed in the package.
6. The method as in claim 1, wherein the series of images, the associated motion path metadata and the volumetric metadata label are received as a package, and wherein the series of images are compressed in the package.
6. The method as in claim 1, wherein the series of images are continuously captured and stored, over a period of time.
5. The method as in claim 1, wherein the series of images were recorded during a continuous capturing and storing of images in a single camera, over a period of time along a path of motion of the single camera.
7. The method as in claim 1, wherein the method further comprises: conforming the series of images to a desired motion path; and updating the first motion path metadata based on the conformed series of images.
7. The method as in claim 1, wherein the motion path metadata indicates displacement, along a path used during recording of the series of images, from one image to the next image in the series of images, and the motion path metadata is used at playback time to select for display the desired viewpoint on the scene, and wherein the recording, at playback time, supports (1) display of a single image at the desired viewpoint and (2) display of the series of images as a movie.
8. The method as in claim 7, wherein the conforming vertically crops at least one or more images in the series of images and wherein the updating of the first motion path metadata updates the first motion path metadata based on changes, as a result of the vertical cropping, to the series of images.
8. The method as in claim 1, wherein the desired viewpoint is determined from one of (1) manual user selection from a user interface or (2) sensor based tracking of a user's face or head or (3) a predetermined set of one or more viewpoints provided by a content creator.
9. The method as in claim 1, wherein the method further comprises: adjusting a position of one or more images in the series of images to smooth one or more displacements along the desired motion path from image to image in the series of images.
9. The method as in claim 8, wherein the sensor based tracking automatically determines the desired viewpoint from a location, detected by the sensor, of a viewer's head.
10. The method as in claim 8, wherein the method further comprises: displaying one or more guides, during capture of the series of images, to guide a user, who moves the single camera, along the desired motion path.
“…the series of images can be conformed to a desired motion path and the motion path metadata can be updated based on the conformed series of images…” Col 2 Lines 47-60
11. The method as in claim 1, wherein the first motion path metadata indicates displacement, along the path, from one image to the next image in the series of images.
“…the series of images can be conformed to a desired motion path and the motion path metadata can be updated based on the conformed series of images…” Col 2 Lines 47-60
12. A method performed by a data processing system, the method comprising: receiving a series of images with associated motion path metadata and a volumetric metadata label indicating the series of images represent a volumetric image of a scene, the series of image further including distance metadata that provides an estimate of a distance between one or more objects in the scene and a single camera and/or dynamic range metadata that indicates a dynamic range in each image in a set of images in the series of images, the dynamic range for each image indicating a luminance range; determining a desired viewpoint of the volumetric image; determining from the desired viewpoint a selected image based on the series of images; and displaying the selected image.
10. A non-transitory machine readable medium storing executable program instructions which when executed by a data processing system, cause the data processing system to perform a method comprising: receive a series of images with associated motion path metadata and a volumetric metadata label indicating the series of images represent a volumetric image of a scene; determine a desired viewpoint of the volumetric image;
determine from the desired viewpoint a selected image based on the series of images and upon a comparison of the desired viewpoint to the motion path metadata; display the selected image.
13. The method as in claim 12, wherein a volumetric image comprises a set of images of the scene from different camera positions or viewpoints.
11. The machine-readable storage medium of claim 10, wherein a volumetric image comprises a set of images of the scene from different camera positions or viewpoints.
14. The method as in claim 12, wherein determining the selected image is based upon a comparison of the desired viewpoint to the motion path metadata.
12. The machine-readable storage medium of claim 10, wherein determining the selected image is based upon a comparison of the desired viewpoint to the motion path metadata.
15. The method as in claim 12, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a rate based upon movement of the camera along the path.
13. The machine-readable storage medium of claim 10, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a rate based upon movement of the camera along the path.
16. The method as in claim 12, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a predetermined rate used for displaying video.
14. The machine-readable storage medium of claim 10, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a predetermined rate used for displaying video.
17. The method as in claim 12, wherein the series of images were recorded during a continuous capturing and storing of images in a single camera, over a period of time along a path of motion of the single camera.
15. The machine-readable storage medium of claim 10, wherein the series of images were recorded during a continuous capturing and storing of images in a single camera, over a period of time along a path of motion of the single camera.
18. The method as in claim 12, wherein the series of images, the associated motion path metadata and the volumetric metadata label are received as a package, and wherein the series of images are compressed in the package.
16. The machine-readable storage medium of claim 10, wherein the series of images, the associated motion path metadata and the volumetric metadata label are received as a package, and wherein the series of images are compressed in the package.
19. The method as in claim 12, wherein the motion path metadata indicates displacement, along a path used during recording of the series of images, from one image to the next image in the series of images, and the motion path metadata is used at playback time to select for display the desired viewpoint on the scene, and wherein the recording, at playback time, supports (1) display of a single image at the desired viewpoint and (2) display of the series of images as a movie.
17. The machine-readable storage medium of claim 10, wherein the motion path metadata indicates displacement, along a path used during recording of the series of images, from one image to the next image in the series of images, and the motion path metadata is used at playback time to select for display the desired viewpoint on the scene, and wherein the recording, at playback time, supports (1) display of a single image at the desired viewpoint and (2) display of the series of images as a movie.
20. The method as in claim 12, wherein the desired viewpoint is determined from one of (1) manual user selection from a user interface or (2) sensor-based tracking of a user's face or head or (3) a predetermined set of one or more viewpoints provided by a content creator.
18. The machine-readable storage medium of claim 10, wherein the desired viewpoint is determined from one of (1) manual user selection from a user interface or (2) sensor based tracking of a user's face or head or (3) a predetermined set of one or more viewpoints provided by a content creator.
Election/Restrictions
9. Restriction is required under 35 U.S.C. 121 and 372.
10. This application contains the following inventions or groups of inventions which are not so linked as to form a single general inventive concept under PCT Rule 13.1.
In accordance with 37 CFR 1.499, applicant is required, in reply to this action, to elect a single invention to which the claims must be restricted.
Group 1, claim(s) 1-6, 11, 13-20, drawn to a method for improving the chances of being able to perform viewpoint interpolations
Group 2 claim(s) 7-8, and 10, drawn to a system for rendering the system more flexible
Group 3 claim 9, drawn to eliminating large jumps between adjacent images
Group 4 claims 1 and 12, adjusting the luminance values of image data
11. Claims 1-20 lack unity of invention because the groups do not share the same or corresponding technical feature.
Claim Rejections - 35 USC § 103
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. Claim(s) 1 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hong et al., [US Pub. No.: 2017/0061687 A1] in view of Mecca et al., [US Pub. No.: 2021/0044788A1].
Re. Claim 1, Hong et al., [US Pub. No.: 2017/0061687 A1] discloses:
A method performed by a data processing system [Server 12 includes one or more processors |0039], the method comprising: receiving, from a single camera, a series of images of a scene captured as the single camera is moved along a path relative to the scene [Fig. 3 camera travel path for capturing images| 0047];
storing first motion path metadata about the path [the memory 14 stores the medical scan data, presets, and/or the instructions for the server 12 to perform rendering |0040];
and/or dynamic range metadata that indicates a dynamic range in each image in a set of images in the series of images, the dynamic range for each image indicating a luminance range [The values for the window/level (e.g., range of scalar values of interest), transfer function, illumination model, color maps, camera field-of-view, or other rendering settings are acquired by the server 12. See also The server 12 also receives the preset. The preset is a collection of any number of rendering parameters. The illumination model is equivalent to luminance range as described in applicants specification at para. 0013,0048 |0038];
associating the series of images with the first motion path metadata [Fig.3 generate a sequence of images to display as a movie. The sequence is for images along a path. FIG. 3 shows an example path with nodes 30.];
and associating a metadata label with the series of images, the metadata label indicating that the series of images represent a volumetric image of the scene [The path is provided as part of the preset, provided with or as part of the data set for the volume, is created by the user, or is created by the server 12. |0048].
Hong does not distinctly disclose:
storing distance metadata that provides an estimate of a distance between one or more objects in the scene and the single camera;
In the same field of endeavor Mecca et al., [US Pub. No.: 2021/0044788A1] discloses:
storing distance metadata that provides an estimate of a distance between one or more objects in the scene and the single camera [a 3D volumetric reconstruction using differential parameterisation of the volume with the signed distance function merging multi-view and photometric stereo imaging data… See Also an octree implementation capable of re-tracing visibility information as well as shadows that are crucially important in scenes with multiple discrete objects.| 0182, 0186 Fig.7 and Fig.23];
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine Hong with Mecca to provide a 3D volumetric reconstruction using differential parameterization of the volume with the signed distance function merging multi-view and photometric stereo imaging data.
Re. Claim 2, Hong discloses:
The method as in claim 1, wherein a volumetric image comprises a set of images of the scene from different camera positions or viewpoints [each node has two offset viewpoints for stereoscopic viewing. The server 12 renders multiple view directions from each of two viewpoints for a given node 30 or location. Images are rendered at each position for offset viewpoints to produce stereoscopic image pairs. |0055].
Re. Claim 3, Hong discloses:
The method as in claim 1, wherein the capturing is performed at a rate based upon movement of the camera along the path [A movie sequence for imaging while progressing along the path is pre-generated.|0025].
Re. Claim 4, Hong discloses:
The method as in claim 1, wherein the capturing is performed at a predetermined frame rate used for displaying video [high quality rendered frames are generated for each point in time along each render path |0047].
Re. Claim 5, Hong discloses:
The method as in claim 1, wherein the series of images, the associated first motion path metadata and the metadata label are assembled into a package, and wherein the series of images are compressed in the package [the different viewpoint renderings are packed together into an atlas before encoding.|0108].
Re. Claim 6, Hong discloses:
The method as in claim 1, wherein the series of images are continuously captured and stored, over a period of time [Fig. 3 images are captured along a path continuously].
Re. Claim 7, Hong discloses:
The method as in claim 1, wherein the method further comprises: conforming the series of images to a desired motion path [the user at the client alters the view direction interactively, the parts of the already provided video images for the selected view direction are extracted and displayed to the user.|0030];
and updating the first motion path metadata based on the conformed series of images [the result is the user being able to interactively alter the view direction for one or more locations without requiring re-rendering.|0030].
Re. Claim 9, Hong discloses:
The method as in claim 1, wherein the method further comprises: adjusting a position of one or more images in the series of images to smooth one or more displacements along the desired motion path from image to image in the series of images [The video graph provides the ability to smoothly transition between segments, whether segments for spatial locations or segments for rendering parameters. High quality pre-rendered data exploration using video may avoid the need of costly servers or intensive computation power at runtime for real-time interactivity.|0029].
.
Re. Claim 10, Hong discloses:
The method as in claim 8, wherein the method further comprises: displaying one or more guides, during capture of the series of images, to guide a user, who moves the single camera, along the desired motion path [The user selects the location along the video graph, which relates the corresponding pre-rendered image or images to the location. By clicking on a point of interest on one of the branches of the graph or navigating through the views, the movie player jumps to the corresponding frame of the movie.|0028].
Re. Claim 11, Hong discloses:
The method as in claim 1, wherein the first motion path metadata indicates displacement, along the path, from one image to the next image in the series of images [a single line may be divided into portions to form different segments 40, such as shown in FIGS. 9 and 10 where the center branch is treated as a separate segment 40 while being a continuous line from the main branch.| Fig. 9-10, 0091].
Re. Claim 12, Hong discloses:
A method performed by a data processing system [Server 12 includes one or more processors |0039], the method comprising: receiving a series of images with associated motion path metadata and a volumetric metadata label indicating the series of images represent a volumetric image of a scene [Fig. 3 camera travel path for capturing images| 0047], the series of image further including distance metadata that provides an estimate of a distance between one or more objects in the scene and a single camera and/or dynamic range metadata that indicates a dynamic range in each image in a set of images in the series of images, the dynamic range for each image indicating a luminance range [The values for the window/level (e.g., range of scalar values of interest), transfer function, illumination model, color maps, camera field-of-view, or other rendering settings are acquired by the server 12. See also The server 12 also receives the preset. The preset is a collection of any number of rendering parameters. The illumination model is equivalent to luminance range as described in applicants specification at para. 0013,0048 |0038];
determining a desired viewpoint of the volumetric image [the user indicates with the user input 24 a node 30 for which other view directions are desired |0076];
determining from the desired viewpoint a selected image based on the series of images [The user selects the location along the video graph, which relates the corresponding pre-rendered image or images to the location. By clicking on a point of interest on one of the branches of the graph or navigating through the views, the movie player jumps to the corresponding frame of the movie.|0028], of a viewer’s head [user input via interface 26 |0071-0072];
and displaying the selected image [The user input 24 is then used to indicate the desired view direction through rotation or direct entry of an angle. The change in view direction is then applied to the image for the node or applied to images of multiple nodes or spatial locations/times.|0076].
Re. Claim 13, Hong discloses:
The method as in claim 12, wherein a volumetric image comprises a set of images of the scene from different camera positions or viewpoints [each node has two offset viewpoints for stereoscopic viewing. The server 12 renders multiple view directions from each of two viewpoints for a given node 30 or location. Images are rendered at each position for offset viewpoints to produce stereoscopic image pairs. |0055].
Re. Claim 14, Hong discloses:
The method as in claim 12, wherein determining the selected image is based upon a comparison of the desired viewpoint to the motion path metadata [the user at the client alters the view direction interactively, the parts of the already provided video images for the selected view direction are extracted and displayed to the user.|0030].
Re. Claim 15, Hong discloses:
The method as in claim 12, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a rate based upon movement of the camera along the path [A movie sequence for imaging while progressing along the path is pre-generated.|0025]..
Re. Claim 16, Hong discloses:
The method as in claim 12, wherein the series of images is a series of images captured by a single camera as the camera moved along a path, and wherein the series of images were captured at a predetermined rate used for displaying video [high quality rendered frames are generated for each point in time along each render path |0047]..
Re. Claim 17, Hong discloses:
The method as in claim 12, wherein the series of images were recorded during a continuous capturing and storing of images in a single camera, over a period of time along a path of motion of the single camera [Fig. 3 images are captured along a path continuosusly].
Re. Claim 18, Hong discloses:
The method as in claim 12, wherein the series of images, the associated motion path metadata and the volumetric metadata label are received as a package, and wherein the series of images are compressed in the package [the different viewpoint renderings are packed together into an atlas before encoding.|0108].
Re. Claim 20, Hong discloses:
The method as in claim 12, wherein the desired viewpoint is determined from one of (1) manual user selection from a user interface or (2) sensor-based tracking of a user's face or head or (3) a predetermined set of one or more viewpoints provided by a content creator [user input uses touch screen |0028 0071-0072].
Allowable Subject Matter
14. Claims 8 and 19 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HOWARD D BROWN JR whose telephone number is (571)272-4371. The examiner can normally be reached Monday - Friday 7:30AM - 5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sathyanarayanan Perungavoor can be reached at 5712727455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
HOWARD D. BROWN JR
Primary Examiner
Art Unit 2488
/HOWARD D BROWN JR/Examiner, Art Unit 2488
Prosecution Insights