DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . It is responsive to the submission dated 07/14/2026. Claims 1-9 are presented for examination of which, claims 1, 8 and 9 are independent claims.
Response to Arguments
2. Applicant’s arguments, see pages 5-7 of Applicant’s Remarks, filed 07/14/2026, with respect to the obviousness rejections under 35 USC 103 of the claims have been fully considered and are persuasive. The rejections of these claims have been withdrawn since the amendments remedy the previous issues.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1-9 are rejected under 35 U.S.C. 103 as being unpatentable over Arai (US 20200082603) in view of Tanaka (US 10291875).
Considering claim 1, Arai discloses a playback apparatus (see fig. 1) comprising:
one or more memories storing instructions (items 102 or 103, fig. 1); and one or more processors (item 101, fig. 1) executing the instructions (see paras. 24-27 of Arai) to:
play back a virtual viewpoint image from a predetermined start timecode to a predetermined end timecode (for examples, Arai discloses: The virtual viewpoint path managed based on frames and time lines is configured to define a motion of the virtual viewpoint in a video image generated by sequentially playing back multiple virtual viewpoint images or multiple CG images. A frame holds information … on clock times (time codes) of scenes as well as the positions and the orientations of the virtual viewpoints. Each clock time of the scene is expressed by a time code while defining the clock time of start of a game targeted for image capturing as a frame at 00 hour, 00 minute, 00 second and 00, for example. See para. 43. Examples of the method of generating the virtual viewpoint image include a “slow” mode, a “switching” mode, a “stop” mode, and the like. The “slow” mode is suitable for a case where a time interval between the key frames are extremely short or a case where a distance between the virtual viewpoints at two ends of the precipitous section is shorter than a predetermined distance. See para. 55. The “stop” mode is designed to move the virtual viewpoint during a set playback time period in a state of stopping time of the three-dimensional model between the key frames detected as the precipitous virtual viewpoint path, and to release the stop of the time of the three-dimensional model after moving the virtual viewpoint. .. This “stop” mode is considered to be useful in a case where the distance between the virtual viewpoints at the two ends of the precipitous section is larger than the predetermined distance. See para. 57),
the virtual viewpoint image being generated using three-dimensional shape data generated from images captured from a plurality of positions and virtual viewpoint information (for examples, Arai discloses: A virtual viewpoint imaging apparatus group 211 carries out synchronous image capturing by using multiple imaging apparatuses installed in such a way as to surround a playing field and the like, and outputs captured images obtained by the image capturing to the virtual viewpoint image generating unit 212. See para. 32. The virtual viewpoint image generating unit 212 generates a three-dimensional model from a multiple view image obtained by the synchronous image capturing with the virtual viewpoint imaging apparatus group 211. Then, the virtual viewpoint image is generated by mapping textures based on the virtual viewpoints (the positions, the orientations, and the angles of view of the virtual viewpoints) in the virtual viewpoint path created by the virtual viewpoint path creating unit 207. See para. 41, wherein the generated 3D model image obtained from the multiple view image corresponds to the use of the three-dimensional shape data generated from the captured images to generate the virtual viewpoint image); and
control the virtual viewpoint information (e.g., Akai teaches: The virtual viewpoint path editing unit 20 determines a parameter group of a series of virtual viewpoint parameters that control a virtual viewpoint and constitute a virtual viewpoint path. See para. 30).
In addition, Arai discloses a user, via an editing controller unit on a user interface, during play back, controls/edits the virtual viewpoint path by using the virtual viewpoint parameters associated with designated key frames and the editing unit transmits the information on the frames targeted/designated for editing constantly to the virtual viewpoint image generating unit for display. See paras. 45-46. This corresponds to one manner for controlling the virtual viewpoint information in playing back the virtual viewpoint image.
Furthermore, Arai discloses to create the virtual viewpoint path, the virtual viewpoint operating unit 201 obtains the time code, the position and the orientation information on the virtual viewpoint, and the like to be designated as the key frame information based on a user operation and registers these pieces of information with the key frame setting unit 202. The above-mentioned operation to register the key frame information is repeated at least twice, and the obtained information is stored in the key frame storing unit 203. Then, the virtual viewpoint variation measuring unit 204 detects a precipitous section in the virtual viewpoint path based on the key frame information stored in the key frame storing unit 203. See paras. 49-51. And, in a case where the virtual viewpoint variation measuring unit 204 detects any precipitous section in the virtual viewpoint path, the method of generating the virtual viewpoint image include a “stop” mode that is designed to move the virtual viewpoint during a set playback time period in a state of stopping time of the three-dimensional model between the key frames detected as the precipitous virtual viewpoint path, and to release the stop of the time of the three-dimensional model after moving the virtual viewpoint. In the case of setting the playback speed of the three-dimensional model to zero as mentioned above, the virtual viewpoint parameters corresponding to the playback time are associated with one time record of the captured image. This “stop” mode is considered to be useful in a case where the distance between the virtual viewpoints at the two ends of the precipitous section is larger than the predetermined distance, such as in a case of switching from a viewpoint on one side of a field to a viewpoint of another side thereof being the opposite side. These image generation methods may be automatically determined on a system side based on a time interval between the key frames, or a distance between the positions of the virtual viewpoints See paras. 52-58.
Here, according to Arai, it is implied that the image generating unit causes the system automatically initiates the designated start timecode in response to detecting a stopping time of the three-dimensional model between the key frames detected as the precipitous virtual viewpoint path. This process is construed to encompass controlling the initialization of starting a predetermined start timecode is in response to an end of playback of the virtual viewpoint image in a manner that is different from the aforementioned controlling the virtual viewpoint information in playing back the virtual viewpoint image.
As such, it is submitted that the Arai reference obviously encompasses all the characteristics necessary for controlling the play back of a virtual viewpoint image from a predetermined start timecode to a predetermined end timecode, as stated above, such that the initialized predetermined start timecode is in response to an end of playback of the virtual viewpoint image, wherein the virtual viewpoint information is controlled differently in playing back the virtual viewpoint image and in initializing the timecode.
Although Arai discloses substantial features of the claimed invention, Arai lacks the details for, but Tanaka discloses: in a case where the virtual viewpoint image is being played back, the virtual viewpoint information is controlled based on an operation accepted from a user (for example, Tanaka discloses From the time T1 to the time T2 (e.g., during playback of the virtual viewpoint image), the still image virtual-viewpoint content generation unit 119 generates still image virtual-viewpoint content 45 (e.g., the virtual viewpoint information) and outputs it from the image output unit 125. For example, when the user makes an instruction for a generation of still image virtual-viewpoint content by a predetermined operation at the time T1, a notification of this instruction is made to the image control unit 115 from the video generation control unit 118. The image control unit 115, after receiving this instruction notification, causes the generation of the still image virtual-viewpoint content by the still image virtual-viewpoint content generation unit 119 to start, and causes the generation of the live image by the live image content generation unit 121 to stop. See col. 9 lines 41-54 of Tanaka), and,
in a case where the timecode of the virtual viewpoint image is initialized, the virtual viewpoint information is controlled without being based on the operation accepted from the user (for example, Tanaka discloses: From the time T0 to the time T1, the live image content generation unit 121 generates the live image content 44, and the live audio content generation unit 123 generates the live audio content 40 and the live audio content 41. These contents are multiplexed in the multiplexing unit 124, and the result is outputted from the image output unit 125 as video content. The time stamps of the image data used to create the live image content 44 are tn0 to tn1. Between T0 and T1, the difference between the time stamp and the reproduction time of the image data used for generating live image content remains as the difference between the time of playback initiation and the time stamp (Δt=T0−tn0). For this reason, in the automatic-time-stamp-determination mode, it is determined that the audio data will not be reused. Also, the synchronization control unit 116 makes a notification to the audio control unit 117 that live image content is to be generated. See col. 9 lines 21-37 of Tanaka).
Accordingly, the ordinary skilled in the art, before the effective filing date of the claimed invention would have found it obvious to modify play back system as taught by Arai to incorporate: controlling the virtual viewpoint image and the virtual viewpoint information in the same conventional manner as taught by Tanaka such that, in a case where the virtual viewpoint image is being played back, the virtual viewpoint information is controlled based on an operation accepted from a user, and, in a case where the timecode of the virtual viewpoint image is initialized, the virtual viewpoint information is controlled without being based on the operation accepted from the user. Such a modification would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. Combining the teachings of Arai with those of Tanaka is beneficial in that it is possible to determine input audio for generating virtual surround sound in accordance with virtual-viewpoint content, and provide the user with an improved sense of presence by rotating the audio similarly to the virtual viewpoint. Also, by providing a plurality of reference coordinates, it is possible to leave room for the user to select among a variety of senses of presence. See col. 16 lines 7-15 of Tanaka.
As per claim 2, Arai, as modified by Tanaka, discloses: in a case where the timecode of the virtual viewpoint image is to be initialized, pause the playback of the virtual viewpoint image (e.g., causing the system to initiate a “stop” mode to move the virtual viewpoint during a set playback time period in a state of stopping time of the three-dimensional model between the key frames detected as the precipitous virtual viewpoint path, and to release the stop of the time of the three-dimensional model after moving the virtual viewpoint; see paras. 56-57) and display an indication that the playback of the virtual viewpoint image is in a paused state (e.g., the virtual viewpoint variation measuring unit 204 notifies the user of detection of the precipitous section in the virtual viewpoint path by displaying a specific icon or text image. See para. 53). See also col. 9 lines 41-54 of Tanaka, and the rejection of claim 1 above for reasons of obviousness.
As per claim 3, Arai, as modified by Tanaka, discloses: in a case where the timecode of the virtual viewpoint image is to be initialized, set predetermined values to a position and orientation of a virtual point of view included in the virtual viewpoint information (e.g., the playback speed is set substantially equal to a moving speed of the virtual viewpoint between precedent and subsequent frames or that the playback speed is set to an arbitrary moving speed within a range not exceeding a threshold set in order to detect a precipitous virtual viewpoint path. See para. 61).
As per claim 4, Arai, as modified by Tanaka, discloses: accept an operation made by a user (see para 33);
in a case where the virtual viewpoint image is being played back, control the virtual viewpoint information based on the accepted operation (see paras. 33-34, 37-39 and 45-47); and
in a case where the timecode of the virtual viewpoint image is to be initialized, control the virtual viewpoint information without being based on the accepted operation (see paras. 52-58, wherein causing the system to automatically switch the image generation methods from a “slow” mode to a “stop” mode based on a time interval between the key frames, a distance between the positions of the virtual viewpoints, and the like (see para. 58), the Arai reference obviously encompasses controlling the virtual viewpoint information without being based on the accepted operation, as claimed).
As per claim 5, Arai, as modified by Tanaka, discloses: in a case where the virtual viewpoint image is to be repeatedly played back, control the virtual viewpoint information differently in repeating playing back the virtual viewpoint image and in initializing the timecode of the virtual viewpoint image (e.g., Arai discloses to create the virtual viewpoint path, the virtual viewpoint operating unit 201 obtains the time code, the position and the orientation information on the virtual viewpoint, and the like to be designated as the key frame information based on a user operation and registers these pieces of information with the key frame setting unit 202. The above-mentioned operation to register the key frame information is repeated at least twice, and the obtained information is stored in the key frame storing unit 203. Then, the virtual viewpoint variation measuring unit 204 detects a precipitous section in the virtual viewpoint path based on the key frame information stored in the key frame storing unit 203. See paras. 49-51).
As per claim 6, Arai, as modified by Tanaka, discloses: in a case where the virtual viewpoint image is to be repeatedly played back, initialize the timecode of the virtual viewpoint image in response to a fact that the virtual viewpoint image is repeatedly played back a predetermined number of times. See paras. 49-51.
As per claim 7, Arai, as modified by Tanaka, discloses: in a case where the virtual viewpoint image is to be repeatedly played back, initialize the timecode of the virtual viewpoint image in response to a fact that the virtual viewpoint image is played back for a predetermined time. See paras. 49-51.
The invention of claim 8 contains features that correspond in scope with the limitations recited claim 1. As the limitations of claim 1 were found obvious over the teachings of Arai, it is readily apparent that the applied prior arts perform the underlying elements. As such, the limitations of claim 8 are, therefore, subject to rejections under the same rationale as claim 1.
The subject-matter of independent claim 9 corresponds in terms of a computer readable medium to that of independent method claim 1, and the rationale raised above to reject the later also apply, mutatis mutandis, to the former.
Conclusion
5. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ogasawara (US 12346545) discloses an image generation unit 206 generates the virtual viewpoint image from the three-dimensional model based on the control-processed virtual camera. Specifically, for each point configuring the three-dimensional model, an appropriate pixel value is obtained from the multi-viewpoint image and a coloring process is performed. Also, by arranging a colored three-dimensional model in a three-dimensional virtual space and projecting to the virtual viewpoint and rendering, the virtual viewpoint image is generated. However, the method of generating the virtual viewpoint image is not limited to this, and various methods such as a method for generating the virtual viewpoint image by the projective transformation of the captured image without using a three-dimensional model can be used. Note that the model generation unit 205 and image generation unit 206 may be configured as a device different from the image display apparatus 104 and connected to the image display apparatus 104.
6. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to WESNER SAJOUS whose telephone number is (571) 272-7791. The examiner can normally be reached on M-F 10:00 TO 7:30 (ET).
Examiner interviews are available via telephone and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice or email the Examiner directly at wesner.sajous@uspto.gov.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached on 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WESNER SAJOUS/Primary Examiner, Art Unit 2612
WS
08/26/2026
Prosecution Insights