DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/11/2025 has been entered.
Applicant’s amendment/response filed 11/11/2025 has been entered and made of record. Claims 1, 6, and 17-18 were amended. Claims 1-11 and 16-18 are pending in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-6, 8-11, and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iwakiri (US 2018/0359458) in view of Lininger (US 8339394) and Cappello et al. (US 2020/0035019).
Regarding claim 1, Iwakiri teaches/suggests: An information processing apparatus comprising:
one or more memories storing instructions (Iwakiri Fig. 20: ROM 2002 and RAM 2003); and
one or more processors executing the instructions (Iwakiri Fig. 20: processor 2001) to:
generate first viewpoint information indicating a position of a first virtual viewpoint and a line-of-sight direction from the first virtual viewpoint (Iwakiri [0042] “The image generating apparatus 101 accepts, from the terminal 102, the virtual camera control information as information about the position and/or line-of-sight direction of a viewpoint for generating a virtual viewpoint image”); and
control, in a case where second viewpoint information is acquired, to output the first viewpoint information without the second viewpoint information for generating a virtual viewpoint image (Iwakiri [0078] “If the monitoring virtual viewpoint mode is the camera operation input monitoring mode (NO in step S506), the second virtual-viewpoint information holding unit 414 acquires virtual camera path information for camera operation input monitoring” [0076] “If the monitoring virtual viewpoint mode is the prediction monitoring mode (YES in step S504), the second virtual-viewpoint information holding unit 414 acquires virtual camera path information for prediction monitoring” [The prediction monitoring meets the limitation “without the second viewpoint information.”]),
the second viewpoint information being generated based on a user operation and indicating a position of a second virtual viewpoint and a line-of-sight direction from the second virtual viewpoint (Iwakiri [0043] “The terminal 102 accepts, from the user, information about the position and/or line-of-sight direction of a viewpoint for generating a virtual viewpoint image”).
Iwakiri further teaches/suggests a three-dimensional model of a subject (Iwakiri [0042] “generating a virtual viewpoint image using a three-dimensional model generated based on a plurality of captured images obtained by capturing an object from a plurality of directions”). Iwakiri is silent regarding:
generate, prior to generation of a three-dimensional model of a subject, first viewpoint information without the three-dimensional model;
Lininger, however, teaches/suggests prior to generation of a three-dimensional model (Lininger col. 9 line 66 – col. 10 line 19 “The 3D program space 106 may be geolocated, before or after creating the 3D model 100, to establish the real-world location of the modeled structure”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the 3D model of Iwakiri to be generated after the viewpoint information is generated as taught/suggested by Lininger because the 3D model may be generated either before or after. As such, Iwakiri as modified by Lininger teaches/suggests:
generate, prior to generation of a three-dimensional model of a subject, first viewpoint information without the three-dimensional model (Iwakiri [0042] “The image generating apparatus 101 accepts, from the terminal 102, the virtual camera control information as information about the position and/or line-of-sight direction of a viewpoint for generating a virtual viewpoint image” Lininger col. 9 line 66 – col. 10 line 19 “The 3D program space 106 may be geolocated, before or after creating the 3D model 100, to establish the real-world location of the modeled structure”);
Iwakiri as modified by Lininger does not teach/suggest:
acquire event information;
generate, prior to generation of a three-dimensional model of a subject, first viewpoint information based on the event information without the three-dimensional model;
Cappello, however, teaches/suggests:
acquire event information (Cappello [0076] “the detected events may correspond to e.g. a goal, offside, foul, throw-in, corner, free-kick, etc.”);
generate first viewpoint information based on the event information (Cappello [0079] “the view processor 506 may determine a corresponding virtual camera angle from which that event is to be viewed in the graphical representation of the scene”);
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the predicted viewpoint of Iwakiri as modified by Lininger to be based on the event as taught/suggested by Cappello for viewing. As such, Iwakiri as modified by Lininger and Cappello teaches/suggests:
generate, prior to generation of a three-dimensional model of a subject, first viewpoint information based on the event information without the three-dimensional model (Lininger col. 9 line 66 – col. 10 line 19 “The 3D program space 106 may be geolocated, before or after creating the 3D model 100, to establish the real-world location of the modeled structure” Cappello [0079] “the view processor 506 may determine a corresponding virtual camera angle from which that event is to be viewed in the graphical representation of the scene”);
Regarding claim 2, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 1, wherein the event information is information indicating an action of the subject or an event caused by the subject (Cappello [0080] “the virtual camera angle may be determined based on one or more players that are detected as contributing to a detected event”). The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 3, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 2, wherein the event information is information including an event occurrence position indicating a position where the action of the subject is identified or a position where the event caused by the subject is identified (Cappello [0080] “the virtual camera angle may be determined based on one or more players that are detected as contributing to a detected event”), and wherein the one or more processors further execute the instructions to generate the first viewpoint information indicating the position of the first virtual viewpoint and the line-of-sight direction from the first virtual viewpoint so that the event occurrence position is included in a virtual viewpoint image (Iwakiri [0087] “A predicted monitoring path 701 indicates a path along which a monitoring virtual camera 704 moves from a movement start position 702” [0042] “The image generating apparatus 101 accepts, from the terminal 102, the virtual camera control information as information about the position and/or line-of-sight direction of a viewpoint for generating a virtual viewpoint image” Cappello [0079] “the view processor 506 may determine a corresponding virtual camera angle from which that event is to be viewed in the graphical representation of the scene”). The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 5, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 2, wherein the event information is information including an occurrence time of the action of the subject or the event caused by the subject (Cappello [0080] “the virtual camera angle may be determined based on one or more players that are detected as contributing to a detected event” [0042] “This may correspond to the same football match at a later time”). The occurrence time is an inherent and/or implicit feature of the detected event. The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 6, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 1, wherein the one or more processors further execute the instructions to:
acquire position information about the subject (Cappello [0080] “the virtual camera angle may be determined based on one or more players that are detected as contributing to a detected event”); and
generate, based on the position information about the subject independent of the three-dimensional model, the first viewpoint information indicating the position of the first virtual viewpoint and the line-of-sight direction from the first virtual viewpoint so that the subject is included in a virtual viewpoint image (Lininger col. 9 line 66 – col. 10 line 19 “The 3D program space 106 may be geolocated, before or after creating the 3D model 100, to establish the real-world location of the modeled structure” Cappello [0079]-[0080] “the view processor 506 may determine a corresponding virtual camera angle from which that event is to be viewed in the graphical representation of the scene … the virtual camera angle may be determined based on one or more players that are detected as contributing to a detected event”).
The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 8, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 6, wherein a plurality of images is images of the subject captured from different directions (Iwakiri [0042] “generating a virtual viewpoint image using a three-dimensional model generated based on a plurality of captured images obtained by capturing an object from a plurality of directions”), and wherein the one or more processors further execute the instructions to acquire the position information about the subject by using a stereo matching method based on the plurality of images (Cappello [0047] “The location of the player in the video may be identified using computer vision or machine learning”). The concept and advantages of stereo matching are well known and expected in the art (Official Notice). It would have been obvious for the location of the player in Smith to be identified using stereo matching for depth estimation. The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 9, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 1, wherein the one or more processors further execute the instructions to acquire the event information based on a plurality of images acquired by a plurality of imaging apparatuses installed to generate a virtual viewpoint image (Iwakiri [0033] “The image capturing apparatuses 100 are arranged to capture the entire or partial range of the arena such as a soccer field” Cappello [0076] “the virtual camera angle may be determined based on events that are detected as occurring within the video”). The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 10, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 9, wherein the one or more processors further execute the instructions to acquire the event information by using a learning model that inputs the plurality of images and outputs the event information (Cappello [0077] “The events may be detected, for example, using machine learning. For example, a machine learning model may be trained with video clips of known events … Once trained, the output of the model may be used by the view processor 506 to determine a virtual camera angle from which that event is to be captured”). The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 11, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 1, wherein the one or more processors further execute the instructions to:
acquire sound information (Cappello [0078] “the events may be detected based on e.g. the audio associated with the video”); and
acquire position information about the subject and the event information based on the acquired sound information (Cappello [0078] “using a model that has been trained with speech samples and labels indicating the type of event associated with audio of that nature” [0080] “the virtual camera angle may be determined based on one or more players that are detected as contributing to a detected event”).
The same rationale to combine as set forth in the rejection of claim 1 above is incorporated herein.
Regarding claim 16, Iwakiri as modified by Lininger and Cappello teaches/suggests: The information processing apparatus according to claim 1, wherein the one or more processors further execute the instructions to:
generate the three-dimensional model of the subject based on a plurality of images acquired by a plurality of imaging apparatuses installed to generate a virtual viewpoint image (Iwakiri [0042] “generating a virtual viewpoint image using a three-dimensional model generated based on a plurality of captured images obtained by capturing an object from a plurality of directions” [0033] “The image capturing apparatuses 100 are arranged to capture the entire or partial range of the arena such as a soccer field”); and
generate the virtual viewpoint image based on the generated three-dimensional model of the subject, the position of the first virtual viewpoint, and the line-of-sight direction from the first virtual viewpoint (Iwakiri [0042] “The image generating apparatus 101 accepts, from the terminal 102, the virtual camera control information as information about the position and/or line-of-sight direction of a viewpoint for generating a virtual viewpoint image … generating, as an image, the appearance of a scene from a virtual viewpoint using the three-dimensional shape (model) of the target scene”).
Claim 17 recites limitation(s) similar in scope to those of claim 1, and is rejected for the same reason(s).
Claim 18 recites limitation(s) similar in scope to those of claim 1, and is rejected for the same reason(s). Iwakiri as modified by Lininger and Cappello further teaches/suggests a non-transitory computer-readable storage medium storing a computer program (Iwakiri Fig. 20: ROM 2002 and/or RAM 2003).
Claim(s) 4 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Iwakiri (US 2018/0359458) in view of Lininger (US 8339394) and Cappello et al. (US 2020/0035019) as applied to claim 1 above, and further in view of Tanaka et al. (US 2018/0036642).
Regarding claim 4, Iwakiri as modified by Lininger and Cappello does not teach/suggest: The information processing apparatus according to claim 3, wherein the one or more processors further execute the instructions to generate the first viewpoint information indicating the position of the first virtual viewpoint and the line-of-sight direction from the first virtual viewpoint so that the event occurrence position is at a center of the virtual viewpoint image. Tanaka, however, teaches/suggests at a center of the virtual viewpoint image (Tanaka [0144] “the position and direction of the virtual camera are determined to show the player character at the center of the screen”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the predicted viewpoint of Iwakiri as modified by Lininger and Cappello such that the one or more players contributing to the event are at the center of the screen as taught/suggested by Tanaka for focus.
As such, Iwakiri as modified by Lininger, Cappello, and Tanaka teaches/suggests generate the first viewpoint information indicating the position of the first virtual viewpoint and the line-of-sight direction from the first virtual viewpoint so that the event occurrence position is at a center of the virtual viewpoint image (Cappello [0080] “the virtual camera angle may be determined based on one or more players that are detected as contributing to a detected event” Tanaka [0144] “the position and direction of the virtual camera are determined to show the player character at the center of the screen”).
Regarding claim 7, Iwakiri as modified by Lininger and Cappello does not teach/suggest: The information processing apparatus according to claim 6, wherein the one or more processors further execute the instructions to generate the first viewpoint information indicating the position of the first virtual viewpoint and the line-of-sight direction from the first virtual viewpoint so that the subject is at a center of the virtual viewpoint image. Tanaka, however, teaches/suggests generate the first viewpoint information indicating the position of the first virtual viewpoint and the line-of-sight direction from the first virtual viewpoint so that the subject is at a center of the virtual viewpoint image (Tanaka [0144] “the position and direction of the virtual camera are determined to show the player character at the center of the screen”). The same rationale to combine as set forth in the rejection of claim 4 above is incorporated herein.
Response to Arguments
Applicant's arguments filed 11/11/2025 have been fully considered but they are moot in view of the new ground(s) of rejection set forth in this Office action.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2011/0301760 – 3D model generated before or after decluttering
US 2020/0184710 – 3D model generated after haze removal
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANH-TUAN V NGUYEN whose telephone number is 571-270-7513. The examiner can normally be reached on M-F 9AM-5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached on 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANH-TUAN V NGUYEN/
Primary Examiner, Art Unit 2619