DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 26th December, 2025 has been entered.
Response to Amendment
This action is in response to the amendment filed on 26th December, 2025. Claims 1, 6, 9, and 18-20 have been amended. Claims 7-8 and 12 have been cancelled. Claims 21-22 have been added. Claims 1-6, 9-11, and 13-22 remain rejected in the application.
Response to Arguments
Applicant's arguments with respect to Claims 1 and 18-20 filed on 26th December, 2025, with respect to the rejection under 35 U.S.C. § 103, regarding that the prior art does not teach the limitation(s): "the representative image is an image decided according to a state of a third subject in the imaging region" has been fully considered, but are moot because of new grounds for rejection. It has now been taught by the combination of Hanamoto-19 and Rafati.
Regarding arguments to Claims 2-6, 9-17, and 21-22, they directly/indirectly depend on independent Claims 1 and 18-20 respectively. Applicant does not argue anything other than independent Claims 1 and 18-20. The limitations in those claims, in conjunction with combination, was previously established as explained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-6, 9-10, 14, 16-22 are rejected under 35 U.S.C. 103 as being unpatentable over Hanamoto et al. (US 20190213791 A1, previously cited), hereinafter referenced as Hanamoto-19, in view of Rafati et al. (US 20180336419 A1), hereinafter referenced as Rafati.
Regarding Claim 1, Hanamoto-19 discloses an image processing apparatus (Hanamoto-19, [0067]: teaches an image processing apparatus 100) comprising:
a processor (Hanamoto-19, [0126]: teaches one or more processors in a computer of the system); and
a memory connected to or built in the processor (Hanamoto-19, [0126]: teaches the system/apparatus including a storage medium with "one or more processors in a computer of the system or the apparatus to read and execute the program"; [0127]: teaches "the storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like"), wherein
the processor is configured to: acquire a representative image corresponding to a virtual viewpoint moving image generated (Hanamoto-19, [0071]: teaches the system generating a virtual viewpoint video image <read on virtual viewpoint moving image>, where information specifies "a virtual viewpoint in the virtual viewpoint video image data, that is, a path along which the gaze point moves (hereinafter, gaze point path), which is the direction in which the virtual camera faces, and a path along which the virtual camera moves (hereinafter, camera path) are specified by a user"; [0073]: teaches arranging "generated thumbnail images <read on representative image> along the camera path drawn on the static 2D map onto which the object 202 is projected is performed" as shown in FIG. 5; Note: it should be noted that in the specification, paragraph [0142] defines a representative image as a thumbnail image) based on
PNG
media_image1.png
639
384
media_image1.png
Greyscale
a plurality of captured images obtained by imaging an imaging region (Hanamoto-19, [0061]: teaches "video image data <read on captured images> captured from multiple viewpoints (here, ten viewpoints corresponding to each of the ten cameras) is acquired"; [0062]: teaches "a two-dimensional image of a still image (hereinafter, called “static 2D map”) that captures an image capturing scene (here, field of the rugby ground) <read on imaging an imaging region> of the acquired multi-viewpoint video image data from a bird's eye is generated"; Note: it should be noted that in the specification, paragraph [0086] provides an example of an imaging region, which is a soccer stadium) and
a plurality of pieces of viewpoint information, based on the plurality of captured images and the plurality of pieces of viewpoint information (Hanamoto-19, [0059]: teaches the virtual camera setting area providing "a display field for displaying information (for example, angle information) <read on pieces of viewpoint information> relating to the image capturing direction of a virtual camera in the virtual camera setting area 320"); and
output data for displaying the representative image on a display in a size different from the virtual viewpoint moving image (Hanamoto-19, [0084]: teaches a selected thumbnail 1002 <read on representative image> size being increased <read on size difference between representative image and virtual viewpoint moving image> by a pinch operation of the user and displayed <read on output data> as shown in FIGS. 10A-10C), wherein
PNG
media_image2.png
560
389
media_image2.png
Greyscale
the representative image is an image [[decided according to a state]] of a third subject in the imaging region (Hanamoto-19, [0070]: teaches the results of "projecting the object 202 of the player holding the ball <read on third subject> onto the static 2D map of the field 201 shown in FIG. 2"; [0056]: teaches "on a field 201 where a game is played, a player and a ball, each as an object 202, exist and ten cameras 203 are arranged so as to surround the field 201"; [0073]: teaches thumbnail arrangement processing, where arranging "the generated thumbnail images <read on representative image> along the camera path drawn on the static 2D map onto which the object 202 is projected is performed" and "in the bird's eye image display area 300, a state where a plurality of thumbnail images is put side by side at regular time intervals along the camera path drawn on the static 2D map is displayed").
PNG
media_image3.png
373
410
media_image3.png
Greyscale
However, Hanamoto-19 does not expressly disclose
the representative image is an image decided according to a state of a third subject in the imaging region.
Rafati discloses
the representative image is an image decided according to a state of a third subject in the imaging region (Rafati, [0076]: teaches a selection operation 332 that "selects a best shot based on the shot relevancy metric <read on state of third subject>," where "the frame relevancy metric for each frame may take into account, for example, one or more of the same, similar, or additional content features as those discussed above in calculating the shot relevancy metric, such as the average texture value of a shot, the average number of “skin-colored” pixels per frame in the shot, the number of faces present the shot, the number of eyes present in the shot, whether the eyes are open or closed, or the size of the faces or other human features visible in each shot" as shown in FIG. 3; [0078]: teaches a selection operation 336 selecting "one or more of the best representative frames from the winning shot(s) <read on representative image being decided>," where "the one or more representative frames selected may be recommended or automatically selected as thumbnail images").
PNG
media_image4.png
390
529
media_image4.png
Greyscale
Rafati is analogous art with respect to Hanamoto-19 because they are from the same field of endeavor, namely generating thumbnails for videos. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a selection operation for candidate thumbnails based on frame comparison relevancy as taught by Rafati into the teaching of Hanamoto-19. The suggestion for doing so would allow the system to categorize relevant frames into subgroups, where a best frame for each viewpoint can be selected automatically or manually, thereby offering the user choice and improving the overall user experience. Therefore, it would have been obvious to combine Rafati with Hanamoto-19.
Regarding Claim 18, it recites the limitations that are identical in scope to Claim 1. As shown in the rejection, the combination of Hanamoto-19 and Rafati disclose the limitations of Claim 1. Thus, Claim 18 is met by the combination of Hanamoto-19 and Rafati according to the mapping presented in the rejection of Claim 1 under the same rationale.
Regarding Claim 19, it recites the limitations that are similar in scope to Claim 1, but in an image processing method. As shown in the rejection, the combination of Hanamoto-19 and Rafati discloses the limitations of Claim 1. Additionally, Hanamoto-19 discloses an image processing method (Hanamoto-19, [0064]: teaches a Multi-stereo method <read on image processing method>, which uses triangulation to generate a virtual viewpoint video image) comprising:…
Thus, Claim 19 is met by Hanamoto-19 according to the mapping presented in the rejection of Claim 1, given the image processing apparatus corresponds to an image processing method.
Regarding Claim 20, it recites the limitations that are similar in scope to Claim 1, but in a non-transitory computer-readable storage medium. As shown in the rejection, the combination of Hanamoto-19 and Rafati discloses the limitations of Claim 1. Additionally, Hanamoto-19 discloses a non-transitory computer-readable storage medium storing a program executable by a computer to perform a process (Hanamoto-19, [0127]: teaches the system/apparatus "that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’)") comprising:…
Thus, Claim 20 is met by Hanamoto-19 according to the mapping presented in the rejection of Claim 1, given the image processing apparatus corresponds to a non-transitory computer-readable storage medium.
Regarding Claim 2, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Additionally, Hanamoto-19 further discloses wherein the representative image is
an image related to a first frame among a plurality of frames including a first subject in the imaging region in the virtual viewpoint moving image (Hanamoto-19, [0062]: teaches "at step 402, a two-dimensional image of a still image (hereinafter, called “static 2D map”) that captures an image capturing scene (here, field of the rugby ground) <read on imaging region> of the acquired multi-viewpoint video image data from a bird's eye is generated"; [0064]: teaches "at step 404, in all the frames <read on first frame among plurality of frames> included in the set target range, the position and the three-dimensional shape (hereinafter, 3D shape) of the object 202 <read on first subject> are estimated," where "information on the estimated position and 3D shape of the object is saved in the storage unit 103 as object information" and "in a case where a plurality of objects exists in the image capturing scene <read on image>, estimation of the position and the 3D shape is performed for each object").
Regarding Claim 3, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 2. Additionally, Hanamoto-19 further discloses wherein the first subject is
a subject decided based on a time included in the virtual viewpoint moving image (Hanamoto-19, [0070]: teaches "the position and the shape of the object 202 <read on subject> make a transition along the time axis, and therefore, all the objects within the time frame <read on included in virtual viewport> set by a user are projected").
Regarding Claim 5, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Additionally, Hanamoto-19 further discloses wherein the processor is configured to
acquire the representative image based on an edition result of the plurality of pieces of viewpoint information (Hanamoto-19, [0080]: teaches camera path adjustment processing, where "a thumbnail image 1001 selected by a user is highlighted by, for example, a thick frame," and "by selecting in advance “Camera” in the dropdown list 326, the height and the moving speed of the virtual camera in the frame of interest, which is located at the position corresponding to the thumbnail image in relation to the selection, are displayed <read on edition result> in the display fields 324 and 325, respectively"; FIGS. 10A-10C teaches the camera path adjustment process; Note: it should be noted that in the specification, paragraph [0192] defines an edition result as being the result of editing a viewpoint path).
Regarding Claim 6, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 5. Additionally, Hanamoto-19 further discloses wherein the plurality of pieces of viewpoint information include
a plurality of viewpoint paths (Hanamoto-19, [0073]: teaches the image processing apparatus 100 displaying a plurality of virtual viewpoint video images based on at least one of the camera path and the gaze point path <read on viewpoint paths> on a display screen), and
the edition result includes a result of edition performed with respect to the plurality of viewpoint paths (Hanamoto-19, [0080]: teaches camera path adjustment processing, where "a thumbnail image 1001 selected by a user is highlighted by, for example, a thick frame," and "by selecting in advance “Camera” in the dropdown list 326, the height and the moving speed of the virtual camera in the frame of interest, which is located at the position corresponding to the thumbnail image in relation to the selection, are displayed <read on edition result> in the display fields 324 and 325, respectively"; [0071]: teaches "information specifying a virtual viewpoint in the virtual viewpoint video image data, that is, a path along which the gaze point moves (hereinafter, gaze point path), which is the direction in which the virtual camera faces, and a path along which the virtual camera moves (hereinafter, camera path) <read on performed with respect to viewpoint paths>").
Regarding Claim 9, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Additionally, Hanamoto-19 further discloses wherein the plurality of pieces of viewpoint information include
a plurality of viewpoint paths (Hanamoto-19, [0073]: teaches the image processing apparatus 100 displaying a plurality of virtual viewpoint video images based on at least one of the camera path and the gaze point path <read on viewpoint paths> on a display screen), and
the processor is configured to acquire the representative image based on a positional relationship among the plurality of viewpoint paths (Hanamoto-19, [0083]: teaches "in accordance with the movement of the selected thumbnail <read on acquired representative image> by the one-finger drug operation, the movement path of the virtual camera is changed"; [0084]: teaches "the height of the virtual camera is changed in accordance with a change in the size of the selected thumbnail by the two-finger pinch operation (the interval is increased or narrowed by two fingers)," where "the size of the selected thumbnail and the height of the virtual camera at the position be interlocked <read on positional relationship> with each other").
Regarding Claim 10, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 9. Additionally, Hanamoto-19 further discloses wherein the positional relationship is
a positional relationship among the plurality of viewpoint paths with respect to a second subject in the imaging region (Hanamoto-19, [0084]: teaches "the height of the virtual camera is changed in accordance with a change in the size of the selected thumbnail by the two-finger pinch operation (the interval is increased or narrowed by two fingers)," where "the size of the selected thumbnail and the height of the virtual camera at the position be interlocked <read on positional relationship> with each other"; [0064]: teaches estimating the position and 3D shape for a plurality of objects <read on viewpoint paths with respect to second subject> in the image capturing scene <read on imaging region>; Note: it should be noted that the positional relationship being based on the second subject is being interpreted as the virtual viewpoint framing a plurality of objects such that the objects are not outside the viewable frame).
Regarding Claim 14, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Additionally, Hanamoto-19 further discloses wherein the representative image is
an image showing a content of the virtual viewpoint moving image (Hanamoto-19, [0079]: teaches generated thumbnail images 801 being arranged "at regular time intervals along the camera path, and as a result of this, a state is brought about where almost all the thumbnail images overlap another thumbnail image" as shown in FIGS. 8A-8C).
PNG
media_image5.png
628
417
media_image5.png
Greyscale
Regarding Claim 16, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Additionally, Hanamoto-19 further discloses wherein the processor is configured to:
output first data for displaying the representative image on a first display (Hanamoto-19, [0084]: teaches a selected thumbnail 1002 <read on representative image> size being increased by a pinch operation of the user and displayed <read on output first data on first display> as shown in FIGS. 10A-10C), and
output second data for displaying the virtual viewpoint moving image corresponding to the representative image on at least one of the first display or a second display according to selection of the representative image displayed on the first display (Hanamoto-19, [0085]: teaches "the moving speed of the virtual camera is changed <read on output second data> in accordance with addition of a predetermined icon to the selected thumbnail <read on selection of representative image> by the two-finger swipe operation" for a viewpoint video <read on virtual viewpoint moving image>).
Regarding Claim 17, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Additionally, Hanamoto-19 further discloses wherein the processor is configured to
store the representative image and the virtual viewpoint moving image in a state of being associated with each other in the memory (Hanamoto-19, [0085]: teaches "a relationship <read on association> between each thumbnail image <read on representative image>, the moving speed of the virtual camera, and the reproduction time of the virtual viewpoint video image and the upper portion indicates the state before the moving speed is changed and the lower portion indicates the state after the moving speed is changed" as shown in FIG. 11B).
PNG
media_image6.png
274
354
media_image6.png
Greyscale
Regarding Claim 21, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Hanamoto-19 does not expressly disclose the limitations of Claim 21; however, Rafati discloses wherein the processor is configured to
acquire the representative image based on a difference degree among the plurality of pieces of viewpoint information (Rafati, [0076]: teaches a selection operation 332 that "selects a best shot <read on representative image> based on the shot relevancy metric <read on difference degree>," where "the frame relevancy metric for each frame may take into account, for example, one or more of the same, similar, or additional content features as those discussed above in calculating the shot relevancy metric, such as the average texture value of a shot, the average number of “skin-colored” pixels per frame in the shot, the number of faces present the shot, the number of eyes present in the shot, whether the eyes are open or closed, or the size of the faces or other human features visible in each shot" as shown in FIG. 3).
Rafati is analogous art with respect to Hanamoto-19 because they are from the same field of endeavor, namely generating thumbnails for videos. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a selection operation for candidate thumbnails based on frame comparison relevancy as taught by Rafati into the teaching of Hanamoto-19. The suggestion for doing so would allow the system to categorize relevant frames into subgroups, where a best frame for each viewpoint can be selected automatically or manually, thereby offering the user choice and improving the overall user experience. Therefore, it would have been obvious to combine Rafati with Hanamoto-19.
Regarding Claim 22, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. Additionally, Hanamoto-19 further discloses wherein the plurality of pieces of viewpoint information include
a plurality of viewpoint paths (Hanamoto-19, [0073]: teaches the image processing apparatus 100 displaying a plurality of virtual viewpoint video images based on at least one of the camera path and the gaze point path <read on viewpoint paths> on a display screen), and
[[a difference degree is a difference degree among the plurality of viewpoint paths.]]
However, Hanamoto-19 does not expressly disclose
a difference degree is a difference degree among the plurality of viewpoint paths.
Rafati discloses
a difference degree is a difference degree among the plurality of viewpoint paths (Rafati, [0076]: teaches a selection operation 332 that "selects a best shot based on the shot relevancy metric <read on difference degree>," where "the frame relevancy metric for each frame may take into account, for example, one or more of the same, similar, or additional content features as those discussed above in calculating the shot relevancy metric, such as the average texture value of a shot, the average number of “skin-colored” pixels per frame in the shot, the number of faces present the shot, the number of eyes present in the shot, whether the eyes are open or closed, or the size of the faces or other human features visible in each shot" as shown in FIG. 3; [0064]: teaches comparing two frames with very different imagery <read on viewpoint paths>, where "the correlation value falls below a predetermined threshold value and the frames are partitioned into two different shots, 308 and 310 respectively"; Note: it should be noted that it is being interpreted that the calculations for the frame relevancy metric being done for each frame is being interpreted as a calculated difference degree for each viewpoint).
Rafati is analogous art with respect to Hanamoto-19 because they are from the same field of endeavor, namely generating thumbnails for videos. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a selection operation for candidate thumbnails based on frame comparison relevancy as taught by Rafati into the teaching of Hanamoto-19. The suggestion for doing so would allow the system to categorize relevant frames into subgroups, where a best frame for each viewpoint can be selected automatically or manually, thereby offering the user choice and improving the overall user experience. Therefore, it would have been obvious to combine Rafati with Hanamoto-19.
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Hanamoto et al. (US 20190213791 A1, previously cited), hereinafter referenced as Hanamoto-19, in view of Rafati et al. (US 20180336419 A1), hereinafter referenced as Rafati as applied to Claims 2 and 1 above respectively, and further in view of Umemura (US 20200169713 A1, previously cited).
Regarding Claim 4, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 2. The combination of Hanamoto-19 and Rafati does not expressly disclose the limitations of Claim 4; however, Umemura discloses wherein the first frame is
a frame decided based on a size of the first subject in the virtual viewpoint moving image (Umemura, [0080]: teaches an image generation unit 120 arranging "foreground models generated based on the data acquired from the storage unit 118 in a virtual space," where "deviations in the positions, orientations, and sizes of the objects <read on first subject> are reduced even in the cases in which the models of the plurality of objects captured at different image capturing locations are arranged in the same virtual space"; [0076]: teaches a virtual viewpoint image <read on virtual viewpoint moving image> being generated by the image generation unit 120, where "the image generation unit 120 may adjust the positions, orientations, and sizes of the objects using adjustment information input based on a user operation via the instruction unit 119"; [0077]: teaches the image processing system 20 storing position information about image capturing target objects, such as position and direction, to adjust display sizes <read on frame based on size> of said objects).
Umemura is analogous art with respect to Hanamoto-19, in view of Rafati because they are from the same field of endeavor, namely generating virtual viewpoints. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to generate and arrange foreground models based on acquired data of a virtual space as taught by Umemura into the teaching of Hanamoto-19, in view of Rafati. The suggestion for doing so would allow for given virtual viewpoints to have proper framing of said virtual objects, thereby resulting in a desired output. Therefore, it would have been obvious to combine Umemura with Hanamoto-19, in view of Rafati.
Regarding Claim 15, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. The combination of Hanamoto-19 and Rafati does not expressly disclose the limitations of Claim 15; however, Umemura discloses wherein the plurality of pieces of viewpoint information include
first viewpoint information and second viewpoint information which have different viewpoints (Umemura, [0069]: teaches generating virtual viewpoint images "containing a plurality of image capturing targets captured at different image capturing locations <read on different viewpoints>"; [0120]: teaches a "user viewing the virtual viewpoint image can compare with ease the plurality of image capturing targets that is different in at least one of the image capturing location and the image capturing time point," where "since the virtual viewpoint image is generated based on the position and direction of the virtual viewpoint determined by a user operation, the user can compare the plurality of image capturing targets from various viewpoints <read on first and second viewpoint information>"), and
the first viewpoint information and the second viewpoint information include information related to different time points (Umemura, [0045]: teaches generating a virtual viewpoint image "that contains images of the plurality of objects captured at the plurality of different image capturing time points <read on different time points> and the background image and corresponds to the virtual viewpoint specified by the viewpoint information").
Umemura is analogous art with respect to Hanamoto-19, in view of Rafati because they are from the same field of endeavor, namely generating virtual viewpoints. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to generate and arrange foreground models based on acquired data of differing views of a 3D object(s) as taught by Umemura into the teaching of Hanamoto-19, in view of Rafati. The suggestion for doing so would allow for given virtual viewpoints to have proper framing of said virtual objects, thereby resulting in a desired output. Therefore, it would have been obvious to combine Umemura with Hanamoto-19, in view of Rafati.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Hanamoto et al. (US 20190213791 A1, previously cited), hereinafter referenced as Hanamoto-19, in view of Rafati et al. (US 20180336419 A1), hereinafter referenced as Rafati as applied to Claim 1 above respectively, and further in view of Ota (US 20190278803 A1, previously cited).
Regarding Claim 11, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. The combination of Hanamoto-19 and Rafati does not expressly disclose the limitations of Claim 11; however, Ota discloses wherein the processor is configured to:
search a plurality of the virtual viewpoint moving images for a search condition conformation virtual viewpoint moving image that conforms to a given search condition (Ota, [0059]: teaches "by taking part of the camera paths of the virtual viewpoint video images A, B, and C as a search target, it is made possible to extract a virtual viewpoint video image whose virtual viewpoint parameter <read on search condition conformation virtual viewpoint moving image> of part of the frames (here, the first or last frame) coincides with or resembles the search condition <read on given search condition>"), and
acquire the representative image based on the search condition conformation virtual viewpoint moving image (Ota, FIG. 2 teaches metadata 201 of the virtual viewpoint video image data, which contains the virtual viewpoint parameter 211 and the video image-attached information 212 <read on acquiring representative image>; [0039]: teaches making use "of another piece of information included in the video image-attached information as a search condition, not limited to the time information" of a virtual viewpoint video image).
Ota is analogous art with respect to Hanamoto-19, in view of Rafati because they are from the same field of endeavor, namely generating virtual viewpoints. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to extract virtual viewpoint video images based on search condition parameters as taught by Ota into the teaching of Hanamoto-19, in view of Rafati. The suggestion for doing so would allow for easier searching for desired viewpoints, such as viewing angles, thereby obtaining a desired result. Therefore, it would have been obvious to combine Ota with Hanamoto-19, in view of Rafati.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Hanamoto et al. (US 20190213791 A1, previously cited), hereinafter referenced as Hanamoto-19, in view of Rafati et al. (US 20180336419 A1), hereinafter referenced as Rafati as applied to Claim 1 above respectively, and further in view of Hanamoto (US 20200366878 A1, previously cited), hereinafter referenced as Hanamoto-20.
Regarding Claim 13, the combination of Hanamoto-19 and Rafati discloses the image processing apparatus of Claim 1. The combination of Hanamoto-19 and Rafati does not expressly disclose the limitations of Claim 13; however, Hanamoto-20 discloses wherein the representative image is
an image decided according to an attribute of a person involved in the virtual viewpoint moving image (Hanamoto-20, [0083]: teaches "object IDs in the left column are IDs for identifying <read on attribute> the objects and codes such as “player
N
” <read on person> and “ball” are assigned to the respective objects"; [0082]: teaches "illustrating the positions of players, a ball, and a virtual camera 701 in a specific frame" as shown in FIG. 7A; [0082]: further teaches "since the positions of objects such as the players and the ball are held in the metadata generated in advance in S309, a distance
L
c
between the virtual camera 701 and each object and a distance
L
b
between the ball and each object can be calculated" for a virtual viewpoint image <read on virtual viewpoint moving image>; [0073]: teaches "thumbnail images <read on representative image> and metadata of image data on a match are displayed on the display-operation unit 115 of the user terminal 110," where "the user selects the contents corresponding to a desired scene from the thumbnail images and the metadata displayed on the display-operation unit 115").
Hanamoto-20 is analogous art with respect to Hanamoto-19, in view of Rafati because they are from the same field of endeavor, namely generating virtual viewpoints. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to utilize and assign codes to identified objects as taught by Hanamoto-20 into the teaching of Hanamoto-19, in view of Rafati. The suggestion for doing so would allow for streamlined categorization for future searches, thereby making it easier to find desired virtual viewpoints. Therefore, it would have been obvious to combine Hanamoto-20 with Hanamoto-19, in view of Rafati.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Furukawa (US 20200327724 A1) discloses generating 3D shape data on a moving object on virtual viewpoint images by determining effective and ineffective pixels of a texture image;
Ito (US 20190132529 A1) discloses obtaining a 3D shape model that represents a shape of a captured object;
Kato (US 20200084431 A1) discloses generating a virtual viewpoint image that includes specified viewpoint information;
Matsushita et al. (US 20180048810 A1) discloses generating a virtual viewpoint image where the system determines if occlusion is occurring in at least one of the captured images;
Mizuno (US 20180063514 A1) discloses obtaining virtual viewpoint information that indicates a position and/or a direction of a virtual viewpoint that are not set by a user;
Sakakima (US 20190279418 A1) discloses generating a virtual viewpoint image based on historical information that illustrates a change in a time-axis direction of a virtual viewpoint;
Tanaka (US 20190335154 A1) discloses generating a virtual viewpoint image according to a virtual viewpoint having a moving speed based on the acquired image data; and
Ota (US 20220408069 A1) discloses obtaining viewpoint information that specifies the similarity between virtual viewpoints of a virtual viewpoint image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KARL TRUONG whose telephone number is (703)756-5915. The examiner can normally be reached 10:30 AM - 7:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.D.T./Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614