DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
2. Claims 1, 11, and 20 are amended.
3. Claims 2-10, 12-19 are as previously presented.
Claim Rejections - 35 USC § 103
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1, 3-9, 11, 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Heinen (US 20200312029 A1), hereinafter Heinen, in view of Park (US 10818057 B2), hereinafter Park..
Regarding claim 1, Heinen teaches a method, comprising: obtaining, using at least one image capture device, at least one panoramic image of a live-event production scene (Fig. 29a, paragraph 344, a panoramic camera for shooting and obtaining panoramic images; paragraph 136, wherein camera images may be recorded live or near live for places such as sporting events, which is interpreted as the panoramic image being of a live-event production scene); receiving an indication to add a real-time graphic to the at least one panoramic image before transmission of the at least one panoramic image to an end user, wherein the real-time graphic comprises a graphic that is to appear in a production video (paragraph 171, wherein scenes can be annotated with 2D or 3D objects in real time during a live stream which is interpreted as adding real-time graphics to the image, and wherein the live stream is interpreted as a live production video; Fig. 8e, paragraph 235 wherein a user can be notified of an annotation and view it when it has been added, which is interpreted as the annotated graphics being added before being transmitted to an end user); placing the real-time graphic within the at least one panoramic image during the live event production (Fig. 8e, paragraph 235, wherein an annotation containing text and visuals can be added onto the scene, which is defined as a panoramic object; paragraph 253-254 wherein annotations can be added during a live stream, which is interpreted as placing the real-time graphics during a live event production); and verifying placement of the real-time graphic onto the at least one panoramic image by comparing a location of the real-time graphic within the at least one panoramic image with a location of a physical marker within the live-action production scene corresponding to the at least one panoramic image (paragraph 146, wherein real world objects can act as markers for virtual objects to be placed in a 360-degree scene, which is interpreted as real-time graphics having a location corresponding to a location of a physical marker within the scene, and wherein having virtual objects be placed and displayed relative to the markers is interpreted as verifying placement of the real-time graphics by comparing its location to the marker in the panoramic image).
Heinen does not teach generating a broadcast frame comprising the real-time graphic from the at least one panoramic image having the real-time graphic, wherein the generating comprises identifying a region of interest within the at least one panoramic image having the real-time graphic and converting the region of interest to the broadcast frame.
Park teaches generating a broadcast frame (Col. 21, lines 3-13, wherein the terminal electronic device outputting VR content based off of a received image, defined as a panoramic image, is interpreted as generating an output image frame; Fig. 12, Col. 22 lines 34-55, wherein the electronic device can be operated in a network environment suggests that the output image frame can be broadcast image frame sent through the network) comprising the real-time graphic from the at least one panoramic image having the real-time graphic (Fig. 6, Col. 15 line 52 – Col. 16 line 12, wherein the panoramic image can have patch images added to it; Col. 5 lines 55-64, wherein the image to be edited can be captured in real time, Fig. 9 lines 4-25, wherein the patch to place onto the captured image is determined via user input, where the patch graphic can be changed or adjusted at will is interpreted as having a user edit the image by placing graphics on the image, wherein the graphics can be modified and placed in real time; Col. 21, lines 16-20, wherein the terminal electronic device both outputs the virtual VR content based on the provided image and provides a way to apply the patch image to the content, which suggests that the electronic device outputs an image frame based on the panoramic image having a patch image), wherein the generating comprises identifying a region of interest within the at least one panoramic image having the real-time graphic and converting the region of interest to the broadcast frame (Col. 21, lines 3-29, wherein the terminal electronic device may output content based on an image provided to divide and output at least a partial region of the image, where the provided image is defined as a panoramic image, which is interpreted as identifying a region of interest of a panoramic image and converting the region to the broadcast frame).
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Heinen to incorporate the teachings of Park for this method of obtaining and modifying a panoramic image. Both references discuss analogous art in editing panoramic, spherical images by adding graphics to the image. Furthermore, both discuss viewing and editing spherical images in VR environments. Heinen discusses various user interfaces and tools for users to create and modify VR environments and 360-degree images more easily, without requiring technical knowledge, with those techniques being described as fully nonlimiting and capable of being combined or augmented with other features. Furthermore, Park largely discloses a specific editing process for modifying spherical content, namely generating a patch image in order to cover up structure images, such as parts of a camera, tripod, or other device used to capture the image that a user may wish to edit out, providing a method to edit spherical content more easily and intuitively, for the purposes of editing out those structure images. Given that the teachings of Heinen also describe ways to modify and edit spherical images captured by cameras or other image capture devices, and that the teachings of Heinen for editing spherical images are described as nonlimiting, it would be obvious to one of ordinary skill in the art to also incorporate the teachings of Park which also teaches another novel method to edit spherical images, to specifically edit out structure images such as cameras, tripods, or other image capture devices in an intuitive manner. For that reason, it would be obvious to combine these references.
Regarding claim 3, Heinen in view of Park discloses the method of claim 1. Additionally, Heinen teaches the method of claim 1, comprising adjusting the placement of the real-time graphic responsive to the verifying (paragraph 146, wherein placing objects within the scene relative to physical markers via an interaction method is interpreted as adjusting the placement of the real-time objects in response to verifying their placement compared to a physical marker).
Regarding claim 4, Heinen in view of Park discloses the method of claim 1. Additionally, Park teaches the method of claim 1, wherein the physical marker is not included in the at least one panoramic image (Col. 12, lines 25-35, wherein a structure image corresponding to a central point of the spherical image, where a structure image is defined as a physical structure in the image such as a tripod or hand, is interpreted as a physical marker, and the structure being covered up by a patch image suggests it is not included in the panoramic image).
The motivation to combine would be the same as that set forth for claim 1.
Regarding claim 5, Heinen in view of Park discloses the method of claim 1. Additionally, Heinen teaches the method of claim 1, wherein the obtaining comprises obtaining metadata with the at least one panoramic image (paragraph 357, wherein capturing content like 360-degree images may include obtaining additional meta information).
Regarding claim 6, Heinen in view of Park discloses the method of claim 1. Additionally, Heinen teaches the method of claim 1, wherein the obtaining comprises obtaining a plurality of panoramic images and generating, from the plurality of panoramic image, a full three-dimensional image (Fig. 29a, paragraph 344, wherein the camera obtains multiple images which are stitched together to form a 360-degree panoramic image, which is interpreted as a full three-dimensional image).
Regarding claim 7, Heinen in view of Park discloses the method of claim 1. Additionally, Heinen teaches the method of claim 1, wherein the placing the real-time graphic comprises placing the real-time graphic between dynamic entities within the at least one panoramic image (paragraph 311-312, wherein moving objects within a scene can be tracked, which is interpreted as dynamic entities, and suggests that real-time graphics placed within a panoramic scene can be placed between dynamic entities).
Regarding claim 8, Heinen in view of Park discloses the method of claim 1. Additionally, Park teaches the method of claim 1, wherein the adding the real-time graphic comprises adjusting at least one characteristic of the real-time graphic before placement within the at least one panoramic image (Col. 13, lines 8-18, wherein a patch image being applied to spherical content is interpreted as a real-time graphic being added to a panoramic image, and wherein the patch image may be adjusted before being applied to the image).
The motivation to combine would be the same as that set forth for claim 1.
Regarding claim 9, Heinen in view of Park disclose the method of claim 1. Additionally, Heinen teaches the method of claim 1, wherein the real-time graphic is derived from information captured by one or more sensors (Fig. 29c, paragraph 346, wherein a 3D scanner containing sensors is used to scan and generate object models that can be inputted into a panoramic scene; paragraph 146, wherein virtual objects from the 3D scanner can be inserted into the scene).
Regarding claim 11, Heinen teaches a system, comprising: at least one image capture device; a processor operatively coupled to the at least one image capture device; a memory device that stores instructions (Fig. 1A, computing systems including a processor and memory) that, when executed by the processor, cause the information handling device to: obtain, using at least one image capture device, at least one panoramic image of a live-event production scene (Fig. 29a, paragraph 344, a panoramic camera for shooting and obtaining panoramic images; paragraph 136, wherein camera images may be recorded live or near live for places such as sporting events, which is interpreted as the panoramic image being of a live-event production scene); receiving an indication to add a real-time graphic to the at least one panoramic image before transmission of the at least one panoramic image to an end user, wherein the real-time graphic comprises a graphic that is to appear in a production video (paragraph 171, wherein scenes can be annotated with 2D or 3D objects in real time during a live stream which is interpreted as adding real-time graphics to the image, and wherein the live stream is interpreted as a live production video; Fig. 8e, paragraph 235 wherein a user can be notified of an annotation and view it when it has been added, which is interpreted as the annotated graphics being added before being transmitted to an end user); placing the real-time graphic within the at least one panoramic image during the live event production (Fig. 8e, paragraph 235, wherein an annotation containing text and visuals can be added onto the scene, which is defined as a panoramic object; paragraph 253-254 wherein annotations can be added during a live stream, which is interpreted as placing the real-time graphics during a live event production); and verifying placement of the real-time graphic onto the at least one panoramic image by comparing a location of the real-time graphic within the at least one panoramic image with a location of a physical marker within the live-event production scene corresponding to the at least one panoramic image (paragraph 146, wherein real world objects can act as markers for virtual objects to be placed in a 360-degree scene, which is interpreted as real-time graphics having a location corresponding to a location of a physical marker within the scene, and wherein having virtual objects be placed and displayed relative to the markers is interpreted as verifying placement of the real-time graphics by comparing its location to the marker in the panoramic image).
Heinen does not teach generating a broadcast frame comprising the real-time graphic from the at least one panoramic image having the real-time graphic, wherein the generating comprises identifying a region of interest within the at least one panoramic image having the real-time graphic and converting the region of interest to the broadcast frame.
Park teaches generating a broadcast frame (Col. 21, lines 3-13, wherein the terminal electronic device outputting VR content based off of a received image, defined as a panoramic image, is interpreted as generating an output image frame; Fig. 12, Col. 22 lines 34-55, wherein the electronic device can be operated in a network environment suggests that the output image frame can be broadcast image frame sent through the network) comprising the real-time graphic from the at least one panoramic image having the real-time graphic (Fig. 6, Col. 15 line 52 – Col. 16 line 12, wherein the panoramic image can have patch images added to it; Col. 5 lines 55-64, wherein the image to be edited can be captured in real time, Fig. 9 lines 4-25, wherein the patch to place onto the captured image is determined via user input, where the patch graphic can be changed or adjusted at will is interpreted as having a user edit the image by placing graphics on the image, wherein the graphics can be modified and placed in real time; Col. 21, lines 16-20, wherein the terminal electronic device both outputs the virtual VR content based on the provided image and provides a way to apply the patch image to the content, which suggests that the electronic device outputs an image frame based on the panoramic image having a patch image), wherein the generating comprises identifying a region of interest within the at least one panoramic image having the real-time graphic and converting the region of interest to the broadcast frame (Col. 21, lines 3-29, wherein the terminal electronic device may output content based on an image provided to divide and output at least a partial region of the image, where the provided image is defined as a panoramic image, which is interpreted as identifying a region of interest of a panoramic image and converting the region to the broadcast frame).
The motivation to combine would be the same as that set forth for claim 1.
Regarding claim 13, Heinen in view of Park discloses the system of claim 11. Additionally, Heinen teaches the system of claim 11, comprising adjusting the placement of the real-time graphic responsive to the verifying (paragraph 146, wherein placing objects within the scene relative to physical markers via an interaction method is interpreted as adjusting the placement of the real-time objects in response to verifying their placement compared to a physical marker).
Regarding claim 14, Heinen in view of Park discloses the system of claim 11. Additionally, Park teaches the system of claim 11, wherein the physical marker is not included in the at least one panoramic image (Col. 12, lines 25-35, wherein a structure image corresponding to a central point of the spherical image, where a structure image is defined as a physical structure in the image such as a tripod or hand, is interpreted as a physical marker, and the structure being covered up by a patch image suggests it is not included in the panoramic image).
The motivation to combine would be the same as that set forth for claim 1.
Regarding claim 15, Heinen in view of Park discloses the system of claim 11. Additionally, Heinen teaches the system of claim 11, wherein the obtaining comprises obtaining metadata with the at least one panoramic image (paragraph 357, wherein capturing content like 360-degree images may include obtaining additional meta information).
Regarding claim 16, Heinen in view of Park discloses the system of claim 11. Additionally, Heinen teaches the system of claim 11, wherein the obtaining comprises obtaining a plurality of panoramic images and generating, from the plurality of panoramic image, a full three-dimensional image (Fig. 29a, paragraph 344, wherein the camera obtains multiple images which are stitched together to form a 360-degree panoramic image, which is interpreted as a full three-dimensional image).
Regarding claim 17, Heinen in view of Park discloses the system of claim 11. Additionally, Heinen teaches the system of claim 11, wherein the placing the real-time graphic comprises placing the real-time graphic between dynamic entities within the at least one panoramic image (paragraph 311-312, wherein moving objects within a scene can be tracked, which is interpreted as dynamic entities, and suggests that real-time graphics placed within a panoramic scene can be placed between dynamic entities).
Regarding claim 18, Heinen in view of Park discloses the system of claim 11. Additionally, Park teaches the system of claim 11, wherein the adding the real-time graphic comprises adjusting at least one characteristic of the real-time graphic before placement within the at least one panoramic image (Col. 13, lines 8-18, wherein a patch image being applied to spherical content is interpreted as a real-time graphic being added to a panoramic image, and wherein the patch image may be adjusted before being applied to the image).
The motivation to combine would be the same as that set forth for claim 1.
Regarding claim 19, Heinen in view of Park disclose the system of claim 11. Additionally, Heinen teaches the system of claim 11, wherein the real-time graphic is derived from information captured by one or more sensors (Fig. 29c, paragraph 346, wherein a 3D scanner containing sensors is used to scan and generate object models that can be inputted into a panoramic scene; paragraph 146, wherein virtual objects from the 3D scanner can be inserted into the scene).
Regarding claim 20, Heinen teaches a product, comprising: a non-transitory computer-readable storage device that stores executable code that, when executed by a processor, (Fig. 1A, computing systems including a processor and memory that run software programs) causes the product to: obtain, using at least one image capture device, at least one panoramic image of a live-event production scene (Fig. 29a, paragraph 344, a panoramic camera for shooting and obtaining panoramic images; paragraph 136, wherein camera images may be recorded live or near live for places such as sporting events, which is interpreted as the panoramic image being of a live-event production scene); receiving an indication to add a real-time graphic to the at least one panoramic image before transmission of the at least one panoramic image to an end user, wherein the real-time graphic comprises a graphic that is to appear in a production video (paragraph 171, wherein scenes can be annotated with 2D or 3D objects in real time during a live stream which is interpreted as adding real-time graphics to the image, and wherein the live stream is interpreted as a live production video; Fig. 8e, paragraph 235 wherein a user can be notified of an annotation and view it when it has been added, which is interpreted as the annotated graphics being added before being transmitted to an end user); placing the real-time graphic within the at least one panoramic image during the live event production (Fig. 8e, paragraph 235, wherein an annotation containing text and visuals can be added onto the scene, which is defined as a panoramic object; paragraph 253-254 wherein annotations can be added during a live stream, which is interpreted as placing the real-time graphics during a live event production); and verifying placement of the real-time graphic onto the at least one panoramic image by comparing a location of the real-time graphic within the at least one panoramic image with a location of a physical marker within the live-action production scene corresponding to the at least one panoramic image (paragraph 146, wherein real world objects can act as markers for virtual objects to be placed in a 360-degree scene, which is interpreted as real-time graphics having a location corresponding to a location of a physical marker within the scene, and wherein having virtual objects be placed and displayed relative to the markers is interpreted as verifying placement of the real-time graphics by comparing its location to the marker in the panoramic image).
Heinen does not teach generating a broadcast frame comprising the real-time graphic from the at least one panoramic image having the real-time graphic, wherein the generating comprises identifying a region of interest within the at least one panoramic image having the real-time graphic and converting the region of interest to the broadcast frame.
Park teaches generating a broadcast frame (Col. 21, lines 3-13, wherein the terminal electronic device outputting VR content based off of a received image, defined as a panoramic image, is interpreted as generating an output image frame; Fig. 12, Col. 22 lines 34-55, wherein the electronic device can be operated in a network environment suggests that the output image frame can be broadcast image frame sent through the network) comprising the real-time graphic from the at least one panoramic image having the real-time graphic (Fig. 6, Col. 15 line 52 – Col. 16 line 12, wherein the panoramic image can have patch images added to it; Col. 5 lines 55-64, wherein the image to be edited can be captured in real time, Fig. 9 lines 4-25, wherein the patch to place onto the captured image is determined via user input, where the patch graphic can be changed or adjusted at will is interpreted as having a user edit the image by placing graphics on the image, wherein the graphics can be modified and placed in real time; Col. 21, lines 16-20, wherein the terminal electronic device both outputs the virtual VR content based on the provided image and provides a way to apply the patch image to the content, which suggests that the electronic device outputs an image frame based on the panoramic image having a patch image), wherein the generating comprises identifying a region of interest within the at least one panoramic image having the real-time graphic and converting the region of interest to the broadcast frame (Col. 21, lines 3-29, wherein the terminal electronic device may output content based on an image provided to divide and output at least a partial region of the image, where the provided image is defined as a panoramic image, which is interpreted as identifying a region of interest of a panoramic image and converting the region to the broadcast frame).
The motivation to combine would be the same as that set forth for claim 1.
7. Claims 2, 12 are rejected under 35 U.S.C. 103 as being unpatentable over Heinen in view of Park as applied to claims 1, 11 above, and further in view of Peterson (US 8692849 B2), hereinafter Peterson.
Regarding claim 2, Heinen in view of Park discloses the method of claim 1. Additionally, Park teaches generating a first sphere from the at least one panoramic image (Fig. 6, wherein spherical content viewing image 610 is interpreted as a first sphere; Col. 15 lines 12-20); and generating a second sphere, wherein the generating a second sphere comprises adding the real-time graphic into the second sphere (Fig. 6, Col. 15 line 52 – Col. 16 line 12, wherein the top view image 630 where patches are added onto is interpreted as a second sphere).
Neither Heinen nor Park teaches generating a single model by fusing the first sphere with the second sphere having the real-time graphic.
Peterson teaches generating a single model by fusing a panoramic image with a
second image having the real-time graphic (Fig. 4, Col. 9 lines 34-49, wherein adjustment layers are interpreted as an image having real-time graphic edits, and saving the layers as a single panoramic image file is interpreted as fusing the images together).
It would be obvious to one of ordinary skill before the effective filing date of the claimed invention to have modified Heinen in view of Park to incorporate the teachings of Peterson for this method of generating and editing a spherical, panoramic image. Heinen discusses generating a 3D panoramic scene by using a camera to capture and stitch together images, as well as a method for users to add text and visuals to the scene. Park discusses a method for capturing and editing spherical, 360-degree images by adding image patches to the spherical image, and provides a method of editing spherical content more easily. Peterson, similarly, discusses adding adjustment layers consisting of graphical edits on top of panoramic images, in order to adjust factors such as exposure, color balance, and other various inconsistencies found in the stitching process of creating a panoramic image, in order to create a more seamless image. Seeing as Heinen and Park both discuss generating spherical images by stitching together images captured by various cameras and editing its content, it would be obvious to incorporate the teachings of Peterson, for adding a layer of graphical edits on top of an existing spherical panoramic image, and fusing the two into a single image in order to create a more seamless and better-looking panoramic image.
Regarding claim 12, Heinen in view of Park discloses the system of claim 11. Additionally, Park teaches generating a first sphere from the at least one panoramic image (Fig. 6, wherein spherical content viewing image 610 is interpreted as a first sphere; Col. 15 lines 12-20); and generating a second sphere, wherein the generating a second sphere comprises adding the real-time graphic into the second sphere (Fig. 6, Col. 15 line 52 – Col. 16 line 12, wherein the top view image 630 where patches are added onto is interpreted as a second sphere).
Neither Heinen nor Park teaches generating a single model by fusing the first sphere with the second sphere having the real-time graphic.
Peterson teaches generating a single model by fusing a panoramic image with a
second image having the real-time graphic (Fig. 4, Col. 9 lines 34-49, wherein adjustment layers are interpreted as an image having real-time graphic edits, and saving the layers as a single panoramic image file is interpreted as fusing the images together).
The motivation to combine would be the same as that set forth for claim 2.
8. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Heinen in view of Park as applied to claim 1 above, and further in view of Irie (US 9760974 B2), hereinafter Irie.
Regarding claim 10, Heinen in view of Park discloses the method of claim 1. Additionally, Irie teaches the method of claim 1, wherein excerpted region of interest comprises a two-dimensional rectified region of interest (Fig. 10A-D; Col. 9 line 53 - Col. 10 line 27, wherein a predetermined area is interpreted as an excerpted region of interest, and a projection view of the predetermined area T represented by (x, y) coordinates is interpreted as a two-dimensional rectified region).
It would be obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Heinen in view of Park to incorporate the teachings of Irie for this method of finding a two-dimensional rectified region of interest of a three-dimensional spherical image. Both Heinen and Park both disclose methods of capturing and stitching images to create panoramic images, as well as methods for editing that image. Similarly, Irie also discloses a method of capturing a panoramic image, as well as discussing displaying a rectified, two-dimensional area of a three-dimensional panoramic image, to allow users to easily view and edit a portion of the panoramic image on a user device. As all three references discuss ways for a user to edit a panoramic image, and Irie discloses a more in-depth way of viewing specific regions of a panoramic image for easier editing capabilities, it would be obvious to combine these references.
Response to Arguments
9. The applicant's arguments filed July 10th, 2026 have been fully considered but they are not persuasive. Applicant argues that there is there is not enough of an adequate motivation to combine the two references of Heinen et al (US 20200312029 A1), hereinafter Heinen, and Park et all (US 10818057 B1), hereinafter Park. Specifically, applicant argues that Park does not adequately teach “generating a broadcast frame”. Additionally, Applicant argues that neither reference teaches the amended claim limitation specifying a “live-event production scene”.
Examiner respectfully disagrees. Examiner replies that in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Additionally, Examiner replies that, during patent examination, the pending claims must be given their broadest reasonable interpretation consistent with the specification. See MPEP § 2111. Also, it is improper to import claim limitations from the specification. See MPEP § 2111.01(II).
Furthermore, in response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007).
In this case, both Heinen and Park both clearly discuss editing panoramic images by placing graphics on the image in real time. Heinen discusses editing 360-degree spherical images by annotating them with 2D and 3D objects, where the editing and viewing of the image can be done in real time (Fig. 8, paragraph 171). As “Spherical and panoramic images or videos can be created using a spherical or panorama camera” (paragraph 146), it is interpreted that spherical images or videos can be created using a panorama camera, and that the spherical images are analogous with panoramic images. Additionally, these features are stated to be nonlimiting, and combinable with other examples within the disclosure, or with other features entirely. Similarly, Park clearly discusses editing spherical content by applying patch images to the content (Fig. 8, Col. 17 lines 41-60), wherein the spherical content is an image captured by cameras configured to capture wide angled and 360-degree images (Fig. 1, Col. 5 line 7-28), which is interpreted as being a panoramic image. As the image to be edited can be captured in real time (Col. 5 lines 55-64), and the patch to place onto the captured image is determined via user input, where the patch graphic can be changed or adjusted at will (Fig. 9, Col. 18 lines 4-25), it is interpreted as having a user edit the image by placing graphics on the image, wherein the graphics can be modified and placed in real time. Furthermore, examiner states that Park discloses generating an output frame, as it discloses outputting virtual VR content based on an image provided from the image acquisition device (Col. 21 lines 3-13). Given that the terminal electronic device is described as both outputting the virtual VR content based on the provided image and provides a way to apply the patch image to the content (Col. 21, lines 16-20), it suggests that the electronic device outputs an image frame based on the panoramic image having a patch image. As the electronic device can be operated in a network environment (Fig. 12, Col. 22 lines 34-55), it suggests that the output image frame can be broadcast image frame sent through the network.
Examiner additionally explains that there is ample reason for someone of ordinary skill in the art to have combined these two references. As stated above, both references clearly teach analogous art in editing panoramic, spherical images by adding graphics to the image. Furthermore, both discuss viewing and editing spherical images in VR environments. Heinen is directed towards letting users design and create AR/VR content more easily, and without requiring “significant knowhow in the software engineering space” (paragraph 137). The techniques disclosed in Heinen discuss various user interfaces and tools for users to create and modify VR environments and 360-degree images more easily, without requiring technical knowledge. It also reiterates that the techniques described are fully nonlimiting, as described in paragraph 169 and paragraph 171, and are capable of being implemented using different sets of 2D or 3D objects, with other input devices such as head-mounted displays, and can be combined or augmented with other features. Furthermore, Park largely discloses a specific editing process for modifying spherical content, namely generating a patch image in order to cover up structure images, such as parts of a camera, tripod, or other device used to capture the image that a user may wish to edit out. As described in the “Technical Problem” section of the embodiment (Col. 1 lines 23-35), the invention discussed in Park aims to provide a method to edit spherical content more easily and intuitively, for the purposes of editing out those structure images. Given that the teachings of Heinen describe ways to modify and edit spherical images captured by cameras or other image capture devices, and that the teachings of Heinen for editing spherical images are described as nonlimiting, it would be obvious to one of ordinary skill in the art to also incorporate the teachings of Park which also teaches a method to edit spherical images, and more specifically to edit out structure images such as cameras, tripods, or other image capture devices in an intuitive manner. For that reason, it would be obvious to combine these references.
Furthermore, while the examiner agrees that Park fails to teach a “live-event production scene” as claimed in the amended claims, the examiner states that Heinen does teach a “live-event production scene”. Heinen explicitly states that a user may be presented with camera images of another place that may be “live or near live conditions” and gives a live baseball game as an example (paragraph 136). As Heinen discusses showing users 360-degree spherical images of those live streamed events, it is interpreted as obtaining and displaying a panoramic image of a live-event production scene. As these live streams can have spherical images annotated with 2D or 3D graphics in real time, wherein those annotations can be consumed by users in real time (paragraph 171), it is interpreted as the real-time graphics being added to the panoramic image comprising a graphic that is to appear in the production video, wherein a live stream is interpreted as a live produced video.
In conclusion, the rejections set forth in the previous Office Action are shown to have been proper, and the claims are rejected above. To the extent that new citations and parenthetical remarks can be considered new grounds of rejection, such new grounds are necessitated by applicant’s amendments to the claim. Therefore, the present office action is made final.
Conclusion
10. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
11. Any inquiry concerning this communication or earlier communications from the examiner should be directed to JORDAN W YICK whose telephone number is (571)272-4063. The examiner can normally be reached M-F 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JORDAN WAN YICK/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights