DETAILED ACTION
This Office Action is in response to the amendment filed on March 27, 2026. Claims 1-15 are pending and are examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments made to original claims 1, 8, 10, and 14 have been fully considered.
Response to Argument
Applicant's arguments and amendments received March 27, 2026 have been fully considered.
With regard to 35 U.S.C. § 103, Applicant argues that the cited prior art fails to disclose that each toggleable overlay is associated with a respective unique identifier and that the video stream includes position, size, and unique identifier coordinates. This language corresponds to the newly amended language of claims 1, 8, 10, and 14.
As such, these have been considered but they are directed to newly amended language, which is addressed below. See the rejection below for how newly added references read on the newly amended language as well as the examiner's interpretation of the cited art in view of the presented claim set.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-15 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 2019/0373243 (“Jeong”) in view of U.S. Patent Publication No. 2021/0219013 (“Deshpande”) and U.S. Patent Publication No. 2015/0195571 (“Boyce”).
With respect to claim 8, Jeong discloses the invention substantially as claimed, including:
A system for encoding one or more video streams (see Figs. 1, 3, items 110, 130, S320, ¶¶33-35, 37-43, describing a system for encoding video streams), the system comprising:
one or more processors (see citations with respect to element above and ¶¶31, 110-111, describing that the system may include one or more processors); and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors (see citations with respect to elements above, describing that the system may include one or more non-transitory computer-readable media storing instructions executable by the one or more processors), wherein the instructions, when executed, cause the system to perform operations comprising:
receiving a plurality of image frames (see Figs. 1, 3, items 110 (and output therefrom), “Video signal”, S310 (and output therefrom), ¶¶36-40, 42, 54, describing that the encoder receives video signal input containing multiple images, i.e., a plurality of image frames);
determining one or more overlay image frames, each overlay image frame comprising a plurality of separately toggleable overlays, each toggleable overlay being associated with a respective unique identifier, wherein the overlay image frame has a same size as an image frame of the plurality of image frames (see citations with respect to element above and ¶¶41, 62-72, 74-77, 99, 101, describing that an overlay image is determined including multiple, i.e., a plurality of, overlays which may be turned on or off, i.e., the overlays are separately toggleable, and each of which are associated with overlay-related metadata, including an overlay identifier indicating a unique identifier associated with the overlay, e.g., “overlay_id”, and overlay region information indicating the position and size of the region in which each overlay of the individual overlays is rendered);
for each image frame among at least some of the plurality of image frames:
associating the image frame with an overlay image frame of the one or more overlay image frames (see ¶¶41, 63, 71, 76, describing that the overlay image is rendered on a 360-degree video, i.e., the image frame (of the video) is associated with an overlay image frame, paragraphs 71 and 76 make clear that an image frame is associated with the overlay frame as it indicates that syntax indicates the position of the overlay within the projected or packed picture, i.e., image frame); and
adding metadata to a … image frame, wherein the metadata comprises, for each toggleable overlay of the plurality of toggleable overlays: position data identifying a position of the toggleable overlay in the overlay image frame, size data identifying a size of the toggleable overlay in the overlay image frame, and identification data corresponding to the unique identifier of the toggleable overlay (see citations with respect to element above and ¶¶41, 63-64, 66-80, 99, describing adding metadata to the projected/packed picture, i.e., image frame, and that the metadata includes, for each overlay (which is detailed in the elements above to be toggleable), position data identifying the position of the overlay in the overlay image frame, the size of the overlay in the overlay image frame, and identification data corresponding to the unique identifier of the overlay); and
encoding the one or more video stream, wherein the encoding comprises:
encoding the plurality of image frames into a first plurality of encoded image frames … (see ¶¶43, 54, describing encoding the plurality of image frames into a plurality of encoded image frames); and
encoding the one or more overlay image frames (see ¶¶41, 54, 63, describing that the overlay images may be compressed/encoded and transmitted to the decoder for decoding).
Jeong focuses its description on the overlays and does not explicitly describe the encoding of the captured image frames/video using a GOP structure, nor does it disclose where in the image, the metadata is included, i.e., it does not disclose encoding the plurality of image frames into a first plurality of encoded image frames using a first group of picture (GOP) structure and adding metadata to a header of image frame.
However, in the same field of endeavor, Deshpande discloses that it was known to encode the image frames using a GOP structure, i.e.:
encoding the plurality of image frames into a first plurality of encoded image frames using a first group of picture (GOP) structure (see Fig. 2A, ¶¶3, 23, describing that it was known to code, i.e., encode, image frames using GOP structure);
As detailed above, Jeong focuses its description on overlays and does not explicitly describe the encoding of the captured image frames/video associated with the overlays. At the time of filing, one of ordinary skill would have been familiar with coding of video image and associated overlays and their need for compression/encoding in order to limit bandwidth usage and minimize delay. Accordingly, such a person would have understood that, as evidenced by Deshpande, coding of the primary images/video using GoPs would have been beneficial. Accordingly, one of ordinary skill in the art at the time of filing would have been motivated to include coding of images using GoPs, as taught by Deshpande, in the coding system of Jeong in order to obtain these advantages. Moreover, to such a person, doing so would have represented nothing more than the combination of prior art elements according to predictable results and/or the simple substitution of one known element for another to obtain predictable results.
Therefore, it would have been obvious to one having ordinary skill in the art at the time of filing to include a mechanism for coding/decoding images using GoPs in the image header in the coding system of Jeong as taught by Deshpande.
However, in the same field of endeavor, Boyce discloses that it was known to add metadata to the header of an image frame, i.e.:
adding metadata to a header of image frame… (see ¶¶49-50, describing that it was known for overlay control information, i.e., metadata, to be included in a picture header).
As detailed above, Jeong focuses its description on overlays and does not explicitly describe where in the image the overlay metadata is located. At the time of filing, one of ordinary skill in the art at the time of filing would have understood the potential locations for metadata used to associate such associated overlays with the primary images/video, including, as evidenced by Boyce, in the image header. Accordingly, to such a person, including the overlay metadata in the image header would have represented nothing more than the combination of prior art elements according to predictable results and/or the simple substitution of one known element for another to obtain predictable results.
Therefore, it would have been obvious to one having ordinary skill in the art at the time of filing to include a mechanism for including overlay metadata in the image header in the coding system of Jeong/Deshpande as taught by Boyce.
With respect to claim 9, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of independent claim 8. Jeong/Deshpande/Boyce additionally discloses:
further implemented on a non-transitory computer-readable storage medium having stored thereon instructions for implementing when executed on one or more devices having processing capabilities (see Jeong ¶¶31, 110-111, describing that the system may be implemented on a non-transitory computer-readable storage medium having instructions stored thereon for implementing the described method on a processor).
The reasons for combining the cited prior art with respect to claim 8 also apply to claim 9.
With respect to claim 1, claim 1 discloses the elements of claim 8 in method form rather than system form. Accordingly, the disclosure cited with respect to claim 8 also applies to claim 1.
With respect to claim 2, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of independent claim 1. Jeong/Deshpande/Boyce additionally discloses:
wherein the encoded image frames and the encoded overlay frames are included in a first video stream, wherein the method further comprises setting each image frame of the plurality of image frames as a no-display frame;
wherein the encoding comprises:
using a block-based codec supporting skip-blocks;
for each delta-encoded image frame of the first plurality of encoded image frames:
determining an overlay image frame among the one or more overlay image frames;
setting the overlay image frame as a display frame;
delta-frame encoding the overlay image frame into an encoded overlay frame referencing the delta-encoded image frame, wherein each pixel block of the encoded overlay frame not corresponding to any of the plurality of toggleable overlays is set as a skip-block; and
including the encoded overlay frame in the first video stream as auxiliary data associated with the delta-encoded image frame (see citations and arguments with respect to claim 8 above and Boyce ¶¶7, 15-17, 21, 24, describing that that the video and overlay may be coded using prediction, i.e., delta-frame encoding, sent to the decoder in a bitstream separately, such that only the combined overlay and video frames are displayed rather than the primary video frame itself – i.e., setting such frames as no-display, and that the overlay frame may be included as auxiliary data associated with the predicted/coded image data, and Boyce ¶¶24, describing that combining streams may include skipping any portions of the overlay frame that do not correspond to the toggleable overlays).
At the time of filing, one of ordinary skill would have been familiar with the ways to combine frames and overlays for display and have understood that, as evidenced by Boyce, such combination may include the encoding described in Boyce, e.g., combining overlay and primary images for display, using prediction coding, and the skipping of any portions of an overlay that do not include overlay elements. Accordingly, to one of ordinary skill in the art at the time of filing doing so during the combining of Jeong/Deshpande/Boyce’s overlay and video images would have represented nothing more than the combination of prior art elements according to predictable results and/or the simple substitution of one known element for another to obtain predictable results.
Therefore, it would have been obvious to one having ordinary skill in the art at the time of filing to include a mechanism for using prediction coding, combining overlay and primary images for display, and skipping any portions of the overlay that do not include overlay elements in the combination of images/overlays of Jeong/Deshpande/Boyce as taught by Boyce.
With respect to claim 3, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of independent claim 1. Jeong/Deshpande/Boyce additionally discloses:
wherein the first plurality of encoded image frames are included in a first video stream and the encoded one or more overlay frames are included in a second video stream, wherein the step of associating the image frame with an overlay image frame of the one or more overlay image frames comprises:
including synchronization data in the first video stream to associate the image frame with an overlay image frame of the one or more overlay image frames (see citations and arguments with respect to claim 8 above, and Boyce ¶¶30, 37, 39-40, describing that the video and overlay frames may be in separate bitstreams and may be associated by including synchronization data which indicates dependency to associate the overlay image with the primary image).
At the time of filing, one of ordinary skill would have been familiar with the ways to stream overlay and primary data and have understood that, as evidenced by Boyce, such streaming may include the method described in Boyce, e.g., sending them in separate scalable video streams and including synchronization data in a stream to associate the two. Accordingly, to one of ordinary skill in the art at the time of filing doing so during the combining of Jeong/Deshpande/Boyce’s overlay and video images would have represented nothing more than the combination of prior art elements according to predictable results and/or the simple substitution of one known element for another to obtain predictable results.
Therefore, it would have been obvious to one having ordinary skill in the art at the time of filing to include a mechanism for sending primary and overlay images in separate scalable video streams and including synchronization data in a stream to associate the two in the coding system of Jeong/Deshpande/Boyce as taught by Boyce.
With respect to claim 4, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of dependent claim 3. Jeong/Deshpande/Boyce additionally discloses:
wherein the step of encoding one or more overlay image frames comprises using a second GOP structure different from the first GOP structure (see citations and arguments with respect to claims 8 and 3 above, describing that the encoding of the overlay images may be in a separate stream from the encoding of primary pictures, i.e., they may be encoded using a second GOP structure different from the first GOP structure).
The reasons for combining the cited prior art with respect to claims 8 and 2-3 also apply to claim 4.
With respect to claim 5, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of dependent claim 3. Jeong/Deshpande/Boyce additionally discloses:
further comprising the steps of:
encoding the plurality of image frames into a second plurality of encoded image frames using a third GOP structure;
including the second plurality of encoded image frames in a third video stream; and
including the synchronization data in the third video stream (see citations and arguments with respect to claims 8, 2, and 3 above and Boyce ¶¶3-4, 15, 28, 39-41, 50, describing that it was known to encode the image frames into separate layers with separate streams, i.e., an additional layer may be a second plurality of frames using a third GOP structure in a third video stream, and that overlay information may be associated with these layers, i.e., synchronization data is included in the third stream).
The reasons for combining the cited prior art with respect to claims 8 and 2-3 also apply to claim 5.
With respect to claim 6, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of dependent claim 5. Jeong/Deshpande/Boyce additionally discloses:
wherein the encoding of the plurality of image frames into the first plurality of encoded image frames differs from the encoding of the plurality of image frames into the second plurality of encoded image frames in at least one of:
encoding quality, frame rate, GOP structure, codec, or resolution (see citations and arguments with respect to claims 8 and 2-4 above and Boyce ¶¶30, describing that the primary image frames and overlay/auxiliary image frames may be separately encoded with different frame rates, different GOP structures, etc.).
The reasons for combining the cited prior art with respect to claims 8 and 2-3 also apply to claim 6.
With respect to claim 7, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of dependent claim 6. Jeong/Deshpande/Boyce additionally discloses:
wherein the step of encoding one or more overlay image frames comprises using a scalable video coding, SVC, codec (see citations and arguments with respect to claims 8 and 2-4 above and Boyce ¶¶2-3, 7, 16-17, describing that the video and overlay image frames may be coded using a scalable video coding codec).
The reasons for combining the cited prior art with respect to claims 8 and 2-3 also apply to claim 7.
With respect to claim 14, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of independent claim 8. Jeong/Deshpande/Boyce additionally discloses:
A system decoding a video stream (see citations and arguments with respect to preamble of claim 8 above), the system comprising:
one or more processors (see citations and arguments with respect to corresponding element of claim 8 above); and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors (see citations and arguments with respect to corresponding element of claim 8 above), wherein the instructions, when executed, cause the system to perform operations comprising:
receiving one or more encoded overlay image frames, each encoded overlay image frame comprising a plurality of separately toggleable overlays, each toggleable overlay being associated with a respective unique identifier (see citations and arguments with respect to corresponding element of claim 8 above, describing that the encoder encodes and the decoder receives such image frames);
receiving a plurality of encoded image frames in a first encoded video stream (see citations and arguments with respect to corresponding element of claim 8 above, describing that the encoder encodes and the decoder receives such image frames);
receiving first metadata indicating identifiers of one or more of the plurality of toggleable overlays to be visible in the decoded video stream (see citations and arguments with respect to corresponding element of claim 8 above, describing that the encoder encodes and the decoder receives such metadata);
decoding the plurality of encoded image frames into a first plurality of image frames using a GOP structure (see citations and arguments with respect to corresponding element of claim 8 above including Boyce ¶¶28, 30, describing decoding/reconstructing the encoded image frames into a plurality of frames using a GOP structure);
decoding the one or more encoded overlay image frames into one or more overlay image frames, wherein each overlay image frame has a same size as an image frame of the first plurality of image frames (see citations and arguments with respect to corresponding element of claim 8 above);
upon determining that an image frame of the first plurality of image frames is associated with an overlay frame of the one or more overlay image frames:
extracting second metadata from the header of the image frame, the second metadata comprises, for each toggleable overlay of the plurality of toggleable overlays: position data identifying a position of the toggleable overlay in the overlay frame, size data identifying a size of the toggleable overlay in the overlay frame, and identification data corresponding to an identifier of the toggleable overlay (see citations and arguments with respect to corresponding element of claim 8 above); and
upon the first metadata comprising an identifier of a toggleable overlay of the plurality of overlays, determining an image frame with overlays by including image data from the image frame and image data from overlay frame corresponding to the toggleable overlay using the position data and size data from the second metadata, and including the image frame with overlays in the decoded video stream (see Jeong ¶¶41, 52, 63, describing combining the overlay image and video/picture/image data according to the metadata, which as detailed in claim 8 above, identifies the toggleable overlays, position, and size and including the overlays accordingly in the image/video).
The reasons for combining the cited prior art with respect to claim 8 also apply to claim 14.
With respect to claim 15, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of independent claim 14. Jeong/Deshpande/Boyce additionally discloses:
further implemented on a non-transitory computer-readable storage medium having stored thereon instructions for implementing when executed on one or more devices having processing capabilities (see citations and arguments with respect to corresponding element of claim 9 above).
The reasons for combining the cited prior art with respect to claim 8 also apply to claim 15.
With respect to claim 10, claim 10 discloses the elements of claim 14 in method form rather than system form. Accordingly, the disclosure with respect to claim 14 also applies to claim 10.
With respect to claim 11, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of dependent claim 10. Jeong/Deshpande/Boyce additionally discloses:
wherein the step of receiving one or more encoded overlay image frames comprises:
receiving a plurality of delta-encoded overlay frames, each delta-encoded overlay frame associated with an encoded image frame of the plurality of encoded image frames, wherein each pixel block of the delta-encoded overlay frames not comprising image data of any of the plurality of toggleable overlays is set as a skip-block;
wherein the step of determining an image frame with overlays comprises:
delta-frame decoding an overlay image frame associated with the image frame using the image frame as reference, wherein the delta-frame decoding comprises, upon the first metadata not comprising an identifier of a toggleable overlay of the plurality of overlays, setting pixel blocks corresponding to the toggleable overlay as skip-blocks using the position data and size data from the second metadata; wherein the image frame with overlays is a display frame; and
wherein each delta-encoded overlay image frame is included as auxiliary data in the first encoded video stream, wherein each image frame of the plurality of image frames is a no-display frame (see citations and arguments with respect to claim 2 above).
The reasons for combining the cited prior art with respect to claims 8 and 2 also apply to claim 11.
With respect to claim 12, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of independent claim 10. Jeong/Deshpande/Boyce additionally discloses:
wherein the one or more encoded overlay image frames are received in a second encoded video stream different from the first encoded video stream, wherein the first encoded video stream comprises synchronization data indicating association of each overlay image frame of the one or more overlay image frames with an image frame of the first plurality of image frames (see citations and arguments with respect to claim 3 above).
The reasons for combining the cited prior art with respect to claim 8 also apply to claim 12.
With respect to claim 13, Jeong discloses the invention substantially as claimed. As described above, Jeong in view of Deshpande and Boyce discloses all the elements of dependent claim 12. Jeong/Deshpande/Boyce additionally discloses:
wherein the step of determining an image frame with overlays comprises:
extracting first image data from overlay frame corresponding to the toggleable overlay;
identifying second image data in the image frame to be replaced by the first image data using the position data and size data from the second metadata; and
determining the image frame with overlays by replacing the second pixel blocks with the first pixel blocks in the image frame (see citations and arguments with respect to claims 8 and 12 above, describing that the system combines the video/image and the overlay image by extracting associated metadata from the overlay frame corresponding to the overlay (which is toggleable as described in claim 8), uses the metadata to determine the location in which the overlay will over lay image data, i.e., second image data is replace by the first image data using the position data and size data from the metadata, and combining the image/video and overlay accordingly, i.e., determining the image frame with overlays by over laying/replacing the second pixel blocks with the first pixel blocks in the image frame).
The reasons for combining the cited prior art with respect to claim 8 also apply to claim 13.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LINDSAY JANE KILE UHL whose telephone number is (571)270-0337. The examiner can normally be reached 8:30 AM-5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached on (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
LINDSAY J UHL
Primary Examiner
Art Unit 2481
/LINDSAY J UHL/Primary Examiner, Art Unit 2481
Prosecution Insights