Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Applicant’s response amends claims Claim 1 and 18, thereby providing claims 1-20 pending.
Response to Arguments
Applicant's arguments filed 15 Jan 2026 have been fully considered but they are not persuasive.
Applicant argues Tsukagoshi does not teach the limitation of instantiating a first number of video decoder instances to be executed by video decoding hardware implemented in circuitry on the basis “Tsukagoshi, demultiplexer 203 demultiplexes a received transport stream (TS) to extract encoded video frames, and then stores the encoded video frames to compressed data buffer (cpb) 204. That is, the video frames as stored in cpb 204 are still encoded, i.e., compressed. Moreover, FIG. 37 depicts decoder 205 (a single decoder), which is clearly separate from demultiplexer 203. Decoder 205 of Tsukagoshi retrieves encoded video data from cpb 204 and decodes the video data.:” (Remarks, 7).
Applicant’s specification does not define the term “decoding,” accordingly, under the broadest reasonable interpretation, decoding includes the process of extracting multiple streams from a single bitstream. Additionally, the term “a first number” includes a single decoder as well as multiple decoders.
Accordingly, Tsukagoshi teaches the claimed limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Tsukagoshi (US 2021/0258589) in view of Potetsianakis US 2024/0056614).
For claims 1 and 18, Tsukagoshi discloses a method and a device ([0245] e.g. device 200) of decoding media data, the method comprising:
instantiating a first number of video decoder instances to be executed by video decoding hardware implemented in circuitry ([0250] The stream processing units 232-2 to 232-N extract the encoded image data of the picture included in the ES payload by performing the same process as the stream processing unit 232-1, and transfer the extracted encoded image data to the stream combining unit 233.);
determining properties of a plurality of video media streams, the properties indicating that each of the plurality of video media streams is available for streaming selection ([0247] The PID processing unit 231 performs filtering based on the packet identifier (PID) according to the decoding capability, and extracts a predetermined number of video streams including at least the base stream);
decoding, by the video decoder instances, the second number of input video media streams to form the second number of decoded video media streams ([0247] For example, the stream processing unit 232-1 processes the base stream, and the stream processing units 232-2 to 232-N process the enhancement stream); and
outputting data of the second number of decoded video media streams ([0250] e.g. transfer the extracted encoded image data to the stream combining unit 233.).
Tsukagoshi does not expressly disclose selecting a second number of input video media streams from the plurality of video media streams according to the determined properties of the second number of input video media streams;
Potetsianakis teaches selecting a second number of input video media streams from the plurality of video media streams according to the determined properties of the second number of input video media streams ([0035] The resulting data is transferred to an elementary stream buffer (EB) 310, which includes respective stream buffers (SB) 312-1-312-N. SB 312-1-312-N may correspond to buffers 202 as described previously. In each SB 312, the encoded streams for each layer are reconstructed and output to reassembly device 205, which may be a multiplexer. Reassembly device 205 then reassembles base layer 110 and any enhancement layers 112 and outputs the combined bitstream . . . .). It would be obvious to a person with ordinary skill in the art to combine Tsukagoshi’s plurality stream teachings with Potetsianakis multiplexing “selecting” teachings for the predictable benefit of providing scalable video streams.
2. Tsukagoshi discloses further comprising receiving configuration data indicating the first number ([0224] The switch unit 144 selectively extracts the PES packets generated by the PES packetizing units 143-1 to 143-N based on a packet identifier (PID)).
3. Tsukagoshi discloses wherein receiving the configuration data comprises receiving the configuration data from a media player application ([0247] The PID processing unit 231 performs filtering based on the packet identifier (PID) according to the decoding capability [0263] In this case, the decoding unit 254 analyzes the VPS and the SPS, detects, for example, the level designation value “sublayer_level_idc” of the bit rate of each sublayer, and checks whether or not the decoding can be performed within the decoding capability.).
4. Tsukagoshi discloses wherein receiving the configuration data comprises receiving one or more supplemental enhancement information (SEI) messages with the input video media streams ([0143] A timing P01 is a timing at which first byte data of the picture “2” serving as the last display picture of the first sequence is input to the cpb 1. At the timing P01, the decoder detects the SEI of the encoded image data of the picture “2,” recognizes that the picture “2” is the last picture (access unit) before switching, and detects a control technique of a subsequent picture (access unit).).
5. Tsukagoshi discloses further comprising: receiving a third number of received video media streams; and formatting the received video media streams to form the second number of input video media streams ([0247] For example, the stream processing unit 232-1 processes the base stream, and the stream processing units 232-2 to 232-N process the enhancement stream).
6. While Tsukagoshi does not, Potetsianakis teaches wherein formatting the received video media stream comprises inserting a first video object from a first received video media stream to a side of a second video object from a second video media stream ([0006] For example, some applications pack different video objects, e.g. different viewpoints of the same scene, cube-map faces of 360 video format, point cloud data and video data, etc. inside the same picture, while these different objects do not necessarily have to be displayed together nor at the same time and, in some cases, these objects are in the end not displayed at all when pruned by the rendering step. One advantage of such packing of different video objects into the same frames is that the different objects are time-locked (synchronised) by design of video elementary streams, i.e. by belonging to a single frame corresponding to a certain decoding timestamp, hence referring to a single point in time of the media timeline. But when these different video objects are packed together in the same picture). It would be obvious to a person with ordinary skill in the art to combine Tsukagoshi’s plurality stream teachings with Potetsianakis teaching for the predictable benefit of providing different objects from different streams available to an application all at the same time as a single output picture after decoding.
7. While Tsukagoshi does not, Potetsianakis teaches wherein formatting the received video media streams comprises inserting a first video object from a first received video media stream above or below a second video object from a second video media stream ([0006] For example, some applications pack different video objects, e.g. different viewpoints of the same scene, cube-map faces of 360 video format, point cloud data and video data, etc. inside the same picture, while these different objects do not necessarily have to be displayed together nor at the same time and, in some cases, these objects are in the end not displayed at all when pruned by the rendering step. One advantage of such packing of different video objects into the same frames is that the different objects are time-locked (synchronised) by design of video elementary streams, i.e. by belonging to a single frame corresponding to a certain decoding timestamp, hence referring to a single point in time of the media timeline. But when these different video objects are packed together in the same picture). It would be obvious to a person with ordinary skill in the art to combine Tsukagoshi’s plurality stream teachings with Potetsianakis teaching for the predictable benefit of providing different objects from different streams available to an application all at the same time as a single output picture after decoding.
8. While Tsukagoshi does not, Potetsianakis teaches wherein formatting the received video media streams comprises appending a first video object from a first received video media stream to a second video object from the first received video media stream ([0006] For example, some applications pack different video objects, e.g. different viewpoints of the same scene, cube-map faces of 360 video format, point cloud data and video data, etc. inside the same picture, while these different objects do not necessarily have to be displayed together nor at the same time and, in some cases, these objects are in the end not displayed at all when pruned by the rendering step. One advantage of such packing of different video objects into the same frames is that the different objects are time-locked (synchronised) by design of video elementary streams, i.e. by belonging to a single frame corresponding to a certain decoding timestamp, hence referring to a single point in time of the media timeline. But when these different video objects are packed together in the same picture). It would be obvious to a person with ordinary skill in the art to combine Tsukagoshi’s plurality stream teachings with Potetsianakis teaching for the predictable benefit of providing different objects from different streams available to an application all at the same time as a single output picture after decoding.
9. While Tsukagoshi does not, Potetsianakis teaches wherein formatting the received video media streams comprises stacking a first video object from a first received video media stream on top of a second video object from the first received video media stream ([0006] For example, some applications pack different video objects, e.g. different viewpoints of the same scene, cube-map faces of 360 video format, point cloud data and video data, etc. inside the same picture, while these different objects do not necessarily have to be displayed together nor at the same time and, in some cases, these objects are in the end not displayed at all when pruned by the rendering step. One advantage of such packing of different video objects into the same frames is that the different objects are time-locked (synchronised) by design of video elementary streams, i.e. by belonging to a single frame corresponding to a certain decoding timestamp, hence referring to a single point in time of the media timeline. But when these different video objects are packed together in the same picture). It would be obvious to a person with ordinary skill in the art to combine Tsukagoshi’s plurality stream teachings with Potetsianakis teaching for the predictable benefit of providing different objects from different streams available to an application all at the same time as a single output picture after decoding.
10. Tsukagoshi discloses further comprising receiving configuration data from a media application indicating how the input video media streams are to be combined to form a single output video media stream ([0247] FIG. 38 illustrates an exemplary configuration of the demultiplexer 203. The demultiplexer 203 includes a PID processing unit 231, N stream processing units 232-1 to 232-N, and a stream combining unit 233).
11. Tsukagoshi discloses further comprising receiving information of a supplemental enhancement information (SEI) message indicating how the input video media streams are to be combined to form a single output video media stream ([0248] The stream processing unit 232-1 includes a section parser 241, a PES packet parser 242, a PES header extracting unit 243, and a PES payload extracting unit 244. The section analyzing unit 241 analyzes section data of a target video stream, acquires presence information of the AU timing control SEI in the encoded image data based on, for example, the temporal control descriptor, and transfers the presence information to the CPU 201.).
12. While Tsukagoshi does not, Potetsianakis teaches wherein outputting the data of the one or more decoded video media streams comprises appending a first decoded picture of a first video media stream above, below, to the left of, or to the right of a second decoded picture of a second video media stream ([0006] For example, some applications pack different video objects, e.g. different viewpoints of the same scene, cube-map faces of 360 video format, point cloud data and video data, etc. inside the same picture, while these different objects do not necessarily have to be displayed together nor at the same time and, in some cases, these objects are in the end not displayed at all when pruned by the rendering step. One advantage of such packing of different video objects into the same frames is that the different objects are time-locked (synchronised) by design of video elementary streams, i.e. by belonging to a single frame corresponding to a certain decoding timestamp, hence referring to a single point in time of the media timeline. But when these different video objects are packed together in the same picture). It would be obvious to a person with ordinary skill in the art to combine Tsukagoshi’s plurality stream teachings with Potetsianakis teaching for the predictable benefit of providing different objects from different streams available to an application all at the same time as a single output picture after decoding..
13. Tsukagoshi discloses wherein the properties of the plurality of the video media streams comprise profile, tier, and level requirements and hypothetical reference decoder (HRD) requirements ([0145] Further, it is recognized that a substream of an upper layer including an access unit (AU) having a display timing later than a display timing of a current access unit (AU) is newly added to the cpb (if a current frame rate is indicated by P, a change in the frame rate from P to N: P<N) or that a substream of an upper layer including an access unit (AU) having a display timing later than a display timing of a current access unit (AU) is not newly input to the cpb (if a current frame rate is indicated by N, a change in the frame rate from N to P: P<N), and a parameter of a subsequent access unit (AU) is checked. [0141] FIG. 12 illustrates an example of Hypothetical Reference Decoder (HRD) control of the encoder 102 in the sequence switching portion illustrated in FIG. 11. In the following description, the base stream is assumed to be a substream 1 (Encoding of Substream 1), and the enhancement stream is assumed to be a substream 2 (Encoding of Substream 2).).
14. Tsukagoshi discloses wherein each of the first number of video decoder instances includes a respective coded picture buffer (CPB) and a respective decoded picture buffer (DPB) ([0151] In the example of FIG. 12, the substream 1 is decoded in the order of the pictures “0,” “2,” “4,” “6,” “8,” “10,” “12,” . . . , and the substream 2 is decoded in the order of the pictures “15,” “17,” “19,” . . . . In other words, only the pictures of the substream 1 are decoded in the first sequence, and the pictures of the substream 1 and the pictures of the substream 2 are alternately decoded in the second sequence. The decoded image data of each picture is input to an uncompressed data buffer (a decoded picture buffer (dpb)) and read and output from the dpb at a timing of a “display image 1” illustrated in FIG. 12.).
15. Tsukagoshi discloses wherein each of the first number of video decoder instances includes a respective coded picture buffer (CPB), and wherein the first number of video decoder instances shares a common decoded picture buffer (DPB).
16. Tsukagoshi discloses wherein the first number of video decoder instances includes a shared common coded picture buffer (CPB) and a shared common decoded picture buffer (DPB) ([0192] The HRD setting unit 123 is supplied with “cpb_removal_delay” and “dpb_output_delay” of the picture of each video stream calculated by the buffer delay control unit 122 and supplied with information of the number of streams from the CPU 101. The HRD setting unit 123 performs a HRD setting based on the information.).
17. Tsukagoshi discloses wherein selecting the second number of input video media streams comprises: determining decoding capabilities of the first number of video decoder instances; determining rendering capabilities for rendering the second number of decoded video media streams; and selecting the second number of the input video media streams that can be decoded according to the decoding capabilities and rendered according to the rendering capabilities ([0247] FIG. 38 illustrates an exemplary configuration of the demultiplexer 203. The demultiplexer 203 includes a PID processing unit 231, N stream processing units 232-1 to 232-N, and a stream combining unit 233. The PID processing unit 231 performs filtering based on the packet identifier (PID) according to the decoding capability, and extracts a predetermined number of video streams including at least the base stream. The video streams extracted by the PID processing unit 231 are transferred to the corresponding stream processing units. For example, the stream processing unit 232-1 processes the base stream, and the stream processing units 232-2 to 232-N process the enhancement stream.).
19. Tsukagoshi discloses wherein the memory comprises a common decoded picture buffer (DPB) to store decoded pictures for each of the video decoder instances ([0151] The decoded image data of each picture is input to an uncompressed data buffer (a decoded picture buffer (dpb)) and read and output from the dpb at a timing of a “display image 1” illustrated in FIG. 12.).
20. Tsukagoshi discloses, wherein the memory comprises a common coded picture buffer (CPB) to store encoded pictures for each of the video decoder instances ([0156] A stair-like solid line a11 indicates a transition of a data amount of the substream 1 generated by encoding, and each step corresponds to one picture unit. The height of the step indicates a data amount generated by encoding. A stair-like solid line b11 indicates a transition of a data amount in the cpb 1 (compressed data buffer) consumed by decoding . . . ).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
DESHPANDE; SACHIN G. US 20220109865 A1 SYSTEMS AND METHODS FOR SIGNALING PICTURE BUFFER INFORMATION FOR INTRA RANDOM ACCESS POINT PICTURE SUB-BITSTREAMS IN VIDEO CODING
Wang; Ye-kui US 20230085717 A1 SIGNALING OF CODED PICTURE BUFFER INFORMATION IN VIDEO BITSTREAMS
Stockhammer; Thomas et al. US 20240373049 A1 VIDEO DECODING ENGINE FOR PARALLEL DECODING OF MULTIPLE INPUT VIDEO STREAMS
DESHPANDE; SACHIN G. US 20220385918 A1 SYSTEMS AND METHODS FOR SIGNALING BUFFERING PERIOD INFORMATION IN VIDEO CODING
NISHI; Takahiro et al. US 20230108110 A1 ENCODER, DECODER, ENCODING METHOD, AND DECODING METHOD
TSUKAGOSHI; Ikuo US 20160301940 A1 CODING APPARATUS, CODING METHOD, TRANSMISSION APPARATUS, AND RECEPTION APPARATUS
Wang; Ye-Kui US 20220217359 A1 Sequence-level HRD Parameters
Wang; Ye-Kui US 20220217393 A1 SEI Message For Single Layer OLS
Narasimhan; Mandayam et al. US 20220007032 A1 INDIVIDUAL TEMPORAL LAYER BUFFER MANAGEMENT IN HEVC TRANSPORT
DRUGEON; Virginie et al. US 20220368929 A1 ENCODER, DECODER, ENCODING METHOD, AND DECODING METHOD
SEREGIN; Vadim et al. US 20210385472 A1 DECODED PICTURE BUFFER (DPB) OPERATIONS AND ACCESS UNIT DELIMITER (AUD)
DRUGEON; Virginie et al. US 20220103843 A1 ENCODER, DECODER, ENCODING METHOD, AND DECODING METHOD
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEIL MIKESKA whose telephone number is (571)272-3917. The examiner can normally be reached M-F: 6a - 2p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached at (571) 272-2988. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NEIL R MIKESKA/Primary Examiner, Art Unit 2485
Prosecution Insights