DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/23/2026 has been entered.
Response to Arguments
Applicant's arguments, Remarks, pg. 10, filed 1/23/2026, with respect to the status of the claims are hereby acknowledged. Claims 1-2, 5-8, 10, 12, 31-32, 34-39, 41-44, and 48-51 remain pending in this application.
Applicant's arguments, Remarks, pg. 10, filed 1/23/2026, with respect to the rejection of the claims on under 35 U.S.C. 103 have been fully considered. The examiner notes that the applicant’s arguments are directed to newly amended limitations. Therefore, the examiner will set forth a new grounds of rejection and rely, part, on the prior art of record and newly found prior art references.
Applicant sets forth three main arguments, in Remarks pg. 10-13, with respect to the newly amended claims as follows:
First, the cited references at least fail to disclose or suggest "wherein each selected boundary point was included in a transcoded stream received by one of the plurality of segmenters" of claim 1…Second, the cited references at least fails to disclose or suggest "a computing device" of claim 1. Specifically, the cited references do not disclose or suggest a computing device that "receiv[es], ... from a plurality of segmenters, a plurality of selected boundary points,""determin[es] ... a sync point,""determin[es] ... a lagging segmenter," and "caus[es] ... the lagging segmenters to send, to the computing device, a second selected boundary point" as claim 1 recites...Third, the cited references at least fail to disclose or suggest "causing ... the lagging segmenter to send, to the computing device, a second selected boundary point for ending the current segment later than the first selected boundary point" of claim 1.
On the issue of obviousness, test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). More importantly, on the issue of obviousness, the Supreme Court stated that when a patent simply arranges old elements with each performing the same function it had been known to perform and yields no more than one would expect from such an arrangement, the combination is obvious. KSR International Co. v. Teleflex Inc., 550 U.S. 398, 417, 82 USPQ2d 1385 (2007) (citing Sakraida v. AG Pro, Inc., 425 U.S. 273, 96 S. Ct. 1532, 47 L. Ed. 2d 784 (1976)). The Court further reiterated that the analysis of a rejection on obviousness grounds need not seek out precise teachings directed to the specific subject matter of the challenged claim, for a court can take account of the inferences and creative steps that a person of ordinary skill in the art would employ. See id. at 418. The obvious analysis cannot be confined by a formalistic conception of the words teaching, suggestion, and motivation. Id. at 419. Further, the Court stated that common sense teaches, however, that familiar items may have obvious uses beyond their primary purposes, and in many cases a person of ordinary skill will be able to fit the teachings of multiple patents together like pieces of a puzzle. See id. at 420.
The examiner respectfully disagrees with the applicant’s characterization of the teachings of the prior art. For example, the applicant’s arguments in the first argument states, inter alia, that with respect to Van Veldhuisen “…assuming (without conceding) that Van Veldhuisen's "transcoding modules 120" (also referred to as "transcoder 102") could each include its own "packager 121" embedded inside it, as the Office Action alleges, Van Veldhuisen would still fail to disclose or suggest "wherein each selected boundary point was included in a transcoded stream received by one of the plurality of segmenters" because Van Veldhuisen is silent about its "transcoding modules 120" ever receiving a "transcoded stream" that includes "each selected boundary point" as claim 1 recites regarding the "plurality of segmenters….” The applicant’s arguments do not appear to appreciate the significant teaching value of Van Veldhuisen when viewed as a whole. The applicant’s arguments do not appear to take into consideration Van Veldhuisen’s disclosure of the operation of the transcoders. For example, Van Veldhuisen para 35-42 disclose that a transcoder is able to receive a plurality of input streams or a single input streams in order to generate multiple profiles, or stream representations, may be output from a single input stream 101, each profile/stream representation having its own resolution and bit rate parameters. Based on Van Veldhuisen, the streams output from the transcoders can be further manipulated by the described packagers. Van Veldhuisen teaches various embodiments that take into consideration that segmentation can be performed at the transcoder or in a transcoder and separate packager that is also referred to as “a segmenter, a fragmenter, or an encapsulator, receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol…”For example, Van Veldhuisen para 33 teaches “…part of the transcoding process by the disclosed transcoding module 120 includes marking the boundary of a media segment per configuration and passing it as embedded private metadata in an output stream 119, e.g., an MPEG 2 transport stream, to the packager 121. As described in more detail below, the signaling between the transcoder and the packager identifying segment boundaries may be instantaneous decode refresh (IDR) based signaling. The packager 121, also referred to as a segmenter, a fragmenter, or an encapsulator, receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol…” and Van Veldhuisen recognizes a known problem relating to adaptive bit rate streaming (a plurality of transcoded streams with a different bitrate) comprising proposed boundary points for segments associated with each transcoded stream wherein Van Veldhuisen addresses the known problem of maintaining synchronicity of segment boundary points that identify locations where switching between stream representations in the output group with respect to transcoding a content stream into a plurality of streams wherein each of the plurality of streams is transcoded with a different bitrate (Van Veldhuisen para 58-66); Van Veldhuisen teaches that if the transcoder has the embedded packager option, then the output streams within an output group can be further segmented into ABR streams before being output from the transcoder 102 (para 51) such that Van Veldhuisen’s disclosure reads on applicant’s limitations comprising a plurality of segmenters comprising two segmenters. Therefore, the analysis of the claim in relation to Van Veldhuisen identifies a plurality of segmenters and an initial impression of the claim limitations appear to identify at least different time positions for each of the segmenters. In relation to having different bitrate as claimed, as discussed above, Van Veldhuisen further teaches “[t]o support adaptive bit rate streaming, it may be desirable that the transcoder 102 outputs multiple streams within an output group that are scene aligned and GOP/IDR synchronized at the beginning of each segment… Thus, multiple output streams generated from the same input stream, such as multiple bit rate versions of the same input stream, are identified with misaligned timestamps. As a result, output streams generated from the same input but output from two different chassis may not be aligned, as desired, when accessing the same time on a specific frame. Switching between streams, therefore, is not continuous and interruptions in the presentation to a subscriber will occur” (para 62-65) which a person of ordinary skill in the art would have understood that at particular times, the produced segments may need to be synchronized to infer that not all the segments will be scene aligned but some of the plurality of segments for particular streams may not need to be realigned to be synchronized. A person of ordinary skill in the art would have reasonably inferred that Van Veldhuisen’s invention comprises a plurality of segmenters to produce segmented content for each of the plurality of ABR streams at different bit rates wherein Van Veldhuisen Fig. 1 and para 26-33 teaches element 120 disclosed as a plurality of transcoders and packagers referred to a segmenter, a fragmenter, or an encapsulator and Van Veldhuisen further teaches that in order to output streams that are scene aligned, Van Veldhuisen teaches “Embodiments are disclosed that enable each of the transport streams within the same adaptive streaming group to have the same segment and/or fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, the disclosed techniques may be used when chunk boundary points must be synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch.” See Van Veldhuisen para 67. Most importantly, Van Veldhuisen teaches “The disclosed adaptive bit rate transcoder 102 may output transport streams with an IDR at the segment/fragment boundary. Stream transcoding/encoding by the transcoder module 120, while continuous, may then be broken up in to small chunks by a packager, each starting with the IDR. A chunk is a discrete section of content that can be independently decoded. The chunk may include initialization information. A chunk is also referred to herein as a segment or a fragment. See para 58. Van Veldhuisen also teaches “Segment boundary points identify locations where switching between stream representations in the output group maintains synchronicity. A segment may have the max duration and the last fragment just before the max duration is the segment boundary. Thus, a segment may be equal to a fragment or may include a plurality of fragments where a segment boundary point is determined by choosing a fragment boundary within the segment” (See Van Veldhuisen para 60 ) and further teaches “As per the above, when referenced herein, a fragment boundary point or segment boundary point may refer to the same thing. For example, a fragment and a segment, when equal to each other, may include a segment boundary point that is defined by either of the fragment or the segment, i.e., the fragment boundary point is the segment boundary point since the segment=1 fragment. When a segment comprises a plurality of fragments, the segment boundary point may be defined by the boundary point of a particular fragment within the segment. Thus, as used herein, a segment boundary point can be understood in terms of the segment as a whole, i.e., the segment boundary point, or in terms of a fragment within the segment, i.e., a fragment boundary point.” (para 61). All things considered, Van Veldhuisen teaches the limitations for a plurality of transcoded streams, each transcoded with a different bitrate (i.e., adaptive bitrate) and further recites a plurality of proposed boundary points indicating time positions for each of the plurality of transcoded streams.
Applicant further sets forth additional arguments stating “…the Office Action alleges that the combination of the "transcoding modules 120" and the "packager 121" corresponds to the "plurality of segmenters" as recited in claim 1, but the Office Action fails to specify which device, if any, of Van Veldhuisen corresponds to the "computing device" of claim 1. Even assuming (without conceding) that Van Veldhuisen's "transcoder 102" or "transcoding modules 120" could somehow be equated with the "plurality of segmenters" of claim 1, Van Veldhuisen is silent about any additional "computing device" that "receiv[es] ... a plurality of selected boundary points" from Van Veldhuisen's "transcoder 102" or "transcoding modules 120," as claim 1 requires with the "plurality of segmenters." Instead, Van Veldhuisen explains that the segment alignment is performed internally within the "transcoder 120," which would obviate the need for any additional computing device that "receiv[es] ... a plurality of selected boundary points" from the transcoder 120,""determin[es] ... a sync point,""determin[es] ... a lagging segmenter," and "caus[e] ... the lagging segmenters to send, to the computing device, a second selected boundary point," as in claim 1 (Van Veldhuisen, paragraph [0085], "The disclosed transcoder 102 may align GOP/IDR among multiple output streams, making it easier for the packager to segment aligned streams"). The examiner also respectfully disagrees with applicant’s arguments. Based on the combined teachings of the prior art of record, a person of ordinary skill in the art would have reasonably inferred that applicant’s claimed “computing device” corresponds to the devices disclosed in the prior art for delivering adaptive bit rate streaming to receiving devices.
Lastly, applicant argues the following:
Third, the cited references at least fail to disclose or suggest "causing ... the lagging segmenter to send, to the computing device, a second selected boundary point for ending the current segment later than the first selected boundary point" of claim 1. Notably, the Office Action states, "Regarding claim 1, Regarding "a method comprising: ... causing the lagging segmenter to send, to the computing device, a subsequent selected boundary point" (Office Action, pp. 2-3) but fails to identify exactly which of the cited references discloses or suggests "causing . ..the lagging segmenter to send, to the computing device, a ...selected boundary point." In other words, the Office Action's analysis omits any citation regarding this feature of claim 1. Applicant submits that the cited references do not disclose or suggest "causing, by the computing device, the lagging segmenter to send, to the computing device, a second selected boundary point for ending the current segment later than the first selected boundary point" of claim 1.
The examiner respectfully disagrees. The limitation with respect toa lagging segmenter is understood by a person of ordinary skill in the art to a segmenter that is not aligned, by reasons of latency, faults, failures, and/or processing issues as discussed in the prior art. The combination of prior art renders obvious that a plurality of segmenters providing adaptive bitrate streams. The examiner has set forth finds of fact based on the prior art relating to adaptive bitrate streaming comprising a plurality of segmenters that take into consideration alignment issues and is able to compensate for alignment issues by modifying segment lengths. For example, Nemiroff teaches the use of boundary points for synchronizing/proper alignment of video streams (e.g., a sync point corresponds to a selected timing position such that each of the multiple transcoders, for transcoding each of the plurality of content streams, will be able to use the same timing position for a specific boundary point and results in determining a relationship between the sync point and each of the plurality of timing positions). [See also Office Action, pg. 6-7, dated 5/19/2018 Jones, MUKTA KAR ET AL, and Eduardo of record incorporated herein and made of record but not relied upon in order to avoid duplicative references for supporting the understanding of encoder boundary points (EBP)].
Furthermore, Shaffer discloses two or more transcoding nodes for generating ABR streams (para 14-29) and further teaches that each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. See Shaffer paragraphs 22-29 teaching in part:
[0022] Additionally, the synchronization control 16 and 18 further can adjust chunk boundaries in response to detecting a cadence-changing event. Such boundary adjustments may be referred to herein, in some examples, as variable or dynamic boundary placement. For example, a given synchronization control 16 or 18 can be configured to insert an adjusted boundary at a location that is different from expected (e.g., based on synchronization parameters) in response to the cadence-changing event. If an ingested media stream already includes marked chunk boundaries, a closest boundary to the adjusted boundary location that is inserted can be removed also in response to the cadence-changing event. A boundary identifier can be inserted at the determined adjusted boundary location. As an example, for an ABR video stream implemented according to H.264 standard, the boundary identifier can be implemented as an instantaneous decode refresh (IDR) access unit also referred to as an IDR frame.
[0023] As an example, assuming a fixed chunk duration and in the absence of a cadence-changing event, the chunk boundaries fall at deterministic locations in the stream (e.g., at regularly spaced intervals). In response to a cadence-changing event, the synchronization control 16 and 18 can adjust the cadence according to an offset that is calculated. In some examples, the offset can be calculated based on a difference between the originally scheduled, boundary location (assuming the cadence-changing event did not occur) and the location of the adjusted boundary location in response to the cadence-changing event. Thus, the cadence can describe a sequence of scheduled boundary locations including an amount of offset, which scheduled boundary locations can be described relative to an arbitrary initial point in the timeline of the media source (e.g., modulo the nominal chunk duration). The amount of offset can thus be cumulative to adjust the cadence for any number of cadence-changing events.
See also Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time.
Additionally, the examiner cites the teachings of Mutton teaching segmented parallel encoding with frame-aware variable size chunking wherein a particular segmenter, that is not aligned with other segmenters providing adaptive bit rate streaming, is able to change the boundary points to include portions of the next adjacent segment in order to properly align video content in adaptive bitrate streaming (Abstract, para 19 and 234-241).
All things considered, the examiner will set forth a new grounds of rejection based, in part, on the prior art of record in order to address the newly amended limitations.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-2, 5-8, 10, 12 and 31-32, 34-39, 41-44, 48-51 are rejected under 35 U.S.C. 103 as being unpatentable over Van Veldhuisen; Brenda US 20150281752 A1 (hereafter Van Veldhuisen) and in further view of Narayanan; Karthik et al. US 8532171 B1 (hereafter Narayanan) and further in view of Nemiroff; Robert S. et al. US 20150281751 A1 (hereafter Nemiroff) and in further view of SHAFFER; GARY US 20150201012 A1 (hereafter Shaffer) and in further view of Mutton; James A. US 20130114744 A1 (hereafter Mutton) and in further view of Forehand; Joseph et al. US 20170048536 A1 (hereafter Forehand) and in further view of Tweedale; Paul et al. US 20150113100 A1 (hereafter Tweedale) and in further view McCormick; Anne K. et al. US 20140351455 A1 (hereinafter McCormick).
Regarding claim 1, Regarding “a method comprising: receiving, by a computing device from a plurality of segmenters, a plurality of selected boundary points indicating time positions, in a content item, for ending current segments of a plurality of transcoded streams, of the content item, having different bitrates, wherein each selected boundary point was included in a transcoded stream received by one of the plurality of segmenters; determining, by the computing device, a sync point based on a common boundary point of the plurality of selected boundary points; determining, by the computing device, a lagging segmenter that sent a first selected boundary point, of the plurality of selected boundary points, that is earlier, in the content item, than the sync point; and causing, by the computing device, the lagging segmenter to send, to the computing device, a second selected boundary point for ending the current segment later than the first selected boundary point” Van Veldhuisen para 33 teaches “…part of the transcoding process by the disclosed transcoding module 120 includes marking the boundary of a media segment per configuration and passing it as embedded private metadata in an output stream 119, e.g., an MPEG 2 transport stream, to the packager 121. As described in more detail below, the signaling between the transcoder and the packager identifying segment boundaries may be instantaneous decode refresh (IDR) based signaling. The packager 121, also referred to as a segmenter, a fragmenter, or an encapsulator, receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol…” and Van Veldhuisen recognizes a known problem relating to adaptive bit rate streaming (a plurality of transcoded streams with a different bitrate) comprising proposed boundary points for segments associated with each transcoded stream wherein Van Veldhuisen addresses the known problem of maintaining synchronicity of segment boundary points that identify locations where switching between stream representations in the output group with respect to transcoding a content stream into a plurality of streams wherein each of the plurality of streams is transcoded with a different bitrate (Van Veldhuisen para 58-66); Van Veldhuisen teaches that if the transcoder has the embedded packager option, then the output streams within an output group can be further segmented into ABR streams before being output from the transcoder 102 (para 51) such that Van Veldhuisen’s disclosure reads on applicant’s limitations comprising a plurality of segmenters comprising two segmenters. Therefore, the analysis of the claim in relation to Van Veldhuisen identifies a plurality of segmenters and an initial impression of the claim limitations appear to identify at least different time positions for each of the segmenters. In relation to having different bitrate as claimed, as discussed above, Van Veldhuisen further teaches “[t]o support adaptive bit rate streaming, it may be desirable that the transcoder 102 outputs multiple streams within an output group that are scene aligned and GOP/IDR synchronized at the beginning of each segment… Thus, multiple output streams generated from the same input stream, such as multiple bit rate versions of the same input stream, are identified with misaligned timestamps. As a result, output streams generated from the same input but output from two different chassis may not be aligned, as desired, when accessing the same time on a specific frame. Switching between streams, therefore, is not continuous and interruptions in the presentation to a subscriber will occur” (para 62-65) which a person of ordinary skill in the art would have understood that at particular times, the produced segments may need to be synchronized to infer that not all the segments will be scene aligned but some of the plurality of segments for particular streams may not need to be realigned to be synchronized. A person of ordinary skill in the art would have reasonably inferred that Van Veldhuisen’s invention comprises a plurality of segmenters to produce segmented content for each of the plurality of ABR streams at different bit rates wherein Van Veldhuisen Fig. 1 and para 26-33 teaches element 120 disclosed as a plurality of transcoders and packagers referred to a segmenter, a fragmenter, or an encapsulator and Van Veldhuisen further teaches that in order to output streams that are scene aligned, Van Veldhuisen teaches “Embodiments are disclosed that enable each of the transport streams within the same adaptive streaming group to have the same segment and/or fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, the disclosed techniques may be used when chunk boundary points must be synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch.” See Van Veldhuisen para 67. Most importantly, Van Veldhuisen teaches “The disclosed adaptive bit rate transcoder 102 may output transport streams with an IDR at the segment/fragment boundary. Stream transcoding/encoding by the transcoder module 120, while continuous, may then be broken up in to small chunks by a packager, each starting with the IDR. A chunk is a discrete section of content that can be independently decoded. The chunk may include initialization information. A chunk is also referred to herein as a segment or a fragment. See para 58. Van Veldhuisen also teaches “Segment boundary points identify locations where switching between stream representations in the output group maintains synchronicity. A segment may have the max duration and the last fragment just before the max duration is the segment boundary. Thus, a segment may be equal to a fragment or may include a plurality of fragments where a segment boundary point is determined by choosing a fragment boundary within the segment” (See Van Veldhuisen para 60 ) and further teaches “As per the above, when referenced herein, a fragment boundary point or segment boundary point may refer to the same thing. For example, a fragment and a segment, when equal to each other, may include a segment boundary point that is defined by either of the fragment or the segment, i.e., the fragment boundary point is the segment boundary point since the segment=1 fragment. When a segment comprises a plurality of fragments, the segment boundary point may be defined by the boundary point of a particular fragment within the segment. Thus, as used herein, a segment boundary point can be understood in terms of the segment as a whole, i.e., the segment boundary point, or in terms of a fragment within the segment, i.e., a fragment boundary point.” (para 61). Van Veldhuisen does not use the term lagging (e.g., causing, by the computing device, the lagging segmenter to send, to the computing device, a second selected boundary point for ending the current segment later than the first selected boundary point) when teaching the synchronization of a plurality of adaptive bit rate streams for the same content, however, Van Veldhuisen teaches the limitations for a plurality of transcoded streams, each transcoded with a different bitrate (i.e., adaptive bitrate) and further recites a plurality of proposed boundary points indicating time positions for each of the plurality of transcoded streams.
Regarding the deficiency of Van Veldhuisen relating to the term “lagging”, Narayanan teaches (col. 4 ll. 29-61 – element 203 computing device receives multiple streams from encoder element 201 comprising streams of content; Narayanan Fig. 5 and col. 5 ll. 10 to col. 6 ll. 10 teaching different streams of different qualities encoded with different boundary points); Narayanan teaches (col. 4 ll. 63-67 to col. 5 ll. 1-9 – as the streams 202 arrive at computing device 203, the ABR stream creator analyzes each stream and determines key frames in order to synchronize all the streams and col. 5 ll. 10-24 and 6 ll. 11-21 – key frames occur at a particular timing position; col. 7 ll. 4-67 – time-stamp and frame-type determination wherein time stamp information is employed for syncing purposes as discussed in previous embodiments in addition to key-frame identification). A person of ordinary skill in the art would reasonably infer the need to synchronize the producing of ABR content especially when the content is produced live. The examiner interprets the teachings of Narayanan as providing a significant teaching value disclosing the need to produce ABR streams with matching boundary points in the event that the production of ABR streams is not synchronized, in part, because of delay timing issues such as latency. Whereas Narayanan teaches the use of time-stamp and frame-type determination, Narayanan does not explicitly refer to the timing position comprising a “boundary point” and the lagging segementer, the examiner will reference how the prior art would have interpreted Narayanan to disclose boundary points based on known elements and additionally how the combined references disclose a threshold used to segment respective content streams. In reviewing Narayanan Fig. 5, a person of ordinary skill in the art would reasonably infer that the encoders for Stream 1 and Stream 3 will receive packet 502 earlier in time (T.sub.L) than Stream 501n such that by the time the encoders for Stream 1 and Stream 3 receive a future packet (e.g., 503), the encoders will be processing a packet 503 and associated boundary point sooner than the encoder for Stream N. The inverse is also evident from the teachings of Narayanan such that any encoder that receives packet 503 later in time will be lagging behind encoders that receive packet 503 sooner in time such that some encoders are lagging behind in processing the same packets that were already received by the encoders that received the packets earlier in time. Stated differently, the encoders that receive the corresponding packets 502 earlier in time (e.g., at t.sub.1) will also be processing packets with corresponding boundary points sooner in time (i.e., will be processing corresponding packets and transmitting respective boundary points earlier in time). A person of ordinary skill in the art would have understood that based on the teachings of the prior art, that in order to determine whether segments corresponding to different versions of streams generated from adaptive bitrates, then the timing information from each stream would need to be obtained and wherein boundary information as understood in the prior art comprises encoder boundary points. Narayanan further teaches (col. 7 ll. 4-67 – related streams sent separately received at receiver 203 are synchronized by treating the streams as independent streams wherein constant key frame intervals and inconsistent key frame intervals, the key frame determination and duration of a chunk can be decided on the video data collected from a single stream in order to determine a sync point for each of the plurality of related streams and processing at least one of the received related streams; once a large lag is determined, the ABR stream creator follows the syncing procedure; see col. 4 ll. 62-67 to col. 5 ll. 1-9 analyze each stream to determine key frames in order to synchronize all the streams. All things considered, a person of ordinary skill in the art would have understood the timing the boundary points of each stream are analyzed based on which encoders are receiving the same respective packet earlier in time and/or determine which encoders are lagging in order to perform synchronization of all the stream segments. A person of ordinary skill in the art would understand that the synching procedure of Narayanan would identify a stream which lags the synch point in order to make the lagging segment synchronize.
With respect Narayanan not using the same terms as “boundary points” as discussed above, Nemiroff teaches adaptive streaming systems and methods that streams content at varying levels of quality corresponding to differing coding rates and further teaches timing positions comprising a “boundary point” para 0087-0097 – timing information is transmitted with Encoder Boundary Points for each content stream; see also para 0105-0106 – identifying, within the transcoding process in the transcoding processor 209, a time associated with boundary point data that indicates a time at which a component within the transcoding processor 209 processed or output the segment with the boundary point wherein the boundary point timestamp is based on the acquisition time and can be inserted in to a chunk boundary point frame or, in the EBP example, the boundary point timestamp may be inserted into an EBP structure; see also para 0033; para 0067-0068 transcoding module 120 includes marking the boundary of a media segment per configuration and passing it as embedded private metadata in an output stream 119 to the packager 121 wherein the signaling between the transcoder and the segmenter identifying segment boundaries and receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol; para 0067 – each of the transport streams within the same adaptive streaming group to have the same segment and fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, chunk boundary points are synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch; para 0068 – teaches EBP and timing positions correspond wherein the embodiments described apply to a Network Time Protocol (NTP) sync for Encoder Boundary Point (EBP) implementations. Nemiroff recognizes that time information in a plurality of streams may vary between transcoding modules and interfere with proper alignment of frames within a plurality of streams such that switching between streams would be irregular. See also Nemiroff teaches para 0108-0111 – a sync point corresponds to a selected timing position such that each of the multiple transcoders, for transcoding each of the plurality of content streams, will be able to use the same timing position for a specific boundary point and results in determining a relationship between the sync point and each of the plurality of timing positions; see also para 0022 - the timing information utilized for each stream should be the exact same value across all output streams in the same group for synching between streams teaches that based on the disclosed embodiments, processing, is accomplished based on the relationship of synchronizing timing information, at least one of the plurality of encoded content streams. As discussed above, a person of ordinary skill in the art would have understood the timing the boundary points of each stream are analyzed in order to perform synchronization of all the stream segments. As such, Nemiroff teaches the use of boundary points for synchronizing/proper alignment of video streams (e.g., a sync point corresponds to a selected timing position such that each of the multiple transcoders, for transcoding each of the plurality of content streams, will be able to use the same timing position for a specific boundary point and results in determining a relationship between the sync point and each of the plurality of timing positions). [See also Office Action, pg. 6-7, dated 5/19/2018 Jones, MUKTA KAR ET AL, and Eduardo of record incorporated herein and made of record but not relied upon in order to avoid duplicative references for supporting the understanding of encoder boundary points (EBP)].
With respect to the deficiency of Van Veldhuisen and Narayanan as discussed above with respect to the term lagging, the examiner relies on the teachings of Nemiroff and Shaffer. For example, Nemiroff does disclose that during the synchronization process, some segmenters, while active, are only used as backup. For example, Nemiroff para 145-146 teaches “Two transcoder chassis 902, 903 are shown in FIG. 9B, one transcoder 902 is the primary and one transcoder 902 is the backup, and only one 902 is sending streams to the packager. If the primary 902 fails, then the backup chassis 902 will send the streams to the packager.” Furthermore, Nemiroff para 107, 154-155 teaches using a master/slave encoders to maintain synchronization comprising frame rate conversion and GOP synchronization. More importantly,
[0170] In embodiments, active resynchronization techniques may be employed. It may be desirable to actively synchronize more than one frame rate conversion module; in some cases it is necessary to synchronize the modules. For instance, synchronizing the frame rate conversion modules may be necessary when there are four (4) 1280×720p25 outputs in a group, as one frame rate conversion module may not be capable of processing more than two (2) 1280×720p25 output channels. During active synchronization, the slave channels are constantly comparing the PTS values of the slave channel, with the PTS values of the master output channel. When a mismatch is detected, the slave channels will drop an extra frame in the frame rate conversion, bringing the channel back in sync with the master. The frame drop resynchronization may be done multiple times to bring a slave channel back in sync with the master channel. For example, in the case of 30 Hz to 10 Hz, for example, the frame drop resynchronization may need to be done two times in order to bring the channel back in sync with the master. In some designs, once the two channels are synchronized, they may remain synchronized because there is no any variance in the input channel within a transcoding processor 209.
[0172] In some implementations, it is necessary that each segment/fragment boundary on the output streams remain synchronized. In implementations it is necessary that each stream mark its segment/fragment boundary, place an IDR on the first frame after the boundary, and ensure that the frame in the output stream matches in PTS (and time) the frame from the master output channel. In general, a setup configuration will provide a range for placing these boundary points in the output stream. The disclosed techniques may then include placing the fragment and segment boundaries in locations that provide for the best video quality. Note that the larger these ranges are, the better the opportunity may be for the transcoding processor 209 to place the segment/fragment boundaries in a location that has little or no adverse effect on the video quality.
[0205] The master chassis can be either ahead or behind the slave chassis due to a lack of input synchronization to the multiple Chassis. The master chassis can also be ahead or behind the slave chassis due to the database arrival to the slave chassis, which may be delivered with various delays. Therefore, synchronization may be desirable for both a slave chassis that is running behind the master, and/or a slave chassis that is running ahead of the master.
Based on Nemiroff’s teachings as discussed above in para 145-146, 107, 154-155, 170-172, 205, a person of ordinary skill in the art would have reasonably inferred that in a master/slave arrangement, only one master encoder is actively providing media content segmentation. The examiner will further rely on the teachings of Tweedale below to dovetail with the teachings of Van Veldhuisen, Narayanan, and Nemiroff.
Whereas Nemiroff above discloses that a slave chassis is resynchronized even when not actively producing content for segmentation, Nemiroff also discloses that it is necessary to actively synchronize more than one frame conversion module (Nemiroff para 170 during active synchronization, the slave channels are constantly comparing the PTS values of the slave channel and when a mismatch is detected, the slave channels will drop an extra frame in the frame rate conversion, bringing the channel back in sync with the master). Therefore, Nemiroff identifies a solution for a known problem is keeping slave elements synchronized.
In an analogous art, with respect to the limitation “causing, by the computing device, the lagging segmenter to send, to the computing device, a second selected boundary point for ending the current segment later than the first selected boundary point,” whereas Shaffer does not refer to master and slave transcoders as discussed in Nemiroff, Shaffer discloses two or more transcoding nodes for generating ABR streams (para 14-29) and further teaches that each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. See Shaffer paragraphs 22-29 teaching in part:
[0022] Additionally, the synchronization control 16 and 18 further can adjust chunk boundaries in response to detecting a cadence-changing event. Such boundary adjustments may be referred to herein, in some examples, as variable or dynamic boundary placement. For example, a given synchronization control 16 or 18 can be configured to insert an adjusted boundary at a location that is different from expected (e.g., based on synchronization parameters) in response to the cadence-changing event. If an ingested media stream already includes marked chunk boundaries, a closest boundary to the adjusted boundary location that is inserted can be removed also in response to the cadence-changing event. A boundary identifier can be inserted at the determined adjusted boundary location. As an example, for an ABR video stream implemented according to H.264 standard, the boundary identifier can be implemented as an instantaneous decode refresh (IDR) access unit also referred to as an IDR frame.
[0023] As an example, assuming a fixed chunk duration and in the absence of a cadence-changing event, the chunk boundaries fall at deterministic locations in the stream (e.g., at regularly spaced intervals). In response to a cadence-changing event, the synchronization control 16 and 18 can adjust the cadence according to an offset that is calculated. In some examples, the offset can be calculated based on a difference between the originally scheduled, boundary location (assuming the cadence-changing event did not occur) and the location of the adjusted boundary location in response to the cadence-changing event. Thus, the cadence can describe a sequence of scheduled boundary locations including an amount of offset, which scheduled boundary locations can be described relative to an arbitrary initial point in the timeline of the media source (e.g., modulo the nominal chunk duration). The amount of offset can thus be cumulative to adjust the cadence for any number of cadence-changing events.
See also Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. For example, Shaffer teaches:
[0034] By way of example, the synchronization control 60 can include an event processor 78 that is configured to handle cadence-changing events such as disclosed herein. For example, the event processor 78 can employ an event API 80 that is utilized by external applications and/or devices to signal a time when an impending cadence-changing event is to occur. For example, such signaling can be communicated to the node 50 via the network 60 using an appropriate signaling protocol as disclosed herein. In other examples, such signaling can be provided via other mechanisms and other communication interfaces (not shown). The event processor 78 can also include a detector 82 that is configured to generate a detection signal in response to detecting the occurrence of the cadence-changing event (e.g., provide an alert or flag to the synchronization logic). The detection signal can include timing information (e.g., a PTS value) corresponding to when the cadence-changing event is to occur. As disclosed herein, cadence-changing events can include a program boundary, a scene change, a change in the data source that is driving the input or one or more other events that can specify a change in the chunk boundary of the data stream that is generated by the node 50.
[0035] The synchronization logic 70 can also employ rules 74 that can define how synchronization is to be implemented in response to the detection (e.g., by the event processor) of a cadence-changing event, which can include dynamic modification to one or more chunk boundary. For example, the rules 74 can include instruction for removing a closest chunk boundary from the data stream to the occurrence of the cadence-changing event time (e.g., PTS) and placing an adjusted chunk boundary at a location (e.g., time) corresponding to the cadence-changing event. In the examples disclosed herein, the location corresponding to the cadence-changing event is demonstrated as occurring at the time of the cadence-changing event. In other examples, the location could be shifted in time by a known amount, forward or backward in time. The rules 74 can further control how the synchronization logic 70 determines a chunk duration from the cadence-changing event to the next chunk boundary. The rules 74, for example can be different for different ABR protocols.
[0036] The synchronization control 60 can receive and send messages via a communication interface 68. The communication interface 68 can be configured to communicate with one or more other nodes 64 via a communication link, demonstrated as the network 66. For example, the network 66 utilized to communicate messages for the inter-node communications can be implemented via a transport layer protocol (e.g., TCP, UDP, stream control transmission protocol), such as can be employed for communicating multicast messages among the respective nodes of a node group assigned to a given service.
[0037] As a further example, the synchronization control 60 can also include a message control 84 that is configured to control sending messages via the communication interface 62. As disclosed herein, the message control 84 can be configured to send a synchronization message to the other node(s) 64. The message control can send the synchronization message periodically such as at a rate that commensurate with a chunk duration to provide a synchronization message with each chunk that is generated. In other examples, the synchronization message can be sent at other rates that can be greater than or less than the rate that each chunk is generated. Additionally or alternatively, the message control 84 can be configured to generate the synchronization in conjunction with detecting a cadence-changing event.
In order to clarify the inferences that a person of ordinary skill in the art would have made in view of the combination of prior art, the examiner cites the teachings of Mutton teaching segmented parallel encoding with frame-aware variable size chunking wherein a particular segmenter, that is not aligned with other segmenters providing adaptive bit rate streaming, is able to change the boundary points to include portions of the next adjacent segment in order to properly align video content in adaptive bitrate streaming (Abstract, para 19 and 234-241).
All things considered, the prior art recognizes an embodiment wherein a plurality of encoders are used in parallel wherein one or more encoders finish encoding at the same time and the timing of the a additional encoder is delayed, then the delayed encoder is resynchronized to match the encoding of the first and second encoder in order to maintain synchronicity of all the encoders (Forehand Abstract, para 42-43, 52). More importantly, Forehand explicitly teaches what Shaffer does not state when using multiple transcoding elements. See Forehand “[0007] While redundancy is often supported with a good encoder quickly being swapped for a encoder with a detected failure condition, this may introduce a delay or loss of content with respect to a stream corresponding to the failed encoder. While segments are normally published in a synchronized manner, if a fault condition with regard to an encoder is detected, the publication of segments encoded by the functional encoders is not delayed for purposes of synchronization with the segment normally produced by the failed encoder. In this way a fault with one encoder does not adversely impact the other streams being produced with synchronization between the publication of segments for functional encoders proceeding in a synchronized manner.” See also Forehand para 9-13, 37, 43-46, 52-55 and 69 determining a failure condition of an encoder and when an encoder can be brought back online/active state.
See also prior art made of record but not relied upon: Jones; Anthony Richard US 20150256906 A1 disclosing known elements with respect to creating chunks of video content from a plurality of segments/fragments comprising boundary points relating to timing information and recognizing the need to synchronize/align a plurality of streams of video content generated from the same video with adaptive bit rate processing.
In an analogous art, Tweedale is cited for the significant teaching value of a known solution for dynamically enabling/disabling components of a transcoder and packager in an invention that teaches a cluster of transcoders in combination with a cluster of packagers are utilized in redundancy formation (Fig. 3 and para 42) wherein sub-systems of processing units are independently controlled (i.e., activated or deactivated) based on the incoming source media characteristics (para 35-36 – no segmentation is required by packager; para 40 – processing units subsystems are independently and dynamically disabled; para 44 element 402 and further disclosing that if no packager processing is needed, the incoming media may be minimally processed and uploaded is interpreted as the packager performing at least some functions of the packager while other functions of the packager are disabled; determination is made whether segmentation is required (understood and disabling or enabling segment generator; see also para 38 disclosing a packager comprises elements for recording 204, decoding 206, encoding 208, segmenting 210 and uploading 212). The applicant has claimed a packager with a sub-system comprising a packet listener, a sequence list generator, and a segment generator.
A teaching, suggestion, or motivation to modify Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Forehand, and Tweedale is further evidenced in how McCormick dovetails with the teachings of Van Veldhuisen. For example, Van Veldhuisen teaches para 33 teaches “…part of the transcoding process by the disclosed transcoding module 120 includes marking the boundary of a media segment per configuration and passing it as embedded private metadata in an output stream 119, e.g., an MPEG 2 transport stream, to the packager 121. As described in more detail below, the signaling between the transcoder and the packager identifying segment boundaries may be instantaneous decode refresh (IDR) based signaling. The packager 121, also referred to as a segmenter, a fragmenter, or an encapsulator, receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol…” More importantly, McCormick para 30-32 teaches:
[0030] In accordance with various embodiments, first encapsulator 106a and second encapsulator 106b are configured to: implement common algorithms such that the same decisions are made when possible in parallel by first encapsulator 106a and second encapsulator 106b at these decision points without any need for coordination, report decisions (or would-be decisions, if known ahead of time) to the encapsulation coordinator 116 either proactively or on-demand, and accept instruction from the encapsulation coordinator 116 to enforce a decision in order to ensure aligned decisions or correct misaligned decisions across the parallel encapsulations of first encapsulator 106a and second encapsulator 106b. Encapsulating entities that implement these processes are coordinated by encapsulation coordinator 116 to produce equivalent output, and can be re-synchronized by encapsulation coordinator 116 in the event they become out of synchronization.
[0031] The set of decisions made (or about to be made) by an encapsulating entity at these decision points are referred to as synchronization information (SyncInfo). In at least one embodiment, the SyncInfo further includes an identifier for the encoder boundary point at which the decisions are taken. In a particular embodiment, the identifier is the earliest Presentation Timestamp (ePTS) of the segment or fragment (e.g., HLS Segment, Microsoft Smooth Fragment, Dynamic Adaptive Streaming over HTTP (DASH) Segment, etc.).
[0032] In various embodiments, encapsulation coordinator 116 may periodically poll each of first encapsulator 106a and second encapsulator 106b for their latest encapsulation synchronization information, receive synchronization information from each of first encapsulator 106a and second encapsulator 106b, and store a predetermined amount of the synchronization information in a history. Encapsulation coordinator 116 may then compare the synchronization information associated with each redundant encapsulating instance generated by first encapsulator 106a and second encapsulator 106b to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized. If encapsulation coordinator 116 determines that the encapsulating instance of first encapsulator 106a and second encapsulator 106b are not synchronized, encapsulation coordinator 116 may generate corrected synchronization information and send the corrected synchronization information to one or more of first encapsulator 106a and second encapsulator 106b. Upon receiving the corrected synchronization information, first encapsulator 106a or second encapsulator 106b may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct. In at least one embodiment, the corrected synchronization information includes the synchronization information associated with the correctly functioning encapsulator as a starting decision point for generating segments and/or fragments.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Veldhuisen for segmenting a video stream using adaptive bitrate encoding in relation to when segments from different encoders are not synchronized at a particular time position and further incorporate the teachings of Narayanan for segmenting multiple media streams of differing qualities (e.g., bit rates) comprising a plurality of encoders which utilizes timing data (e.g., time-stamp and frame-type determination) in determining timing positions for frames of the corresponding video streams within one of the received media streams for synchronizing multiple related streams because Nemiroff’s invention recognizes the benefit of adaptive streaming systems and methods that streams content at varying levels of quality corresponding to differing coding rates comprising timing positions for boundary point to enable the computing device to identify the timing information and boundary information in packets corresponding to each segment and/or fragment boundary and facilitate the processing, synchronization, and alignment of the plurality of streams. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Veldhuisen, Narayanan, and Nemiroff by further incorporating known elements of Shaffer’s and Mutton’s invention comprising two or more transcoding nodes for generating ABR streams wherein each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is different (leading or lagging) in time in order to take into account a plurality of delays/latency when producing video streams for the same content but with each stream having different adaptive bit rates with a plurality of transcoding segmenters for resolving different time positions for a plurality of proposed boundary points wherein a particular timing position is different (i.e., leading or lagging) in the content of other proposed boundary points in order to establish a offset to synchronize segments produced by independent segmenters in order to improve the combination of Veldhuisen, Narayanan, and Nemiroff by taking into consideration any disruptions that will occur to any individual segmenter as taught by Forehand in the synchronization/alignment of different stream segments. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, and Forehand to address known problems of using parallel encoders for segmenting a video stream using adaptive bitrate encoding in relation to when segments from different encoders are not synchronized at a particular time position (i.e., plurality of encoders which takes into account a plurality of delays/latency when producing video streams for the same content but with each stream having different adaptive bit rates wherein the chassis with a plurality of segmenters for resolving different time positions for a plurality of proposed boundary points wherein a particular timing position is later in the content of other proposed boundary points in order to establish a offset to synchronize segments produced by independent segmenters) because McCormick recognizes the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, and Forehand to address known problems of using parallel encoders for segmenting a video stream using adaptive bitrate encoding in relation to when segments from different encoders are not synchronized at a particular time position (i.e., plurality of encoders which takes into account a plurality of delays/latency when producing video streams for the same content but with each stream having different adaptive bit rates wherein the chassis with a plurality of segmenters for resolving different time positions for a plurality of proposed boundary points wherein a particular timing position is later in the content of other proposed boundary points in order to establish a offset to synchronize segments produced by independent segmenters) by further incorporating known elements of Tweedale’s invention for dynamically enabling/disabling components of a transcoder and packager in an invention that teaches a cluster of transcoders in combination with a cluster of packagers are utilized in redundancy formation in order to utilize a single segmenter in an active state until an abnormality/failure is detected and then activate other synchronized segmenters to conserve processing power and wherein McCormick recognizes the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct.
Regarding claim 2, “further comprising: causing the segmenters that sent the common boundary point to segment corresponding transcoded streams of the plurality of transcoded streams at the sync point” is further rejected on obviousness grounds as discussed in the rejection of claim 1 wherein the prior art as discussed in Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, Forehand, Tweedale and McCormick wherein the prior art teaches (McCormick para 30-32 teaches the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct; See also Shaffer, discloses two or more transcoding nodes for generating ABR streams (para 14-29) and Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to mark a segment boundary independent of the other transcoders and to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. See also the synchronization of segments wherein chunks are aligned to enable switching between ABR streams (see Nemiroff para 0067 – each of the transport streams within the same adaptive streaming group to have the same segment and fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, chunk boundary points are synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch; para 0068 – teaches EBP and timing positions correspond wherein the embodiments described apply to a Network Time Protocol (NTP) sync for Encoder Boundary Point (EBP) implementations; See also Van Veldhuisen identifying a known problem with respect to mismatch of alignment time between different streams caused by a plurality of known delays (para 46--67, 77-87, 98-108 mismatch of time alignment between different stream representations.
Regarding claim 5, “wherein each transcoded stream of the plurality of streams corresponds to an encoding of the content stream” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2 wherein the prior art as discussed in Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, Forehand, Tweedale and McCormick wherein Veldhuisen addresses the known problem of maintaining synchronicity of segment boundary points that identify locations where switching between stream representations in the output group with respect to transcoding a content stream into a plurality of streams wherein each of the plurality of streams is transcoded with a different bitrate (Van Veldhuisen para 58-66); Van Veldhuisen teaches that if the transcoder has the embedded packager option, then the output streams within an output group can be further segmented into ABR streams before being output from the transcoder 102 (para 51). See also Shaffer, discloses two or more transcoding nodes for generating ABR streams (para 14-29) and Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to mark a segment boundary independent of the other transcoders and to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. See also the combination of Van Veldhuisen Narayanan, Nemiroff, Shaffer, Forehand, Tweedale and McCormick as discussed in the rejection of claim 1 wherein the combination renders obvious use of EBP for each received video stream previously encoded or not encoded to produce variable bit rate encoding.
Regarding claim 6, “wherein the common boundary point is a most common time position of the plurality of selected boundary points” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2 and 5 as discussed in the combination of Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, Forehand, Tweedale and McCormick wherein the prior art to McCormick teaches para 30-32 teaches the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct. Furthermore, Narayanan teaches (col. 7 ll. 4-67 – related streams sent separately received at receiver 203 are synchronized by treating the streams as independent streams wherein constant key frame intervals and inconsistent key frame intervals, the key frame determination and duration of a chunk can be decided on the video data collected from a single stream in order to determine a sync point for each of the plurality of related streams and processing at least one of the received related streams; once a large lag is determined, the ABR stream creator follows the syncing procedure; see col. 4 ll. 62-67 to col. 5 ll. 1-9 analyze each stream to determine key frames in order to synchronize all the streams). See also Van Veldhuisen identifying a known problem with respect to mismatch of alignment time between different streams caused by a plurality of known delays (para 46--67, 77-87, 98-108 mismatch of time alignment between different stream representations. See also Shaffer, discloses two or more transcoding nodes for generating ABR streams (para 14-29) and Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to mark a segment boundary independent of the other transcoders and to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time.
Regarding claim 7, “wherein the causing the lagging segmenter to send the second selected boundary point comprises: sending, by the computing device to the lagging segmenter, an instruction for sending, to the computing device, a subsequent selected boundary point in a transcoded stream corresponding to the lagging segmenter” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2 and 5-6 wherein the prior art as discussed in Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, Forehand, Tweedale and McCormick wherein Veldhuisen addresses the known problem of maintaining synchronicity of segment boundary points that identify locations where switching between stream representations in the output group with respect to transcoding a content stream into a plurality of streams wherein each of the plurality of streams is transcoded with a different bitrate (Van Veldhuisen para 58-66); Van Veldhuisen teaches that if the transcoder has the embedded packager option, then the output streams within an output group can be further segmented into ABR streams before being output from the transcoder 102 (para 51). See also Shaffer, discloses two or more transcoding nodes for generating ABR streams (para 14-29) and Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to mark a segment boundary independent of the other transcoders and to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. See also the combination of Van Veldhuisen Narayanan, Nemiroff, Shaffer, Mutton, Forehand, and McCormick as discussed in the rejection of claim 1 wherein the combination renders obvious use of EBP for each received video stream previously encoded or not encoded to produce variable bit rate encoding. Furthermore, the combination of prior art to Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, Forehand and McCormick disclose use of EBP for each received video stream previously encoded to produce variable bit rate encoding and wherein Narayanan teaches col. 4 ll. 36-37 (the streams 202 are different qualities (bit rates) of the same content).
Regarding claim 8, “wherein each transcoded stream, of the plurality of transcoded streams, is associated with a multicast group of a transcoder, and wherein each selected boundary point, of the plurality of selected boundary points, was selected by the transcoder and inserted into a corresponding transcoded stream of the plurality of transcoded streams” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2 and 5-7 wherein Narayanan (col. 6:28-37) teaches media streams associated with a unicast or multicast. See also Shaffer para 26, 36, 55 disclosing use of multicast transmission. See also Van Veldhuisen para 26, 37, 264, 272 teaching multicast transmission of streams.
Regarding method claims 10, “further comprising: after receiving, from the lagging segmenter, the second selected boundary point in a transcoded stream corresponding to the lagging segmenter, causing the lagging segmenter to segment the transcoded stream corresponding to the lagging segmenter at the sync point” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2 and 5-8 wherein the prior art as discussed in Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, Forehand and McCormick wherein the prior art to McCormick para 30-32 teaches the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct. See also Shaffer, discloses two or more transcoding nodes for generating ABR streams (para 14-29) and Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to mark a segment boundary independent of the other transcoders and to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. See also the synchronization of segments wherein chunks are aligned to enable switching between ABR streams (see Nemiroff para 0067 – each of the transport streams within the same adaptive streaming group to have the same segment and fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, chunk boundary points are synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch; para 0068 – teaches EBP and timing positions correspond wherein the embodiments described apply to a Network Time Protocol (NTP) sync for Encoder Boundary Point (EBP) implementations; See also Van Veldhuisen identifying a known problem with respect to mismatch of alignment time between different streams caused by a plurality of known delays (para 46--67, 77-87, 98-108 mismatch of time alignment between different stream representations. See also the prior art recognizes an embodiment wherein at least three encoders are used in parallel wherein two of the three encoders finish encoding at the same time and the timing of the third encoder is delayed, then the third encoder is resynchronized to match the encoding of the first and second encoder in order to maintain synchronicity of all the encoders (Forehand Abstract, para 42-43, 52). In order to clarify the inferences that a person of ordinary skill in the art would have made in view of the combination of prior art, the examiner cites the teachings of Mutton teaching segmented parallel encoding with frame-aware variable size chunking wherein a particular segmenter, that is not aligned with other segmenters providing adaptive bit rate streaming, is able to change the boundary points to include portions of the next adjacent segment in order to properly align video content in adaptive bitrate streaming (Abstract, para 19 and 234-241).
Regarding the method of claim 12, “further comprising: receiving, by the computing device from the plurality of segmenters, a plurality of subsequent selected boundary points; and based on a determination that at least one subsequent selected boundary point, of the plurality of subsequent selected boundary points, is later, in the content item, than a common time position indicated by the plurality of subsequent selected boundary points, causing: at least one segmenter, of the plurality of segmenters, that sent the at least one subsequent selected boundary point to await further segmenting instructions from the computing device.” is further rejected on obviousness grounds as discussed in the rejection of claims 1-2, 5-8, and 10 and wherein based on the teachings of the combination of references teach the addition of new encoder boundary point and wherein the prior art as discussed in Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton, Forehand, Tweedale and McCormick wherein the prior art teaches (McCormick para 30-32 teaches the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct. See also the combination of prior art as discussed in the rejection of claim 1 that teaches the synchronization of segments wherein chunks are aligned to enable switching between ABR streams (Shaffer, discloses two or more transcoding nodes for generating ABR streams (para 14-29) and Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to mark a segment boundary independent of the other transcoders and to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time; see also Nemiroff para 0067 – each of the transport streams within the same adaptive streaming group to have the same segment and fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, chunk boundary points are synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch; para 0068 – teaches EBP and timing positions correspond wherein the embodiments described apply to a Network Time Protocol (NTP) sync for Encoder Boundary Point (EBP) implementations; see also Van Veldhuisen para 108 teaches a chassis with a plurality of transcoding processors must slew (i.e., modify) the offset to a new offset based on a recently captured time and the modified/updated offset value is used to compute the time value for a boundary point frame, such as the time for insertion in to the EBP at the segment and fragment boundary. See also Tweedale is cited for the significant teaching value of a known solution for dynamically enabling/disabling components of a transcoder and packager in an invention that teaches a cluster of transcoders in combination with a cluster of packagers are utilized in redundancy formation (Fig. 3 and para 42) wherein sub-systems of processing units are independently controlled (i.e., activated or deactivated) based on the incoming source media characteristics (para 35-36 – no segmentation is required by packager; para 40 – processing units subsystems are independently and dynamically disabled; para 44 element 402 and further disclosing that if no packager processing is needed, the incoming media may be minimally processed and uploaded is interpreted as the packager performing at least some functions of the packager while other functions of the packager are disabled; determination is made whether segmentation is required (understood and disabling or enabling segment generator; see also para 38 disclosing a packager comprises elements for recording 204, decoding 206, encoding 208, segmenting 210 and uploading 212). A person of ordinary skill in the art would have appreciated the benefit of a chassis with a plurality of segmenters for resolving different time positions for a plurality of proposed boundary points wherein a particular timing position is later in the content of other proposed boundary points in order to establish a offset to synchronize segments produced by independent segmenters on a chassis which takes into account a plurality of delays/latency when producing video streams for the same content but with each stream having different adaptive bit rates and wherein components are deactivated until required for processing video streams.
Regarding apparatus claims 31-32, 34-37, 48-49 are rejected on obviousness grounds as discussed in the rejection of claims 1-2, 5-8, 10 and 12 because the elements of the method claims are taught by the methods, apparatus, and systems of the combination of prior art and because the steps of the method are easily converted into elements of an apparatus and system comprising computer components.
Regarding claim 38, “a system comprising: one or more transcoders; a plurality of segmenters; and a computing device, wherein the one or more transcoders are configured to: receive one or more content streams of a content item; generate a plurality of transcoded streams of the content item by transcoding the one or more content streams at a different bitrate for each transcoded stream of the plurality of transcoded streams” Van Veldhuisen Fig. 1 teaches transcoders 102/120 for performing multi-rate, multi-resolution conversions on the input stream to produce one or more multi-bit rate streams and outputs transport streams with metadata to packagers 121 for segment creation;
Regarding “insert, into each transcoded stream of the plurality of transcoded streams, one or more boundary points for segmenting the transcoded stream, wherein each of the one or more boundary points indicates a different time position in the content item; and send, to the plurality of segmenters, the plurality of transcoded streams, wherein the plurality of transcoded streams comprise the one or more boundary points that are selected and inserted by the one or more transcoders, wherein each segmenter of the plurality of segmenters is configured to: receive, from the one or more transcoders, a transcoded stream of the plurality of transcoded streams, wherein the transcoded stream comprises one or more selected boundary points that were, before the segmenter received the transcoded stream from at least one of the one or more transcoders, selected by the at least one of the one or more transcoders and inserted into the transcoded stream by the at least one of the one or more transcoders; send, to the computing device, a selected boundary point of the one or more selected boundary points; and after sending the selected boundary point, await further instructions, from the computing device, before sending any subsequent selected boundary points of the one or more selected boundary points to the computing device, and wherein the computing device comprises: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the computing device to: receive, from the plurality of segmenters, a plurality of selected boundary points; and determine a sync point based on a common boundary point of the plurality of selected boundary points; determine a lagging segmenter that sent a first selected boundary point, of the plurality of selected boundary points, for ending a current segment of the transcoded, wherein the first selected boundary point is earlier, in the content item, than the sync point; and cause the lagging segmenter to send, to the computing device, a second selected boundary point for ending the current segment later than the first selected boundary point while segmenters that sent the common boundary point await further segmenting instructions” Van Veldhuisen para 33 teaches “…part of the transcoding process by the disclosed transcoding module 120 includes marking the boundary of a media segment per configuration and passing it as embedded private metadata in an output stream 119, e.g., an MPEG 2 transport stream, to the packager 121. As described in more detail below, the signaling between the transcoder and the packager identifying segment boundaries may be instantaneous decode refresh (IDR) based signaling. The packager 121, also referred to as a segmenter, a fragmenter, or an encapsulator, receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol…” and Van Veldhuisen recognizes a known problem relating to adaptive bit rate streaming (a plurality of transcoded streams with a different bitrate) comprising proposed boundary points for segments associated with each transcoded stream wherein Van Veldhuisen addresses the known problem of maintaining synchronicity of segment boundary points that identify locations where switching between stream representations in the output group with respect to transcoding a content stream into a plurality of streams wherein each of the plurality of streams is transcoded with a different bitrate (Van Veldhuisen para 58-66); Van Veldhuisen teaches that if the transcoder has the embedded packager option, then the output streams within an output group can be further segmented into ABR streams before being output from the transcoder 102 (para 51) such that Van Veldhuisen’s disclosure reads on applicant’s limitations comprising a plurality of segmenters. Van Veldhuisen further teaches “[t]o support adaptive bit rate streaming, it may be desirable that the transcoder 102 outputs multiple streams within an output group that are scene aligned and GOP/IDR synchronized at the beginning of each segment… Thus, multiple output streams generated from the same input stream, such as multiple bit rate versions of the same input stream, are identified with misaligned timestamps. As a result, output streams generated from the same input but output from two different chassis may not be aligned, as desired, when accessing the same time on a specific frame. Switching between streams, therefore, is not continuous and interruptions in the presentation to a subscriber will occur” (para 62-65) which a person of ordinary skill in the art would have understood that at particular times, the produced segments may need to be synchronized to infer that not all the segments will be scene aligned but some of the plurality of segments for particular streams may not need to be realigned to be synchronized. A person of ordinary skill in the art would have reasonably inferred that Van Veldhuisen’s invention comprises a plurality of segmenters to produce segmented content for each of the plurality of ABR streams at different bit rates wherein Van Veldhuisen further discloses Fig. 1 and para 26-33 teaches element 120 disclosed as a plurality of transcoders and packagers referred to a segmenter, a fragmenter, or an encapsulator and Van Veldhuisen further teaches that in order to output streams that are scene aligned, Van Veldhuisen teaches “Embodiments are disclosed that enable each of the transport streams within the same adaptive streaming group to have the same segment and/or fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, the disclosed techniques may be used when chunk boundary points must be synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch.” See Van Veldhuisen para 67. Most importantly, Van Veldhuisen teaches “The disclosed adaptive bit rate transcoder 102 may output transport streams with an IDR at the segment/fragment boundary. Stream transcoding/encoding by the transcoder module 120, while continuous, may then be broken up in to small chunks by a packager, each starting with the IDR. A chunk is a discrete section of content that can be independently decoded. The chunk may include initialization information. A chunk is also referred to herein as a segment or a fragment. See para 58. Van Veldhuisen also teaches “Segment boundary points identify locations where switching between stream representations in the output group maintains synchronicity. A segment may have the max duration and the last fragment just before the max duration is the segment boundary. Thus, a segment may be equal to a fragment or may include a plurality of fragments where a segment boundary point is determined by choosing a fragment boundary within the segment” (See Van Veldhuisen para 60 ) and further teaches “As per the above, when referenced herein, a fragment boundary point or segment boundary point may refer to the same thing. For example, a fragment and a segment, when equal to each other, may include a segment boundary point that is defined by either of the fragment or the segment, i.e., the fragment boundary point is the segment boundary point since the segment=1 fragment. When a segment comprises a plurality of fragments, the segment boundary point may be defined by the boundary point of a particular fragment within the segment. Thus, as used herein, a segment boundary point can be understood in terms of the segment as a whole, i.e., the segment boundary point, or in terms of a fragment within the segment, i.e., a fragment boundary point.” (para 61). All things considered, Van Veldhuisen teaches the limitations for a plurality of transcoded streams, each transcoded with a different bitrate (i.e., adaptive bitrate) and further recites a plurality of proposed boundary points indicating time positions for each of the plurality of transcoded streams.
Van Veldhuisen does not explicitly (in haec verba) reference/disclose “after sending the selected boundary point, await further instructions, from the computing device, before sending any subsequent selected boundary points of the one or more selected boundary points to the computing device” or “a subsequent selected boundary point while segmenters that sent the common boundary point await further segmenting instructions”), however, as will be evidenced from the combination of prior art references that will be discussed infra, a person of ordinary skill in the art would have reasonably inferred that if segmenters are continually being synchronized and are being polled to provide boundary point information for performing segmenting, the time between polling requests sent from a controller for controlling a cluster of segmenters would be considered awaiting further segmenting instructions.
Regarding the deficiency of Van Veldhuisen as discussed above (i.e., “await further instructions,” “await further segmenting instructions” in relation to boundary points and segmenters as claimed), and the inferences drawn above in Van Veldhuisen (i.e., a person of ordinary skill in the art would have reasonably inferred that if segmenters are continually being synchronized and are being polled to provide boundary point information for performing segmenting, the time between polling requests sent from a controller for controlling a cluster of segmenters would be considered awaiting further segmenting instructions), Narayanan teaches (col. 4 ll. 29-61 – element 203 computing device receives multiple streams from encoder element 201 comprising streams of content; Narayanan Fig. 5 and col. 5 ll. 10 to col. 6 ll. 10 teaching different streams of different qualities encoded with different boundary points); Narayanan teaches (col. 4 ll. 63-67 to col. 5 ll. 1-9 – as the streams 202 arrive at computing device 203, the ABR stream creator analyzes each stream and determines key frames in order to synchronize all the streams and col. 5 ll. 10-24 and 6 ll. 11-21 – key frames occur at a particular timing position; col. 7 ll. 4-67 – time-stamp and frame-type determination wherein time stamp information is employed for syncing purposes as discussed in previous embodiments in addition to key-frame identification). A person of ordinary skill in the art would reasonably infer the need to synchronize the producing of ABR content especially when the content is produced live. The examiner interprets the teachings of Narayanan as providing a significant teaching value disclosing the need to produce ABR streams with matching boundary points in the event that the production of ABR streams is not synchronized, in part, because of delay timing issues such as latency. Whereas Narayanan teaches the use of time-stamp and frame-type determination, Narayanan does not explicitly refer to the timing position in relation to a “boundary point,” the examiner will reference how the prior art would have interpreted Narayanan to disclose boundary points based on known elements and additionally how the combined references disclose a threshold used to segment respective content streams and leading and/or lagging segmenters. In reviewing Narayanan Fig. 5, a person of ordinary skill in the art would reasonably infer that the encoders for Stream 1 and Stream 3 will receive packet 502 earlier in time (T.sub.L) than Stream 501n such that by the time the encoders for Stream 1 and Stream 3 receive a future packet (e.g., 503), the encoders will be processing a packet 503 and associated boundary point sooner than the encoder for Stream N. The inverse is also evident from the teachings of Narayanan such that any encoder that receives packet 503 later in time will be lagging behind encoders that receive packet 503 sooner in time such that some encoders are lagging behind in processing the same packets that were already received by the encoders that received the packets earlier in time. Stated differently, the encoders that receive the corresponding packets 502 earlier in time (e.g., at t.sub.1) will also be processing packets with corresponding boundary points sooner in time (i.e., will be processing corresponding packets and transmitting respective boundary points earlier in time). A person of ordinary skill in the art would have understood that based on the teachings of the prior art, that in order to determine whether segments corresponding to different versions of streams generated from adaptive bitrates, then the timing information from each stream would need to be obtained and wherein boundary information as understood in the prior art comprises encoder boundary points. Narayanan further teaches (col. 7 ll. 4-67 – related streams sent separately received at receiver 203 are synchronized by treating the streams as independent streams wherein constant key frame intervals and inconsistent key frame intervals, the key frame determination and duration of a chunk can be decided on the video data collected from a single stream in order to determine a sync point for each of the plurality of related streams and processing at least one of the received related streams; once a large lag is determined, the ABR stream creator follows the syncing procedure; see col. 4 ll. 62-67 to col. 5 ll. 1-9 analyze each stream to determine key frames in order to synchronize all the streams. A person of ordinary skill in the art would have understood the timing the boundary points of each stream are analyzed based on which encoders are receiving the same respective packet earlier in time and/or determine which encoders are lagging in order to perform synchronization of all the stream segments. A person of ordinary skill in the art would understand that the synching procedure of Narayanan would identify a stream which lags the synch point in order to make the lagging segment synchronize.
With respect Narayanan not using the same terms as “boundary points” as discussed above, Nemiroff teaches adaptive streaming systems and methods that streams content at varying levels of quality corresponding to differing coding rates and further teaches timing positions comprising a “boundary point” para 0087-0097 – timing information is transmitted with Encoder Boundary Points for each content stream; see also para 0105-0106 – identifying, within the transcoding process in the transcoding processor 209, a time associated with boundary point data that indicates a time at which a component within the transcoding processor 209 processed or output the segment with the boundary point wherein the boundary point timestamp is based on the acquisition time and can be inserted in to a chunk boundary point frame or, in the EBP example, the boundary point timestamp may be inserted into an EBP structure; see also para 0033; para 0067-0068 transcoding module 120 includes marking the boundary of a media segment per configuration and passing it as embedded private metadata in an output stream 119 to the packager 121 wherein the signaling between the transcoder and the segmenter identifying segment boundaries and receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol; para 0067 – each of the transport streams within the same adaptive streaming group to have the same segment and fragment boundary on the same key or IDR frame for downstream seamless switching. In other words, chunk boundary points are synchronized across each output stream representation such that the chunk boundary point is placed on the same picture (same PTS) in each output stream representation or profile. In embodiments, an IDR synch is complemented by a boundary point synch; para 0068 – teaches EBP and timing positions correspond wherein the embodiments described apply to a Network Time Protocol (NTP) sync for Encoder Boundary Point (EBP) implementations. Nemiroff recognizes that time information in a plurality of streams may vary between transcoding modules and interfere with proper alignment of frames within a plurality of streams such that switching between streams would be irregular. See also Nemiroff teaches para 0108-0111 – a sync point corresponds to a selected timing position such that each of the multiple transcoders, for transcoding each of the plurality of content streams, will be able to use the same timing position for a specific boundary point and results in determining a relationship between the sync point and each of the plurality of timing positions; see also para 0022 - the timing information utilized for each stream should be the exact same value across all output streams in the same group for synching between streams teaches that based on the disclosed embodiments, processing, is accomplished based on the relationship of synchronizing timing information, at least one of the plurality of encoded content streams. Nemiroff does not explicitly use the same terms such as “await” in relation to segmenting, however, as discussed above, a person of ordinary skill in the art would have understood the timing the boundary points of each stream are analyzed in order to perform synchronization of all the stream segments. As such, Nemiroff teaches the use of boundary points for synchronizing/proper alignment of video streams (e.g., a sync point corresponds to a selected timing position such that each of the multiple transcoders, for transcoding each of the plurality of content streams, will be able to use the same timing position for a specific boundary point and results in determining a relationship between the sync point and each of the plurality of timing positions). [See also Office Action, pg. 6-7, dated 5/19/2018 Jones, MUKTA KAR ET AL, and Eduardo of record incorporated herein and made of record but not relied upon in order to avoid duplicative references for supporting the understanding of encoder boundary points (EBP)].
With respect to the deficiency of Van Veldhuisen and Narayanan (i.e., “await further instructions,” “await further segmenting instructions” in relation to boundary points and segmenters as claimed), Nemiroff does disclose that during the synchronization process, some segmenters, while active, are only used as backup and also introduces the teachings of master/slave encoders understood and slave devices being controlled by a master controller which would await instructions for performing functions. For example, Nemiroff para 145-146 teaches “Two transcoder chassis 902, 903 are shown in FIG. 9B, one transcoder 902 is the primary and one transcoder 902 is the backup, and only one 902 is sending streams to the packager. If the primary 902 fails, then the backup chassis 902 will send the streams to the packager.” Furthermore, Nemiroff para 107, 154-155 teaches using a master/slave encoders to maintain synchronization comprising frame rate conversion and GOP synchronization. More importantly,
[0170] In embodiments, active resynchronization techniques may be employed. It may be desirable to actively synchronize more than one frame rate conversion module; in some cases it is necessary to synchronize the modules. For instance, synchronizing the frame rate conversion modules may be necessary when there are four (4) 1280×720p25 outputs in a group, as one frame rate conversion module may not be capable of processing more than two (2) 1280×720p25 output channels. During active synchronization, the slave channels are constantly comparing the PTS values of the slave channel, with the PTS values of the master output channel. When a mismatch is detected, the slave channels will drop an extra frame in the frame rate conversion, bringing the channel back in sync with the master. The frame drop resynchronization may be done multiple times to bring a slave channel back in sync with the master channel. For example, in the case of 30 Hz to 10 Hz, for example, the frame drop resynchronization may need to be done two times in order to bring the channel back in sync with the master. In some designs, once the two channels are synchronized, they may remain synchronized because there is no any variance in the input channel within a transcoding processor 209.
[0172] In some implementations, it is necessary that each segment/fragment boundary on the output streams remain synchronized. In implementations it is necessary that each stream mark its segment/fragment boundary, place an IDR on the first frame after the boundary, and ensure that the frame in the output stream matches in PTS (and time) the frame from the master output channel. In general, a setup configuration will provide a range for placing these boundary points in the output stream. The disclosed techniques may then include placing the fragment and segment boundaries in locations that provide for the best video quality. Note that the larger these ranges are, the better the opportunity may be for the transcoding processor 209 to place the segment/fragment boundaries in a location that has little or no adverse effect on the video quality.
[0205] The master chassis can be either ahead or behind the slave chassis due to a lack of input synchronization to the multiple Chassis. The master chassis can also be ahead or behind the slave chassis due to the database arrival to the slave chassis, which may be delivered with various delays. Therefore, synchronization may be desirable for both a slave chassis that is running behind the master, and/or a slave chassis that is running ahead of the master.
Based on Nemiroff’s teachings as discussed above in para 145-146, 107, 154-155, 170-172, 205, a person of ordinary skill in the art would have reasonably inferred that in a master/slave arrangement, only one master encoder is actively providing media content segmentation. The examiner will further rely on the teachings of Tweedale below to dovetail with the teachings of Van Veldhuisen, Narayanan, and Nemiroff.
Whereas Nemiroff above discloses that a slave chassis is resynchronized even when not actively producing content for segmentation, Nemiroff also discloses that it is necessary to actively synchronize more than one frame conversion module (Nemiroff para 170 during active synchronization, the slave channels are constantly comparing the PTS values of the slave channel and when a mismatch is detected, the slave channels will drop an extra frame in the frame rate conversion, bringing the channel back in sync with the master). Therefore, Nemiroff identifies a solution for a known problem is keeping slave elements synchronized.
In an analogous art, whereas Shaffer does not refer to master and slave transcoders as discussed in Nemiroff, Shaffer discloses two or more transcoding nodes for generating ABR streams (para 14-29) and further teaches that each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. See Shaffer paragraphs 22-29 teaching in part:
[0022] Additionally, the synchronization control 16 and 18 further can adjust chunk boundaries in response to detecting a cadence-changing event. Such boundary adjustments may be referred to herein, in some examples, as variable or dynamic boundary placement. For example, a given synchronization control 16 or 18 can be configured to insert an adjusted boundary at a location that is different from expected (e.g., based on synchronization parameters) in response to the cadence-changing event. If an ingested media stream already includes marked chunk boundaries, a closest boundary to the adjusted boundary location that is inserted can be removed also in response to the cadence-changing event. A boundary identifier can be inserted at the determined adjusted boundary location. As an example, for an ABR video stream implemented according to H.264 standard, the boundary identifier can be implemented as an instantaneous decode refresh (IDR) access unit also referred to as an IDR frame.
[0023] As an example, assuming a fixed chunk duration and in the absence of a cadence-changing event, the chunk boundaries fall at deterministic locations in the stream (e.g., at regularly spaced intervals). In response to a cadence-changing event, the synchronization control 16 and 18 can adjust the cadence according to an offset that is calculated. In some examples, the offset can be calculated based on a difference between the originally scheduled, boundary location (assuming the cadence-changing event did not occur) and the location of the adjusted boundary location in response to the cadence-changing event. Thus, the cadence can describe a sequence of scheduled boundary locations including an amount of offset, which scheduled boundary locations can be described relative to an arbitrary initial point in the timeline of the media source (e.g., modulo the nominal chunk duration). The amount of offset can thus be cumulative to adjust the cadence for any number of cadence-changing events.
See also Shaffer paragraphs 30-42 teaching each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is later in time. For example, Shaffer teaches:
[0034] By way of example, the synchronization control 60 can include an event processor 78 that is configured to handle cadence-changing events such as disclosed herein. For example, the event processor 78 can employ an event API 80 that is utilized by external applications and/or devices to signal a time when an impending cadence-changing event is to occur. For example, such signaling can be communicated to the node 50 via the network 60 using an appropriate signaling protocol as disclosed herein. In other examples, such signaling can be provided via other mechanisms and other communication interfaces (not shown). The event processor 78 can also include a detector 82 that is configured to generate a detection signal in response to detecting the occurrence of the cadence-changing event (e.g., provide an alert or flag to the synchronization logic). The detection signal can include timing information (e.g., a PTS value) corresponding to when the cadence-changing event is to occur. As disclosed herein, cadence-changing events can include a program boundary, a scene change, a change in the data source that is driving the input or one or more other events that can specify a change in the chunk boundary of the data stream that is generated by the node 50.
[0035] The synchronization logic 70 can also employ rules 74 that can define how synchronization is to be implemented in response to the detection (e.g., by the event processor) of a cadence-changing event, which can include dynamic modification to one or more chunk boundary. For example, the rules 74 can include instruction for removing a closest chunk boundary from the data stream to the occurrence of the cadence-changing event time (e.g., PTS) and placing an adjusted chunk boundary at a location (e.g., time) corresponding to the cadence-changing event. In the examples disclosed herein, the location corresponding to the cadence-changing event is demonstrated as occurring at the time of the cadence-changing event. In other examples, the location could be shifted in time by a known amount, forward or backward in time. The rules 74 can further control how the synchronization logic 70 determines a chunk duration from the cadence-changing event to the next chunk boundary. The rules 74, for example can be different for different ABR protocols.
[0036] The synchronization control 60 can receive and send messages via a communication interface 68. The communication interface 68 can be configured to communicate with one or more other nodes 64 via a communication link, demonstrated as the network 66. For example, the network 66 utilized to communicate messages for the inter-node communications can be implemented via a transport layer protocol (e.g., TCP, UDP, stream control transmission protocol), such as can be employed for communicating multicast messages among the respective nodes of a node group assigned to a given service.
[0037] As a further example, the synchronization control 60 can also include a message control 84 that is configured to control sending messages via the communication interface 62. As disclosed herein, the message control 84 can be configured to send a synchronization message to the other node(s) 64. The message control can send the synchronization message periodically such as at a rate that commensurate with a chunk duration to provide a synchronization message with each chunk that is generated. In other examples, the synchronization message can be sent at other rates that can be greater than or less than the rate that each chunk is generated. Additionally or alternatively, the message control 84 can be configured to generate the synchronization in conjunction with detecting a cadence-changing event.
All things considered, Shaffer recognizes an embodiment wherein at least three encoders are used in parallel wherein two of the three encoders finish encoding at the same time and the timing of the third encoder is delayed, then the third encoder is resynchronized to match the encoding of the first and second encoder in order to maintain synchronicity of all the encoders (Forehand Abstract, para 42-43, 52). More importantly, Forehand explicitly teaches what Shaffer does not state when using multiple transcoding elements. See Forehand “[0007] While redundancy is often supported with a good encoder quickly being swapped for a encoder with a detected failure condition, this may introduce a delay or loss of content with respect to a stream corresponding to the failed encoder. While segments are normally published in a synchronized manner, if a fault condition with regard to an encoder is detected, the publication of segments encoded by the functional encoders is not delayed for purposes of synchronization with the segment normally produced by the failed encoder. In this way a fault with one encoder does not adversely impact the other streams being produced with synchronization between the publication of segments for functional encoders proceeding in a synchronized manner.” See also Forehand para 9-13, 37, 43-46, 52-55 and 69 determining a failure condition of an encoder and when an encoder can be brought back online/active state.
See also prior art made of record but not relied upon: Jones; Anthony Richard US 20150256906 A1 disclosing known elements with respect to creating chunks of video content from a plurality of segments/fragments comprising boundary points relating to timing information and recognizing the need to synchronize/align a plurality of streams of video content generated from the same video with adaptive bit rate processing.
In order to clarify the inferences that a person of ordinary skill in the art would have made in view of the combination of prior art, the examiner cites the teachings of Mutton teaching segmented parallel encoding with frame-aware variable size chunking wherein a particular segmenter, that is not aligned with other segmenters providing adaptive bit rate streaming, is able to change the boundary points to include portions of the next adjacent segment in order to properly align video content in adaptive bitrate streaming (Abstract, para 19 and 234-241).
In an analogous art, Tweedale dovetails with the deficiency of Shaffer wherein Tweedale is cited for the significant teaching value of a known solution for dynamically enabling/disabling components of a transcoder and packager is potentially interpreted as enabling a segmenter to await further instructions for segmenting in an invention that teaches a cluster of transcoders in combination with a cluster of packagers are utilized in redundancy formation (Fig. 3 and para 42) wherein sub-systems of processing units are independently controlled (i.e., activated or deactivated) based on the incoming source media characteristics (para 35-36 – no segmentation is required by packager; para 40 – processing units subsystems are independently and dynamically disabled; para 44 element 402 and further disclosing that if no packager processing is needed, the incoming media may be minimally processed and uploaded is interpreted as the packager performing at least some functions of the packager while other functions of the packager are disabled; determination is made whether segmentation is required (understood and disabling or enabling segment generator; see also para 38 disclosing a packager comprises elements for recording 204, decoding 206, encoding 208, segmenting 210 and uploading 212). The applicant has claimed a packager with a sub-system comprising a packet listener, a sequence list generator, and a segment generator.
A teaching, suggestion, or motivation to modify Van Veldhuisen, Narayanan, Nemiroff, Shaffer, Forehand, and Tweedale is further evidenced in how McCormick dovetails with the teachings of Van Veldhuisen. For example, Van Veldhuisen teaches para 33 teaches “…part of the transcoding process by the disclosed transcoding module 120 includes marking the boundary of a media segment per configuration and passing it as embedded private metadata in an output stream 119, e.g., an MPEG 2 transport stream, to the packager 121. As described in more detail below, the signaling between the transcoder and the packager identifying segment boundaries may be instantaneous decode refresh (IDR) based signaling. The packager 121, also referred to as a segmenter, a fragmenter, or an encapsulator, receives the transcoded video output from the transcoder and packages the video based on a delivery protocol, such as a proprietary deliver protocol…” More importantly, McCormick para 30-32 teaches:
[0030] In accordance with various embodiments, first encapsulator 106a and second encapsulator 106b are configured to: implement common algorithms such that the same decisions are made when possible in parallel by first encapsulator 106a and second encapsulator 106b at these decision points without any need for coordination, report decisions (or would-be decisions, if known ahead of time) to the encapsulation coordinator 116 either proactively or on-demand, and accept instruction from the encapsulation coordinator 116 to enforce a decision in order to ensure aligned decisions or correct misaligned decisions across the parallel encapsulations of first encapsulator 106a and second encapsulator 106b. Encapsulating entities that implement these processes are coordinated by encapsulation coordinator 116 to produce equivalent output, and can be re-synchronized by encapsulation coordinator 116 in the event they become out of synchronization.
[0031] The set of decisions made (or about to be made) by an encapsulating entity at these decision points are referred to as synchronization information (SyncInfo). In at least one embodiment, the SyncInfo further includes an identifier for the encoder boundary point at which the decisions are taken. In a particular embodiment, the identifier is the earliest Presentation Timestamp (ePTS) of the segment or fragment (e.g., HLS Segment, Microsoft Smooth Fragment, Dynamic Adaptive Streaming over HTTP (DASH) Segment, etc.).
[0032] In various embodiments, encapsulation coordinator 116 may periodically poll each of first encapsulator 106a and second encapsulator 106b for their latest encapsulation synchronization information, receive synchronization information from each of first encapsulator 106a and second encapsulator 106b, and store a predetermined amount of the synchronization information in a history. Encapsulation coordinator 116 may then compare the synchronization information associated with each redundant encapsulating instance generated by first encapsulator 106a and second encapsulator 106b to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized. If encapsulation coordinator 116 determines that the encapsulating instance of first encapsulator 106a and second encapsulator 106b are not synchronized, encapsulation coordinator 116 may generate corrected synchronization information and send the corrected synchronization information to one or more of first encapsulator 106a and second encapsulator 106b. Upon receiving the corrected synchronization information, first encapsulator 106a or second encapsulator 106b may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct. In at least one embodiment, the corrected synchronization information includes the synchronization information associated with the correctly functioning encapsulator as a starting decision point for generating segments and/or fragments.
See also prior art made of record: MacInnis para 33-38 teaches managing a plurality of transcoders wherein “…each transcoder device 134a-n provides an interface between the transcoder device 134a-n and the transcode manager 123. For example, the controller 149 may allow for a transcoder device 134a-n to be enabled or disabled. The controller 149 may indicate a transcoding progress of the transcoder device 134a-n to the transcode manager 123. The controller 149 may also maintain a workload queue of a transcoder device 134a-n that indicates an amount of work scheduled to be performed by a transcoder device 134a-n. Transcoding progress, for example, may indicate whether a transcoder is busy or not….”
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Veldhuisen for segmenting a video stream using adaptive bitrate encoding in relation to when segments from different encoders are not synchronized at a particular time position and further incorporate the teachings of Narayanan for segmenting multiple media streams of differing qualities (e.g., bit rates) comprising a plurality of encoders which utilizes timing data (e.g., time-stamp and frame-type determination) in determining timing positions for frames of the corresponding video streams within one of the received media streams for synchronizing multiple related streams because Nemiroff’s invention recognizes the benefit of adaptive streaming systems and methods that streams content at varying levels of quality corresponding to differing coding rates comprising timing positions for boundary point to enable the computing device to identify the timing information and boundary information in packets corresponding to each segment and/or fragment boundary and facilitate the processing, synchronization, and alignment of the plurality of streams. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Van Veldhuisen, Narayanan, and Nemiroff by further incorporating known elements of Shaffer’s and Mutton’s invention comprising two or more transcoding nodes for generating ABR streams wherein each of the transcoding nodes is able to extend a current segment of the content item until a time position that is at or after the time position of a proposed boundary point obtained from a different transcoding node that is different (leading or lagging) in time in order to take into account a plurality of delays/latency when producing video streams for the same content but with each stream having different adaptive bit rates with a plurality of transcoding segmenters for resolving different time positions for a plurality of proposed boundary points wherein a particular timing position is different (i.e., leading or lagging) in the content of other proposed boundary points in order to establish a offset to synchronize segments produced by independent segmenters in order to improve the combination of Veldhuisen, Narayanan, and Nemiroff by taking into consideration any disruptions that will occur to any individual segmenter as taught by Forehand in the synchronization/alignment of different stream segments. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton and Forehand to address known problems of using parallel encoders for segmenting a video stream using adaptive bitrate encoding in relation to when segments from different encoders are not synchronized at a particular time position (i.e., plurality of encoders which takes into account a plurality of delays/latency when producing video streams for the same content but with each stream having different adaptive bit rates wherein the chassis with a plurality of segmenters for resolving different time positions for a plurality of proposed boundary points wherein a particular timing position is later in the content of other proposed boundary points in order to establish a offset to synchronize segments produced by independent segmenters) because McCormick recognizes the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Veldhuisen, Narayanan, Nemiroff, Shaffer, Mutton and Forehand to address known problems of using parallel encoders for segmenting a video stream using adaptive bitrate encoding in relation to when segments from different encoders are not synchronized at a particular time position (i.e., plurality of encoders which takes into account a plurality of delays/latency when producing video streams for the same content but with each stream having different adaptive bit rates wherein the chassis with a plurality of segmenters for resolving different time positions for a plurality of proposed boundary points wherein a particular timing position is later in the content of other proposed boundary points in order to establish a offset to synchronize segments produced by independent segmenters) by further incorporating known elements of Tweedale’s invention for dynamically enabling/disabling components of a transcoder and packager in an invention that teaches a cluster of transcoders in combination with a cluster of packagers are utilized in redundancy formation in order to utilize a single segmenter in an active state until an abnormality/failure is detected and then activate other synchronized segmenters to conserve processing power and wherein McCormick recognizes the benefit of periodically polling each of first encapsulator and second encapsulator for their latest encapsulation synchronization information to determine whether the encapsulating instances are either the same or offset (i.e. equivalent) from one another within a predetermined window and thus synchronized such that upon receiving the corrected synchronization information, first encapsulator or second encapsulator may use the corrected synchronization information to begin encapsulating content such that it is in synchronization with the encapsulator that is determined to be correct.
Regarding system claims 38-39, 41-44, 50-51, the claims are rejected on obviousness grounds as discussed in the rejection of claims 1-2, 5-8, 10 and 12 because the elements of the method claims are taught by the methods, apparatus, and systems of the combination of prior art and because the steps of the method are easily converted into elements of an apparatus and system comprising computer components.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALFONSO CASTRO whose telephone number is (571)270-3950. The examiner can normally be reached on Monday to Friday from 10am to 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nathan Flynn can be reached. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALFONSO CASTRO/Primary Examiner, Art Unit 2421