DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Remarks
This is a reply to the application filed on 06/09/2025, in which, claims 1-20 remain pending in the present application with claims 1, 16, and 20 being independent claims.
When making claim amendments, the applicant is encouraged to consider the references in their entireties, including those portions that have not been cited by the examiner and their equivalents as they may most broadly and appropriately apply to any particular anticipated claim amendments.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the invention was patented or described in a printed publication in this or a foreign country or in public use or on sale in this country, more than one year prior to the date of application for patent in the United States.
Claims 1-4, 6-14, and 16-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Dahl et al. (US 20220264168 A1, hereinafter referred to as “Dahl”).
Regarding claim 1, Dahl discloses a method comprising:
receiving a first video from a first publisher, the first video characterized by a first file size and a source resolution (see Dahl, paragraph [0026]: “publish an internet video stream (e.g., by generating an HLS manifest file specifying available renditions of the input video) with a video-specific encoding ladder for an input video that is predicted to maximize quality at any of the bitrates included in the video-specific encoding ladder without performing additional encodes to determine the quality of the video at various bitrates and resolutions”);
partially decoding the first video, to generate a proxy video representation of the first video, the proxy video representation characterized by a second file size less than the first file size (see Dahl, paragraph [0111]: “the computer system can produce video thumbnails or thumbnail images by selectively decoding AV segments that contain the video frames that include the thumbnail. For example, a video thumbnail can be displayed shortly after publication of the video by specifying a time interval for the video thumbnail and selectively transcoding the segments corresponding to the video thumbnail in each rendition offered by the computer system immediately after publication”);
selecting a set of resolutions for the first video (see Dahl, paragraph [0068]: “The system retrieves viewing data that can include a set of audience bandwidths, resolutions, and/or viewing conditions”);
for a first resolution in the set of resolutions:
accessing a first model associated with the first resolution, the first model configured to derive target bitrates based on target viewing qualities (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
passing the proxy video representation to the first model (see Dahl, paragraph [0048]: “The system can also execute multiple convex hull estimation models, wherein each convex hull estimation model outputs estimated convex hulls that indicate quality-maximizing resolutions for the input video when encoded over a range of bitrates and viewed in a particular viewing condition”); and
receiving a first target bitrate for the first resolution returned by the first model (see Dahl, paragraph [0047]: “model outputs an accurate convex hull (e.g., a convex hull that actually represents the quality maximizing resolution for the input video over a series of bitrates)”);
for a second resolution in the set of resolutions:
accessing a second model associated with the second resolution, the second model configured to derive target bitrates and target viewing qualities (see Dahl, paragraph [0065]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
passing the proxy video representation to the second model (see Dahl, paragraph [0065]: “the system can select bitrate-resolution pairs from any of the estimated convex hulls corresponding to the various viewing conditions in the predicted audience of the input video”); and
receiving a second target bitrate for the second resolution returned by the second model (see Dahl, paragraph [0065]: “for each viewing condition in a set of viewing conditions, generate a viewing-condition-specific set of bitrate-resolution pairs based on the set of video features via a convex hull estimation model corresponding to the viewing condition”);
defining a first rendition, for the first video, characterized by the first resolution and the first target bitrate (see Dahl, paragraph [0054]: “To initiate calculation of a convex hull of a training video, the system can encode an initial rendition of the training video at a low bitrate as a first step in the trial encoding process (e.g., 200 kbps and 180p)”);
defining a second rendition, for the first video, characterized by the second resolution and the second target bitrate (see Dahl, paragraph [0054]: “The system can then evaluate the quality of the rendition according to the quality metric. Subsequently, the system can increase the bitrate and/or resolution and again evaluate the rendition according to the quality metric”);
generating an encoding ladder identifying the first rendition and the second rendition (see Dahl, paragraph [0099]: “Upon selecting a set of bitrate-resolution pairs for the video-specific encoding ladder of an input video, the system can generate an encoding ladder for the video segment including the top bitrate-resolution pair, the bottom bitrate-resolution pair, and/or the subset of bitrate-resolution pairs”); and
publishing the encoding ladder for access by a set of video players for playback of the first video (see Dahl, paragraph [0099]: “publishing a manifest file representing the encoding ladder for an internet stream”).
Regarding claim 2, Dahl discloses the method of Claim 1, wherein partially decoding the first video comprises:
deriving a first set of entropy characteristics from the first video, the first set of entropy characteristics representing visual activity between frames of the first video (see Dahl, paragraph [0019]: “the system can intelligently sample frames from the input video that are more representative of the visual and content characteristics of the video in order to improve the accuracy of subsequent feature extraction (e.g., by sampling from each identified scene in the input video according to scene detection algorithms)”);
selecting a subset of pixels, in frames in the first video, representing the first set of entropy characteristics (see Dahl, paragraph [0036]: “extract single frames distributed evenly in the input video in order to calculate visual complexity features and content features for the input video”);
assembling the subset of pixels into a series of proxy frames (see Dahl, paragraph [0036]: “extract a set of consecutive series of frames from the input video in order to calculate motion features for the input video”); and
assembling the subset of proxy frames into the proxy video representation (see Dahl, paragraph [0036]: “the system can sample sequences of frames from the input video that best represent the input video for the purpose of various metrics and/or models”).
Regarding claim 3, Dahl discloses the method of Claim 1:
further comprising accessing a first target viewing quality associated with the first publisher (see Dahl, paragraph [0048]: “the system can access audience viewing condition data”);
wherein accessing the first model comprises:
accessing the first model associated with the first resolution and configured to derive target bitrates based on the first target viewing quality (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”); and
wherein accessing the second model comprises:
accessing the second model associated with the second resolution and configured to derive target bitrates based on the first target viewing quality (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”).
Regarding claim 4, Dahl discloses the method of Claim 1:
further comprising:
for a third resolution in the set of resolutions:
accessing a third model associated with the third resolution, the third model configured to derive target bitrates based on the first target viewing quality (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
passing the proxy video representation to the third model (see Dahl, paragraph [0065]: “the system can select bitrate-resolution pairs from any of the estimated convex hulls corresponding to the various viewing conditions in the predicted audience of the input video”); and
receiving a third target bitrate for the third resolution returned by the third model (see Dahl, paragraph [0065]: “for each viewing condition in a set of viewing conditions, generate a viewing-condition-specific set of bitrate-resolution pairs based on the set of video features via a convex hull estimation model corresponding to the viewing condition”);
calculating a predicted viewing quality of a third rendition of the first video transcoded according to the third target bitrate and the third resolution (see Dahl, paragraph [0065]: “the system can select bitrate-resolution pairs from any of the estimated convex hulls corresponding to the various viewing conditions in the predicted audience of the input video”);
calculating a difference between the predicted viewing quality and the first target viewing quality (see Dahl, paragraph [0075]: “the system selects the top bitrate-resolution pair in the video-specific encoding ladder for the video in order to constrain the encoding space for the input video to bitrates that provide meaningful differences in video quality”); and
in response to the predicted viewing quality falling below the first target viewing quality and in response to the difference between the predicted viewing quality and the target viewing quality exceeding a threshold difference:
calculating a fourth bitrate, greater than the third bitrate, based on the difference between the predicted viewing quality and the first target viewing quality (see Dahl, paragraph [0075]: “the system selects the top bitrate-resolution pair in the video-specific encoding ladder for the video in order to constrain the encoding space for the input video to bitrates that provide meaningful differences in video quality”); and
defining a fourth rendition, for the first video, characterized by the third resolution and fourth target bitrate (see Dahl, paragraph [0054]: “The system can then evaluate the quality of the rendition according to the quality metric. Subsequently, the system can increase the bitrate and/or resolution and again evaluate the rendition according to the quality metric”); and
wherein generating the encoding ladder comprises generating the encoding ladder further identifying the fourth rendition (see Dahl, paragraph [0085]: “the system can estimate a number of renditions included in the video-specific encoding ladder from the estimated convex hull of the input video based on audience bandwidth data and/or audience viewing condition data”).
Regarding claim 6, Dahl discloses the method of Claim 1:
wherein receiving the first video comprises receiving the first video comprising a first mezzanine segment for a first video file (see Dahl, paragraph [0101]: “identifying a first consecutive subset of mezzanine segments in the set of mezzanine segments coinciding with the first playback interval in the audio-video file”);
wherein partially decoding the first video to generate the proxy video representation of the first video comprises partially decoding the first mezzanine segment to generate the proxy video representation of the first mezzanine segment (see Dahl, paragraph [0163]: “the computer system (e.g., a worker assigned to transcode the playback segment) includes a decoder and encoder that can transcode a mezzanine segment into a rendition segment in a requested rendition. For example, a mezzanine segment may be encoded using H.264 at 30 Mbps with AAC audio and 1280 by 720 pixel resolution and a playback segment may be requested in H.264 at 15 Mbps with AAC audio and 640 by 480 pixel resolution. In this case, the method S200 can include transcoding the mezzanine segment to a rendition segment using the H.264 codec”);
further comprising:
receiving a second video comprising a second mezzanine segment for the first video file (see Dahl, paragraph [0143]: “generates mezzanine segments that each include a segment of encoded audio data, a segment of encoded video data, a start time and duration and/or end time of the segment, and a sequence number of the segment such that each mezzanine segment is individually addressable and can be retrieved and transcoded individually from the mezzanine cache”);
partially decoding the second mezzanine segment to generate a second proxy video representation of the second mezzanine segment (see Dahl, paragraph [0163]: “the computer system (e.g., a worker assigned to transcode the playback segment) includes a decoder and encoder that can transcode a mezzanine segment into a rendition segment in a requested rendition. For example, a mezzanine segment may be encoded using H.264 at 30 Mbps with AAC audio and 1280 by 720 pixel resolution and a playback segment may be requested in H.264 at 15 Mbps with AAC audio and 640 by 480 pixel resolution. In this case, the method S200 can include transcoding the mezzanine segment to a rendition segment using the H.264 codec”);
for the first resolution in the set of resolutions:
passing the second proxy video representation to the first model (see Dahl, paragraph [0048]: “The system can also execute multiple convex hull estimation models, wherein each convex hull estimation model outputs estimated convex hulls that indicate quality-maximizing resolutions for the input video when encoded over a range of bitrates and viewed in a particular viewing condition”); and
receiving a third target bitrate for the first resolution returned by the first model (see Dahl, paragraph [0047]: “model outputs an accurate convex hull (e.g., a convex hull that actually represents the quality maximizing resolution for the input video over a series of bitrates)”);
for the second resolution in the set of resolutions:
passing the second proxy video representation to the second model (see Dahl, paragraph [0065]: “the system can select bitrate-resolution pairs from any of the estimated convex hulls corresponding to the various viewing conditions in the predicted audience of the input video”); and
receiving a fourth target bitrate for the second resolution returned by the second model (see Dahl, paragraph [0065]: “for each viewing condition in a set of viewing conditions, generate a viewing-condition-specific set of bitrate-resolution pairs based on the set of video features via a convex hull estimation model corresponding to the viewing condition”);
defining a third rendition of the second video based on the first resolution and the third target bitrate (see Dahl, paragraph [0054]: “The system can then evaluate the quality of the rendition according to the quality metric. Subsequently, the system can increase the bitrate and/or resolution and again evaluate the rendition according to the quality metric”); and
defining a fourth rendition of the second video based on the second resolution and the fourth target bitrate (see Dahl, paragraph [0054]: “The system can then evaluate the quality of the rendition according to the quality metric. Subsequently, the system can increase the bitrate and/or resolution and again evaluate the rendition according to the quality metric”); and
wherein generating the encoding ladder comprises generating the encoding ladder identifying the third rendition and the fourth rendition (see Dahl, paragraph [0085]: “the system can estimate a number of renditions included in the video-specific encoding ladder from the estimated convex hull of the input video based on audience bandwidth data and/or audience viewing condition data”).
Regarding claim 7, Dahl discloses the method of Claim 6:
wherein receiving the first video comprises receiving the first video comprising the first mezzanine segment from a live video stream (see Dahl, paragraph [0101]: “identifying a first consecutive subset of mezzanine segments in the set of mezzanine segments coinciding with the first playback interval in the audio-video file”); and
wherein receiving the second video comprises receiving the second video comprising the second mezzanine segment from the live video stream (see Dahl, paragraph [0143]: “generates mezzanine segments that each include a segment of encoded audio data, a segment of encoded video data, a start time and duration and/or end time of the segment, and a sequence number of the segment such that each mezzanine segment is individually addressable and can be retrieved and transcoded individually from the mezzanine cache”).
Regarding claim 8, Dahl discloses the method of Claim 6, further comprising:
transcoding the first mezzanine segments into a first rendition segment in the first resolution and the first target bitrate (see Dahl, paragraph [0101]: “concurrently transcoding the mezzanine segment into a rendition segment in the first rendition and transmitting the rendition segment coinciding with the first playback interval to the first computational device via the first stream”) in response to receiving a first request for a first playback segment, corresponding to the first mezzanine segment, in the first rendition (see Dahl, paragraph [0105]: “When the computer system receives a request for a playback segment from an instance of an AV player, the computer system: maps the playback segment to coincident rendition segments; and identifies whether mezzanine segments corresponding to the coincident rendition segments were previously transcoded and stored in memory (e.g., in a database, a rendition cache) or are currently queued for transcoding in the requested rendition”); and
transcoding the second mezzanine segments into a second rendition segment in the first resolution and the third target bitrate (see Dahl, paragraph [0101]: “concurrently transcoding the mezzanine segment into a rendition segment in the first rendition and transmitting the rendition segment coinciding with the first playback interval to the first computational device via the first stream”) in response to receiving a second request for a second playback segment, corresponding to the second mezzanine segment, in the third rendition (see Dahl, paragraph [0105]: “When the computer system receives a request for a playback segment from an instance of an AV player, the computer system: maps the playback segment to coincident rendition segments; and identifies whether mezzanine segments corresponding to the coincident rendition segments were previously transcoded and stored in memory (e.g., in a database, a rendition cache) or are currently queued for transcoding in the requested rendition”).
Regarding claim 9, Dahl discloses the method of Claim 1, further comprising:
deriving a set of entropy characteristics from the proxy video representation, the set of entropy characteristics representing visual activity between frames for the first video (see Dahl, paragraph [0019]: “the system can intelligently sample frames from the input video that are more representative of the visual and content characteristics of the video in order to improve the accuracy of subsequent feature extraction (e.g., by sampling from each identified scene in the input video according to scene detection algorithms)”);
interpreting a visual complexity of the first video based on the set of entropy characteristics of the proxy video representation (see Dahl, paragraph [0037]: “the system can extract features representative of the visual complexity, motion, content, and/or any other characteristic of the input video based on a number of visual complexity, motion, and/or content specific metrics and/or models”); and
setting a count of resolutions, for the set of resolutions, proportional to the visual complexity of the first video (see Dahl, paragraph [0037]: “system can generate a video-level feature vector that acts as a representation of the input video. The system can then input this representation of the input video into the convex hull estimation model in order to estimate a set of bitrate-resolution pairs that are estimated to maximize the quality of the input video at each given bitrate without performing any trial encodes of the input video”).
Regarding claim 10, Dahl discloses the method of Claim 1, further comprising:
accessing a set of metadata for the first video (see Dahl, paragraph [0035]: “the system can extract metadata from the input video”);
extracting a set of nonvisual characteristics of the first video from the set of metadata (see Dahl, paragraph [0037]: “extracting … a set of content features”);
predicting a viewership count for the first video based on the set of nonvisual characteristics of the first video (see Dahl, paragraph [0212]: “predict increased viewer demand for a particular playback segment at a particular bitrate and resolution pair at a future time based on trends in viewership for the video and/or other similar videos”); and
setting a count of resolutions, for the set of resolutions, proportional to the viewership count (see Dahl, paragraph [0212]: “predict increased viewer demand for a particular playback segment at a particular bitrate and resolution pair at a future time based on trends in viewership for the video and/or other similar videos; and preemptively initiate re-transcoding of segments of the video in preparation for increased demand for the playback segments of the video at this bitrate and resolution”).
Regarding claim 11, Dahl discloses the method of Claim 1:
further comprising:
identifying a set of keyframes in the first video, the set of keyframes comprising a first keyframe and a second keyframe (see Dahl, paragraph [0167]: “for each transcoded audio-video segment in the stream of audio-video segments, responsive to identifying that the transcoded audio-video segment includes a segment timestamp between the first keyframe timestamp and the second keyframe timestamp, store the transcoded AV segment in the rendition cache as a rendition AV segment, wherein each segment corresponds to a range of bytes in the rendition segment”); and
for a set of frames between the first keyframe and the second keyframe (see Dahl, paragraph [0102]: “identifying a set of keyframe timestamps corresponding to keyframes in the audio-video file”):
deriving a set of entropy characteristics representing visual activity in frames in the set of frames (see Dahl, paragraph [0019]: “the system can intelligently sample frames from the input video that are more representative of the visual and content characteristics of the video in order to improve the accuracy of subsequent feature extraction (e.g., by sampling from each identified scene in the input video according to scene detection algorithms)”); and
selecting a set of pixels from the set of frames based on the set of entropy characteristics (see Dahl, paragraph [0036]: “extract single frames distributed evenly in the input video in order to calculate visual complexity features and content features for the input video”); and
wherein partially decoding the first video to generate the proxy video representation of the first video comprises generating the proxy video representation comprising the set of pixels (see Dahl, paragraph [0036]: “the system can sample sequences of frames from the input video that best represent the input video for the purpose of various metrics and/or models”).
Regarding claim 12, Dahl discloses the method of Claim 1, further comprising:
accessing a first set of metadata for the first video (see Dahl, paragraph [0035]: “the system can extract metadata from the input video”);
receiving a second video from the first publisher (see Dahl, claim 10, “ingesting a second video, the second video succeeding the video in a video stream”);
accessing a second set of metadata for the second video (see Dahl, paragraph [0035]: “the system can extract metadata from the input video”); and
in response to detecting correspondence between the first set of metadata and the second set of metadata:
generating a second encoding ladder for the second video, the third encoding ladder identifying the first rendition and the second rendition (see Dahl, paragraph [0030]: “select particular frames from the input video to extract features of the input video for analysis and to generate the video-specific encoding ladder for the input video”; and paragraph [0035]: “the system can extract metadata from the input video and correlate the metadata with the variability of the video”); and
publishing the second encoding ladder for access by video players for playback of the second video (see Dahl, paragraph [0099]: “publishing a manifest file representing the encoding ladder for an internet stream”).
Regarding claim 13, Dahl discloses the method of Claim 1, further comprising, in response to receiving a first request for a first playback segment of the first video in the first rendition from a video player in the set of video players:
initiating transcoding of a mezzanine segment, in a set of mezzanine segments of the first video, into the first playback segment in the first rendition by a first worker (see Dahl, paragraph [0101]: “assigning a first worker to transcode the first playback segment in the first rendition in Block S230; and initiating a first stream between the first worker and the first computational device in Block S240. The method S200 further includes, at the first worker: identifying a first consecutive subset of mezzanine segments in the set of mezzanine segments coinciding with the first playback interval in the audio-video file in Block S250”);
releasing the first playback segment from the first worker for distribution to the video player (see Dahl, paragraph [0101]: “The method additionally includes, for each mezzanine segment in the consecutive subset of mezzanine segments: concurrently transcoding the mezzanine segment into a rendition segment in the first rendition and transmitting the rendition segment coinciding with the first playback interval to the first computational device via the first stream in Block S260”); and
storing the first playback segment in the first rendition in a rendition cache (see Dahl, paragraph [0167]: “store the transcoded AV segment in the rendition cache as a rendition AV segment, wherein each segment corresponds to a range of bytes in the rendition segment”).
Regarding claim 14, Dahl discloses the method of Claim 1, further comprising:
receiving a second video from a second publisher, the second video characterized by a third file size and a second source resolution (see Dahl, paragraph [0026]: “publish an internet video stream (e.g., by generating an HLS manifest file specifying available renditions of the input video) with a video-specific encoding ladder for an input video that is predicted to maximize quality at any of the bitrates included in the video-specific encoding ladder without performing additional encodes to determine the quality of the video at various bitrates and resolutions”);
partially decoding the second video to generate a second proxy video representation of the second video, the second proxy video representation characterized by a fourth file size less than the third file size (see Dahl, paragraph [0111]: “the computer system can produce video thumbnails or thumbnail images by selectively decoding AV segments that contain the video frames that include the thumbnail. For example, a video thumbnail can be displayed shortly after publication of the video by specifying a time interval for the video thumbnail and selectively transcoding the segments corresponding to the video thumbnail in each rendition offered by the computer system immediately after publication”);
accessing a set of historic viewership characteristics for videos published by the second publisher (see Dahl, paragraph [0069]: “the system can: generate a video-level feature vector for the set of historical videos”);
predicting a set of viewer characteristics of the second video based on the set of historic viewership characteristics (see Dahl, paragraph [0070]: “the system can also estimate, based on device audience data from historical internet stream of historical video, the viewing condition of each viewer of these historical videos”);
deriving a set of target resolutions based on the set of viewer characteristics and the set of resolutions (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
for a third resolution in the set of target resolutions:
accessing a third model associated with the third resolution, the third model configured to derive target bitrates based on target viewing qualities (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
passing the proxy video representation to the third model (see Dahl, paragraph [0048]: “The system can also execute multiple convex hull estimation models, wherein each convex hull estimation model outputs estimated convex hulls that indicate quality-maximizing resolutions for the input video when encoded over a range of bitrates and viewed in a particular viewing condition”); and
receiving a third target bitrate for the third resolution returned by the third model (see Dahl, paragraph [0047]: “model outputs an accurate convex hull (e.g., a convex hull that actually represents the quality maximizing resolution for the input video over a series of bitrates)”);
defining a third rendition for the second video characterized by the third resolution and the third target bitrate (see Dahl, paragraph [0054]: “To initiate calculation of a convex hull of a training video, the system can encode an initial rendition of the training video at a low bitrate as a first step in the trial encoding process (e.g., 200 kbps and 180p)”);
generating a second encoding ladder identifying the third rendition for the second video (see Dahl, paragraph [0099]: “Upon selecting a set of bitrate-resolution pairs for the video-specific encoding ladder of an input video, the system can generate an encoding ladder for the video segment including the top bitrate-resolution pair, the bottom bitrate-resolution pair, and/or the subset of bitrate-resolution pairs”); and
publishing the second encoding ladder for access by the set of video players for playback of the second video (see Dahl, paragraph [0099]: “publishing a manifest file representing the encoding ladder for an internet stream”).
Claim 16 is rejected for the same reasons as discussed in claim 1 above. In addition, Dahl also discloses deriving a first set of entropy characteristics from the first video, the first set of entropy characteristics representing visual activity in frames of the first video (see Dahl, paragraph [0017]: “generate a video-specific encoding ladder (e.g., a manifest file, such as an HLS manifest) specific to an input video that improves video quality (e.g., compared to a fixed bitrate ladder) over a range of bitrates and resolutions based on visual-, motion-, and content-related features of the video”);
partially decoding the first video according to the first set of entropy characteristics (see Dahl, paragraph [0019]: “the system can intelligently sample frames from the input video that are more representative of the visual and content characteristics of the video in order to improve the accuracy of subsequent feature extraction”);
transcoding a first video segment of the first video into a first rendition segment, in the first rendition (see Dahl, paragraph [0101]: “transcode the first playback segment in the first rendition”);
transcoding the first video segment of the first video into a second rendition segment, in the second rendition (see Dahl, paragraph [0103]: “accessing the set of mezzanine segments from the mezzanine cache; and identifying a first consecutive subset of mezzanine segments in the set of mezzanine segments coinciding with the first playback interval in the livestream”); and
publishing the first rendition segment and the second rendition segment for access by video players to stream rendition segments of the first video (see Dahl, paragraph [0106]: “compile a full set of transcoded rendition segments for the AV file, wherein each rendition segment is transcoded in (near) real-time following a first request for this rendition segment from an AV player instance after—rather than before—the AV file is published for streaming”).
Regarding claim 17, Dahl discloses the method of Claim 16:
further comprising:
accessing a target viewing quality associated with the first publisher (see Dahl, paragraph [0048]: “the system can access audience viewing condition data”);
for a third resolution in the set of resolutions, during a first time period:
accessing a third model associated with the third resolution, the third model configured to derive target bitrates based on the target viewing quality (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
passing the proxy video representation to the third model (see Dahl, paragraph [0065]: “the system can select bitrate-resolution pairs from any of the estimated convex hulls corresponding to the various viewing conditions in the predicted audience of the input video”); and
receiving a third target bitrate for the third resolution returned by the third model (see Dahl, paragraph [0065]: “for each viewing condition in a set of viewing conditions, generate a viewing-condition-specific set of bitrate-resolution pairs based on the set of video features via a convex hull estimation model corresponding to the viewing condition”);
calculating a predicted viewing quality of a third rendition characterized by the third target bitrate and the third resolution (see Dahl, paragraph [0065]: “the system can select bitrate-resolution pairs from any of the estimated convex hulls corresponding to the various viewing conditions in the predicted audience of the input video”);
calculating a difference between the predicted viewing quality and the target viewing quality (see Dahl, paragraph [0075]: “the system selects the top bitrate-resolution pair in the video-specific encoding ladder for the video in order to constrain the encoding space for the input video to bitrates that provide meaningful differences in video quality”);
in response to the predicted viewing quality falling below the target viewing quality and in response to the difference between the predicted viewing quality and the target viewing quality exceeding a threshold difference:
calculating a fourth bitrate, greater than the third bitrate, based on the difference between the predicted viewing quality and the target viewing quality (see Dahl, paragraph [0075]: “the system selects the top bitrate-resolution pair in the video-specific encoding ladder for the video in order to constrain the encoding space for the input video to bitrates that provide meaningful differences in video quality”); and
defining a fourth rendition, for the first video, characterized by the third resolution and fourth target bitrate (see Dahl, paragraph [0054]: “The system can then evaluate the quality of the rendition according to the quality metric. Subsequently, the system can increase the bitrate and/or resolution and again evaluate the rendition according to the quality metric”); and
wherein generating the encoding ladder comprising generating the encoding ladder further identifying the fourth rendition (see Dahl, paragraph [0085]: “the system can estimate a number of renditions included in the video-specific encoding ladder from the estimated convex hull of the input video based on audience bandwidth data and/or audience viewing condition data”).
Regarding claim 18, Dahl discloses the method of Claim 16:
further comprising:
identifying a set of keyframes in the first video, the set of keyframes comprising a first keyframe and a second keyframe (see Dahl, paragraph [0167]: “for each transcoded audio-video segment in the stream of audio-video segments, responsive to identifying that the transcoded audio-video segment includes a segment timestamp between the first keyframe timestamp and the second keyframe timestamp, store the transcoded AV segment in the rendition cache as a rendition AV segment, wherein each segment corresponds to a range of bytes in the rendition segment”); and
for a set of frames between the first keyframe and the second keyframe, selecting a set of pixels from the set of frames based on the set of entropy characteristics (see Dahl, paragraph [0036]: “extract single frames distributed evenly in the input video in order to calculate visual complexity features and content features for the input video”); and
wherein partially decoding the first video to generate the proxy video representation of the first video comprises generating the proxy video representation comprising the set of pixels (see Dahl, paragraph [0036]: “the system can sample sequences of frames from the input video that best represent the input video for the purpose of various metrics and/or models”).
Regarding claim 19, Dahl discloses the method of Claim 16, further comprising:
during a first time period:
accessing a set of historic viewership characteristics for videos published by the first publisher (see Dahl, paragraph [0069]: “the system can: generate a video-level feature vector for the set of historical videos”);
predicting a first set of viewer characteristics based on the set of historic viewership characteristics (see Dahl, paragraph [0070]: “the system can also estimate, based on device audience data from historical internet stream of historical video, the viewing condition of each viewer of these historical videos”); and
deriving a target viewership quality based on the first set of viewer characteristics (see Dahl, paragraph [0066]: “the system can: access historical audience data for a set of similar videos and/or currently available audience data for the input video itself and predict a distribution of audience bandwidths representing likely viewers of the input video. Thus, the system can estimate the effect of each bitrate-resolution pair included in the estimated convex hull of the input video on the aggregate viewing quality for viewers of the input video by multiplying the quality score corresponding to each bitrate-resolution pair by the number of viewers in a segment of the distribution of audience bandwidths that are predicted to view the bitrate-resolution pair; and
during a second time period:
receiving a second set of viewer characteristics from video players streaming playback segments of the first video, the second set of viewer characteristics representing characteristics of playback requests received from video players for the first video (see Dahl, paragraph [0105]: “When the computer system receives a request for a playback segment from an instance of an AV player, the computer system: maps the playback segment to coincident rendition segments; and identifies whether mezzanine segments corresponding to the coincident rendition segments were previously transcoded and stored in memory (e.g., in a database, a rendition cache) or are currently queued for transcoding in the requested rendition”);
calculating a first deviation between the second set of viewer characteristics and the first set of viewer characteristics (see Dahl, paragraph [0075]: “the system selects the top bitrate-resolution pair in the video-specific encoding ladder for the video in order to constrain the encoding space for the input video to bitrates that provide meaningful differences in video quality”);
in response to the first deviation between the second set of viewer characteristics and the first set of viewer characteristics exceeding a threshold deviation:
for the first resolution in the set of resolutions:
accessing a third model associated with the first resolution, the first model configured to derive target bitrates based on target viewer characteristics (see Dahl, paragraph [0048]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
passing the proxy video representation and the second set of viewer characteristics to the third model (see Dahl, paragraph [0048]: “The system can also execute multiple convex hull estimation models, wherein each convex hull estimation model outputs estimated convex hulls that indicate quality-maximizing resolutions for the input video when encoded over a range of bitrates and viewed in a particular viewing condition”); and
receiving a third target bitrate for the first resolution returned by the third model (see Dahl, paragraph [0047]: “model outputs an accurate convex hull (e.g., a convex hull that actually represents the quality maximizing resolution for the input video over a series of bitrates)”); and
for the second resolution in the set of resolutions:
accessing a fourth model associated with the second resolution, the fourth model configured to derive target bitrates based on target viewer characteristics (see Dahl, paragraph [0065]: “the system can access audience viewing condition data and better select bitrate-resolution pairs for a video-specific encoding ladder of the input video that improve the viewing quality for the most viewers across multiple different viewing conditions”);
passing the proxy video representation and the second set of viewer characteristics to the fourth model (see Dahl, paragraph [0065]: “the system can select bitrate-resolution pairs from any of the estimated convex hulls corresponding to the various viewing conditions in the predicted audience of the input video”); and
receiving a fourth target bitrate for the second resolution based on the fourth model (see Dahl, paragraph [0065]: “for each viewing condition in a set of viewing conditions, generate a viewing-condition-specific set of bitrate-resolution pairs based on the set of video features via a convex hull estimation model corresponding to the viewing condition”);
defining a third rendition, for the first video, characterized by the first resolution and the third target bitrate (see Dahl, paragraph [0054]: “To initiate calculation of a convex hull of a training video, the system can encode an initial rendition of the training video at a low bitrate as a first step in the trial encoding process (e.g., 200 kbps and 180p)”);
defining a fourth rendition, for the first video, characterized by the second resolution and the fourth target bitrate (see Dahl, paragraph [0054]: “To initiate calculation of a convex hull of a training video, the system can encode an initial rendition of the training video at a low bitrate as a first step in the trial encoding process (e.g., 200 kbps and 180p)”);
replacing the first rendition with the third rendition in the encoding ladder (see Dahl, paragraph [0099]: “Upon selecting a set of bitrate-resolution pairs for the video-specific encoding ladder of an input video, the system can generate an encoding ladder for the video segment including the top bitrate-resolution pair, the bottom bitrate-resolution pair, and/or the subset of bitrate-resolution pairs”);
replacing the second rendition with the fourth rendition in the encoding ladder (see Dahl, paragraph [0099]: “Upon selecting a set of bitrate-resolution pairs for the video-specific encoding ladder of an input video, the system can generate an encoding ladder for the video segment including the top bitrate-resolution pair, the bottom bitrate-resolution pair, and/or the subset of bitrate-resolution pairs”);
transcoding a first video segment of the first video into a third rendition segment in the third rendition (see Dahl, paragraph [0101]: “transcode the first playback segment in the first rendition”);
transcoding the first video segment of the first video into a fourth rendition segment in the fourth rendition (see Dahl, paragraph [0101]: “transcode the first playback segment in the first rendition”); and
publishing the third rendition segment and the fourth rendition segment for access by video players to stream rendition segments of the first video (see Dahl, paragraph [0106]: “compile a full set of transcoded rendition segments for the AV file, wherein each rendition segment is transcoded in (near) real-time following a first request for this rendition segment from an AV player instance after—rather than before—the AV file is published for streaming”).
Claim 20 is rejected for the same reasons as discussed in claim 1 above. In addition, Dahl also discloses partially decoding the first video, based on visual characteristics in the first video (see Dahl, paragraph [0019]: “the system can intelligently sample frames from the input video that are more representative of the visual and content characteristics of the video in order to improve the accuracy of subsequent feature extraction”),
accessing a set of historic viewership characteristics for videos published by the first publisher (see Dahl, paragraph [0069]: “the system can: generate a video-level feature vector for the set of historical videos”);
predicting a set of viewer characteristics based on the set of historic viewership characteristics (see Dahl, paragraph [0070]: “the system can also estimate, based on device audience data from historical internet stream of historical video, the viewing condition of each viewer of these historical videos”); and
deriving a target viewership quality based on the set of viewer characteristics (see Dahl, paragraph [0066]: “the system can: access historical audience data for a set of similar videos and/or currently available audience data for the input video itself and predict a distribution of audience bandwidths representing likely viewers of the input video. Thus, the system can estimate the effect of each bitrate-resolution pair included in the estimated convex hull of the input video on the aggregate viewing quality for viewers of the input video by multiplying the quality score corresponding to each bitrate-resolution pair by the number of viewers in a segment of the distribution of audience bandwidths that are predicted to view the bitrate-resolution pair.
Allowable Subject Matter
Claims 5 and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIENRU YANG whose telephone number is (571)272-4212. The examiner can normally be reached Monday-Friday 10AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI TRAN can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NIENRU YANG
Examiner
Art Unit 2484
/NIENRU YANG/Examiner, Art Unit 2484
/THAI Q TRAN/Supervisory Patent Examiner, Art Unit 2484
Prosecution Insights