DETAILED ACTIONNotice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant Response to Official Action
The response filed on 10/1/2025 has been entered and made of record.
Acknowledgment
Claims 1, 4, 9, 11, 13-14, 16, and 18-20, amended on 10/1/2025, are acknowledged by the examiner.
Response to Arguments
Applicant’s arguments with respect to claims 1, 16, 18, and their dependent claims have been considered but they are moot in view of the new grounds of rejection necessitated by amendments initiated by the applicant. Examiner addresses the main arguments of the Applicant as below.
Regarding the 35 U.S.C. 112(a) rejection, the amendment filed on 10/1/2025 addresses few issues. As a result, the 35 U.S.C. 112(a) rejection for claims 16 and 18-20 is withdrawn.
Regarding the 35 U.S.C. 112(b) rejection, the amendment filed on 10/1/2025 addresses few issues. As a result, the 35 U.S.C. 112(b) rejection for claims 16 and 18-20 is withdrawn.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) ELEMENT IN CLAIM FOR A COMBINATION.—An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as "configured to" or "so that"; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: a stall analysis function in claim 4, an origin agent in claims 8-9, a live encoder in claim 11, a mixed-integer linear programming (MILP) model in claim 14.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejection – 35 U.S.C. § 112
The following is a quotation of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by the inventor of carrying out his invention.
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of pre-AIA 35 U.S.C. 112, second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4, 8-9, 11, and 14 are rejected under 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph, as failing to comply with the written description requirement. The claims contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention. After reviewing the specification there appears to be no corresponding structure for the stall analysis function, the origin agent, the live encoder, the mixed-integer linear programming (MILP) model, and the distributed database. An amendment that includes a memory comprising program instructions executable by a processor to cause the processor to implement these functions would address the rejections.
Claims 4, 8-9, 11, and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matters, which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. It is not clear what is a corresponding structure for the stall analysis function, the origin agent, the live encoder, the mixed-integer linear programming (MILP) model, and the distributed database. Therefore, the claims are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102(e), (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 1-13 and 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Lobanov (US Patent Application Publication US 2021/0176530 A1), (“Lobanov”), in view of Amirpour et al. (US Patent US 12,108,055 B2), (“Amirpour”), in view of Mani et al. (US Patent US 10,587,523 B2), (“Mani”).
Regarding claim 1, Lobanov meets the claim limitations as follow.
A method for optimizing a bitrate ladder for live streaming (a method and apparatus for estimating bitrate) [Lobanov: para. 0002], the method comprising (a method) [Lobanov: para. 0002]:receiving a client-side input (The neural network may receive a training data set comprised of a plurality of CDN server logs. The CDN server logs may be associated with known QoE metrics collected from a plurality of user devices) [Lobanov: para. 0002] and an origin-side input ((receiving Quality of Experience (QoE) metric information) [Lobanov: Claim 1]; (actual values of the QoE metrics 404 collected from user device for the same N minute interval) [Lobanov: 0032; Fig. 4]) during a first interval in a timeslot ((activity during an N minute interval) [Lobanov: 0031; Figs. 6-9]; (a timestamp of the first record) [Lobanov: 0044]), the client-side input comprising a content delivery network (CDN) log from a client (A CDN log segment 302 representing user device activity during an N minute interval) [Lobanov: 0031], the origin-side input comprising a quality measure from an origin server ((actual values of bitrate, buffering occurrences and buffering duration QoE metrics from a user device corresponding to each CDN log segment. Bitrate may be measured using a network analyzer) [Lobanov: 0054]; (an origin server) [Lobanov: para. 0017; Fig. 1]), wherein the timeslot comprises a duration selected to (Estimates may be made every N minutes using N minute segments in the CDN log. The CDN log may cover a long interval, for example, an hour, several hours or a day. Alternatively, it may be more interesting to estimate buffering events and other QoE metrics in short intervals, for example N minute intervals. Long interval and short interval estimates may be combined so as to provide estimate trends of QoE metrics over time. That combination estimate may provide invaluable information for further analyses by OTT providers. It allows correlation of low QoE with time of day, recognizing isolated events from consistently low QoE etc.) [Lobanov: 0037] avoid a late reaction to network bandwidth fluctuation on a client side;during the first interval (during an N minute interval) [Lobanov: 0031; Figs. 6-9], extracting from the CDN log (filter the CDN log and to extract only records related to) [Lobanov: 0062] a frequency of requests ((all requests in an N minute interval) [Lobanov: 0041]; (requests from the device for some interval or repeated requests for the same video segments) [Lobanov: 0029]) for each bitrate in a bitrate ladder in the timeslot (bitrate for an N minute interval) [Lobanov: 0049] and a duration of a recent stall event for the client's player ((duration of video freeze during an analysis interval) [Lobanov: 0030]; (pause/stop modes and can send player status back as telemetry for analysis. Adding start, stop, and pause telemetry to input data of the neural network model helps it to distinguish player inactivity from network communication issues and therefore improves estimation accuracy) [Lobanov: 0052]; (the plurality of CDN server logs are enhanced with play, stop or pause statuses of the user devices) [Lobanov: Claim 9]); selecting (Following user selection) [Lobanov: 0066], during a second interval in the timeslot ((CDN log segments 402 each covering an N minute interval) [Lobanov: 0032; Fig. 4]; (In an embodiment, a 5 minute interval may be used. Other intervals may also be used) [Lobanov: 0031]), an optimized bitrate ladder comprising an optimal set of bitrates (OSB) for a representation in a manifest using an optimization function (A player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer. If it is too low, or empties, the picture and sound may freeze and break up. However, compliance with DASH and HLS allows the player to detect low buffer levels and calculate the bandwidth available and switch to the highest stream available to keep the buffer full) [Lobanov: 0015], the optimization function (Video players use sophisticated buffer management algorithms to keep a buffer filled at an optimal level) [Lobanov: 0029] taking as input the quality measure (Video profile selection and video profile switching events may be extracted from a CDN log. Requests for video segment(s) directly identify a video profile of requested segment) [Lobanov: 0029] and a coefficient value (average bitrate QoE metrics) [Lobanov: 0002] determined using the frequency of requests ((all requests in an N minute interval) [Lobanov: 0041]; (requests from the device for some interval or repeated requests for the same video segments) [Lobanov: 0029]) and the duration of the recent stall event (average bitrate, number of buffering events and duration of video freeze during an analysis interval) [Lobanov: 0030]; and sending the optimized bitrate ladder to the origin server (The model processes input data and produces an estimate of QoE metrics 308 every N minutes. Output of the model is sent to a system that monitors quality of viewer experience) [Lobanov: 0036; Fig. 3] for live encoding a next segment (In an embodiment, video/audio distribution may include live video and audio. For example, video and audio signals of a live cloud based game. In this embodiment, gameplay based training data may be relied upon in addition to other parameters and metrics disclosed herein. For example, player control input data ( or a delta thereof) may be used as a metric for ML and for metric estimation. A user device used for gaming purposes may include a virtual reality or augmented reality headset which may access the Internet via wireless or wired methods.) [Lobanov: 0070].
In the same field of endeavor, Amirpour further discloses the claim limitations as follows:
a set of bitrates (Per-title encoding of a segment of the plurality of segments may be performed in step 1306, which may result in construction of a bitrate ladder) [Amirpour: col. 11, line 13-15; Figs. 2-13]
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov with Amirpour to program the system to implement of Amirpour’s method.
Therefore, the combination of Lobanov with Amirpour will enable the system to improve the bitrate ladders, which results in suboptimal video delivery for the end-users [Amirpour: col. 5, line 12-22].
Lobanov and Amirpour do not explicitly disclose the following claim limitations (Emphasis added).
avoid a late reaction to network bandwidth fluctuation on a client side.
However, in the same field of endeavor Mani further discloses the deficient claim limitations as follows:
avoid a late reaction to network bandwidth fluctuation on a client side (The probing time interval for subsequent probes may be increased every time significant bandwidth degradation is detected. As sending probe packets requires network bandwidth, the quality of the media stream may be affected each time probing is performed (even though it can help mitigate packet loss). Thus by increasing the time between probing intervals when bandwidth degradation is detected, fluctuations in the quality of the media stream may be reduced) [Mani: col. 12, line 33-41].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov and Amirpour with Mani to program the system to implement of Mani’s method.
Therefore, the combination of Lobanov and Amirpour with Mani will enable to reduce the fluctuations in the quality of the media stream [Mani: col. 12, line 13-15].
Regarding claim 2, Lobanov meets the claim limitations as set forth in claim 1.Lobanov further meets the claim limitations as follow.
selecting the coefficient value (data-rate video stream (video profile) is selected) [Lobanov: para. 0018] based on an average difference of quality (average bitrate QoE metrics) [Lobanov: 0002] and an average difference of bitrate (average bitrate) [Lobanov: 0030].
Regarding claim 3, Lobanov meets the claim limitations as set forth in claim 2.Lobanov further meets the claim limitations as follow.
wherein the coefficient value is selected to decrease one or both of (the player to detect low buffer levels and calculate the bandwidth available and switch to) [Lobanov: para. 0015] the average difference of quality (average bitrate QoE metrics) [Lobanov: 0002] and the average difference of bitrate (average bitrate) [Lobanov: 0030].
In the same field of endeavor, Amirpour further discloses the claim limitations as follows:
wherein the coefficient value is selected to decrease one or both of the average difference of quality and the average difference of bitrate ((DNN compression techniques, and other DNN compression techniques) to reduce the required bitstream for delivery of the associated DNN) [Amirpour: col. 5, line 46-49]; (As described herein, SPTE may include constructing bitrate ladder 108 as a base layer, training an enhancement layer 109 to generate an associated DNN for each representation, using DNN compression to reduce the bitstream for delivery of the associated DNN(s). In some examples, a scene-cut detection algorithm may be used to reduce the number) [Amirpour: col. 6, line 3-9]; (To reduce the bitrate overhead for streaming content-aware video super-resolution DNN, a context-adaptive binary arithmetic coding for DNN compression (e.g., DeepCABAC) may be used. DeepCABAC is known to reach up to 63x compression ratio of a DNN with no accuracy loss. Experimental results show about 40% bitrate reduction for GPU-available end-users, while provides the video content for CPU-only users as per usual) [Amirpour: col. 5, line 54-61]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov with Amirpour to program the system to implement of Amirpour’s method.
Therefore, the combination of Lobanov with Amirpour will enable the system to improve the bitrate ladders, which results in suboptimal video delivery for the end-users [Amirpour: col. 5, line 12-22].
Regarding claim 4, Lobanov meets the claim limitations as set forth in claim 1.Lobanov further meets the claim limitations as follow.
determining the coefficient value (determining an actual quality of viewer experience) [Lobanov: para. 0018] using a stall analysis function capable of (pause/stop modes and can send player status back as telemetry for analysis. Adding start, stop, and pause telemetry to input data of the neural network model helps it to distinguish player inactivity from network communication issues and therefore improves estimation accuracy) [Lobanov: 0052]; (the plurality of CDN server logs are enhanced with play, stop or pause statuses of the user devices) [Lobanov: Claim 9] determining the coefficient value (determining an actual quality of viewer experience) [Lobanov: para. 0018] and a binary variable based on a threshold mean stall duration.
Lobanov does not explicitly disclose the following claim limitations (Emphasis added).
a binary variable based on a threshold mean stall duration.
However, in the same field of endeavor Amirpour further discloses the deficient claim limitations as follows:
a binary variable (To reduce the bitrate overhead for streaming content-aware video super-resolution DNN, a context-adaptive binary arithmetic coding for DNN compression (e.g., DeepCABAC) may be used) [Amirpour: col. 5, line 54-61] based on a threshold mean stall duration (While specific examples have been provided above, it is understood that the present invention can be applied with a wide variety of inputs, thresholds, ranges, and other factors) [Amirpour: col. 12, line 53-55].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov with Amirpour to program the system to implement of Amirpour’s method.
Therefore, the combination of Lobanov with Amirpour will enable the system to improve the bitrate ladders, which results in suboptimal video delivery for the end-users [Amirpour: col. 5, line 12-22].
Regarding claim 5, Lobanov meets the claim limitations as set forth in claim 3.Lobanov further meets the claim limitations as follow.
wherein the OSB comprises a new OSB (A player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer) [Lobanov: 0015] when the binary variable comprises a True value ((Binary data may first be encoded as 0 and 1) [Lobanov: para. 0024]; (A player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer. If it is too low, or empties, the picture and sound may freeze and break up. However, compliance with DASH and HLS allows the player to detect low buffer levels and calculate the bandwidth available and switch to the highest stream available to keep the buffer full) [Lobanov: 0015]).
Regarding claim 6, Lobanov meets the claim limitations as set forth in claim 3.Lobanov further meets the claim limitations as follow.
wherein the OSB comprises a previously selected OSB (A player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer) [Lobanov: 0015] when the binary variable comprises a False value ((Binary data may first be encoded as 0 and 1) [Lobanov: para. 0024]; (if a stationary device, such as a Smart TV, experiences network congestion, the data delivered may not reach the device in time and buffer underruns may occur, resulting in the infamous "buffering, please wait" icon, thus negatively affecting the viewer experience) [Lobanov: para. 0014]).
Regarding claim 7, Lobanov meets the claim limitations as set forth in claim 1.Lobanov further meets the claim limitations as follow.
wherein the CDN log comprises a URL of a HTTP request message (When the App connects to a CDN HTTP server and provides a request for a list of available movies, called a master playlist, the App appends the ID to the server Uniform Resource Locater (URL), used as an address of information, which could be server_name, file name or the like. A URL may look like http://server_name/master. m3u8?uid=xxx. The added ID is appended following the ?uid=) [Lobanov: 0015], the duration of the recent stall event (duration of video freeze during an analysis interval) [Lobanov: 0030] included in the URL in common media client data (CMCD) format (Referring to FIG. 8, the master play list 800 is in the form of an M3U computer file which provides information of a program ID having different available bandwidth streams. At the first line in the file, line 802, #EXTM3U designates that the file 800 is an M3U file. At line 804, #EXT-X-STREAM-INF specifies parameter values which are comma delimited, including a program ID 822 having a value of 1 and a bandwidth value 834 set to 2000000. Line 806 indicates a URL of the media stream identified by line 804. The URL identifies UID 846. Line 808 specifies an #EXT-X-I-FRAME-STREAM-INF which identifies an I-frame file having the same program ID 824 and bandwidth 836 identified in line 804. Line 808 identifies the UID 848. At line 810, #EXT-X-STREAM-INF specifies a bandwidth value 838 of 1500000 for the same program ID 1 826. Line 812 specifies a URL of a media stream having a bandwidth as indicated by line 810. Line 814 specifies an #EXT-X-IFRAME-STREAM-INF having the same program ID 828, a bandwidth 840 of 1500000 and UID 850. Line 816 specifies an #EXT-X-STREAM-INF having a same program ID 830 and lower bandwidth 842 of 500000. Line 818 indicates a URL specifying UID 852. Line 820 specifies the #EXT-XI-FRAME-STREAM-INF for the same program ID 832, bandwidth 844 of 500000 and UID 854. The UID 846-854 is provided to the server each time a URL is requested by a user device.) [Lobanov: 0067; Figs 8-9].
Regarding claim 8, Lobanov meets the claim limitations as set forth in claim 3.Lobanov further meets the claim limitations as follow.
wherein the origin server (an origin server) [Lobanov: para. 0017; Fig. 1] comprises an origin agent ((origin services) [Lobanov: Fig. 1]; (AI model) [Lobanov: para. 0002; Figs. 1-6]) and the quality measure (bitrate QoE metrics) [Lobanov: 0002] comprises a measure of quality of a previously encoded segment by the origin server's live encoder ((The model processes input data and produces an estimate of QoE metrics 308 every N minutes. Output of the model is sent to a system that monitors quality of viewer experience) [Lobanov: 0036; Fig. 3]; (A player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer) [Lobanov: 0015]).
Regarding claim 9, Lobanov meets the claim limitations as set forth in claim 7.Lobanov further meets the claim limitations as follow.
wherein the origin agent ((origin services) [Lobanov: Fig. 1]; (AI model) [Lobanov: para. 0002; Figs. 1-6]) is deployed as a plug-in at the origin server and configured to measure (A method and apparatus for estimating bitrate, buffering events and/or other Quality of Experience (QoE) metrics of video reception, based on content distribution network (CDN) logs, are provided. The method and apparatus may rely on an artificial intelligence (AI) model, including a neural network. The neural network may receive a training data set comprised of a plurality of CDN server logs. The CDN server logs may be associated with known QoE metrics collected from a plurality of user devices. Once the neural network is trained, a CDN server log, without associated QoE metrics, may be received as input. With the trained neural network and the CDN server log, buffering events and average bitrate QoE metrics may be estimated for one or more user devices without explicitly receiving QoE metrics from the user device) [Lobanov: para. 0002; Figs. 1-6] – Note: AI model can be a plug-in to the server) perceptual quality.
Lobanov does not explicitly disclose the following claim limitations (Emphasis added).
measure perceptual quality.
However, in the same field of endeavor Amirpour further discloses the deficient claim limitations as follows:
perceptual quality (Different video contents require content-fit bitrate ladders to reach a certain perceived video quality. For a video sequence that is an "easy to encode" video and perceptually lossless at higher bitrates (e.g., greater than dB PSNR), selecting a high bitrate by using a fixed bitrate ladder will result in bitrate wastage. On the other hand, for a video sequence that is a "hard to encode" video, a high bitrate is preferable to reach an acceptable quality) [Lobanov: para. 0017; Fig. 1].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov with Amirpour to program the system to implement of Amirpour’s method.
Therefore, the combination of Lobanov with Amirpour will enable the system to improve the bitrate ladders, which results in suboptimal video delivery for the end-users [Amirpour: col. 5, line 12-22].
Regarding claim 10, Lobanov meets the claim limitations as set forth in claim 1.Lobanov further meets the claim limitations as follow.
wherein the quality measure (The model processes input data and produces an estimate of QoE metrics 308 every N minutes. Output of the model is sent to a system that monitors quality of viewer experience) [Lobanov: 0036; Fig. 3] comprises one or both of a video multi-method assessment fusion (VMAF) and peak signal-to-noise ratio (PSNR).
Lobanov does not explicitly disclose the following claim limitations (Emphasis added).
comprises one or both of a video multi-method assessment fusion (VMAF) and peak signal-to-noise ratio (PSNR).
However, in the same field of endeavor Amirpour further discloses the deficient claim limitations as follows:
comprises one or both of a video multi-method assessment fusion (VMAF) and peak signal-to-noise ratio (PSNR) (Different video contents require content-fit bitrate ladders to reach a certain perceived video quality. For a video sequence that is an "easy to encode" video and perceptually lossless at higher bitrates (e.g., greater than dB PSNR), selecting a high bitrate by using a fixed bitrate ladder will result in bitrate wastage. On the other hand, for a video sequence that is a "hard to encode" video, a high bitrate is preferable to reach an acceptable quality) [Lobanov: para. 0017; Fig. 1].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov with Amirpour to program the system to implement of Amirpour’s method.
Therefore, the combination of Lobanov with Amirpour will enable the system to improve the bitrate ladders, which results in suboptimal video delivery for the end-users [Amirpour: col. 5, line 12-22].
Regarding claim 11, Lobanov meets the claim limitations as set forth in claim 1.Lobanov further meets the claim limitations as follow.
wherein the origin server (an origin server) [Lobanov: para. 0017; Fig. 1] comprises a live encoder (As shown in FIG. 1, video content 116 gets encoded 118, segmented and packaged 120 and encrypted at or by a content preparation center 108. Prepared content is then placed in an origin server 122) [Lobanov: para. 0017; Fig. 1] and is configured to perform the live encoding of the next segment (In an embodiment, video/audio distribution may include live video and audio. For example, video and audio signals of a live cloud based game. In this embodiment, gameplay based training data may be relied upon in addition to other parameters and metrics disclosed herein. For example, player control input data ( or a delta thereof) may be used as a metric for ML and for metric estimation. A user device used for gaming purposes may include a virtual reality or augmented reality headset which may access the Internet via wireless or wired methods.) [Lobanov: 0070].
Regarding claims 12 and 17, Lobanov meets the claim limitations as set forth in claims 1 and 16. Lobanov further meets the claim limitations as follow.
storing a tuple for each client (stores the values of input parameters) [Lobanov: para. 0040; Figs. 8-9] that experienced a stall event ((duration of video freeze during an analysis interval) [Lobanov: 0030]; (The network layer 222 of the end user device 200 provides for data access 226 and receipt of telemetry 224 from the OTT provider App 204. The controls API 212 is used to start, stop and pause the player. The API 212 provides telemetry 218 and receives control information 220 from the OTT Provider App 204.) [Lobanov: 0021]), the tuple comprising (stores the values of input parameters) [Lobanov: para. 0040; Figs. 8-9] a unique player identifier (Referring to FIG. 8, the master play list 800 is in the form of an M3U computer file which provides information of a program ID having different available bandwidth streams. At the first line in the file, line 802, #EXTM3U designates that the file 800 is an M3U file. At line 804, #EXT-X-STREAM-INF specifies parameter values which are comma delimited, including a program ID 822 having a value of 1 and a bandwidth value 834 set to 2000000. Line 806 indicates a URL of the media stream identified by line 804. The URL identifies UID 846. Line 808 specifies an #EXT-X-I-FRAME-STREAM-INF which identifies an I-frame file having the same program ID 824 and bandwidth 836 identified in line 804. Line 808 identifies the UID 848. At line 810, #EXT-X-STREAM-INF specifies a bandwidth value 838 of 1500000 for the same program ID 1 826. Line 812 specifies a URL of a media stream having a bandwidth as indicated by line 810. Line 814 specifies an #EXT-X-IFRAME-STREAM-INF having the same program ID 828, a bandwidth 840 of 1500000 and UID 850. Line 816 specifies an #EXT-X-STREAM-INF having a same program ID 830 and lower bandwidth 842 of 500000. Line 818 indicates a URL specifying UID 852. Line 820 specifies the #EXT-XI-FRAME-STREAM-INF for the same program ID 832, bandwidth 844 of 500000 and UID 854. The UID 846-854 is provided to the server each time a URL is requested by a user device.) [Lobanov: 0067; Figs 8-9], a stall start time ((the plurality of CDN server logs are enhanced with play, stop or pause statuses of the user devices) [Lobanov: Claim 9]; (duration of video freeze during an analysis interval) [Lobanov: 0030]; (pause/stop modes and can send player status back as telemetry for analysis. Adding start, stop, and pause telemetry to input data of the neural network model helps it to distinguish player inactivity from network communication issues and therefore improves estimation accuracy) [Lobanov: 0052]), and a stall end time ((duration of video freeze during an analysis interval) [Lobanov: 0030] ; (pause/stop modes and can send player status back as telemetry for analysis. Adding start, stop, and pause telemetry to input data of the neural network model helps it to distinguish player inactivity from network communication issues and therefore improves estimation accuracy) [Lobanov: 0052]; (from about 676 ms to 1200 ms, typical audio output is audible. Between roughly 1200 ms and 1250 ms, no audio output is detected 704. After 1250 ms, audio output again becomes typical) [Lobanov: 0060; Fig. 7] – Lobanov teaches the a stall start time and a stall end time).
Regarding claim 13, Lobanov meets the claim limitations as set forth in claim 1.Lobanov further meets the claim limitations as follow.
storing (stores the values of input parameters) [Lobanov: para. 0040; Figs. 8-9] a number of requests received from a given client ((requests from the device for some interval or repeated requests for the same video segments) [Lobanov: 0029]; (all requests in an N minute interval) [Lobanov: 0041]) for each bitrate in the set of bitrates (bitrate for an N minute interval) [Lobanov: 0049].
In the same field of endeavor, Amirpour further discloses the claim limitations as follows:
set of bitrates (Per-title encoding of a segment of the plurality of segments may be performed in step 1306, which may result in construction of a bitrate ladder) [Amirpour: col. 11, line 13-15; Figs. 2-13]
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov with Amirpour to program the system to implement of Amirpour’s method.
Therefore, the combination of Lobanov with Amirpour will enable the system to improve the bitrate ladders, which results in suboptimal video delivery for the end-users [Amirpour: col. 5, line 12-22].
Regarding claim 15, Lobanov meets the claim limitations as set forth in claim 1.Lobanov further meets the claim limitations as follow.
receiving (receiving QoE metrics from the user device) [Lobanov: para. 0002; Figs. 1-6] a HTTP (The network analyzer may be configured to capture HTTP packets exchanged between the user device and the CDN server. The network analyzer may be configured to filter out packets or frames which are not relevant. From the relevant packets, the network analyzer may calculate an average bitrate for one or more N minute segments.) [Lobanov: para. 0059] request from the client ((requests from the device for some interval or repeated requests for the same video segments) [Lobanov: 0029], the request comprising a selected segment and a requested bitrate ((The network analyzer may be configured to capture HTTP packets exchanged between the user device and the CDN server. The network analyzer may be configured to filter out packets or frames which are not relevant. From the relevant packets, the network analyzer may calculate an average bitrate for one or more N minute segments.) [Lobanov: para. 0059]; (all requests in an N minute interval) [Lobanov: 0041]; (bitrate for an N minute interval) [Lobanov: 0049]); and
providing the selected segment at the requested bitrate wherein the requested bitrate is included (To measure the bitrate 626, a network analyzer 624 may be connected to a user device 604 and/or to an ISP router 620 connection line to sniff packets or frames. In FIG. 6, it is assumed that a connection of the user device 604 is wired. With a wired connection, an Ethernet switch 618 may be included between the user device 604 and the ISP provider router 620. The Ethernet network switch 618 may be configured to mirror all packets (received and sent) from port 2 618B to the mirroring port 618C, which is the port with which the network analyzer is connected to. Port 1 618A may couple the Ethernet network switch 618 to the ISP router 620 and Internet 622) [Lobanov: para. 0057] in the OSB or at a lower bitrate wherein the requested bitrate is not included in the OSB (A player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer. If it is too low, or empties, the picture and sound may freeze and break up. However, compliance with DASH and HLS allows the player to detect low buffer levels and calculate the bandwidth available and switch to the highest stream available to keep the buffer full) [Lobanov: 0015].
Regarding claim 16, Lobanov meets the claim limitations as follow.
A distributed computing system (a system that monitors quality of viewer experience) [Lobanov: para. 0036] comprising:a database (layers 500) [Lobanov: para. 0040] configured to store (FIG. 5 shows 5 example layers 500. Layer 1 is an input layer 502 which stores the values of input parameters) [Lobanov: para. 0040; Fig. 5] client stall event information ((duration of video freeze during an analysis interval) [Lobanov: 0030]; (pause/stop modes and can send player status back as telemetry for analysis. Adding start, stop, and pause telemetry to input data of the neural network model helps it to distinguish player inactivity from network communication issues and therefore improves estimation accuracy) [Lobanov: 0052]; (the plurality of CDN server logs are enhanced with play, stop or pause statuses of the user devices) [Lobanov: Claim 9]) and bitrates (bitrate QoE metrics) [Lobanov: 0002]; and one or more processors ((computer) [Lobanov: para. 0012]; (multiple servers) [Lobanov: para. 0017]) configured to:receive a client-side input (The neural network may receive a training data set comprised of a plurality of CDN server logs. The CDN server logs may be associated with known QoE metrics collected from a plurality of user devices) [Lobanov: para. 0002] and an origin-side input ((receiving Quality of Experience (QoE) metric information) [Lobanov: Claim 1]; (actual values of the QoE metrics 404 collected from user device for the same N minute interval) [Lobanov: 0032; Fig. 4]) during a first interval in a timeslot ((activity during an N minute interval) [Lobanov: 0031; Figs. 6-9]; (a timestamp of the first record) [Lobanov: 0044]), the client-side input comprising a CDN log from a client (A CDN log segment 302 representing user device activity during an N minute interval) [Lobanov: 0031], the origin-side input comprising a quality measure from an origin server ((actual values of bitrate, buffering occurrences and buffering duration QoE metrics from a user device corresponding to each CDN log segment. Bitrate may be measured using a network analyzer) [Lobanov: 0054]; (an origin server) [Lobanov: para. 0017; Fig. 1]), wherein the timeslot comprises a duration selected to (Estimates may be made every N minutes using N minute segments in the CDN log. The CDN log may cover a long interval, for example, an hour, several hours or a day. Alternatively, it may be more interesting to estimate buffering events and other QoE metrics in short intervals, for example N minute intervals. Long interval and short interval estimates may be combined so as to provide estimate trends of QoE metrics over time. That combination estimate may provide invaluable information for further analyses by OTT providers. It allows correlation of low QoE with time of day, recognizing isolated events from consistently low QoE etc.) [Lobanov: 0037] avoid a late reaction to network bandwidth fluctuation on a client side;during the first interval (during an N minute interval) [Lobanov: 0031; Figs. 6-9], extracting from the CDN log (filter the CDN log and to extract only records related to) [Lobanov: 0062] a frequency of requests ((all requests in an N minute interval) [Lobanov: 0041]; (requests from the device for some interval or repeated requests for the same video segments) [Lobanov: 0029]) for each bitrate in a bitrate ladder in the timeslot (bitrate for an N minute interval) [Lobanov: 0049] and a duration of a recent stall event for the client's player ((duration of video freeze during an analysis interval) [Lobanov: 0030]; (pause/stop modes and can send player status back as telemetry for analysis. Adding start, stop, and pause telemetry to input data of the neural network model helps it to distinguish player inactivity from network communication issues and therefore improves estimation accuracy) [Lobanov: 0052]; (the plurality of CDN server logs are enhanced with play, stop or pause statuses of the user devices) [Lobanov: Claim 9]); select (Following user selection) [Lobanov: 0066], during a second interval in the timeslot ((CDN log segments 402 each covering an N minute interval) [Lobanov: 0032; Fig. 4]; (In an embodiment, a 5 minute interval may be used. Other intervals may also be used) [Lobanov: 0031]), an optimized bitrate ladder comprising an optimal set of bitrates (OSB) using an optimization function (A player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer. If it is too low, or empties, the picture and sound may freeze and break up. However, compliance with DASH and HLS allows the player to detect low buffer levels and calculate the bandwidth available and switch to the highest stream available to keep the buffer full) [Lobanov: 0015], the optimization function (Video players use sophisticated buffer management algorithms to keep a buffer filled at an optimal level) [Lobanov: 0029] taking as input the quality measure (Video profile selection and video profile switching events may be extracted from a CDN log. Requests for video segment(s) directly identify a video profile of requested segment) [Lobanov: 0029] and a coefficient value (average bitrate QoE metrics) [Lobanov: 0002] determined using the frequency of requests ((all requests in an N minute interval) [Lobanov: 0041]; (requests from the device for some interval or repeated requests for the same video segments) [Lobanov: 0029]) and the duration of the recent stall event (average bitrate, number of buffering events and duration of video freeze during an analysis interval) [Lobanov: 0030]; and send the optimized bitrate ladder to the origin server (The model processes input data and produces an estimate of QoE metrics 308 every N minutes. Output of the model is sent to a system that monitors quality of viewer experience) [Lobanov: 0036; Fig. 3] for live encoding a next segment (In an embodiment, video/audio distribution may include live video and audio. For example, video and audio signals of a live cloud based game. In this embodiment, gameplay based training data may be relied upon in addition to other parameters and metrics disclosed herein. For example, player control input data ( or a delta thereof) may be used as a metric for ML and for metric estimation. A user device used for gaming purposes may include a virtual reality or augmented reality headset which may access the Internet via wireless or wired methods.) [Lobanov: 0070].
In the same field of endeavor, Amirpour further discloses the claim limitations as follows:
a set of bitrates (Per-title encoding of a segment of the plurality of segments may be performed in step 1306, which may result in construction of a bitrate ladder) [Amirpour: col. 11, line 13-15; Figs. 2-13]
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov with Amirpour to program the system to implement of Amirpour’s method.
Therefore, the combination of Lobanov with Amirpour will enable the system to improve the bitrate ladders, which results in suboptimal video delivery for the end-users [Amirpour: col. 5, line 12-22].
Lobanov and Amirpour do not explicitly disclose the following claim limitations (Emphasis added).
avoid a late reaction to network bandwidth fluctuation on a client side.
However, in the same field of endeavor Mani further discloses the deficient claim limitations as follows:
avoid a late reaction to network bandwidth fluctuation on a client side (The probing time interval for subsequent probes may be increased every time significant bandwidth degradation is detected. As sending probe packets requires network bandwidth, the quality of the media stream may be affected each time probing is performed (even though it can help mitigate packet loss). Thus by increasing the time between probing intervals when bandwidth degradation is detected, fluctuations in the quality of the media stream may be reduced) [Mani: col. 12, line 33-41].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lobanov and Amirpour with Mani to program the system to implement of Mani’s method.
Therefore, the combination of Lobanov and Amirpour with Mani will enable to reduce the fluctuations in the quality of the media stream [Mani: col. 12, line 13-15].
Regarding claim 18, Lobanov meets the claim limitations as follow.
A system (a system that monitors quality of viewer experience) [Lobanov: para. 0036] for optimizing a bitrate ladder (a player app in a user device may need to keep its own buffer optimized to maintain a good level of QoE for the viewer) [Lobanov: 0015]; (bitrate QoE metrics) [Lobanov: 0002]) for live streaming (In an embodiment, video/audio distribution may include live video and audio) [Lobanov: 0070], the system (a system) [Lobanov: para. 0036] comprising:
a processor ((computer) [Lobanov: para. 0012]; (multiple servers) [Lobanov: para. 0017]); and
a memory comprising program instructions executable by the processor to cause the processor to implement:
an analytics server ((CDN serve