Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to communication filed 9/16/2025. Claims 1-4, 6, 9, 11, 13-14, 17-18, 20-23, 25, 27, 29 and 33-34 are pending for examination, the rejection cited as stated below.
Claim Objections
2. Claim 23 is objected to because of the following informalities: The claim ends with an “and”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
3. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
4. Claim 34 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows. Claim 34 is drawn to functional descriptive material recorded on a computer-readable medium. However, neither the specification nor the claim limits the recited “computer-readable medium” to be non-transitory. The examiner presumes for the sake of examination that the term “computer-readable medium" comprises both transitory and non-transitory medium.
“A transitory, propagating signal … is not a “process, machine, manufacture, or composition of matter.” Those four categories define the explicit scope and reach of subject matter patentable under 35 U.S.C. § 101; thus, such a signal cannot be patentable subject matter.” (In re Petrus A.C.M. Nuijten; Fed Cir, 2006-1371, 9/20/2007).
Because the full scope of the claim as properly read in light of the disclosure appears to encompass non-statutory subject matter, the claim as a whole is non-statutory. The examiner suggests amending the claim to include the disclosed non-transitory computer readable storage media, while at the same time excluding the transitory media such as signals, carrier waves, etc. Any amendment to the claim should be commensurate with its corresponding disclosure.
Claim Rejections - 35 USC § 112
5. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
6. The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
7. Claims 1-4, 6, 9, 11, 13-14, 17-18, 20-23, 25, 27, 29 and 33-34 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
a) Claim 1 recites “automatically identifying a set of one or more unique video segments from which the video data of each version of the video asset can be constructed, by… comparing the image fingerprint information of each version to identify one or more shared video segments that are used in at least two of the versions, and any version-specific video segments; and determining, for each version of the video asset, a composition of one or more unique video segments from the set that makes up the video data of the respective version.”
First of all, it is unclear whether and how the recited “a set of one or more unique video segments” relates to “a composition of one or more unique video segments.“ For the sake of the examination, Examiner assumes any relationship.
Secondly, it is unclear whether and how the recited “a set of one or more unique video segments” relates to “version-specific video segments”. For the sake of the examination, Examiner interprets as any relationship.
Thirdly, it is unclear whether and how the recited “a composition of one or more unique video segments” relates to “version-specific video segments”. For the sake of the examination, Examiner interprets as any relationship.
Finally, it is unclear between the recited “unique” and “version-specific”. For the sake of the examination, Examiner assumes any relationship.
Claims 2-4, 6, 9, 11, 13-14, 17-18, 20-23, 25, 27, 29 and 33-34 are similarly rejected.
b) Claim 4 recites “and, preferably,
wherein the primary fingerprint is generated using a perceptual Hash function, and the secondary fingerprint is generated from pixel colour information in the image frame; and, preferably,
wherein the secondary image fingerprint comprises a colour histogram with a plurality of colour bins, and the second distance or similarity metric comprises a relative change in each colour bin value of the colour histograms.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
c) Claim 6 recites “and, preferably,
wherein generating the audio fingerprint for each audio slice comprises:
transforming the audio slice from the time domain to the frequency domain to obtain an amplitude spectrum having a plurality of frequency components, each frequency component having a respective amplitude value;
grouping the frequency components into a predefined number of bins, wherein each bin has an aggregated amplitude value;
determining an audio fingerprint based on the sequence of aggregated amplitude values associated with the bins; and
normalising the audio data in each audio slice of the version to have the same perceived loudness before transforming the audio slice to the frequency domain; and, further preferably,
wherein determining the audio fingerprint based on the sequence of aggregated amplitude values comprises:
transforming the sequence of aggregated amplitude values into a sequence of bits; and, preferably, wherein transforming the sequence of aggregated amplitude values into a sequence of bits comprises, normalising the sequence of aggregated amplitude values, and applying a threshold.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
d) Claim 9 recites “and, preferably,
wherein adjusting the transition point between the two consecutive unique audio segments comprises:
detecting the presence or absence of human speech and/or music in each audio slice of the consecutive unique audio segments based on the frequency content of the respective audio slice; and
adjusting the transition point to the timecode of the nearest audio slice in which human speech and/or music is not detected.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s). Claim 11 is similarly rejected during the dependency.
e) Claim 11 recites “and, preferably,
wherein the at least one sound level comprises an overall sound level of the audio slice, optionally or preferably, a Loudness Unit Full Scale (LUFS) level; and/or,
wherein the at least one sound level comprises one or more component sound levels for specific frequency components or bands of interest extracted from the amplitude spectrum of the audio slice, preferably, wherein the specific frequency components or bands of interest are associated with human speech and/or music.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” and/or “optionally” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
f) Claim 14 recites “and, preferably,
generating an interoperable master format (IMF) package for the multiple versions of the video asset based on the extracted sets of unique video and audio segments and the compositions of each version; and further preferably validating the generated IMF package by:
generating each of the multiple versions from the IMF package, each generated version comprising generated video and audio data comprising an aggregation of respective unique video and audio segments from the sets;
generating, for the generated video data of each generated version of the video asset, image fingerprint information for each image frame based on its content;
partitioning the generated audio data of each generated version into a sequence of audio slices of equal time duration, and generating audio fingerprint information for each audio slice of each generated version based on its content; and
comparing, by timecode, the image and audio fingerprint information of each generated version to the image and audio fingerprint information of each respective version master used to generate the IMF package, to determine whether they match or not;
wherein the IMF package is determined to be valid if the generated image and audio data of the generated versions match the video and audio data of the respective version masters used to generate the IMF package.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
g) Claim 18 recites “and, preferably,
generating or amending, for each version, a playback control file for the adaptive bitrate streaming protocol based on the version information so as to reference the unique video segments at each bitrate stored in the one or more servers, optionally wherein the references comprise URLs for retrieving the unique video segments during playback of the video asset; and
uploading the generated or amended playback control files to the one or more servers for use in adaptive bitrate streaming.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
h) Claim 23 recites “and, preferably,
extracting the identified set of unique audio segments from the audio data of the multiple versions of the video asset;
storing the extracted set of unique audio segments together with version information containing the composition for each version; generating an interoperable master format (IMF) package for the multiple versions of the video asset based on the extracted sets of unique video and audio segments and the compositions of each version; and.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
i) Claim 25 recites “and, preferably,
wherein updating the stored sets comprises:
uploading the one or more new unique video segments at each bitrate to the one or more servers together with the updated version information for the new version;
generating or amending a playback control file for the new version based on the version information so as to reference the unique video segments at each bitrate stored in the one or more servers, optionally wherein references comprise URLs for retrieving the unique video segments during playback; and
uploading the generated or amended playback control file to the one or more servers for use in adaptive bitrate streaming.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
j) Claim 27 recites “and, preferably,
wherein the predefined duration of the video chunks is defined by an adaptive bitrate protocol; and/or wherein parts (ii) or (iii) further comprise dividing any unique video segments corresponding to multiple video chunks into individual video chunks.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
k) Claim 29 recites “and, preferably,
wherein the image fingerprint information for each image frame, and where present the audio fingerprint for each audio slice, is associated with a respective timecode, and detecting the multiple versions comprises:
identifying a subset of candidate video assets having a predefined proportion of identical or similar image fingerprint-timecode pairs, and where present audio fingerprint-timecode pairs;
comparing the image fingerprint information of the identified candidate video assets frame-by- frame and by timecode, and where present the audio fingerprint information of the identified candidate video assets slice-by-slice and by timecode; and
identifying those candidate video assets having a predefined proportion of matching video and/or audio content as multiple versions of a video asset; and preferably, identifying those candidate video assets with identical or fully matching video content, and where present identical or fully matching audio content, as duplicate versions; and, further preferably computing, for each pair of compared image frames, and where present each compared pair of audio slices, one or more distance metrics from the respective image and audio fingerprint information; and wherein identifying candidate video assets with a predefined proportion of identical or similar fingerprint-timecode pairs comprises: comparing the one or more distance metrics to one or more respective first threshold values; and wherein identifying those candidate video assets having a predefined proportion of matching video and/or audio content comprises comparing the one or more distance metrics to one or more respective second threshold values, where the one or more second threshold values are lower than the one or more first threshold values.”
The metes and bounds of the scope that the applicant seeks protection with cannot be definitely determined, because the limitations are recited to be “preferably” which does not necessarily happen. As a result, Examiner interprets that these limitations do not further limit the preceding limitation(s).
Claim 33 is similarly rejected.
l) Claim 6 recites “identifying a set of one or more unique audio segments from which the audio data of each version of the video asset can be constructed, by… comparing the audio fingerprints of each version to identify one or more shared audio segments that are used in two or more of the versions, and any version-audio video segments; and determining, for each version of the video asset, a composition of one or more unique audio segments from the set that makes up the audio data of the respective version.”
First of all, it is unclear whether and how the recited “a set of one or more unique audio segments” relates to “a composition of one or more unique audio segments.“ For the sake of the examination, Examiner assumes any relationship.
Secondly, it is unclear whether and how the recited “a set of one or more unique audio segments” relates to “version-specific audio segments”. For the sake of the examination, Examiner interprets as any relationship.
Thirdly, it is unclear whether and how the recited “a composition of one or more unique audio segments” relates to “version-specific audio segments”. For the sake of the examination, Examiner interprets as any relationship.
Finally, it is unclear between the recited “unique” and “version-specific”. For the sake of the examination, Examiner assumes any relationship.
Claims 9 and 11 are similarly rejected.
Claim Rejections - 35 USC § 102
8. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
10. Claims 1, 13, and 34 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Joshua et al (“Content identification”, hereafter Joshua).
As to claim 1, Joshua discloses a computer-implemented method of managing multiple different versions of a video asset (page 4, paragraph 1, “a single program may have between 15 and 60 versions. By fingerprinting multiple pre-existing versions of various assets of differing quality, identifying their differences, and storing only the high-resolution master file and these "deltas," media companies can reduce storage space and quickly and automatically create high-resolution versions that conform to the requirements of any distribution platform”), the method comprising:
automatically identifying a set of one or more unique video segments from which the video data of each version of the video asset can be constructed (see 112 rejection and Examiner’s interpretation stated therein regarding “a set of one or more unique video segments” and “unique”. In addition, it is to be noted that “can be constructed” does not require an actual construction. See citation below), by:
generating, for each version of the video asset, image fingerprint information for each image frame based on its content (page 4, paragraph 1, “a single program may have between 15 and 60 versions. By fingerprinting multiple pre-existing versions of various assets of differing quality, identifying their differences, and storing only the high-resolution master file and these "deltas," media companies can reduce storage space and quickly and automatically create high-resolution versions that conform to the requirements of any distribution platform”; and page 4, paragraph 2, “an adaptive fingerprinting method that can incorporate both precise frame-by-frame searches and rapid, large-scale searches”); and
comparing the image fingerprint information of each version to identify one or more shared video segments that are used in at least two of the versions, and any version-specific video segments (page 4, paragraph 1, “a single program may have between 15 and 60 versions. By fingerprinting multiple pre-existing versions of various assets of differing quality, identifying their differences, and storing only the high-resolution master file and these "deltas," media companies can reduce storage space and quickly and automatically create high-resolution versions that conform to the requirements of any distribution platform”; and page 4, paragraph 2, “an adaptive fingerprinting method that can incorporate both precise frame-by-frame searches and rapid, large-scale searches”; and page 2, “Stored fingerprints typically are indexed and mapped into catalogs, much like directories, that help to speed the matching process. Either a piece of media or a fingerprint can be submitted for analysis and matching against video fingerprints within the DIS.” Therefore both matched and different segments are identified based on comparing the image fingerprint information of each version); and
determining, for each version of the video asset, a composition of one or more unique video segments from the set that makes up the video data of the respective version (see 112 rejection and Examiner’s interpretation stated therein regarding “one or more unique video segments” and “unique”. See page 4, paragraph 1, “a single program may have between 15 and 60 versions. By fingerprinting multiple pre-existing versions of various assets of differing quality, identifying their differences, and storing only the high-resolution master file and these "deltas," media companies can reduce storage space and quickly and automatically create high-resolution versions that conform to the requirements of any distribution platform”).
As to claim 34, see similar rejection to claim 1.
As to claim 13, Joshua discloses the method of claim 1, further comprising:
extracting the identified set of unique video segments from the video data of the multiple versions of the video asset; and storing the extracted set of unique video segments together with version information containing the composition for each version (Joshua, page 4, paragraph 1, “a single program may have between 15 and 60 versions. By fingerprinting multiple pre-existing versions of various assets of differing quality, identifying their differences, and storing only the high-resolution master file and these "deltas," media companies can reduce storage space and quickly and automatically create high-resolution versions that conform to the requirements of any distribution platform”).
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
14. Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Joshua, as applied to claim 1 above, and further in view of Landy (US 2015/0326820).
As to claim 2, Joshua discloses the method of claim 1, wherein comparing the image fingerprint information of each version comprises:
selecting one of the versions as a base version; and comparing the image fingerprint information of the base version to the fingerprint information of each other version frame-by-frame to identify one or more sequences of matching image frames indicative of one or more shared video segments, and to identify any sequences of unique image frames that are indicative of version-specific video segments (page 4, paragraph 1, “a single program may have between 15 and 60 versions. By fingerprinting multiple pre-existing versions of various assets of differing quality, identifying their differences, and storing only the high-resolution master file and these "deltas," media companies can reduce storage space and quickly and automatically create high-resolution versions that conform to the requirements of any distribution platform”; page 2, last two paragraphs, “The fingerprinting engine looks for a series of characteristics, such as color, contrast and borders, that are present in the sampled frames and reduces this information to its minimal representative form. This unique numerical representation of the fundamental characteristics of the set of sampled material is its fingerprint. As fingerprints are created, they are used to populate the iPharro Database Indexing Server (DIS), which in turn serves as the reference for content identification. Queried media content is fingerprinted, and the extracted fingerprint is then compared against those stored on the DIS.”; and page 4, paragraph 2, “an adaptive fingerprinting method that can incorporate both precise frame-by-frame searches and rapid, large-scale searches).
To enhance the rejection, Landy is brought in to further teach a concept of selecting one of multiple versions as a base version; and comparing image fingerprint information of the base version to fingerprint information of each other version frame-by-frame to identify one or more sequences of matching image frames indicative of one or more shared video segments, and to identify any sequences of unique image frames that are indicative of version-specific video segments (See Landy, [0025], “In the illustration of FIG. 5, reference is made to content within individual frames in order to verify that appropriate conversion of frame rates, content, frame size, and the like have been made. In the example, a desired or reference frame 128 is selected in the first version 22. The content 130 of the frame is analyzed and image recognition techniques are employed for determining parameters or characteristics of the content. Such techniques may involve image matching data, such as fingerprints 132. As will be appreciated by those skilled in the art, such fingerprint data may be derived from the image data by reference to individual pixels, lines, regions, or any other parameter of the image, and may be reduced to a simple value that can be easily compared to a corresponding value of frames in the second version. Thus, in corresponding frame 134 in the second version is selected that comprises content 136. The content 136 is used to derive similar data 138 that can be used in a comparison as indicated at reference numeral 140”; [0026], “to compare the same frames in the series of frames comprising each version, beginning with a known frame. Such selection may be made based upon, for example, comparison of an A frame to an A frame, comparison of a B frame to a B frame, or comparison of a D frame to a D frame, to use the 2:3 pulldown discussed above as an example”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Joshua and Landy. The suggestion/motivation of the combination would have been to determine whether a frame is properly converted from the first version to the second version (Landy, [0026]).
As to claim 3, Joshua in view of Landy discloses the method of claim 2, wherein comparing the image fingerprint information of a given pair of image frames comprises:
computing one or more similarity or distance metrics from the image fingerprint information
of the pair of image frames (Landy, [0033], “At step 174 a determination is made whether the desired parameters utilized in the comparison do, in fact, correspond to the tolerance range allowed for the conversion. If the results of the comparison indicate that the parameters fall outside of the tolerance, a notice or report is provided at step 176 that effectively indicates an exception that should be addressed. Such exceptions may be addressed manually or in the automated or semiautomated manners. If the results of the comparison indicated that all desired parameters fall within the tolerances, a notice or report may be generated to this effect as indicated at reference numeral 178”); and
identifying the pair of image frames as matching if the one or more similarity or distance metrics satisfy one or more respective matching conditions (Landy, see citation in the preceding limitation, wherein within the tolerance range indicates a distance metrics satisfy the matching condition).
15. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Joshua in view of Landy, as applied to claim 3 above, and further in view of KANSARA (US 2016/0342844).
As to claim 4, Joshua in view of Landy discloses the method of claim 3, wherein the image fingerprint information comprises a primary image fingerprint generated using a first image fingerprinting technique; and comparing the image fingerprint information of a given pair of image frames comprises: computing a first distance or similarity metric for the primary image fingerprints of the pair of image frames (see citation in rejection to claim 3); and, preferably, wherein the primary fingerprint is generated using a perceptual Hash function, and the secondary fingerprint is generated from pixel colour information in the image frame; and, preferably, wherein the secondary image fingemrint comprises a colour histogram with a plurality of colour bins, and the second distance or similarity metric comprises a relative change in each colour bin value of the colour histograms (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
Joshua in view of Landy, however, does not expressly disclose a secondary image fingerprint generated using a second fingerprinting technique; computing a second distance or similarity metric from the secondary image fingerprints of the pair of image frames; and identifying the pair of image frames as matching if both the primary and secondary fingerprints satisfy a respective matching condition. KANSARA discloses a secondary image fingerprint generated using a second fingerprinting technique; computing a second distance or similarity metric from the secondary image fingerprints of the pair of image frames; and identifying the pair of image frames as matching if both the primary and secondary fingerprints satisfy a respective matching condition ([0140]-[0145], e.g., “Determining a Closeness Vector or Aggregate Closeness Value based on Multiple Closeness Values… If the model frame fingerprint and/or the test frame fingerprint are comprised of more than one type of fingerprint, then the closeness value may be a vector of closeness values and/or an aggregate closeness value… Fingerprint comparison logic 214 may group the first closeness value and the second closeness value into a closeness value vector, wherein each closeness value corresponds to a type of frame fingerprint associated with the model frame and/or the test frame …If the closeness value is a vector of one or more other closeness values, then each closeness value may be compared to a threshold vector. If one or more values in the vector of closeness values is less than or equal to one or more values in the threshold vector, respectively, then fingerprint comparison logic 214 may determine that the model frame and the test frame match”; see also [0146]).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Joshua in view of Landy with KANSARA. The suggestion/motivation of the combination would have been to utilize multiple fingerprint types (KANSARA, [0141]).
16. Claims 6, 14, 20-23, 29 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Joshua, as applied to claim 1 above, and further in view of Stone (US 2019/0362405).
As to claim 6, Joshua discloses the claimed invention substantially as discussed in claim 1, wherein the multiple versions of the video assets further comprise audio data (Joshua, page 2, “audiovisual”), but does not expressly disclose that the method further comprises:
identifying a set of one or more unique audio segments from which the audio data of each version of the video asset can be constructed (it is to be noted that “can be constructed” does not require an actual construction), by:
partitioning the audio data of each version into a sequence of audio slices of equal time duration;
generating, for each version, an audio fingerprint for each audio slice based on its content; and
comparing the audio fingerprints of each version to identify one or more shared audio segments that are used in two or more of the versions and any version-specific audio segments; and
determining, for each version of the video asset, a composition of one or more unique audio segments from the set that makes up the audio data of the respective version; and, preferably,
wherein generating the audio fingerprint for each audio slice comprises: transforming the audio slice from the time domain to the frequency domain to obtain an amplitude spectrum having a plurality of frequency components, each frequency component having a respective amplitude value; grouping the frequency components into a predefined number of bins, wherein each bin has an aggregated amplitude value; determining an audio fingerprint based on the sequence of aggregated amplitude values associated with the bins; and normalising the audio data in each audio slice of the version to have the same perceived loudness before transforming the audio slice to the frequency domain; and, further preferably, wherein determining the audio fingerprint based on the sequence of aggregated amplitude
values comprises: transforming the sequence of aggregated amplitude values into a sequence of bits;
and, preferably, wherein transforming the sequence of aggregated amplitude values into a sequence of bits comprises, normalising the sequence of aggregated amplitude values, and applying a threshold (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
Stone discloses identifying a set of one or more unique audio segments from which the audio data of each version of the video asset can be constructed (see 112 rejection and Examiner’s interpretation stated therein regarding “a set of one or more unique audio segments” and “unique”. In addition, it is to be noted that “can be constructed” does not require an actual construction. See citation below), by:
partitioning the audio data of each version into a sequence of audio slices of equal time duration (Fig. 1, version 190 and version 192 are each partitioned into a sequence of audio slices of equal time duration, as being corresponding to fingerprints in 191 and 193 respectively that contains equally-spaced fingerprints as shown in Fig. 1; see also [0045], “a first version of an audiovisual product 190 and a second version of the audiovisual product 192. An audiovisual product according to the present example contains at least an audio (sound) stream component and a visual (video) stream component. During production of an audiovisual product, modification (e.g. editing) processes are used that create newer versions of an audiovisual product from older versions of the same audiovisual product. For the present purposes, the second version of the audiovisual product 192 in memory 140 is a newer version of the audiovisual product than the first version of the audiovisual product 190”; [0046], “The first and second versions of the audiovisual product 190, 192 stored by the memory 140 have corresponding audio fingerprints-first version audio fingerprints 191 and second version audio fingerprints 193 also stored by memory 140 and depicted in FIG. 1. Each individual audio fingerprint is associated with a temporal location (i.e. a time or time-stamp) within the respective version of the audiovisual product.” [0051], “the temporal location tx contains a section of audiovisual content that is 0.2 seconds long);
generating, for each version, an audio fingerprint for each audio slice based on its content (see citation in the preceding limitation); and
comparing the audio fingerprints of each version to identify one or more shared audio segments that are used in two or more of the versions and any version-specific audio segments ([0049], “In the example of FIG. 1, the map 195 stored by memory 140 illustrates matching regions between the first and second versions 190, 192 using blocks extending across the temporal locations for which a matching region has been identified. In addition, or as an alternative, the visual representation of the map 195 may illustrate new regions rather than matching regions. The new regions may include (i) regions in which a new audiovisual component has been created and added to the pre-existing audiovisual product, and/or (ii) non-matching regions”); and
determining, for each version of the video asset, a composition of one or more unique audio segments from the set that makes up the audio data of the respective version ([0049], “the visual representation of the map 195 may illustrate new regions rather than matching regions. The new regions may include (i) regions in which a new audiovisual component has been created and added to the pre-existing audiovisual product, and/or (ii) non-matching regions”. See also Fig. 1, map “195”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Joshua with Stone. The suggestion/motivation of the combination would have been to utilize fingerprints to find matching audio segments (Stone, abstract).
As to claim 14, Joshua in view of Stone discloses the method of claim 13, wherein the multiple versions of the video assets further comprise audio data (Joshua, page 2, “audiovisual characteristics of a piece of video content are stored”), the method further comprising:
extracting the identified set of unique audio segments from the audio data of the multiple
versions of the video asset; and storing the extracted set of unique audio segments together with version information containing the composition for each version (Stone, ([0049], “In the example of FIG. 1, the map 195 stored by memory 140 illustrates matching regions between the first and second versions 190, 192 using blocks extending across the temporal locations for which a matching region has been identified. In addition, or as an alternative, the visual representation of the map 195 may illustrate new regions rather than matching regions. The new regions may include (i) regions in which a new audiovisual component has been created and added to the pre-existing audiovisual product, and/or (ii) non-matching regions”. See Fig. 1, the map “195”; and In the present context, the map 195, for example, may comprise a database or data structure such as an array or the like storing data representing
locations in one, other or both of the audiovisual products. The locations in the audiovisual products may be indicated, for instance, in the map 195 by time stamps representing the respective playback point in time. Against timestamps, the map 195 may record indicators such as whether there is a match, no match, or such like, for respective timestamps”); and, preferably,
generating an interoperable master format (IMF) package for the multiple versions of the video asset based on the extracted sets of unique video and audio segments and the compositions of each version; and further preferably validating the generated IMF package by: generating each of the multiple versions from the IMF package, each generated version comprising generated video and audio data comprising an aggregation of respective unique video and audio segments from the sets; generating, for the generated video data of each generated version of the video asset, image fingerprint information for each image frame based on its content; partitioning the generated audio data of each generated version into a sequence of audio slices of equal time duration, and generating audio fingerprint information for each audio slice of each generated version based on its content; and
comparing, by timecode, the image and audio fingerprint information of each generated version to the image and audio fingerprint information of each respective version master used to generate the IMF package, to determine whether they match or not; wherein the IMF package is determined to be valid if the generated image and audio data of the generated versions match the video and audio data of the respective version masters used to generate the IMF package (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
As to claim 20, Joshua in view of Stone discloses the method of claim 1, further comprising:
receiving a new version of the video asset, the new version of the video asset comprising
video data including a sequence of image frames (Joshua, see citation in rejection to claim 1. See Stone, [0042], “past and new versions of the audiovisual product”);
generating, for the new version, image fingerprint information for each image frame based on
its content (Joshua, see citation in rejection to claim 1. See Stone, [0042], “past and new versions of the audiovisual product”);
identifying an updated set of unique video segments from which the video data of all versions of the video asset can be constructed (It is to be noted that “can be constructed” does not require an actual construction) by:
comparing the image fingerprint information of each version to identify one or more shared video segments that are used in at least two of the versions, and any version-specific video segments (Joshua, see citation in rejection to claim 1. See Stone, [0042], “past and new versions of the audiovisual product”); and
determining, for each version of the video asset, an updated composition of one or more unique video segments from the updated set that makes up the video data of the respective version (Joshua, see citation in rejection to claim 1. See Stone, [0042], “past and new versions of the audiovisual product”; [0049], “the visual representation of the map 195 may illustrate new regions rather than matching regions. The new regions may include (i) regions in which a new audiovisual component has been created and added to the pre-existing audiovisual product” and [0098], “The map also contains sections, labelled “N”, that exist only in the second version 192 and are thereby new sections relative to the first version 190.”).
As to claim 21, Joshua in view of Stone discloses the method of claim 20, wherein the multiple versions of the video assets further comprise audio data (Joshua, page 2, “audiovisual”), and the method further comprises:
identifying a set of one or more unique audio segments from which the audio data of each version of the video asset can be constructed (it is to be noted that “can be constructed” does not require an actual construction), by:
partitioning the audio data of each version into a sequence of audio slices of equal time duration (Stone, see citation in rejection to claim 6);
generating, for each version, an audio fingerprint for each audio slice based on its content (Stone, see citation in rejection to claim 6); and
comparing the audio fingerprints of each version to identify one or more shared audio segments that are used in two or more of the versions and any version-specific audio segments (Stone, see citation in rejection to claim 6); and
determining, for each version of the video asset, a composition of one or more unique audio segments from the set that makes up the audio data of the respective version,
wherein the new version of the video asset further comprises audio data (Stone, see citation in rejection to claim 6. Also see Stone, [0042], “past and new versions of the audiovisual product), and the method further comprises:
partitioning the audio data of the new version into a sequence of audio slices of equal time duration, and generating audio fingerprint information for each audio slice based on its content (Stone, see citation in rejection to claim 6. Also see Stone, “[0042], “past and new versions of the audiovisual product”); and
identifying an updated set of one or more unique audio segments from which the audio data
of all the versions of the video asset can be constructed (It is to be noted that “can be constructed” does not require an actual construction) by:
comparing the audio fingerprint information of each version to identify one or more shared audio segments that are used in two or more of the versions and any version-specific audio segments (Stone, see citation in rejection to claim 6. Also see Stone, [0042], “past and new versions of the audiovisual product); and
determining, for each version of the video asset, an updated composition of one or more unique audio segments from the updated set that makes up the audio data of the respective version (Stone, see citation in rejection to claim 6. Also see Stone, “[0042], “past and new versions of the audiovisual product”).
As to claim 22, Joshua in view of Stone discloses the method of claim 20, further comprising:
extracting the identified updated set of unique video segments from the video data of the versions of the video asset (Joshua, see citation in rejection to claim 13. See Stone, [0042], “past and new versions of the audiovisual product”); and
storing the updated set of unique video segments together with version information containing the updated composition for each version (Joshua, see citation in rejection to claim 13. See Stone, [0042], “past and new versions of the audiovisual product”).
As to claim 23, Joshua in view of Stone discloses the method of claim 22, further comprising:
extracting the identified updated set of unique audio segments from the audio data of the versions of the video asset (Stone, see citation in rejection to claim 14. see Stone, [0042], “past and new versions of the audiovisual product”); and
storing the extracted updated set of unique audio segments together with version information containing the updated composition for each version (Stone, see citation in rejection to claim 14. see Stone, [0042], “past and new versions of the audiovisual product”); and, preferably.
extracting the identified set of unique audio segments from the audio data of the multiple
versions of the video asset; storing the extracted set of unique audio segments together with version information containing the composition for each version; generating an interoperable master format (IMF) package for the multiple versions of the video asset based on the extracted sets of unique video and audio segments and the compositions of each version; and updating the IMF package for the multiple versions of the video asset based on the updated set of unique video and audio segments and the updated compositions of each version (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
As to claim 29, Joshua in view of Stone discloses the method of claim 1 further comprising:
receiving a plurality of video assets including the multiple versions of the video asset, each video asset comprising video data (Joshua, see citation in rejection to claim 1);
generating, for each video asset received, image fingerprint information for each image frame based on its contents (Joshua, see citation in rejection to claim 1);
where the video assets comprise audio data (Joshua, page 2, “audiovisual”), partitioning the audio data of each video asset into a sequence of audio slices of equal time duration, and generating, for each video asset, an audio fingerprint for each audio slice based on its content (Stone, see citation rejection to claim 6); and
detecting the multiple versions of the video asset in the plurality of video assets based on a comparison of the image fingerprint information, and where present the audio fingerprint information, of each video asset (Stone, see citation in rejection to claim 6); and, preferably,
wherein the image fingerprint information for each image frame, and where present the audio fingerprint for each audio slice, is associated with a respective timecode, and detecting the multiple versions comprises: identifying a subset of candidate video assets having a predefined proportion of identical or similar image fingerprint-timecode pairs, and where present audio fingerprint-timecode pairs; comparing the image fingerprint information of the identified candidate video assets frame-by-
frame and by timecode, and where present the audio fingerprint information of the identified candidate video assets slice-by-slice and by timecode; and identifying those candidate video assets having a predefined proportion of matching video and/or audio content as multiple versions of a video asset; and preferably, identifying those candidate video assets with identical or fully matching video content, and where present identical or fully matching audio content, as duplicate versions; and, further preferably,
computing, for each pair of compared image frames, and where present each compared pair of audio slices, one or more distance metrics from the respective image and audio fingerprint information; and wherein identifying candidate video assets with a predefined proportion of identical or similar fingerprint-timecode pairs comprises: comparing the one or more distance metrics to one or more respective first threshold values; and wherein identifying those candidate video assets having a predefined proportion of matching video and/or audio content comprises comparing the one or more distance metrics to one or more respective second threshold values, where the one or more second threshold values are lower than the one or more first threshold values (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
As to claim 33, Joshua in view of Stone discloses a system for managing multiple different versions of a video asset, comprising one or more processing devices configured with instructions that, when executed by the one or more processing devices, cause the one or more processing devices to perform the method as defined in claim 29 (see similar rejection to claim 29).
17. Claims 9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Joshua in view of Stone, as applied to claim 6 above, and further in view of Woods (US 11392345).
As to claim 9, Joshua in view of Stone discloses the method of claim 6, wherein each unique audio segment has a start and an end timecode, wherein the end of one unique audio segment and the start of the next consecutive unique audio segment in a version defines a transition point between the two consecutive unique audio segments in the version when constructed according to the respective composition (Stone, see citation in rejection to claim 6, e.g., Fig. 1, the map “195”; and [0049], “In the example of FIG. 1, the map 195 stored by memory 140 illustrates matching regions between the first and second versions 190, 192 using blocks extending across the temporal locations for which a matching region has been identified. In addition, or as an alternative, the visual representation of the map 195 may illustrate new regions rather than matching regions. The new regions may include (i) regions in which a new audiovisual component has been created and added to the pre-existing audiovisual product, and/or (ii) non-matching regions”; and [0046], Each individual audio fingerprint is associated with a temporal location (i.e. a time or time-stamp) within the respective version of the audiovisual product”, indicating that the regions are on a temporal axis, therefore each region including the “new region” (i.e., unique audio segment) has a start and an end timecode. It is to be noted that the claimed limitation merely requires “has a start and an end timecode” without requiring that the timecodes are stored or presented. See Fig. 1, wherein the mapped consecutive regions such as the new regions are consecutive in time in the respective version, wherein the end of the previous “new region” and the start of the next consecution “new region” defines a transition point. It is to be noted that the claimed limitation does not require the transition point to be a single time. See also [0048], “In the present context, the map 195, for example, may comprise a database or data structure such as an array or the like storing data representing locations in one, other or both of the audiovisual products. The locations in the audiovisual products may be indicated, for instance, in the map 195 by time stamps representing the respective playback point in time. Against timestamps, the map 195 may record indicators such as whether there is a match, no match, or such like, for respective timestamps”); and, preferably
wherein adjusting the transition point between the two consecutive unique audio segments comprises: detecting the presence or absence of human speech and/or music in each audio slice of the consecutive unique audio segments based on the frequency content of the respective audio slice; and adjusting the transition point to the timecode of the nearest audio slice in which human speech and/or music is not detected (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
But does not expressly disclose that the method further comprising:
adjusting the transition point between two consecutive unique audio segments to avoid one or more of the following: sound above a predefined level, music and human speech at the transition point, based on the audio content at or near the transition point.
Woods discloses a concept of adjusting a transition point between two consecutive audio segments to avoid one or more of the following: sound above a predefined level, music and human speech at the transition point, based on the audio content at or near the transition point (col. 4, lines 16-24., “The optimal point for joining two segments (e.g., via a rapid crossfade) may then be determinized by finding a location with relatively low energy such as, for example, a zero crossing or where the sound envelop has a low value. When the joining point is determined, the corresponding segments are joined to extend the audio track”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Joshua in view of Stone with Woods. The suggestion/motivation of the combination would have been to optimize point for joining two segments (Woods, col. 4, lines 16-24).
As to claim 11, Joshua in view of Stone and Woods discloses the method of claim 9, wherein adjusting the transition point between the two consecutive unique audio segments comprises:
determining at least one sound level for each audio slice of the consecutive unique audio segments; and where the at least one sound level at the transition point is greater than a respective threshold value, adjusting the transition point to the timecode of the nearest audio slice in which the at least one sound level is less than the respective threshold value (Woods, col. 4, lines 16-24., “The optimal point for joining two segments (e.g., via a rapid crossfade) may then be determinized by finding a location with relatively low energy such as, for example, a zero crossing or where the sound envelop has a low value. When the joining point is determined, the corresponding segments are joined to extend the audio track”, wherein a zero or low sound value reflects a threshold, and wherein searching for such a location alone the time line is implied); and, preferably,
wherein the at least one sound level comprises an overall sound level of the audio slice, optionally or preferably, a Loudness Unit Full Scale (LUFS) level; and/or wherein the at least one sound level comprises one or more component sound levels for specific frequency components or bands of interest extracted from the amplitude spectrum of the audio slice, preferably, wherein the specific frequency components or bands of interest are associated with human speech and/or music (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
18. Claims 17-18 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Joshua, as applied to claim 1 above, and further in view of Brueck et al (US 2014/0247885 hereafter Brukck).
As to claim 17, Joshua discloses the claimed invention substantially as discussed in claim 1, but does not expressly disclose encoding the set of unique video segments into multiple bitrates; and storing the set of unique video segments at each bitrate together with version information containing the available bitrates and the composition of unique video segments for each version. Brueck discloses encoding a set of video segments into multiple bitrates; and storing the set of video segments at each bitrate together with version information containing the available bitrates and the composition of video segments for a version ([0025] and [0023], “divide the signal into temporal slices, encode them to various bitrates, and upload them to a remote web server 122”, wherein the signal is a version of a video, and wherein the slices are segments of the video signal, see [0025], “video and/or audio”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Joshua with Brueck. The suggestion/motivation of the combination would have been to provide adaptive streaming (Brueck, [0023]).
As to claim 18, Joshua in view of Brueck discloses the method of claim 17, wherein the step of storing comprises:
uploading the set of unique video segments at each bitrate and the version information to one
or more servers for use in adaptive bitrate streaming (Brueck, [0025] and [0023], “divide the signal into temporal slices, encode them to various bitrates, and upload them to a remote web server 122”, wherein the signal is a version of a video, and wherein the slices are segments of the video signal, see [0025], “video and/or audio”); and, preferably,
generating or amending, for each version, a playback control file for the adaptive bitrate streaming protocol based on the version information so as to reference the unique video segments at
each bitrate stored in the one or more servers, optionally wherein the references comprise URLs for retrieving the unique video segments during playback of the video asset; and uploading the generated or amended playback control files to the one or more servers for use in adaptive bitrate streaming (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
As to claim 27, Joshua in view of Stone discloses the method of claim 1, wherein:
(i) each identified unique video segment in the set is a video chunk of a predefined duration; or
(ii) each identified unique video segment in the set is an integer number of video chunks of a predefined duration; or
(iii) wherein each unique video segment has a start and an end timecode, wherein the end of one unique video segment and the start of the next consecutive unique video segment in a version defines a transition point between the two consecutive unique video segments in the version when constructed according to the respective composition (Stone, see citation in rejection to claim 9), the method further comprising:
adjusting the transition points between consecutive unique video segments such that each identified unique video segment in the set corresponds to an integer number of video chunks of a predefined duration (Brueck, [0005], “The process for preparing media content for adaptive streaming typically involves taking a source content file, such as a movie, or a source content feed (e.g., a broadcast signal), and splitting it into temporal chunks or slices, and encoding each slice at a different quality level. Typically, the different quality levels are achieved by encoding the source media content at different bit rates according to encoding profiles. During playback, and as network conditions fluctuate, a consuming application selects a slice at one of the different quality levels that is best suited for the present network throughput, and the consuming applications adjusts the quality up or down as available bandwidth increases or decreases by selecting subsequent slices at the appropriate quality levels”; see Brueck, [0045], “to receive different portions of the encoded media content from multiple locations. The portions may have an equal duration”, wherein equal duration portions from multiple locations indicates a predefined duration); and, preferably,
wherein the predefined duration of the video chunks is defined by an adaptive bitrate protocol; and/or wherein parts (ii) or (iii) further comprise dividing any unique video segments corresponding to multiple video chunks into individual video chunks (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
19. Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Joshua in view of Stone, as applied to claim 20 above, and further in view of Brueck et al (US 2014/0247885 hereafter Brueck).
As to claim 25, Joshua in view of Stone discloses the claimed invention substantially as discussed in claim 20, including where an updated set comprises one or more new unique video segments that were not present in the previous set (see Joshua, as cited in rejection to claim 1, “deltas”; Stone, [0042], “past and new versions of the audiovisual product”), and, preferably,
wherein updating the stored sets comprises: uploading the one or more new unique video segments at each bitrate to the one or more servers together with the updated version information for the new version; generating or amending a playback control file for the new version based on the
version information so as to reference the unique video segments at each bitrate stored in the one or more servers, optionally wherein references comprise URLs for retrieving the unique video segments during playback; and uploading the generated or amended playback control file to the one or more servers for use in adaptive bitrate streaming (see 112 rejection and Examiner’s interpretation therein that these limitations do not further limit the preceding limitation(s)).
Stone further discloses new
But does not expressly disclose
encoding the set of unique video segments into multiple bitrates;
storing the set of unique video segments at each bitrate together with version information containing the available bitrates and the composition of unique video segments for each version; and
encoding the one or more new version-specific video segments into the multiple bitrates; and
updating the stored sets to include the one or more new unique video segments at each bitrate and the version information for the new version;
Brueck discloses
encoding a set of video segments into multiple bitrates (Brueck, see citation in rejection to claim 17);
storing the set of video segments at each bitrate together with version information containing the available bitrates and the composition of video segments for each version (Brueck, see citation in rejection to claim 17); and
encoding one or more new video segments into the multiple bitrates (Brueck, see citation in rejection to claim 17, repeating the disclosed functionality); and
updating the stored sets to include the one or more new video segments at each bitrate and the version information (Brueck, see citation in rejection to claim 17, repeating the disclosed fucntionality. Also see Brueck, [0053], “Using the container reader 422, the decoupled slicer 306 can scan the source media content, looking for all the video key frames. The decoupled slicer 306 creates an index of the source media content, and the video key frames can become index points. For example, this index can be created when adding the source media content to the content library”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Joshua in view of Stone with Brueck. The suggestion/motivation of the combination would have been to provide adaptive streaming (Brueck, [0023]).
Prior Art Cited but not Applied in the Rejection
20. Below is a list of prior art reference(s) cited but not applied in the rejection:
a) Missale (US 2022/0335245), disclosing detecting similarity between video files using hash-based active recognition of objects, to reduce content storage costs.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUA FAN whose telephone number is (571)270-5311. The examiner can normally be reached on 9-6.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nasser Goodarzi, can be reached at (571) 272-4195. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUA FAN/Primary Examiner, Art Unit 2426
Prosecution Insights