DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is a Final action on the merits in response to the communications filed on 8/28/2025.
Applicant has amended claims 1 – 2, 6, 12 – 13, and 17.
Claims 1 – 19, and 21, are pending in this application.
Examiner Response
Examiner Response to Remarks
Response to Section 101 Rejection.
Response to Section 103 Rejection.
Examiner has considered all information provided in Applicant’s Remarks.
Rejections Under 35 U.S.C. § 101
Applicant’s arguments are persuasive to overcome 35 U.S.C. 101 rejection as there is an audio fingerprint based on an audio signal corresponding to the audio data.
Examiner’s Response to Section 103 Rejections.
Applicant argues Lord in view of Ura does not teach or suggest all of the elements of independent claim 1. Applicant further argues Ura is silent to at least the newly amended bolded features of amended claim 1, and Lord fails to remedy the deficiencies of Ura.
Examiner respectfully disagrees. Applicant has amended claim 1; further search and consideration is necessary and new art has been applied due to amendments. However, Lord in view of Ura teaches Applicant’s claim 1. Applicant uses the term retrieving, however retrieving is no different than collecting; and both Lord and Ura teach collecting data and executing instructions with one or more processors. Ura teaches in ¶ 0066, collecting sensor data from various sensor devices; and Lord teaches collecting sensor data as well in col. 10, lines 54 – 60; Lord also teaches in col. 8, lines 46 – 52, The appending of aggregated metadata can be performed periodically, upon the occurrence of a triggering event (e.g., as may be derived from recovered metadata, whenever new relevant content, metadata, etc., that is associated with extracted information, is found or provided, etc.), or the like or any combination thereof and is likened to new risk element data. Lord further teaches risk elements, and teaches processes for ascertaining context and semantics of the content used to generate metadata. Ura in 0006, further teaches re-learning the model using the initial training data and additional training data and is likened to retraining. Rejection pursuant to 35 U.S.C. § 103 remains.
Claim Rejections – 35 U.S.C. §103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
4. Claims 1 – 19 and 21, are rejected under 35 U.S.C. 103 as being unpatentable over Lord, John D (U.S. Patent No. 9,454,789) in view of Ura, Akira et al. (U.S. Publication No. 2017/037,2229) in view of Bilobrov, Sergiy et al. (EP 3264326 A1).
Claims 1 and 12:
A computer-implemented method for media production risk assessment comprising: receiving, by the one or more processors, an electronic dataset from a user device wherein the electronic dataset comprises audio data corresponding to a media production; Lord teaches in col. 3, lines 22 – 26, The audio processing system may include an audio mixer, an audio CODEC, an audio digital signal processor (DSP), a sequencer, a digital audio workstation (DAW), or the like or any combination thereof. Lord teaches in col. 18, lines 49 – 50, generating audio content, where that audio content is data generated and may be likened to the electronic data generated for use in media production; Lord also teaches in col. 28, lines 33 – 44 metadata obtained from the database and the data may include script data for production. Lord teaches in col. 47, lines 38 – 40, executed from a system's memory (a computer readable medium, such as an electronic, optical or magnetic storage device).
creating, by the one or more processors, an audio fingerprint based on an audio signal corresponding to the audio data: Lord teaches in col. 31, lines 20 – 21, house mixes of pre-recorded media creates new content; Lord teaches in col. 39, lines 15 – 17, the watermark detector can process the captured audio signals generated by the microphone to implement a watermark detection process; Lord teaches in col. 39, lines 60 – 67 and col. 40, lines 1 – 2, the watermark embedder can adjust or otherwise adapt the process by which information is embedded into the produced content using any suitable or desired technique to create the watermarked content in a manner that ensures sufficiently reliable detection and/or reading of information embedded within the watermarked content, in a manner that minimizes or otherwise reduces the perceptibility of the embedded watermark, in a manner that is in accordance with any embedding policy information indicated by the alert, or the like or any combination thereof.
generating, by the one or more processors, via a machine-learning model, a set of risk assessment measures and one or more corresponding conditional annotations, based on the audio fingerprint, the set of risk assessment measures signifying a level of risk for a corresponding one of the one or more identified risk elements; Lord teaches in col. 39, lines 15 – 32, generating audio data from the microphone for watermark detection that may be content theft risk. Lord teaches in col. 34, lines 24 – 33, a classifier determining data that corresponds to watermark types for content theft risks and determining if there is any overlap of the data, where the analysis of overlapping of the data is likened to the set of risk assessment measures. See machine learning model discussed in limitation below; Lord teaches in col. 5, lines 50 – 56, performing one or more content recognition processes on one or more portions of the captured content (e.g., including one or more temporally- or spectrally-segmented portions of the captured content, an audio portion of the captured content, an imagery portion of the captured content, etc.), which include watermark extraction and/or fingerprint recognition.
modifying, by the one or more processors, the electronic dataset based on each of the risk assessment measures to include the one or more corresponding conditional annotations, wherein the one or more corresponding conditional annotations comprises the generated risk assessment measurement for each of the identified risk elements; there is no support in Applicant’s Spec. for “conditional annotations” even though Applicant’s Spec. ¶ 078, recites “annotation of a need for copyright clearance, e.g., absolute vs. conditional”. Lord teaches in col. 33, lines 10 – 12, an insertion function makes changes to embed a watermark signal element determined by perceptual adaptation, and Lord teaches in col. 33, lines 21 – 24, an iterative embedding control module that implements evaluations that control whether iterative embedding is applied to parameters that are also being updated and may be likened to risk assessment measures; Lord teaches in may be likened to risk assessment measures. Lord teaches in col. 34, lines 36 – 37, pre-processing stages transform the audio blocks to a state for further watermark detection, where the transforming is likened to modifying, the audio blocks are likened to the electronic dataset, watermark detection is likened to predicting the risk, and the overlapping of the data is taught in the above limitation to include the one or more corresponding conditional annotations; furthermore there is no support in Applicant’s Specification for “risk assessment elements”.
at least in part by comparing a plurality of semantic tags and metadata corresponding to the risk element data to the one or more potential risk elements detected in the electronic dataset defined risk elements to one or more potential risk elements detected in the electronic dataset; Lord teaches in col. 5, lines 43 – 58, A user can transfer or upload, (i.e., “post”) the captured content to a content hosting service. The content hosting service ingests the posted content and makes it available for sharing with others by giving it a network address and associated metadata. The ingest process may include transcoding of the uploaded or posted content to a form suitable for streaming to others on the Internet. It may also include performing one or more content recognition processes on one or more portions of the captured content (e.g., including one or more temporally- or spectrally-segmented portions of the captured content, an audio portion of the captured content, an imagery portion of the captured content, etc.), which include watermark extraction and/or fingerprint recognition, among other processes for ascertaining context and semantic meaning from the content that can be used to generate richer metadata automatically.
based on the one or more potential risk elements identified in the electronic dataset; Lord teaches in col. 32, lines 9 – 12, probably music/audio licensing issues for each venue/performer/production group if audio is streamed off-site for fingerprinting (artist rights management, copyright, content theft risks).
new risk element data corresponding to the sensor data received from the one or more sensors; Lord teaches in col. 8, lines 46 – 52, the appending of aggregated metadata can be performed periodically, upon the occurrence of a triggering event (e.g., as may be derived from recovered metadata, whenever new relevant content, metadata, etc., that is associated with extracted information, is found or provided, etc.), or the like or any combination thereof. Lord teaches in col. 10, 61 – 64, Digital watermarks can provide synchronization information, such as embedded time codes or timestamps, which enable different video uploads or posts to be synchronized in time. Lord teaches in col. 15, lines 51 – 56, processed (e.g., in an audio processing system at a venue), and played back to the audience at the venue as explained earlier. Watermarking is performed in the intermediate stage (processing stage), with processing performed at the time each new segment of audio becomes available. Digital watermarks correspond to the sensors.
saving, by the one or more processors, the modified electronic dataset in a computer memory; Lord teaches in col. 3, lines 46 – 47, content can be recorded or stored by a tangible media.
While Lord teaches watermark embedding and detecting, fingerprinting, content theft risks, audio and video content, and the perceptual masking model for that watermark type is also predicted and is related to Ura via model predicting; but Lord does not explicitly teach relearning the model. However, Ura teaches the following:
accessing, by one or more processors, sensor data from one or more sensors, wherein the one or more sensors include at least one of: a motion sensor, a stereoscopic array, an environment sensor, a biometric sensor, and/or a microphone; Ura teaches in ¶ 0066, collecting sensor datasets by the machine learning apparatus. See above capturing microphone audio signals;
identifying, by the one or more processors, via the trained machine-learning model; Ura teaches in ¶ 0089, within a single learning step, the machine learning apparatus constructs a model by using training data and evaluates its prediction performance by using test data.
retraining, by the one or more processors, the trained machine-learning model, the new risk element data, and the generated set of risk assessment measures using an iterative training algorithm to reduce error in a subsequently generated set of risk assessment measures by the trained machine-learning model; risk elements and risk assessment measures are taught in the above limitations. Ura teaches in ¶ 0005, when the trained model outputs a poor prediction, the computer reruns the model learning with larger training data and evaluates the resulting model again in iterations where the iterations may reduce error when evaluating the model. Ura teaches in ¶ 0006, a method of re-learning a support vector machine (SVM). The SVM is used in the technical field of detecting scene boundaries in a video, and the proposed method improves its prediction performance. Specifically, the re-learning method learns an SVM with a video as initial training data, generates additional training data by converting the initial training data in its brightness and contrast, and re-learns the SVM using both the initial training data and additional training data.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine content played at an event is identified using watermarking and/or other content recognition combined with contextual metadata, which facilitates identification and correlation with other content and metadata when it is posted to a network of Lord with a predictive model learning device capable of learning a model with training data of Ura to assist businesses in using machine learning models for high accuracy of predictions when retraining models (Ura, Spec. ¶ 0004).
Lord teaches watermark embedding and detecting, fingerprinting, content theft risks, audio and video content, and the perceptual masking model for that watermark type is also predicted and is related to Ura via model predicting; Ura teaches retraining a machine learning model; and Ura is related to Bilobrov via training a machine learning model for predicting data; neither Lord nor Ura explicitly teaches machine learning classifier used to generate a first video fingerprint for a frame in a content item. However, Bilobrov teaches the following:
retrieving, by the one or more processors, via a trained machine-learning model, new risk element data corresponding to the sensor data received from the one or more sensors; Bilobrov teaches in ¶ 0048, the feature-based fingerprinting module can generate multiple fingerprints for a frame (or a set of frames) based on one or more objects that were detected in the frame (or set of frames). For example, a machine learning classifier can be trained to recognize various objects (e.g., landmarks, points of interest, human features, etc.) that are captured in frames. In another example, a machine learning classifier can be trained to recognize the sounds of various objects (e.g., fog horn, bell, human voice, etc.) that are captured in frames. Bilobrov teaches in ¶ 0049, the multiple fingerprints generated using such approaches can be stored and retrieved.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine content played at an event is identified using watermarking and/or other content recognition combined with contextual metadata, which facilitates identification and correlation with other content and metadata when it is posted to a network of Lord and a predictive model learning device capable of learning and relearning model with training data of Ura with systems, methods, and non-transitory computer-readable media that can generate at least one first fingerprint based at least in part on one or more frames corresponding to a test content item of Bilobrov to assist businesses in training a machine learning classifier models to recognize audio and video content (Bilobrov, Spec. ¶ 0048).
Claims 2 and 13:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Ura discloses use of a machine learning model as in Claim 1 above. Lord discloses the following limitation:
wherein the identifying further comprises predicting the level of risk using a machine-learning model trained to recognize similarity between the risk element data and one or more referents of one or more symbol combinations appearing in the electronic dataset; Lord teaches in col. 34, lines 32 – 33, predicting a watermark type that may be likened to content theft risk in col. 31, lines 40 – 41. Lord further teaches col. 16, line 25, masking model that may be used as an algorithm when predicting watermarks. Lord teaches in col. 34, lines 7 – 12, the detector starts by executing a preprocessor on digital audio data stored in a buffer. The preprocessor samples the audio data to the time resolution used by subsequent stages of the detector. It also spawns execution of initial pre-processing modules 902 to classify the audio and determine watermark type. Lord teaches in col. 36, lines 10 – 13, For a watermark structure comprised of bumps, this includes aggregating the bump estimates at the bump locations based on a code symbol mapping to embedding locations.
Claims 3 and 14:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord discloses the following limitation:
correlating the set of risk assessment measures to one or more symbol combinations appearing in a script of the electronic dataset; Lord teaches in col. 49, claim 9, identifying at least one item of metadata associated with the watermark IDs from the received items, wherein the act of correlating the received items of captured content comprises correlating the received items with each other based on the at least one identified item of metadata.
Claims 4 and 15:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord further discloses the following limitation:
generating an annotated version of the script at least in part by adding one or more indications of the risk assessment to each of the one or more symbol combinations. Lord teaches in col. 3, lines 36 – 40, textual inputs which may be likened to an annotated version of the script. Lord teaches in col. 11, lines 25 – 30, this type of audio processing enables the audio that is prepared for streaming to users via the network service to be refined based on the different versions of audio captured at an event and uploaded or posted by users. One type of processing is audio artifacts detection and filtering, where different versions is likened to annotated version.
Claims 5 and 16:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord further discloses the following limitation:
identifying comprises at least one of: analyzing an electronic script or digitally scanning one or more sets by the one or more processors; Lord teaches in col. 7, lines 7 – 13, content metadata can be obtained by analyzing the captured content. Content metadata can be analyzed at the device that captured the content (e.g., as the content is captured, or at some later point in time prior to or upon upload of the captured content) or by some other device associated with the capturing device (e.g., auxiliary device), where content metadata analyzed is likened to analyzing an electronic script.
Claims 6 and 17:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord further discloses the following limitation:
identifying further comprises identifying the potential risk elements using the machine-learning model machine learning component trained to recognize similarity between symbol combinations that connote the risk element data and the symbol combinations; Lord teaches in col. 33, lines 65 – 67, and col. 34, lines 1 – 6, detecting the audio data which may be the watermark or copyright risks; see discussion above provided for algorithms that covers predicting models. Lord further teaches in col. 34, classifying audio data and determining the watermark type that may be a risk and may be likened to the defined risk elements and the symbol combinations. recognizing the similarity operates on an incoming audio signal, which is digitally sampled and buffered in a memory device. Its basic mode is to apply a set of processing stages to each of several time segments (possibly overlapping by some time delay). The stages are configured to re-use operations and avoid unnecessary processing, where possible (e.g., exit detection where watermark is not initially detected or skip a stage where execution of the stage for a previous segment can be re-used).
Claims 7 and 18:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord further discloses the following limitation:
packaging the annotated version of the script in a computer-readable medium with additional content coordinated with the one or more indications of risk assessment; Lord teaches in col. 47, lines 23 – 24, … instructions stored in tangible computer-readable media.
Claims 8 and 19:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Ura further discloses the following limitation:
training, by the one or more processors, the machine-learning model using an iterative training algorithm; Ura teaches in ¶ 0005, the computer iterates these things until the prediction performance reaches a sufficient level.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine content played at an event is identified using watermarking and/or other content recognition combined with contextual metadata, which facilitates identification and correlation with other content and metadata when it is posted to a network of Lord with a predictive model learning device capable of learning a model with training data of Ura to assist businesses in using machine learning models for high accuracy of predictions (Ura, Spec. ¶ 0004).
Claim 9:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord further discloses the following limitation:
receiving an image of the one or more sets from a mobile computing device; Lord teaches in col. 21, lines 63 – 65, WM marked content is available as soon as it is posted; it can be scanned once and necessary databases populated with extracted ID numbers; Lord teaches in col. 5, lines 59 – 64, captured content can be transferred or uploaded from a mobile device (e.g., smartphone) or an auxiliary device (e.g., desktop computer) to a cloud storage system automatically (e.g., as it is captured, after the capture process is complete, etc.) or through any separate process initiated by the user.
Claims 10 and 21:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord further discloses the following limitation:
comprising by the one or more processors adjusting the risk assessment based on context-sensitive factors for referents of symbol combinations appearing in the electronic dataset; Lord teaches in col. 39, lines 46 – 67, and col. 40, lines 1 – 3, the watermark detector… Based on the indication(s) provided by the alert, the watermark embedder can adjust or otherwise adapt the process by which information is embedded into the produced content using any suitable or desired technique to create the watermarked content in a manner that ensures sufficiently reliable detection and/or reading of information embedded within the watermarked content, in a manner that minimizes or otherwise reduces the perceptibility of the embedded watermark, in a manner that is in accordance with any embedding policy information indicated by the alert, or the like or any combination thereof.
Claim 11:
Lord, Ura, and Bilobrov disclose claims 1 and 12. Lord further discloses the following limitation:
comprising by the one or more processors, including in the indication of risk assessment a reference to digital exclusion images; Lord teaches in col. 23, lines 61 – 67, and col. 24, lines 1 – 2, a watermark ID removed from produced content.
Conclusion
The prior art made of record and not relied upon is considered relevant but not applied:
Note: these are additional references found but not used.
- Reference Rouse, Rolly et al. (U.S. Publication No. 2013/032,5870) discloses the ability of people and entities to produce, distribute, and use items of digital content by providing software tools that enable them to (a) clip items (b) store copies, (c) form and store meshes of tags to represent their mindsets about items of content.
- Reference Balakrishnan, Suhrid et al. (U.S. Publication No. 2014/010,9123) discloses illustrative embodiments of generating advertising media plans for broadcast content where the media plans are based on campaign criteria including cost-per-mille and/or reach
- Reference Segal, Aviad (U.S. Publication No. 2008/030,7310) discloses a system for accessing development components, including an online video clip library of templates and an online music clip library, and enabling online production of user-friendly, custom-integrated media products.
- Reference Torres, Adam (U.S. Publication No. 2012/022,6595) discloses A web-based financing system for financing entertainment media production companies implemented by a computer or personal digital assistant, comprising a member database for storing registration information.
- Reference Agrawal, S., Sureka, A. (2013). Copyright Infringement Detection of Music Videos on YouTube by Mining Video and Uploader Meta-data. In: Bhatnagar, V., Srinivasa, S. (eds) Big Data Analytics. BDA 2013. Lecture Notes in Computer Science, vol 8302. Springer, Cham. https://doi.org/10.1007/978-3-319-03689-2_4.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Frank Alston whose telephone number is 703-756-4510. The examiner can normally be reached 9:00 AM – 5:00 PM Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Beth Boswell can be reached on (571) 272-6737. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK MAURICE ALSTON/
Examiner, Art Unit 3625
12/02/2025
/BETH V BOSWELL/Supervisory Patent Examiner, Art Unit 3625