DETAILED ACTION
Claims 1-20 are currently pending.
The previous rejections to claims 1-7 under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, are withdrawn due to Applicant’s amendment.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/11/26 has been entered.
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventors, at the time the application was filed, had possession of the claimed invention.
Claims 1, 8 and 15 have been amended to recite “identify, by an analysis engine, a textual difference between machine generated text and text matching the audio information and the video information of the first multimedia content, wherein identifying the textual difference comprises processing text comprising subtitles or captions associated with the first multimedia content and comparing the text against an audio waveform and visual lip movements of the first multimedia content to detect inconsistencies indicative of machine-generated text”.
Applicant’s support for the recited limitation is in paragraph 29 of the specification which states:
“In some cases, the analysis engine 154 may process text (e.g., subtitles, captions, and the like) associated with the multi-media file based on the AI model to discriminate between the text generated by the individual and/or text that matches an audio and/or video portion of the multimedia file, and machine generated text.” The disclosure generally provides support for comparing subtitles or captions to audio information and/or video information. However, the disclosure does not specify that the comparison of video information is performed from visual lip movements. The only discussion of comparing visual lip movements in Applicant’s specification is with the audio information. A skilled artisan would not recognize comparing subtitles to video information as requiring a visual lip movement comparison because a comparison can also be made through performing optical character recognition on the video information.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 7, 8, 12, 14, 15, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. US Publication 2022/0121868 (hereafter “Chen”) and Non-Patent Literature Vinod et al. “A Novel Deepfake Detection Framework using Audio-Video-Textual Features” (hereafter “Vinod”).
Referring to claims 1, 8 and 15, Chen discloses a system comprising:
a deepfake detection training device comprising a data store storing a plurality of trained models, wherein a first trained model is associated with a first profile (paragraph 65, The analytics database 104 may further store any number of enrollment embeddings for audiovisual profiles. The analytics server 102 may generate audiovisual profiles for particular enrollee-users of the particular service. In some cases, the analytics server 102 generates audiovisual profiles for celebrities or other high-profile people);
a computing device comprising:
a processor; and
memory storing computer-readable instructions that, when executed by the processor, cause the computing device to:
receive, from a data store, validated video files and audio files associated with a first user associated with the first profile (paragraph 71, During an enrollment phase, the server 202 receives enrollment audiovisual data 206 and applies the machine-learning architecture 203 on the enrollment audiovisual data 206 to develop the machine-learning architecture 203 for particular people (e.g. enrolled users of a service, celebrities)) (paragraph 34, The input audiovisual data processed by the analytics server 102 may include enrollment audiovisual data, enrollment audio signals, enrollment visual data);
train, based on the validated video files and validated audio files, a first model to generate the first trained model that identifies characteristics associated with audio, video, textual, and/or other audiovisual characteristics associated with the first user, wherein the characteristics comprise vocal patterns, characteristic facial expressions, and audio passages (paragraph 88, The server receives enrollment audiovisual data samples for the enrollee and applies the machine-learning architecture to generate the various enrollment feature vectors, including, for example, a speaker spoofprint, enrollee voiceprint, a facial spoofprint, and an enrollment faceprint);
store, after training, the first model associated with the first profile in the data store of the deepfake detection training device (paragraph 89, The machine-learning architecture then algorithmically combines the corresponding types of embeddings to generate the voiceprint, faceprint, or speaker/facial sproofprint. The server stores each enrollee embedding into a non-transitory storage medium of the database);
receive, based on identification of the first profile and from the deepfake detection training device, the first trained model (paragraph 88, In step 406, the server places the neural network into an optional enrollment operational phase, and obtains enrollment audiovisual data to generate enrollment embeddings for an enrolled profile);
monitor, based on the first profile, a plurality of content files accesses by the computing device (paragraph 90, In step 408, the server places the neural network architecture into a deployment phase, and receives inbound audiovisual data);
identify, based on the first profile, first multimedia content associated with the first profile (paragraph 90, In some cases, the server receives data inputs containing an identity claim that indicates the particular person);
analyze, based on the first trained model, the first multimedia content, wherein the first multimedia content comprises video information and audio information (paragraph 91, In step 410, the server determines whether the inbound audiovisual data is genuine by applying the machine-learning architecture on the features of the inbound audiovisual data);
identify, by an analysis engine, a difference between audio information and the video information of the first multimedia content (paragraph 91, The machine-learning architecture generates one or more similarity scores based on the similarities or differences between the inbound embeddings and the corresponding enrolled embeddings, which in some cases are the enrolled embeddings associated with the person of the identity claim);
determine a probability that the first multimedia content comprises deepfake content based on a combination of probabilities generated based on audio content and video content of the first multimedia content (paragraph 91, The machine-learning architecture generates one or more similarity scores based on the similarities or differences between the inbound embeddings and the corresponding enrolled embeddings, which in some cases are the enrolled embeddings associated with the person of the identity claim); and
initiate, based on a deepfake indicator associated with the probability, a deepfake indication response (paragraph 67, The graphical user interface of the admin device 103 or other computing device displays some or all of the outputted results data, such as notifications indicating that the audiovisual data of the particular communication event session contains genuine or spoofed data or one or more scores generated by the components of the machine-learning architecture).
While Chen discloses determining a probability that content comprises deepfake content, Chen does not disclose expressly analyzing text to make such a determination.
Vinod discloses memory storing computer-readable instructions that, when executed by the processor, cause the computing device to:
analyze, based on the first trained model, the first multimedia content, wherein the first multimedia content comprises video information, audio information and textual information (page 4, The ensemble architecture utilizes two unimodal networks to identify manipulated video and audio deepfakes. The two cross-modal networks effectively detect the dissonance between audio-video clues and audio-text clues);
identify, by an analysis engine, a textual difference between machine generated text and text matching the audio information and the video information of the first multimedia content, wherein identifying the textual difference comprises processing text comprising subtitles or captions associated with the first multimedia content and comparing the text against an audio waveform and visual lip movements of the first multimedia content to detect inconsistencies indicative of machine-generated text (page 5, Audio-Text Cross-modal Deepfake Detection: This cross-modal validates the authenticity of subtitles. This helps in identifying whether the video has been modified or not. Videocr library [44] extracts the subtitle from the given input video and then analyzed it for spelling issues by employing the PySpark library [45]. Spelling correction [46] is the most prevalent protection against textual adversary attacks. The spell-checked text is compared with the speech retrieved from the audio);
determine a probability that the first multimedia content comprises deepfake content based on a combination of probabilities generated based on audio content, video content, and textual content of the first multimedia content (page 4, The ‘D-Fence‘ layer classifies the input video as genuine if all these classifiers identify it as authentic; else, classified as fake).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to analyze whether subtitles are deepfake content. The motivation for doing so would have been to improve the detection of content that is not authentic. Therefore, it would have been obvious to combine Stemmer with Vinod to obtain the invention as specified in claims1, 8 and 15.
Claims 2-4, 6, 9-11, 13 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. US Publication 2022/0121868 and Non-Patent Literature Vinod et al. “A Novel Deepfake Detection Framework using Audio-Video-Textual Features” as applied to claims 1, 8 and 15 above, and further in view of Mathews US Publication 2022/0269922 (hereafter “Mathews”).
Referring to claims 2, 9 and 16, Chen discloses the deepfake indication response, but does not disclose expressly blocking the first multimedia content from being presented.
Mathews discloses wherein the deepfake indication response comprises blocking the first multimedia content from being presented to a user of the computing device (paragraph 66, At block 526, the example component interface 202 displays an indication that the media is a deepfake and/or prevents the display of the media).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to block multimedia content from being presented to a user. The motivation for doing so would have been to remove fake multimedia content to allow a user to easily view only authentic multimedia content. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claims 2, 9 and 16.
Referring to claim 3, 10 and 17, Chen discloses the deepfake indication response, but does not disclose expressly temporarily blocking the first multimedia content from being presented.
Mathews discloses wherein the deepfake indication response comprises temporarily blocking the first multimedia content from being presented to a user of the computing device until entry of an acknowledgement input (paragraph 40, In some example, the report generator 214 may pause the media and ask a user to confirm that they are aware that the media is a deepfake before continuing to watch, stream, and/or download the media).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to temporarily block multimedia content from being presented to a user. The motivation for doing so would have been to remove fake multimedia content to allow a user to easily view only authentic multimedia content unless they confirm their desire to view the fake multimedia content. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claims 3, 10 and 17.
Referring to claims 4, 11 and 18, Chen discloses the deepfake indication response, but does not disclose expressly soliciting information corresponding to a deepfake response action.
Mathews discloses wherein the instructions further cause the computing device to solicit configuration information from a user, wherein the configuration information corresponds to a deepfake response action associated with the first profile (paragraph 40, In some example, the report generator 214 may pause the media and ask a user to confirm that they are aware that the media is a deepfake before continuing to watch, stream, and/or download the media).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to solicit information corresponding to a deepfake response action. The motivation for doing so would have been to remove fake multimedia content to allow a user to easily view only authentic multimedia content unless they confirm their desire to view the fake multimedia content. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claims 4, 11 and 18.
Referring to claims 6 and 13, Chen discloses training of the first model, but does not disclose expressly that training occurs continuously.
Mathews discloses wherein the training of the first model occurs continuously (paragraph 19, the deepfake media and/or information corresponding to the deepfake media may be transmitted to a server corresponding to the training of the deepfake detection models to further tune or adjust deepfake detection models).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to solicit information corresponding to continuously train a model. The motivation for doing so would have been to improve the detection ability of the model by increasing the amount of training data. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claim 6.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights