DETAILED ACTION
Claims 1-20 are currently pending.
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites “receive, from a data store, validated video files and audio files associated with a first user associated with the first profile, a first model”. The meaning of the first model is unclear because the language fails to link the element to rest of the limitation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 7, 8, 12, 14, 15, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. US Publication 2022/0121868 (hereafter “Chen”) and Stemmer et al. US Publication 2022/0004904 (hereafter “Stemmer”).
Referring to claims 1, 8 and 15, Chen discloses a system comprising:
a deepfake detection training device comprising a data store storing a plurality of trained models, wherein a first trained model is associated with a first profile (paragraph 65, The analytics database 104 may further store any number of enrollment embeddings for audiovisual profiles. The analytics server 102 may generate audiovisual profiles for particular enrollee-users of the particular service. In some cases, the analytics server 102 generates audiovisual profiles for celebrities or other high-profile people);
a computing device comprising:
a processor; and
memory storing computer-readable instructions that, when executed by the processor, cause the computing device to:
receive, from a data store, validated video files and audio files associated with a first user associated with the first profile, a first model (paragraph 71, During an enrollment phase, the server 202 receives enrollment audiovisual data 206 and applies the machine-learning architecture 203 on the enrollment audiovisual data 206 to develop the machine-learning architecture 203 for particular people (e.g. enrolled users of a service, celebrities)) (paragraph 34, The input audiovisual data processed by the analytics server 102 may include enrollment audiovisual data, enrollment audio signals, enrollment visual data);
train, based on the validated video files and validated audio files, the first model to generate the first trained model that identifies characteristics associated with audio, video, textual, and/or other audiovisual characteristics associated with the first user, wherein the characteristics comprise vocal patterns, characteristic facial expressions, and audio passages (paragraph 88, The server receives enrollment audiovisual data samples for the enrollee and applies the machine-learning architecture to generate the various enrollment feature vectors, including, for example, a speaker spoofprint, enrollee voiceprint, a facial spoofprint, and an enrollment faceprint);
store, after training, the first model associated with the first profile in the data store of the deepfake detection training device (paragraph 89, The machine-learning architecture then algorithmically combines the corresponding types of embeddings to generate the voiceprint, faceprint, or speaker/facial sproofprint. The server stores each enrollee embedding into a non-transitory storage medium of the database);
receive, based on identification of the first profile and from the deepfake detection training device, the first trained model (paragraph 88, In step 406, the server places the neural network into an optional enrollment operational phase, and obtains enrollment audiovisual data to generate enrollment embeddings for an enrolled profile);
monitor, based on the first profile, a plurality of content files accesses by the computing device (paragraph 90, In step 408, the server places the neural network architecture into a deployment phase, and receives inbound audiovisual data);
identify, based on the first profile, first multimedia content associated with the first profile (paragraph 90, In some cases, the server receives data inputs containing an identity claim that indicates the particular person);
analyze, based on the first trained model, the first multimedia content, wherein the first multimedia content comprises video information and audio information (paragraph 91, In step 410, the server determines whether the inbound audiovisual data is genuine by applying the machine-learning architecture on the features of the inbound audiovisual data);
identify, by an analysis engine, a difference between audio information and the video information of the first multimedia content (paragraph 91, The machine-learning architecture generates one or more similarity scores based on the similarities or differences between the inbound embeddings and the corresponding enrolled embeddings, which in some cases are the enrolled embeddings associated with the person of the identity claim);
determine a probability that the first multimedia content comprises deepfake content based on a combination of probabilities generated based on audio content and video content of the first multimedia content (paragraph 91, The machine-learning architecture generates one or more similarity scores based on the similarities or differences between the inbound embeddings and the corresponding enrolled embeddings, which in some cases are the enrolled embeddings associated with the person of the identity claim); and
initiate, based on a deepfake indicator associated with the probability, a deepfake indication response (paragraph 67, The graphical user interface of the admin device 103 or other computing device displays some or all of the outputted results data, such as notifications indicating that the audiovisual data of the particular communication event session contains genuine or spoofed data or one or more scores generated by the components of the machine-learning architecture).
While Chen discloses determining a probability that content comprises deepfake content, Chen does not disclose expressly analyzing text to make such a determination.
Stemmer discloses memory storing computer-readable instructions that, when executed by the processor, cause the computing device to:
analyze, based on the first trained model, the first multimedia content, wherein the first multimedia content comprises video information, audio information and textual information (paragraph 103, the deepfake system 600 implements deepfake detectors 620, 660 to run the deepfake detection model for the respective subject and warn of a potential deepfake if an estimated probability for the provided data (e.g., face, voice, text being simulated) (e.g., deepfake source data 605) exceeds a predetermined threshold);
identify, by an analysis engine, a textual difference between machine generated text and text matching the audio information and the video information of the first multimedia content (paragraph 103, the deepfake system 600 implements deepfake detectors 620, 660 to run the deepfake detection model for the respective subject and warn of a potential deepfake if an estimated probability for the provided data (e.g., face, voice, text being simulated) (e.g., deepfake source data 605) exceeds a predetermined threshold);
determine a probability that the first multimedia content comprises deepfake content based on a combination of probabilities generated based on audio content, video content, and textual content of the first multimedia content (paragraph 117, at decision block 850, the processing device may determine whether an output score of the identified deepfake detection model exceeds a determined threshold.); and
initiate, based on a deepfake indicator associated with the probability, a deepfake indication response (paragraph 117, If so, the method 800 proceeds to block 860 where the processing device may warn a user of the computing device of a potential attack using a deepfake).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to analyze whether text is deepfake content. The motivation for doing so would have been to notify a user whether a source of text is not authentic. Therefore, it would have been obvious to combine Stemmer with Chen to obtain the invention as specified in claims1, 8 and 15.
Claims 2-4, 6, 9-11, 13 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. US Publication 2022/0121868 and Stemmer et al. US Publication 2022/0004904 as applied to claims 1, 8 and 15 above, and further in view of Mathews US Publication 2022/0269922 (hereafter “Mathews”).
Referring to claims 2, 9 and 16, Chen discloses the deepfake indication response, but does not disclose expressly blocking the first multimedia content from being presented.
Mathews discloses wherein the deepfake indication response comprises blocking the first multimedia content from being presented to a user of the computing device (paragraph 66, At block 526, the example component interface 202 displays an indication that the media is a deepfake and/or prevents the display of the media).
At the time of the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to block multimedia content from being presented to a user. The motivation for doing so would have been to remove fake multimedia content to allow a user to easily view only authentic multimedia content. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claims 2, 9 and 16.
Referring to claim 3, 10 and 17, Chen discloses the deepfake indication response, but does not disclose expressly temporarily blocking the first multimedia content from being presented.
Mathews discloses wherein the deepfake indication response comprises temporarily blocking the first multimedia content from being presented to a user of the computing device until entry of an acknowledgement input (paragraph 40, In some example, the report generator 214 may pause the media and ask a user to confirm that they are aware that the media is a deepfake before continuing to watch, stream, and/or download the media).
At the time of the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to temporarily block multimedia content from being presented to a user. The motivation for doing so would have been to remove fake multimedia content to allow a user to easily view only authentic multimedia content unless they confirm their desire to view the fake multimedia content. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claims 3, 10 and 17.
Referring to claims 4, 11 and 18, Chen discloses the deepfake indication response, but does not disclose expressly soliciting information corresponding to a deepfake response action.
Mathews discloses wherein the instructions further cause the computing device to solicit configuration information from a user, wherein the configuration information corresponds to a deepfake response action associated with the first profile (paragraph 40, In some example, the report generator 214 may pause the media and ask a user to confirm that they are aware that the media is a deepfake before continuing to watch, stream, and/or download the media).
At the time of the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to solicit information corresponding to a deepfake response action. The motivation for doing so would have been to remove fake multimedia content to allow a user to easily view only authentic multimedia content unless they confirm their desire to view the fake multimedia content. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claims 4, 11 and 18.
Referring to claims 6 and 13, Chen discloses training of the first model, but does not disclose expressly that training occurs continuously.
Mathews discloses wherein the training of the first model occurs continuously (paragraph 19, the deepfake media and/or information corresponding to the deepfake media may be transmitted to a server corresponding to the training of the deepfake detection models to further tune or adjust deepfake detection models).
At the time of the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to solicit information corresponding to continuously train a model. The motivation for doing so would have been to improve the detection ability of the model by increasing the amount of training data. Therefore, it would have been obvious to combine Mathews with Chen to obtain the invention as specified in claim 6.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights