Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
DETAILED ACTION
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/16/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claim 8 is objected to because of the following informalities: Claim 8 recites “avator” which appears to be a misspelling of “avatar” . Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-13 and 15-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
In regards to claims 1 and 13, the preamble of each claims states “processing personal video data” however the body of the claim never references this “personal video data” raising questions as to whether or not the reference to “original video data” and “converted video data” is the “personal video data”
Similarly, claim 15 is rejected for the same reasons with reference to “personal audio data”. In addition, the claim inconsistently references both “audio” and “audio data”, thus “the audio data” lacks an antecedent basis.
Furthermore, regarding claim 15, the preamble of the claim recites a method for processing “audio data” however step d) discloses composing “a converted video data” despite there being no reference to or mention of video data in the claim. This creates ambiguity as there is a disconnection regarding the final step e) of “outputting the converted audio data” . Because of this the true scope of the claim cannot be reasonably determined since it is unclear if this is for processing video, audio or both. As a result, claim 15 (and due to their dependency claims 16 and 17) has not been further examined on the merits. The applicant is advised that if the claim is amended to overcome the indefiniteness issue by removing the reference to video data it would shift the scope to a method for processing only audio data which would require a requirement for restriction separating claims 1-14 from 15-17
In regards to claim 8, step o recites “applying non-PII features of the face to the avatar face”; however, “non-PII features” has already been introduced in base claim 1, thus it is unclear if these are the same features or if these are completely new non-PII features. Furthermore, it is observed that the claim references “the face” however previous references states “a detected face”. Lastly, step p recites “composing a converted video data based on the avator face and/or the generated body characteristic”. It is unclear if a second converted video is being generated as the base claim initially discloses “a converted video”. Furthermore, “the avator face” appears to be a misspelling of “avatar” and lastly, “the generated body characteristic” lacks an antecedent basis as the previous base claims do not disclose generating body characteristics.
In regards to claim 11, the limitation “composing the converted video data using face, avatar, portrait, background, and human voice as designated or randomized” renders the claim indefinite as it is unclear what the applicant is attempting to claim. The claim states composing the converted video using elements such as “face”, “avatar”, “portrait”, “background” and “human voice” however it is unclear what these elements are referencing as they have not been introduced. For example, it is unclear what it means to compose the converted video using face or any other item listed in the set and, it is unclear what steps are used to perform or accomplish this. Furthermore, there hasn’t been any discussion of detecting or identifying any of those elements as the base claims only discloses just extracting PII and non-PII information. Lastly, it is unclear as to what the applicant means when they refer to these elements as “designated or random”. As a result the claim is indefinite and for compact prosecution the examiner will broadly presume the elements are alternatives, the elements are selected randomly or are generated and the elements used by the system to generate the composed video.
Dependent claims depending from a rejected based claim are rejected for inheriting the deficiencies from the claims that they depend from and failing to overcome the rejection of the base claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-11, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over US 20240348747 to Chatterjee et al. (hereinafter “Chaterjee”) in view of US 20210090314 to HUSSEN ABDELAZIZ et al. (hereinafter “Hussen”) and further in view of US 20210335337 to GKOULALAS-DIVANIS et al. (hereinafter “Gkoulalas-Divanis”)
Claim 1
Chaterjee teaches a method for processing personal video data, [e.g. Chatterjee; Abstract, Para. 0012, 0099, 0114] comprising the steps of
a) receiving an original video data; [e.g. Chatterjee; Abstract, Para. 0012, 0100 – Chatterjee discloses receiving video data (e.g. original video data) from a capture unit. ]
b) extracting video frames and an audio from the original video data; [e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses the video data includes a sequence of images (e.g. video frames) and sound data (e.g. audio). ]
c) identifying personal identifiable information (PII) features as well as non-PII features in both the video frames ..; [e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses identifying identification features (e.g. PII feature) and mimic features (e.g. non-PII feature). ]
d) extracting the non-PII features from the video frames ...; [e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses storing a mimic expression features in a data space. ]
e) using the extracted non-PII features, composing a converted video data; [e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses replacing a face with a synthetic face while maintaining the mimic expression (e.g. using the non-PII features extracted) to create modified video data (e.g. converted video data)] and
f) outputting the converted video data. [e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses outputting the modified video data.]
While Chaterjee teaches the method of claim 1 Chaterjee fails to explicitly teach composing a converted video data by identifying PII and non PII features in audio, extracting the non-PII features from audio and using the extracted non-PII features to compose a converted video.
However, Hussen directed to animating avatars from video and audio data teaches identifying non-PII features from audio, extracting the non-PII features from the audio and using non-PII features in both video and audio to generate a converted video. [e.g. Hussen; Abstract, Para. 0028, 0264, 0266, 0273, 0279, 0318-0338 – Hussen discloses animating avatar by utilizing video and audio in separate neural networks for controlling the face of an avatar. Audio is used to further determine facial movement.]
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to include, the features above in the invention as disclosed by Chaterjee in order to provide a more accurate representation of the user's face while speaking when considering both audio and video as disclosed by Hussen in paragraph 0028.
Furthermore, while it is believed that Chatterjee extracts video frames and sound data it is noted that a person having ordinary skill in the art would recognize that the single multimedia file can be separated into separate individual files by demultiplexing/demuxing so that that can be fed into the feature extractors of Hussen yielding to predictable results.
While the combination teaches the method of claim 1 the combinations fails to explicitly teach identifying PII features in audio data.
However, Gkoulalas-Divanis directed to data privacy protection, and more specifically, to a method and system for speaker identity and content de-identification under data privacy guarantees teaches identifying PII and non-PII features in audio and further teaches feature vectors based on linguistic and speaker identity features extracted from voice recordings. [e.g. Gkoulalas-Divanis; Abstract, Para. 0026, 0027, 0069-0105 – Gkoulalas-Divanis discloses transcribing audio into a textual transcription and detecting PII features from the audio and transcription. Gkoulalas-Divanis further discloses training (e.g. learning) feature vectors based on linguistic and speaker identity features extracted from voice recordings (e.g. audio) including pitch, rhythm (e.g. audio feature).]
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to include, the features above in the invention as disclosed by the combination in order to protects personal identities and privacy sensitive personal information in audio which would further enhance the visual privacy protections in the system as disclosed by Chatterjee thus improving anonymization.
Claim 2:
Chaterjee teaches the method of claim 1, wherein step c) further comprises detecting one or more of a face and a body characteristic from the video frames. [e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses detecting facial data (e.g. face characteristics). ]
Claim 3:
Chaterjee teaches the method of claim 2, wherein step d) further comprises g) encoding a detected face or a detected body characteristic; [e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses generating fixed-size vectors (e.g. encoding) from facial data as identity vectors. ] h) learning the detected face or the detected body characteristic.[ e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses projecting the identity vector into a data space (e.g. learning) using neural networks or other machine learning models. ]
Claim 4:
Chaterjee teaches the method of claim 3, wherein step h) further comprises i) learning an affect of the detected face; j) learning an emotion of the detected face; [ e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses projecting the identity vector and mimic expression into a data space using neural networks or other machine learning models. Chatterjee discloses that the artificial identity may maintain features such as emotion if desired which would indicate emotion is learned. ] or k) learning a facial expression of the detected face. [ e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses projecting the mimic expression vector (e.g. facial expression of detected face) into a data space using neural networks or other machine learning models.]
Claim 6:
Chaterjee teaches the method of claim 2, wherein step e) further comprises l) generating a designated or random face based on a detected face, or a designated or random body characteristic based on a detected body characteristic; [ e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses generating artificial/synthetic face (e.g. designated face) consistent with features of the original identity (e.g. detected face). ] m) composing the converted video data based on the designated or random face or the designated or random body characteristic. [ e.g. Chatterjee; Abstract, Para. 0005, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses modifying the video data (e.g. composing converted video data) based on the artificial/synthetic face. ]
Claim 7:
Chaterjee teaches the method of claim 6, wherein the designated face is generated using a neural network as an encoder-decoder structure. [ e.g. Chatterjee; Abstract, Para. 0005, 0007, 0010, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses using techniques such as generative adversarial networks (e.g. a neural network as an encoder-decoder structure) for generating the artificial face. ]
Claim 8:
Chaterjee in view of Hussen teaches the method of claim 2, wherein step e) further comprises n) generating an avatar face; [e.g. Hussen; Abstract, Para. 0028, 0264, 0266, 0273, 0279, 0318-0338 – Hussen discloses generating an avatar face] o) applying non-PII features of the face to the avatar face; [e.g. Hussen; Abstract, Para. 0028, 0264, 0266, 0273, 0279, 0318-0338 – Hussen discloses using the set of characteristics (e.g. non-PII features) to animate the avatar.] p) composing a converted video data based on the avator face and/or the generated body characteristic. [e.g. Hussen; Abstract, Para. 0028, 0264, 0266, 0273, 0279, 0318-0338 – Hussen discloses displaying (e.g. composing a converted video data) an avatar face.]
Claim 9:
While the combination teaches the method of claim 1 and teaches learning audio features the combination fails to explicitly teach transcribing the audio into textual transcriptions and detecting PII features from the audio and transcription.
However, Gkoulalas-Divanis directed to data privacy protection, and more specifically, to a method and system for speaker identity and content de-identification under data privacy guarantees teaches identifying PII features in audio and transcribing and identifying PII features in transcript. [e.g. Gkoulalas-Divanis; Abstract, Para. 0026, 0027, 0069-0105 – Gkoulalas-Divanis discloses transcribing audio into a textual transcription and detecting PII features from the audio and transcription. Gkoulalas-Divanis further discloses training (e.g. learning) feature vectors based on linguistic and speaker identity features extracted from voice recordings (e.g. audio) including pitch, rhythm (e.g. audio feature).]
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to include, the features above in the invention as disclosed by the combination in order to protects personal identities and privacy sensitive personal information in audio which would further enhance the visual privacy protections in the system as disclosed by Chatterjee thus improving anonymization.
Claim 10:
Chaterjee as modified by Gkoulalas-Divanis teaches the method of claim 9, wherein the audio feature is pitch, intensity, speed or fluency of speech. [e.g. Gkoulalas-Divanis; Abstract, Para. 0026, 0027, 0069-0105 –Gkoulalas-Divanis further discloses training (e.g. learning) feature vectors based on linguistic and speaker identity features extracted from voice recordings (e.g. audio) including pitch, rhythm (e.g. audio feature).]. 7, Para. 16 - “…Advanced Encryption Standard symmetric key exchange…”]
Claim 11:
Chaterjee teaches the method of claim 1, wherein step e) further comprises composing the converted video data using face, avatar, portrait, background, and human voice as designated or randomized. [ e.g. Chatterjee; Abstract, Para. 0005, 0007, 0010, 0014, 0015, 0017, 0026, 0099-0115 – Chatterjee discloses an anonymization unit that generates video with seamless transition between generated face (e.g. designated face) and background. ]
Regarding claims 13 and 14 they are they are machine and manufacture claims essentially corresponding to the above recitations, and they are rejected, at least, for the same reasons.
Claims 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over US 20240348747 to Chatterjee et al. (hereinafter “Chaterjee”) in view of US 20210090314 to HUSSEN ABDELAZIZ et al. (hereinafter “Hussen”) and further in view of US 20210335337 to GKOULALAS-DIVANIS et al. (hereinafter “Gkoulalas-Divanis”) and US 20220253676 to VALSTAR et al. (hereinafter “Valstar”)
Claim 5
While the combination teaches the method of claim 1, and it could be argued that body characteristics may broadly be interpreted to mean facial characteristics (which is taught by Chaterjee) the combinations fails to explicitly teach detecting body characteristics comprises one or more of a body motion, a posture, and a gesture
However, Valstar directed to data security and particularly to anonymization of a person in a video or an image while preserving behavior information and to the use of machine learning to identify behavior teaches the detected body characteristics comprises one or more of a body motion, a posture, and a gesture. [e.g. Valstar; Abstract, Claim 13, Para. 0002, 0080 – Valstar discloses audio-visual behavior comprise data relating to the person's body pose, gestures. ]
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to include, the features above in the invention as disclosed by the combination in order to ensure anonymization of a person includes body static body characteristics that can reveal an identity of the person as disclosed by Valstar Para. 0002-0006.
Claim12
While the combination teaches the method of claim 1, the combinations fails to explicitly teach annotation of video.
However, Valstar directed to data security and particularly to anonymization of a person in a video or an image while preserving behavior information and to the use of machine learning to identify behavior teaches annotating converted video. [e.g. Valstar; Abstract, Para. 0039, 0072, 0097, 0100-0102, 0108– Valstar annotating the output video (e.g. converted video data) through an interface (e.g. multi-modal data annotation module). ]
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to include, the features above in the invention as disclosed by the combination in order to indicate the behavior to improve the characterization of the behavior as disclosed by Valstar Para. 0039
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER C HARRIS whose telephone number is (571)270-7841. The examiner can normally be reached Monday through Friday between 8:00 AM to 4:00 PM CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeffrey L Nickerson can be reached on (469) 295-9235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER C HARRIS/Primary Examiner, Art Unit 2432