DETAILED ACTION
The following claims are pending in this office action: 1-22
Claims 1, 11 and 21 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings filed on 09/12/2022 are accepted.
Response to Arguments
Applicant argues that Randall, Kwitek, and Allen are silent regarding “wherein the
avatar emits modified audio derived from the plurality of media data and emulates a plurality of expressions derived from an expression model associated with a user”, and therefore the cited combination fails to teach or suggest the amended limitations.
Applicant's arguments filed 08/18/2026 have been fully considered but they are not persuasive. in view of The rejection has been updated to further rely on Zavesky (US 20240015262 A1) and Chu (US 11127181 B2) to address the newly added avatar-audio-expression limitation. Specifically, Zavesky discloses “transforming an actual representation of the first entity in the recorded interaction to an avatar representation, resulting in an edited interaction of the first entity. Further, the operations can include outputting the edited interaction…by rendering the edited interaction for a second entity.” and “converting the first language into the second language for consumption by the second entity, resulting in a converted audio content, wherein the edited interaction comprises the converted audio content”, Thus, Zavesky teaches or suggests avatar-based modified interaction including modified audio derived from media content.
Further, Chu discloses “Each emotion or each level of one emotion is corresponding to a specific expression group. For example, there are 10 facial expressions for a specific second emotion decision, and the processor 150 may select one from the 10 facial expressions randomly” and “the second emotion decision would be determined through the machine learning technology or the tree-based classification manner, in which the first emotion decisions are the input data of the decision model”.Chu therefore teaches or suggests an avatar that emulates a plurality of expressions derived from an expression model generated using machine learning. Accordingly, when the newly applied teachings of Zavesky and Chu are considered with the previously cited references, the amended limitations are taught or would have been obvious to one of ordinary skill in the art.
Therefore, Applicant’s argument that the prior art is silent regarding the amended avatar-audio-expression limitations is overcome by the introduction of Zavesky and Chu and the rejection is maintained.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/18/2026 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 11-15, 18, 21 and 22 are rejected under 35 U.S.C. §103 as being unpatentable over Randall et al. (US20230269422A1) [hereinafter “Randall”] in view of Kwitek et al (US 11297021 B2) [hereinafter “Kwitek”] and in view of Allen et al. (US 20180113587 A1) [hereinafter “Allen”] and further in view of Zavesky et al(US 20240015262 A1) [hereinafter “Zavesky”] and Chu et al(US 11127181 B2) [hereinafter “Chu”].
As per claim 1, Randall discloses a computer-implemented method for privatizing media feeds comprising:
receiving, by a computing device, a plurality of media data; ( [ Randall, para [0051]” At a second block 204, the data processing system 100 selects portions of the image data on the basis of first data 306 indicating locations within the image data……The first data 306 may be implemented in a number of ways, for example, the first data 306 may include a copy of portions of the image data which are to be selected, pointers to the portions of the image data which are to be selected….”).
monitoring, by the computing device, the plurality of media data; ( [Randall, para [0052]]” The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, to redact or obscure information in the image data. [The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data]. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to identify, (or detect), at least one predetermined class of object represented by the image data”).
determining, by the computing device, at least one vulnerable segment of the plurality of media data, [Randall, para [0052]] “The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, to redact or obscure information in the image data. The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to [identify, (or detect), at least one predetermined class of object] represented by the image data.”).
modifying, by the computing device, the at least one vulnerable segment based on the determination([ Randall, para [0052]] “The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, [to redact or obscure information in the image data]. The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to identify, (or detect), at least one predetermined class of object represented by the image data.”]).
Randall does not disclose
wherein the at least one vulnerable segment is predicted by a machine learning model, indicating the at least one vulnerable segment is a topic or activity associated with a user;
However, Kwitek in the same field of endeavor discloses
wherein the at least one vulnerable segment is predicted by a machine learning model, ( [Kwitek, (col.2 ln25-35)]” The machine learning process 140 learns from the results of the Autonomous rules, and the results of the human screening, to learn from the human screening more about which autonomous rules are properly written, which autonomous rules are suspect, and also to create new autonomous rules for scanning the new content.”) indicating the at least one vulnerable segment is a topic( [Kwitek, (col.3 ln20-25), (col.2 ln15-20)]” The software would recognize patterns and ultimately have predictive behavior elements” and “scan posted materials for improper elements (nudity, curse words, hate speech, fake news and others”)or activity associated with a user; ( [Kwitek, (col.2 ln55-60),(col.2 ln45-50)]” illegal activities or trending events” and “The software…is able to list users or people potentially impacted by the postings or attempted postings”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is predicted by a machine learning model, indicating the at least one vulnerable segment is a topic or activity associated with a user; as suggested by Kwitek. One of ordinary skill in the art would have been motivated to do so because incorporating Kwitek’s ML-based topic and activity classification into Randall’s vulnerable segment determination would predictably improve the accuracy and automation of identifying privacy-sensitive content in media.
The combination of Randall and Kwitek does not disclose
by overlaying an avatar on the user at the at least one vulnerable segment;
wherein the avatar emits modified audio derived from the plurality of media data and emulates a plurality of expressions derived from an expression model associated with the user.
However, Allen in the same field of endeavor discloses
by overlaying( [Allen, [0045]]” applying the media overlay to the media content item (410)”) an avatar( [Allen, [0049]]” an “avatar” of a user is any visual representation of user. The avatar of a user may be based on characteristics derived from images of the user in conjunction with the avatar characteristics…”) on the user( [Allen, [0069]]” For example, FIG. 5A depicts an avatar of a first user with the avatar of a second user on his shirt.”) at the at least one vulnerable segment; ( [Allen, [0050]]” the media overlay in conjunction with a media content item”).
Therefore it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is a prediction of a machine learning model, the prediction indicating the at least one vulnerable segment is a topic or activity associated with a user necessary to be added to a list as suggested by Kwitek to further include by overlaying an avatar on the user at the at least one vulnerable segment;
as taught by Allen. One of ordinary skill in the art would have been motivated to do so because incorporating Allen’s avatar overlay technique into Randall’s media modification framework would predictably provide a privacy-preserving visual substitution for vulnerable content.
The combination of Randall, Kwitek and Allen fails to disclose
wherein the avatar emits modified audio derived from the plurality of media data and emulates a plurality of expressions derived from an expression model associated with the user.
However, Zavesky in the same field of endeavor discloses
wherein the avatar emits modified audio (Zavesky, [Abstract],[claim 14]” transforming an actual representation of the first entity in the recorded interaction to an avatar representation, resulting in an edited interaction of the first entity. Further, the operations can include outputting the edited interaction…by rendering the edited interaction for a second entity.” and “converting the first language into the second language for consumption by the second entity, resulting in a converted audio content, wherein the edited interaction comprises the converted audio content”)derived from the plurality of media data (Zavesky, [0043]) “The evaluation component 106 can evaluate the recorded interaction (e.g., voice, gestures, facial expressions, movements, and so on)”).
Therefore it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is a prediction of a machine learning model, the prediction indicating the at least one vulnerable segment is a topic or activity associated with a user necessary to be added to a list as suggested by Kwitek to further include by overlaying an avatar on the user at the at least one vulnerable segment; as taught by Allen to also include wherein the avatar emits modified audio derived from the plurality of media data as suggested by Zavesky. One of ordinary skill in the art would have been motivated to do so because incorporating Zavesky’s avatar-based interaction editing technique, including converted audio and facial-expression mimicking, into Randall’s modified-media framework would predictably preserve the user’s speech content and expressive cues while replacing the user’s actual representation with an avatar.
The combination of Randall, Kwitek, Allen and Zavesky fails to disclose
emulates a plurality of expressions [derived from an expression model associated with the user]
However, Chu in the same field of endeavor discloses
emulates a plurality of expressions]( Chu, [col.10, ln1-10]” Each emotion or each level of one emotion is corresponding to a specific expression group. For example, there are 10 facial expressions for a specific second emotion decision, and the processor 150 may select one from the 10 facial expressions randomly”) [derived from an expression model associated with the user]( Chu, [col.8, ln25-35], [col.8 ln25-34]” the second emotion decision would be determined through the machine learning technology or the tree-based classification manner, in which the first emotion decisions are the input data of the decision model.”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify the system created by combination of Randall, Kwitek, Allen and Zavesky to include emulates a plurality of expressions [derived from an expression model associated with the user] as suggested by Chu. One of ordinary skill in the art would have been motivated to do so because incorporating Chu’s user-data-driven emotion classification and corresponding avatar facial-expression generation technique into the combined system would predictably enable the avatar to select from multiple facial expressions according to an emotion model responsive to the user data.
As per claim 2, the combination of Randall, Kwitek, Allen, Zavesky and Chu discloses the computer-implemented method of claim 1. Kwitek further discloses wherein determining the at least one vulnerable segment comprises:
utilizing the machine learning model to classify one or more frames of the plurality of media data as vulnerable; ( [Kwitek, (8),(11)]” A computer 100, e.g. either server or client computer, first receives user content being posted online 105, from many different social media users 110. This run through an autonomous screening filter 115, which includes software to scan posted materials for improper elements (nudity, curse words, hate speech, fake news and others). The filter 115 can be updated from time to time, to learn different categories at different times” and “The results are passed through a Machine learning process 140. The machine learning process 140 learns from the results of the Autonomous rules, and the results of the human screening, to learn from the human screening more about which autonomous rules are properly written, which autonomous rules are suspect, and also to create new autonomous rules for scanning the new content.”).
Kwitek does not disclose encrypting and redacting, by the computing device, the one or more frames based on the machine learning model.
However, Randall discloses encrypting[Randall, [0112]]“ modifying the selected portions of the image data comprises encrypting the selected portions of the image data.”) and redacting, [Randall, [0112]]“ the modification of the selected portions of the image data by blurring, replacing, and/or redacting the selected portions of the image data, other examples are also envisaged.”) by the computing device, the one or more frames[Randall, [0003]]“ A video stream may generally be considered to be a sequence of digital signals, or packets of data, representing a plurality of frames of image data”)based on the machine learning model. (Randall, [0168], [claim 1]” A computer-implemented method according to any previous clause, further comprising generating the first data indicating locations within the image data by processing the plurality of frames of the image data using an artificial neural network to identify at least one predetermined class of object represented by the image data” and “ selecting portions of the image data on the basis of first data indicating locations within the image data; modifying the selected portions to generate modified portions of the image data;”). Claim 2 is rejected under the same rationale as claim 1.
As per claim 3, the combination of Randall, Kwitek, Allen, Zavesky and Chu discloses the computer-implemented method of claim 2. Randall further discloses wherein at least one vulnerable material is determined within the one or more frames[Randall, [0094],[0052]]“ However, in other examples, it may be sufficient to modify a subset of the plurality of frames of the image data that include the sensitive information” and “ selected portions of the image data represent objects in the frames of image data which are to be modified”).
Randall fails to disclose by comparing, by the computing device, a plurality of user expressions to a pre-defined expressions list.
However, Kwitek discloses by comparing, by the computing device, a plurality of user expressions to a pre-defined expressions list. [Kwitek, claim1, (8)]“ operating to carry out a first autonomous screening of the content to use rules to determine whether the content meets a posting criteria, and categorizing the content as to whether the autonomous screening indicates that the content has met the posting criteria” and “autonomous screening filter 115, which includes software to scan posted materials for improper elements (nudity, curse words, hate speech, fake news and others). ”). Claim 3 is rejected under the same rationale as claim 2 above.
As per claim 4, the combination of Randall, Kwitek, Allen, Zavesky and Chu discloses the computer-implemented method of claim 1. Randall further discloses wherein modifying the at least one vulnerable segment comprises altering, by the computing device, a video stream element of the at least one vulnerable segment preventing it from being emitted in an original state. ([Randall, para. [0057], [0094], [0070]]:“ The data processing system 100 [modifies, at a third block 206], the selected portions of the image data to generate modified portions…”[0057]… “In the examples described above with respect to FIGS. 4 and 5, the selected portions of the image data which are processed to create the modified portions of the image data 308a to 308c are selected for every frame of the image data in which the [sensitive information to be removed is present]. However, in other examples, it may be sufficient to modify a subset of the plurality of frames of the image data that include the sensitive information which is to be removed. That is to say, the selected portions of the image data may correspond to a subset of the plurality of frames of image data 304a to 304h in the video stream 302 even when more frames of the image data in the video stream 302 contain the sensitive information…”[0094]….” Encrypting the second data 310 may, in some instances, include encrypting the second data 310 according to a single-use encryption key. A single-use encryption key allows the second data 310 to be decrypted once so that it can be used to recover the selected portions. Thereafter, the second data 310 is no longer able to be decrypted by a user to whom it is provided. This allows the selected portions of the image data to be recovered for a single limited and specific circumstance while maintaining the general security of the information represented by the second data 310. T[his is of particular use where the method 200 is implemented for a video stream 302 which contains highly sensitive information…]”[0070]. The examiner interprets the “ altering” of the vulnerable segment as including encryption of the selected portions of the image.
As per claim 5, the combination of Randall, Kwitek, Allen, Zavesky and Chu discloses the computer-implemented method of claim 3. Allen further discloses wherein modifying the at least one vulnerable segment comprises: generating, by the computing device, the avatar configured to emulate the at least one vulnerable material; [Allen, [0049]]“ As used herein, an “avatar” of a user is any visual representation of user.”)and overlaying, by the computing device, the avatar[Allen, claim 1]“ a processor; a user interface coupled to the processor, the user interface comprising an input device and a display screen; and memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving a media content item; retrieving, from the memory, avatar characteristics associated with a user of the system; generating a media overlay, based on the user's avatar characteristics, that includes an avatar of the user; and displaying the media overlay with the media content item on the display screen of the user interface”).
Allen does not disclose over at least one of the one or more frames comprising the at least one vulnerable segment.
However, Randall discloses over at least one of the one or more frames comprising the at least one vulnerable segment. [Randall, [0094],[0052]]“ However, in other examples, it may be sufficient to modify a subset of the plurality of frames of the image data that include the sensitive information” and “ selected portions of the image data represent objects in the frames of image data which are to be modified”). Claim 5 is rejected under the same rationale as claimed 3.
As per claim 11, Randall discloses a computer system for privatizing media feeds, the computer system comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations comprising:
receiving, by a computing device, a plurality of media data; ( [ Randall, para [0051]” At a second block 204, the data processing system 100 selects portions of the image data on the basis of first data 306 indicating locations within the image data……The first data 306 may be implemented in a number of ways, for example, the first data 306 may include a copy of portions of the image data which are to be selected, pointers to the portions of the image data which are to be selected….”).
monitoring, by the computing device, the plurality of media data; ( [Randall, para [0052]]” The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, to redact or obscure information in the image data. [The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data]. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to identify, (or detect), at least one predetermined class of object represented by the image data”).
determining, by the computing device, at least one vulnerable segment of the plurality of media data, [Randall, para [0052]] “The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, to redact or obscure information in the image data. The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to [identify, (or detect), at least one predetermined class of object] represented by the image data.”).
modifying, by the computing device, the at least one vulnerable segment based on the determination([ Randall, para [0052]] “The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, [to redact or obscure information in the image data]. The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to identify, (or detect), at least one predetermined class of object represented by the image data.”]).
Randall does not disclose
wherein the at least one vulnerable segment is predicted by a machine learning model, indicating the at least one vulnerable segment is a topic or activity associated with a user;
However, Kwitek in the same field of endeavor discloses
wherein the at least one vulnerable segment is predicted by a machine learning model, ( [Kwitek, (col.2 ln25-35)]” The machine learning process 140 learns from the results of the Autonomous rules, and the results of the human screening, to learn from the human screening more about which autonomous rules are properly written, which autonomous rules are suspect, and also to create new autonomous rules for scanning the new content.”) indicating the at least one vulnerable segment is a topic( [Kwitek, (col.3 ln20-25), (col.2 ln15-20)]” The software would recognize patterns and ultimately have predictive behavior elements” and “scan posted materials for improper elements (nudity, curse words, hate speech, fake news and others”)or activity associated with a user; ( [Kwitek, (col.2 ln55-60),(col.2 ln45-50)]” illegal activities or trending events” and “The software…is able to list users or people potentially impacted by the postings or attempted postings”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is predicted by a machine learning model, indicating the at least one vulnerable segment is a topic or activity associated with a user; as suggested by Kwitek. One of ordinary skill in the art would have been motivated to do so because incorporating Kwitek’s ML-based topic and activity classification into Randall’s vulnerable segment determination would predictably improve the accuracy and automation of identifying privacy-sensitive content in media.
The combination of Randall and Kwitek does not disclose
by overlaying an avatar on the user at the at least one vulnerable segment;
wherein the avatar emits modified audio derived from the plurality of media data and emulates a plurality of expressions derived from an expression model associated with the user.
However, Allen in the same field of endeavor discloses
by overlaying( [Allen, [0045]]” applying the media overlay to the media content item (410)”) an avatar( [Allen, [0049]]” an “avatar” of a user is any visual representation of user. The avatar of a user may be based on characteristics derived from images of the user in conjunction with the avatar characteristics…”) on the user( [Allen, [0069]]” For example, FIG. 5A depicts an avatar of a first user with the avatar of a second user on his shirt.”) at the at least one vulnerable segment; ( [Allen, [0050]]” the media overlay in conjunction with a media content item”).
Therefore it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is a prediction of a machine learning model, the prediction indicating the at least one vulnerable segment is a topic or activity associated with a user necessary to be added to a list as suggested by Kwitek to further include by overlaying an avatar on the user at the at least one vulnerable segment;
as taught by Allen. One of ordinary skill in the art would have been motivated to do so because incorporating Allen’s avatar overlay technique into Randall’s media modification framework would predictably provide a privacy-preserving visual substitution for vulnerable content.
The combination of Randall, Kwitek and Allen fails to disclose
wherein the avatar emits modified audio derived from the plurality of media data and emulates a plurality of expressions derived from an expression model associated with the user.
However, Zavesky in the same field of endeavor discloses
wherein the avatar emits modified audio (Zavesky, [Abstract],[claim 14]” transforming an actual representation of the first entity in the recorded interaction to an avatar representation, resulting in an edited interaction of the first entity. Further, the operations can include outputting the edited interaction…by rendering the edited interaction for a second entity.” and “converting the first language into the second language for consumption by the second entity, resulting in a converted audio content, wherein the edited interaction comprises the converted audio content”)derived from the plurality of media data (Zavesky, [0043]) “The evaluation component 106 can evaluate the recorded interaction (e.g., voice, gestures, facial expressions, movements, and so on)”) .
Therefore it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is a prediction of a machine learning model, the prediction indicating the at least one vulnerable segment is a topic or activity associated with a user necessary to be added to a list as suggested by Kwitek to further include by overlaying an avatar on the user at the at least one vulnerable segment; as taught by Allen to also include wherein the avatar emits modified audio derived from the plurality of media data as suggested by Zavesky. One of ordinary skill in the art would have been motivated to do so because incorporating Zavesky’s avatar-based interaction editing technique, including converted audio and facial-expression mimicking, into Randall’s modified-media framework would predictably preserve the user’s speech content and expressive cues while replacing the user’s actual representation with an avatar.
The combination of Randall, Kwitek, Allen and Zavesky fails to disclose
emulates a plurality of expressions [derived from an expression model associated with the user]
However, Chu in the same field of endeavor discloses
emulates a plurality of expressions]( Chu, [col.10, ln1-10]” Each emotion or each level of one emotion is corresponding to a specific expression group. For example, there are 10 facial expressions for a specific second emotion decision, and the processor 150 may select one from the 10 facial expressions randomly”) [derived from an expression model associated with the user]( Chu, [col.8, ln25-35], [col.8 ln25-34]” the second emotion decision would be determined through the machine learning technology or the tree-based classification manner, in which the first emotion decisions are the input data of the decision model.”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify the system created by combination of Randall, Kwitek, Allen and Zavesky to include emulates a plurality of expressions [derived from an expression model associated with the user] as suggested by Chu. One of ordinary skill in the art would have been motivated to do so because incorporating Chu’s user-data-driven emotion classification and corresponding avatar facial-expression generation technique into the combined system would predictably enable the avatar to select from multiple facial expressions according to an emotion model responsive to the user data.
As per claim 12, the substance of the claimed invention is identical or substantially similar to that of claim 2. Accordingly, this claim is rejected under the same rationale.
As per claim 13, the substance of the claimed invention is identical or substantially similar to that of claim 3. Accordingly, this claim is rejected under the same rationale.
As per claim 14, the substance of the claimed invention is identical or substantially similar to that of claim 4. Accordingly, this claim is rejected under the same rationale.
As per claim 15, the substance of the claimed invention is identical or substantially similar to that of claim 5. Accordingly, this claim is rejected under the same rationale.
As per claim 18, the combination of Randall, Kwitek, Allen, Zavesky and Chu discloses, The computer system of claim 11. Randall further discloses wherein the plurality of media data comprises one or more of video data, audio data, LIDAR data, sonar data, temperature data, and infrared data. ([Randall, para. [0003]]: “Video streams may also comprise other data, such as audio data, and metadata…” and further teaches that “Image sensors may operate in the visible light spectrum, but may additionally, or alternatively, include sensors which operate outside of the visible light spectrum, for example, the infrared spectrum.”[0038].
The examiner interprets this as disclosing that the “plurality of media data’ includes any data captured from an environment, and therefore reasonably encompasses video, audio, LIDAR, sonar, infrared, and temperature data under the broadest reasonable interpretation.
As per claim 21, Randall discloses a computer program product for privatizing media feeds, the computer program product comprising: one or more computer-readable tangible storage media and program instructions stored on at least one of the one or more tangible storage media, the program instructions executable by a processor capable of performing a method, the method comprising:
receiving, by a computing device, a plurality of media data; ( [ Randall, para [0051]” At a second block 204, the data processing system 100 selects portions of the image data on the basis of first data 306 indicating locations within the image data……The first data 306 may be implemented in a number of ways, for example, the first data 306 may include a copy of portions of the image data which are to be selected, pointers to the portions of the image data which are to be selected….”).
monitoring, by the computing device, the plurality of media data; ( [Randall, para [0052]]” The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, to redact or obscure information in the image data. [The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data]. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to identify, (or detect), at least one predetermined class of object represented by the image data”).
determining, by the computing device, at least one vulnerable segment of the plurality of media data, [Randall, para [0052]] “The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, to redact or obscure information in the image data. The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to [identify, (or detect), at least one predetermined class of object] represented by the image data.”).
modifying, by the computing device, the at least one vulnerable segment based on the determination([ Randall, para [0052]] “The selected portions of the image data represent objects in the frames of image data which are to be modified, for example, [to redact or obscure information in the image data]. The image masks (the first data 306), may be generated automatically using object detection algorithms performed on the image data. For example, image masks can be generated by processing the plurality of frames of the image data 304a to 304h using computer vision techniques to identify, (or detect), at least one predetermined class of object represented by the image data.”]).
Randall does not disclose
wherein the at least one vulnerable segment is predicted by a machine learning model, indicating the at least one vulnerable segment is a topic or activity associated with a user;
However, Kwitek in the same field of endeavor discloses
wherein the at least one vulnerable segment is predicted by a machine learning model, ( [Kwitek, (col.2 ln25-35)]” The machine learning process 140 learns from the results of the Autonomous rules, and the results of the human screening, to learn from the human screening more about which autonomous rules are properly written, which autonomous rules are suspect, and also to create new autonomous rules for scanning the new content.”) indicating the at least one vulnerable segment is a topic( [Kwitek, (col.3 ln20-25), (col.2 ln15-20)]” The software would recognize patterns and ultimately have predictive behavior elements” and “scan posted materials for improper elements (nudity, curse words, hate speech, fake news and others”)or activity associated with a user; ( [Kwitek, (col.2 ln55-60),(col.2 ln45-50)]” illegal activities or trending events” and “The software…is able to list users or people potentially impacted by the postings or attempted postings”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is predicted by a machine learning model, indicating the at least one vulnerable segment is a topic or activity associated with a user; as suggested by Kwitek. One of ordinary skill in the art would have been motivated to do so because incorporating Kwitek’s ML-based topic and activity classification into Randall’s vulnerable segment determination would predictably improve the accuracy and automation of identifying privacy-sensitive content in media.
The combination of Randall and Kwitek does not disclose
by overlaying an avatar on the user at the at least one vulnerable segment;
wherein the avatar emits modified audio derived from the plurality of media data and emulates a plurality of expressions derived from an expression model associated with the user.
However, Allen in the same field of endeavor discloses
by overlaying( [Allen, [0045]]” applying the media overlay to the media content item (410)”) an avatar( [Allen, [0049]]” an “avatar” of a user is any visual representation of user. The avatar of a user may be based on characteristics derived from images of the user in conjunction with the avatar characteristics…”) on the user( [Allen, [0069]]” For example, FIG. 5A depicts an avatar of a first user with the avatar of a second user on his shirt.”) at the at least one vulnerable segment; ( [Allen, [0050]]” the media overlay in conjunction with a media content item”).
Therefore it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is a prediction of a machine learning model, the prediction indicating the at least one vulnerable segment is a topic or activity associated with a user necessary to be added to a list as suggested by Kwitek to further include by overlaying an avatar on the user at the at least one vulnerable segment;
as taught by Allen. One of ordinary skill in the art would have been motivated to do so because incorporating Allen’s avatar overlay technique into Randall’s media modification framework would predictably provide a privacy-preserving visual substitution for vulnerable content.
The combination of Randall, Kwitek and Allen fails to disclose
wherein the avatar emits modified audio derived from the plurality of media data and emulates a plurality of expressions derived from an expression model associated with the user.
However, Zavesky in the same field of endeavor discloses
wherein the avatar emits modified audio (Zavesky, [Abstract],[claim 14]” transforming an actual representation of the first entity in the recorded interaction to an avatar representation, resulting in an edited interaction of the first entity. Further, the operations can include outputting the edited interaction…by rendering the edited interaction for a second entity.” and “converting the first language into the second language for consumption by the second entity, resulting in a converted audio content, wherein the edited interaction comprises the converted audio content”)derived from the plurality of media data (Zavesky, [0043]) “The evaluation component 106 can evaluate the recorded interaction (e.g., voice, gestures, facial expressions, movements, and so on)”) and emulates a plurality of expressions(Zavesky, [0080]” Based on the first facial expressions of the first user, the transformation component 108 can transform a first visual representation of the first user into a first avatar representation that mimics the first facial expressions of the first user.”).
Therefore it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify Randall to include wherein the at least one vulnerable segment is a prediction of a machine learning model, the prediction indicating the at least one vulnerable segment is a topic or activity associated with a user necessary to be added to a list as suggested by Kwitek to further include by overlaying an avatar on the user at the at least one vulnerable segment; as taught by Allen to also include wherein the avatar emits modified audio derived from the plurality of media data as suggested by Zavesky. One of ordinary skill in the art would have been motivated to do so because incorporating Zavesky’s avatar-based interaction editing technique, including converted audio and facial-expression mimicking, into Randall’s modified-media framework would predictably preserve the user’s speech content and expressive cues while replacing the user’s actual representation with an avatar.
The combination of Randall, Kwitek, Allen and Zavesky fails to disclose
a plurality of expressions [derived from an expression model associated with the user]
However, Chu in the same field of endeavor discloses
emulates a plurality of expressions]( Chu, [col.10, ln1-10]” Each emotion or each level of one emotion is corresponding to a specific expression group. For example, there are 10 facial expressions for a specific second emotion decision, and the processor 150 may select one from the 10 facial expressions randomly”) [derived from an expression model associated with the user]( Chu, [col.8, ln25-35], [col.8 ln25-34]” the second emotion decision would be determined through the machine learning technology or the tree-based classification manner, in which the first emotion decisions are the input data of the decision model.”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify the system created by combination of Randall, Kwitek, Allen and Zavesky to include emulates a plurality of expressions [derived from an expression model associated with the user] as suggested by Chu. One of ordinary skill in the art would have been motivated to do so because incorporating Chu’s user-data-driven emotion classification and corresponding avatar facial-expression generation technique into the combined system would predictably enable the avatar to select from multiple facial expressions according to an emotion model responsive to the user data.
As per claim 22, the substance of the claimed invention is identical or substantially similar to that of claim 2. Accordingly, this claim is rejected under the same rationale.
Claims 7, 17 are rejected under 35 U.S.C. §103 as being unpatentable over Randall et al. (US20230269422A1) [hereinafter “Randall”] in view of Kwitek et al (US 11297021 B2) [hereinafter “Kwitek”] and in view of Allen et al. (US 20180113587 A1) [hereinafter “Allen”] and in view of Zavesky et al(US 20240015262 A1) [hereinafter “Zavesky”] and Chu et al(US 11127181 B2) [hereinafter “Chu”] as applied to claim 1, 11 and further in view of Dufaux et al. (Scrambling for Privacy Protection in Video Surveillance Systems) [hereinafter (“Dufaux”)].
As per claim 7, the combination of Randall, Kwitek, Allen, Zavesky and Chu discloses the computer-implemented method of claim 1. The combination fails to disclose determining, by the computing device, a receiving party for the plurality of media data including the modified vulnerable segment; and selecting, by the computing device, a variation of the plurality of media data based on determination.
However, Dufaux in the same field of endeavor discloses determining, by the computing device, a receiving party for the plurality of media data including the modified vulnerable segment; and selecting, by the computing device, a variation of the plurality of media data based on determination. ([Dufaux, pp. 1169-1170]” The same scrambled codestream is transmitted to all terminals independently from their access rights. Unauthorized clients, who do not possess the secret key, can only view a distorted version of the content where privacy-sensitive data is concealed. Conversely, authorized clients, e.g., law-enforcement authorities, can unscramble the codestream and recover the truthful scene”)
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify the system created by combination of the combination of Randall, Kwitek, Allen, Zavesky and Chu to include determining, by the computing device, a receiving party for the plurality of media data including the modified vulnerable segment; and selecting, by the computing device, a variation of the plurality of media data based on determination as suggested by Dufaux. One of ordinary skill in the art would have been motivated to do so because Randall provides redacted content while Dufaux teaches a well-established privacy-preserving approach in which different viewing parties receive different visual variations of the same media based on authorization level. Incorporating Dufaux’s selective-access model into Randall would allow Randall’s redacted outputs to be tailored to specific viewer privileges enhancing the privacy-preservation system by aligning the degree of redaction with the viewer’s authorization rights.
As per claim 17, the substance of the claimed invention is identical or substantially similar to that of claim 7. Accordingly, this claim is rejected under the same rationale.
Claims 6 is rejected under 35 U.S.C. §103 as being unpatentable over Randall et al. (US20230269422A1) [hereinafter “Randall”] in view of Kwitek et al (US 11297021 B2) [hereinafter “Kwitek”] and in view of Allen et al. (US 20180113587 A1) [hereinafter “Allen”] and in view of Zavesky et al(US 20240015262 A1) [hereinafter “Zavesky”] and Chu et al(US 11127181 B2) [hereinafter “Chu”] as applied to claim 1 and further in view of Genay et al. (A. Genay, A. Lécuyer and M. Hachet, "Being an Avatar “for Real”: A Survey on Virtual Embodiment in Augmented Reality," in IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 12, pp. 5071-5090, 1 Dec. 2022, doi: 10.1109/TVCG.2021.3099290.) [hereinafter (“Genay”)] .
As per claim 6, the references as combined above disclose the computer-implemented method of claim 5. The combination fails to disclose wherein the avatar is configured to be embedded in at least one of a virtual reality environment, an augmented reality environment, or a mixed reality environment.
However, Genay discloses wherein the avatar is configured to be embedded in at least one of a virtual reality environment, an augmented reality environment, or a mixed reality environment. ([Genay, abstract]” Virtual self-avatars have been increasingly used in Augmented Reality (AR) where one can see virtual content embedded into physical space.” Therefore, it would have been obvious before the effective filling date of the claimed invention for one of the ordinary skill in the art to modify Randall to further include the avatar is configured to be embedded in at least one of a virtual reality environment, an augmented reality environment, or a mixed reality environment as suggested by Genay. One of the ordinary skill in the art would have been motivated to do so because Randall teaches redacting or modifying vulnerable material, while Lui teaches generating avatars as replacements, and Genay teaches the use of avatars embedded into virtual, augmented, or mixed-reality environments for presenting privacy-preserving representations of individuals. Incorporating these teachings would have predictably improved Randall by enabling avatar-based privacy protection across additional digital environments such as VR/AR.
Claims 9, 19 are rejected under 35 U.S.C. §103 as being unpatentable over Randall et al. (US20230269422A1) [hereinafter “Randall”] in view of Kwitek et al (US 11297021 B2) [hereinafter “Kwitek”] and in view of Allen et al. (US 20180113587 A1) [hereinafter “Allen”] and in view of Zavesky et al(US 20240015262 A1) [hereinafter “Zavesky”] and Chu et al(US 11127181 B2) [hereinafter “Chu”] as applied to claim 2 and 11 and further in view in view of Owens et al. (US 20160321260 A1) [hereinafter “Owens”].
As per claim 9, the combination of Randall, Kwitek, Allen, Zavesky and Chu discloses the computer-implemented method of claim 2. wherein modifying the at least one vulnerable segment comprises: merging, by the computing device, the plurality of media data one or more encrypted[Randall, [0094],[0070]]“ Encrypting the second data 310 may, in some instances, include encrypting the second data 310 according to a single-use encryption key. A single-use encryption key allows the second data 310 to be decrypted once so that it can be used to recover the selected portions”)frames[Randall, [0094]]“ the selected portions of the image data may correspond to a subset of the plurality of frames of image data 304a to 304h in the video stream 302 even when more frames of the image data in the video stream 302 contain the sensitive information”).
Randall does not disclose and the plurality of media data in a video stream comprising permission metadata; wherein the video stream is transmitted based on the permission metadata.
However, Owens in the same field of endeavor discloses the plurality of media data in a video stream comprising permission metadata; ( [Owens, [0029], [0030]])”The content analysis module 104 can analyze content items based on signals indicative of objectionable material in the content items. For example, the signals indicative of objectionable material can include a domain signal relating to a number of links to websites that identify the objectionable material as false, the links provided in response to content items published by a domain associated with a publisher of the content items. As another example, the signals indicative of objectionable material can include a keyword signal relating to a number of responses including one or more keywords indicative of objectionable material to a content item.” [0029] “Based on one or more of the signals indicative of objectionable material, a content item that likely contains objectionable material can be identified.” [0030]. wherein the video stream is transmitted based on the permission metadata([Owens, [0047]]” … The signal analysis module 206 in this regard can prioritize an affirmative decision of a user to receive content items from a domain over any contrary determinations regarding the presence of objectionable material in content items from the domain. In various embodiments, when a content item…… objectionable material”).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to modify the system created by the combination of the combination of Randall, Kwitek, Allen, Zavesky and Chu to further include the plurality of media data in a video stream comprising permission metadata; wherein the video stream is transmitted based on the permission metadata as suggested by Owens. One of the ordinary skill in the art would have been motivated to do so because incorporating Owens’s metadata-based modification and selective transmission, making the combination a predictable improvement for controlling privacy-conditioned media distribution.
As per claim 19, the substance of the claimed invention is identical or substantially similar to that of claim 9. Accordingly, this claim is rejected under the same rationale.
Claims 10, 20 are rejected under 35 U.S.C. §103 as being unpatentable over Randall et al. (US20230269422A1) [hereinafter “Randall”] in view of Kwitek et al (US 11297021 B2) [hereinafter “Kwitek”] and in view of Allen et al. (US 20180113587 A1) [hereinafter “Allen”] and in view of Zavesky et al(US 20240015262 A1) [hereinafter “Zavesky”] and Chu et al(US 11127181 B2) [hereinafter “Chu”] as applied to claim 1 and claim 11 and further in view of Miller et al. (US 202000162236 A1) [hereinafter (“Miller”)].
As per claim 10, the combination of Randall, Kwitek and Allen discloses the computer-implemented method of claim 1. The combination does not explicitly disclose wherein the plurality of media data are maintained on a blockchain comprising a plurality of cryptographically linked blocks.
However, Miller in the same field of endeavor discloses wherein the plurality of media data are maintained on a blockchain comprising a plurality of cryptographically linked blocks.([Miller, para. [[0095], [0111], [0116], [0120]]” In various embodiments, and in response to a user request, the reference link to the copy of the document stored in the document repository may be accessed via the block. The accessed reference link may be employed, via document repository synchronizer 328, to retrieve and/or recall the stored copy of the document.” [0095]” That is, the current block may be added to a [distributed blockchain-like ledger]. The current block may include at least one of the current fingerprint of the image, a reference to a previous block included in the distributed ledger, or a fingerprint of the previous block. The previous block may encode a previous transaction that includes a previous fingerprint of the image that corresponds to the previous state of the image” [0111]. “Metadata associated with the document may be stored in the blockchain”. [0116]. “In various embodiments directed towards maintaining a distributed ledger across a plurality of nodes, ……. component is configured to run only when the image editing application is currently in use.” [0120]). ”
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to integrate into Randall the blockchain comprising a plurality of cryptographically linked blocks as suggested by Miller. One of ordinary skill in the art would have been motivated to do so because both references deal with managing sensitive visual data. Using a blockchain as taught by Miller would provide predictable benefits such as immutability, tamper detection, provenance tracking, and secure audit logs enhancements directly applicable to Randall’s output media management. Thus, applying Miller’s secure ledger to Randall’s media storage would represent a straightforward substitution of one known storage technique with another known secure alternative. Incorporating Miller’s blockchain storage into Randall’s would thus enhance integrity and traceability of redacted media data using a known secure-ledger approach.
As per claim 20, the substance of the claimed invention is identical or substantially similar to that of claim 10. Accordingly, this claim is rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure:
Modica et al., (US 20170358204 A1) discloses method, apparatus, and computer product for processing sensor data.
Tan et al., (US 20240054246 A1) discloses anonymizing caller identity based on voice print match.
Schoen et al., (US 20130151346 A1) discloses redacting portions of advertisements delivered to underage users.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Komi N. AMEVIGBE whose telephone number is (571)272-3381. The examiner can normally be reached Monday-Friday 2pm-10pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carl Colin can be reached at (571) 272-3862. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.N.A./Examiner, Art Unit 2493 /CARL G COLIN/Supervisory Patent Examiner, Art Unit 2493
Prosecution Insights