DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-14 are pending under this Office action.
Claim Objections
Claim 1 is objected to because of the following informalities: the term of “a second device” occurs twice in the independent clam 1, the second occurrence of “a second device” may be “the second device”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-9, and 11-14 are rejected under 35 U.S.C. 103 as being unpatentable over Jung, etc. (US 8203609 B2) in view of Astarabadi, etc. (US 20200358983 A1), further in view of Lee, etc. (US 20140147023 A1) and Zavesky (US 20130141530 A1).
Regarding claim 1, Jung teaches that a personal data processing system (See Jung: Fig. 24, and Col. 21 Lines 20-29, “FIG. 24 illustrates an exemplary embodiment of a system 1600. The system includes an imaging device 1610, an anonymization device 1650, and a broadcast device. The imaging device is operable to capture an image depicting a class of subjects 1605 that may have a temporal presence in an area 1603. The broadcast device is operable to display a human-perceivable indication of a selected anonymization policy to at least a portion of the area. The anonymization policy includes obscuring a property of the class of subjects depicted in the captured image”) comprising:
a first device configured to capture image data including facial image data of a human being (See Jung: Fig. 3, and Col. 13 Lines 16-27, “In an embodiment, the imaging device 320 operable to acquire an image of a subject further includes a digital imaging device operable to acquire an image of a subject of the potential image subject(s) 305. In another embodiment, the imaging device operable to acquire an image of a subject further includes a digital camera operable to acquire an image of a subject. For example, the digital camera may include a lens 322, an imaging element 324, an analog to digital converter 326, and/or a processor 328. In a further embodiment, the imaging device operable to acquire an image of a subject further includes an imaging device operable to capture an image of a subject of the potential image subject(s) 305”),
the facial image data of a human being including a plurality of identification features each having a characteristic (See Jung: Fig. 24, and Col. 21 Lines 19-32, “In a further embodiment, the anonymization device further includes an anonymization device having a selector circuit operable to select an anonymization policy, where the anonymization policy includes obscuring at least one of a variable, a recognizable, a distinguishable, and/or a unique aspect of a property of the class of subjects depicted in the captured image. In another embodiment, the anonymization device further includes an anonymization device having a selector circuit operable to select an anonymization policy, where the anonymization policy includes obscuring at least one of a variable, a recognizable, a distinguishable, and/or a unique aspect of a property of each instance of the class of subjects depicted in the captured image”. Note that the unique aspect of the property of the subject may be the identification features of the face image) and a mimic expression feature having a characteristic,
wherein the first device is further configured to detect the facial image data of the human being in the image data (See Jung: Figs. 3-5, and Col. 14 Lines 66-67 ~ Col. 15 Lines 1-18, “FIG. 5 illustrates an embodiment where the display 440B of the image capture device 420 indicating a "Green" color-based visual indication of the anonymization policy, corresponding to decreasing a fidelity of facial properties and license plate properties depicted in acquired images to be indistinguishable. The decreased fidelity of the illustrated embodiment includes a decreased fidelity of facial properties of human subjects 408A, 408B, and 408C implemented by obscuring portions of their faces and/or heads. The faces and/or heads of human subjects may be located in the image using techniques known to those skilled in the art, including artificial intelligence, and/or facial recognition techniques. FIG. 5 also illustrates a decreased fidelity of the identification number "XYZ 123" property of a license plate of a car subject 406 in the image that was implemented by an obscuring portion over the license identification number. In an embodiment, the image 401 may be used to study human and/or vehicle traffic patterns in the area 403 while preserving the anonymity of human subjects and/or registered owners of vehicles”. Note that using face recognition technique to recognize face in the image may be the face detection),
to determine a descriptor associated with the facial image data of the human being and including information describing the characteristic of the mimic expression feature and the characteristic of each identification feature of a selectable subset of the plurality of identification features,
wherein the selectable subset of the plurality of identification features does not allow a unique identification of the human being (See Jung: Fig. 3, and Col. 13 Lines 17-36, “In an embodiment, the anonymizer circuit 330 operable to generate an anonymized image that includes a decreased fidelity of a property of the subject of the acquired image in response to an anonymization policy further includes an anonymizer circuit operable to generate an anonymized image that includes at least one of a decreased resolution of a property, an obscuring of a property, a blackout of a property, and/or a removal of a property of the subject of the acquired image in response to an anonymization policy. In a further embodiment, the anonymizer circuit further includes an anonymizer circuit operable to generate an anonymized image that includes a decreased fidelity of an aspect, a property, and/or an attribute of the subject of the acquired image in response to an anonymization policy. In another embodiment, the anonymizer circuit further includes an anonymizer circuit operable to generate an anonymized image that includes a decreased fidelity of at least one of a face, a license plate, a label, and/or a recognizable property associated with the subject of the acquired image in response to an anonymization policy”. Note that the blackout or removal properties may be the identification features that does not allow unique identification of the human being),
wherein the first device is further configured to provide an obfuscated image (See Jung: Fig. 3-5, and Col. 14 Lines 58-67 ~ Col. 15 Lines 1-18, “FIG. 5 illustrates an image 401 of the exemplary embodiment of an environment 400 of FIG. 4 with a decreased fidelity of a property of a subject. In an embodiment, the decreased fidelity may be implemented and/or practiced using systems, devices, apparatus, and/or methods disclosed herein. For example, the system 300 described in conjunction with FIG. 3 may be used to implement and/or practice the decreased fidelity of a property of a subject depicted in an image 405. For example, FIG. 5 illustrates an embodiment where the display 440B of the image capture device 420 indicating a "Green" color-based visual indication of the anonymization policy, corresponding to decreasing a fidelity of facial properties and license plate properties depicted in acquired images to be indistinguishable. The decreased fidelity of the illustrated embodiment includes a decreased fidelity of facial properties of human subjects 408A, 408B, and 408C implemented by obscuring portions of their faces and/or heads. The faces and/or heads of human subjects may be located in the image using techniques known to those skilled in the art, including artificial intelligence, and/or facial recognition techniques. FIG. 5 also illustrates a decreased fidelity of the identification number "XYZ 123" property of a license plate of a car subject 406 in the image that was implemented by an obscuring portion over the license identification number. In an embodiment, the image 401 may be used to study human and/or vehicle traffic patterns in the area 403 while preserving the anonymity of human subjects and/or registered owners of vehicles”),
wherein in the obfuscated image the facial image data of the human being is replaced with placeholder data, and
wherein the first device is further configured to provide the obfuscated image and the descriptor to a second device (See Jung: Fig. 3, and Col. 11 Lines 52-66, “FIG. 3 illustrates an exemplary embodiment of an environment 300 in which an embodiment may be implemented. The exemplary environment includes an area 303 that includes, or that may include, one or more subjects 305 whose image may be acquired by an imaging device. The environment also includes a system 310 that includes an imaging device 320, an anonymizer circuit 330, and a display 340. The imaging device is operable to acquire an image of a subject. The anonymizer circuit is operable to generate an anonymized image that includes a decreased fidelity of a property of a subject of the acquired image in response to an anonymization policy. The display is operable to provide a human-perceivable indication of the anonymization policy. In an embodiment, the display includes colored lights indicating the anonymization policy then in effect”. Note that the obscured image generation and display portion, etc. of the system may be regarded as a second device), and
a second device configured to perform, based on jointly processing the obfuscated image and the descriptor (See Jung: Fig. 3, and Col. 12 Lines 51-63, “In an embodiment, the anonymizer circuit 330 operable to generate an anonymized image that includes a decreased fidelity of a property of the subject of the acquired image in response to an anonymization policy further includes an anonymizer circuit operable to: generate an anonymized image that includes a decreased fidelity of a property of the subject of the acquired image in response to an anonymization policy; and restrict a dissemination of the acquired image. For example, a restriction of a dissemination of the acquired image may include blocking a dissemination of the image where the image does not include the decreased fidelity of a property of a subject of the acquired image in response to an anonymization policy. By way of further example, a restriction of a dissemination of the acquired image may include requiring a special permission before a dissemination of the image where the image does not include the decreased fidelity of a property of a subject of the acquired image in response to an anonymization policy”. Note that the anonymized image may be the obfuscated image, and the fidelity of the subject property may be the descriptor which may be the set of features of the facial images), a replacement of the placeholder data of the obfuscated image with artificial facial image data corresponding to the descriptor to produce an anonymized image.
However, Jung fails to explicitly disclose that a mimic expression feature having a characteristic, to determine a descriptor associated with the facial image data of the human being and including information describing the characteristic of the mimic expression feature and the characteristic of each identification feature of a selectable subset of the plurality of identification features, wherein in the obfuscated image the facial image data of the human being is replaced with placeholder data, and a replacement of the placeholder data of the obfuscated image with artificial facial image data corresponding to the descriptor to produce an anonymized image.
However, Astarabadi teaches that a mimic expression feature having a characteristic (See Astarabadi: Figs. 2A-D, and [0179], “The second device can: transform the first face model for the first user and the first feed of facial landmark containers into a first synthetic face image feed that mimics the first user's face depicted in the first video feed; and render this first synthetic face image feed over the first background. Concurrently, the second device can: implement a generic face model (e.g., for a cartoon character, a cat, a dog, a mouse) to transform the third feed of facial landmark containers into a third synthetic face image feed that mimics a generic face (e.g., the cartoon character, the cat, the dog, the mouse); and render this third synthetic face image—adjacent the first synthetic face image feed—over the first background. In this implementation, once the first device (or the remote computer system) generates this new face model for the third face, the second device can download this new face model and transition to generating the third synthetic face image feed according to this new face model rather than the generic face model”. Note that mimic a generic face may be the mimic expressions for the facial image).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Jung to have a mimic expression feature having a characteristic as taught by Astarabadi in order to enable the user to select 2D color profiles, schema, and background to match his/her audience and/or to improve the users confidence when engaging in video calls with others (See Astarabadi: Fig. 1, and [0015], “Furthermore, the first device can enable the first user to elect or customize other synthetic content rendered with a synthetic reconstruction of her face at the second device, such as including: a custom background that differs from the first user's true environment during the video call; or makeup, facial hair schema, hair style, clothing and/or accessory schema that differ from the first user's true appearance during the video call. The second device can then incorporate this synthetic content elected by the first user into the first synthetic video feed during the video call, thereby enabling the first user to control how she is presented to the second user”). Jung teaches a method and system that may capture an image with many subjects on the scene, select the anonymization policies, apply the anonymization policies to the captured images to generate anonymized images with identification information such as human faces blacked out or obscured, and output or display the anonymized image to the user or viewer; while Astarabadi teaches a system and method that may generate a synthetic facial image with mimic expressions to represent the first user and present the synthetic image to the second user . Therefore, it is obvious to one of ordinary skill in the art to modify Jung by Astarabadi to have mimic expression in the generic facial image generation and use the synthetic facial image to anonymize the human images. The motivation to modify Jung by Astarabadi is “Use of known technique to improve similar devices (methods, or products) in the same way”.
However, Jung, modified by Astarabadi, fails to explicitly disclose that to determine a descriptor associated with the facial image data of the human being and including information describing the characteristic of the mimic expression feature and the characteristic of each identification feature of a selectable subset of the plurality of identification features, wherein in the obfuscated image the facial image data of the human being is replaced with placeholder data, and a replacement of the placeholder data of the obfuscated image with artificial facial image data corresponding to the descriptor to produce an anonymized image.
However, Lee teaches that to determine a descriptor associated with the facial image data of the human being and including information describing the characteristic of the mimic expression feature and the characteristic of each identification feature of a selectable subset of the plurality of identification features (See Lee: Fig. 2, and [0045], “FIG. 2 (a) shows an input image, FIG. 2 (b) shows a facial image detected by the face detection unit 110, FIG. 2 (c) shows a facial image that is normalized by the normalization unit 120, FIG. 2 (d) shows a facial image expressing key points extracted by the key point position setting unit 130, and FIG. 2 (e) shows a vector K extracted by the key point descriptor extraction unit 140, which may be expressed as K={k1, . . . , kM} (where k1 denotes a descriptor extracted from the first key point and kM denotes a descriptor extracted from the M-th key point). FIG. 2 (f) shows a case where a change in the facial image is detected by the matching unit 150 and the input facial image IN is recognized as the same face as a pre-stored facial image S1”. Note that the descriptors for the facial images are the facial image key point features aggregated in sets of vector, combining with the second art of Astarabadi, the descriptors may also include the mimic features of the facial images).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Jung to have to determine a descriptor associated with the facial image data of the human being and including information describing the characteristic of the mimic expression feature and the characteristic of each identification feature of a selectable subset of the plurality of identification features as taught by Lee in order to enable accurately recognizing the input face image (See Lee: Fig. 1, and [0009], “In addition, another object of the present disclosure is to improve the performance of face recognition by recognizing a face with reference to descriptors of key points extracted from respectively predetermined positions of previously stored facial images and an input facial image, in a manner that addresses the problems of conventional methods”). Jung teaches a method and system that may capture an image with many subjects on the scene, select the anonymization policies, apply the anonymization policies to the captured images to generate anonymized images with identification information such as human faces blacked out or obscured, and output or display the anonymized image to the user or viewer; while Lee teaches a system and method that may determine the descriptors for the input facial image characterization and use the facial image descriptors to identify the faces of the human subjects accurately. Therefore, it is obvious to one of ordinary skill in the art to modify Jung by Lee to have descriptors associated with the facial images and the descriptors characterize the facial image features including various expressions for facial identifications. The motivation to modify Jung by Lee is “Use of known technique to improve similar devices (methods, or products) in the same way”.
However, Jung, modified by Astarabadi and Lee, fails to explicitly disclose that wherein in the obfuscated image the facial image data of the human being is replaced with placeholder data, and a replacement of the placeholder data of the obfuscated image with artificial facial image data corresponding to the descriptor to produce an anonymized image.
However, Zavesky teaches that wherein in the obfuscated image the facial image data of the human being is replaced with placeholder data (See Zavesky: Fig. 1, and [0011], “Further, the disclosed systems and methods may accommodate consumer-quality media (e.g., social media from mobile devices, video from an event, and user-generated video) as candidates for object replacement. For example, a consumer entertainment application may utilize the disclosed systems and methods for replacement of faces, bodies, clothing, etc., based on a consumer's personal photo collection. The disclosed systems and methods may also be utilized in security and surveillance applications to replace unknown or unapproved faces, products, or objects with generic placeholders. For example, an object representation database may include faces, products, and objects that have not been approved for release or that need to be prevented from display. When the disclosed system is executed, context rules may identify that the particular faces, products, or objects are to be replaced with placeholders. Thus, the system may automatically determine when the particular face, products, or objects should be replaced in an image or video”. Note that when the face image is not approved for use, and placeholder is displayed or placed in the face image position), and
a replacement of the placeholder data of the obfuscated image with artificial facial image data corresponding to the descriptor to produce an anonymized image (See Zavesky: Figs. 3-5, and [0033], “The data provider 310 may also provide context rules 314 for the replacement process 300. For example, the data provider 310 may specify conditions (e.g., if, when, and how) for replacing a source object in the video data 312 with the replacement object 316. To illustrate, the data provider 310 may specify audio or music requirements, video semantic requirements, lighting requirements, texture requirements, other visual requirements, or any combination thereof. The context rules may be any combination of logical statements (e.g., AND, NOT, OR), statistical priors (e.g., choose object A 50% of the time and object B 50% of the time to replace a specific source object), strict geometric constraints (e.g., a team flag should only be substituted in the top third of an image frame of the video data 312), or any combination thereof. Other context rules may be provided from a controlling system that may have external knowledge of demographic information (e.g., viewership age), scheduled transmission time of video data 312 or video stream consumption (e.g. primetime TV broadcast), or any combination thereof. For example, during post-production, context rules 314 provided by a data provider (e.g., a movie director) may specify that a main character's face (e.g., Tom Cruise) should replace a specific stunt double's face in scenes where the movie director wants the final output video to include the main character performing a particular stunt (i.e., in specific video frames)”).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Jung to have wherein in the obfuscated image the facial image data of the human being is replaced with placeholder data, and a replacement of the placeholder data of the obfuscated image with artificial facial image data corresponding to the descriptor to produce an anonymized image as taught by Zavesky in order to enable providing a content producer utilizing a replacing system to perform post-production editing to show a Railway logo in commercial without cost associated with reshooting commercial (See Zavesky: Fig. 2, and [0031], “Computed properties and information on the object (i.e., generated 3D model 218, computed audio and visual context 220, computed color and lighting conditions 222, and computed BRDF and texture conditions 224) may be stored 226 in a database. The stored properties and information may enable the system 100 of FIG. 1 to accurately perform object replacement. For example, the system 100 of FIG. 1 may leverage information stored in the database (e.g., existing objects and corresponding properties) to supplement or augment generating a replacement object (i.e., a second object to replace a target first object)”). Jung teaches a method and system that may capture an image with many subjects on the scene, select the anonymization policies, apply the anonymization policies to the captured images to generate anonymized images with identification information such as human faces blacked out or obscured, and output or display the anonymized image to the user or viewer; while Zavesky teaches a system and method that may determine if there is a need to replace the first object with a second object (face) based on some rule, put a placeholder in the position of the first object, and replace the first object (e.g., a first face) with the second object (e.g., a second face). Therefore, it is obvious to one of ordinary skill in the art to modify Jung by Zavesky to have a placeholder for the anonymized facial image and replacing the placeholder with a anonymized image. The motivation to modify Jung by Zavesky is “Use of known technique to improve similar devices (methods, or products) in the same way”.
Regarding claim 3, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 1 as outlined above. Further, Jung teaches that the personal data processing system according to claim 1, wherein the second device includes a storage configured to store the descriptor and the obfuscated image (See Jung: Fig. 24, and Col. 22 Lines 49-60, “For example, the anonymization policy may be applied to the class of subjects depicted in the captured image before the captured image is saved in a storage device, such as before an anonymized captured image 1642 is saved in a storage device 1640 of the imaging device 1610 and/or before an anonymized captured image 1694 is saved in a storage device 1690 of the anonymization device 1650. In another example, a captured image may be transitorily saved in a volatile memory (not shown) before the anonymization policy is applied and the image persistently saved in a more permanent memory and/or non-volatile media, such as the storage 1640 and/or the storage 1690”).
Regarding claim 4, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 3 as outlined above. Further, Jung teaches that the personal data processing system according to claim 3, wherein the second device is configured to retrieve the descriptor from the storage and to provide at least one of the information or a part of the information of the descriptor or the obfuscated image to a user (See Jung: Fig. 20, and Col. 20 Lines 14-29, “FIG. 20 illustrates an alternative embodiment of the exemplary operational flow 1300 of FIG. 19. The ambiguation operation 1310 may include at least one additional operation. The at least one additional operation may include an operation 1312 and/or an operation 1314. The operation 1312 obscures a person, article, object, face, license plate, and/or label of a subject depicted in a received image in response to a privacy policy. The operation 1314 obscures a property of a subject depicted in a received digital and/or analog image in response to a privacy policy. The communication operation 1320 may include at least one additional operation, such as the operation 1322. The operation 1322 sends a signal indicative of the privacy policy and is receivable by at least one of a remotely located image capture device, a privacy policy indicator at a location physically distant from the device, and/or a local privacy policy indicator”).
Regarding claim 5, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 1 as outlined above. Further, Jung teaches that the personal data processing system of claim 1, wherein the second device is further configured to send the anonymized image to a third device configured to display the anonymized image to a user (See Jung: Figs. 3-5, and Col. 14 Lines 58-67 ~ Col. 15 Lines 1-18, “FIG. 5 illustrates an image 401 of the exemplary embodiment of an environment 400 of FIG. 4 with a decreased fidelity of a property of a subject. In an embodiment, the decreased fidelity may be implemented and/or practiced using systems, devices, apparatus, and/or methods disclosed herein. For example, the system 300 described in conjunction with FIG. 3 may be used to implement and/or practice the decreased fidelity of a property of a subject depicted in an image 405. For example, FIG. 5 illustrates an embodiment where the display 440B of the image capture device 420 indicating a "Green" color-based visual indication of the anonymization policy, corresponding to decreasing a fidelity of facial properties and license plate properties depicted in acquired images to be indistinguishable. The decreased fidelity of the illustrated embodiment includes a decreased fidelity of facial properties of human subjects 408A, 408B, and 408C implemented by obscuring portions of their faces and/or heads. The faces and/or heads of human subjects may be located in the image using techniques known to those skilled in the art, including artificial intelligence, and/or facial recognition techniques. FIG. 5 also illustrates a decreased fidelity of the identification number "XYZ 123" property of a license plate of a car subject 406 in the image that was implemented by an obscuring portion over the license identification number. In an embodiment, the image 401 may be used to study human and/or vehicle traffic patterns in the area 403 while preserving the anonymity of human subjects and/or registered owners of vehicles”. Note that the obscured image 405 with identifiable information blacked out is output and displayed to the user).
Regarding claim 6, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 1 as outlined above. Further, Zavesky teaches that the personal data processing system of claim 1, wherein the image data includes video data (See Zavesky: Fig. 1, and [0015], “Referring to FIG. 1, a particular illustrative embodiment of a system to digitally replace objects in images or video is disclosed and generally designated 100. The system 100 includes a three-dimensional (3D) model generator 120 coupled to a database 130. The 3D model generator 120 may be configured to generate a 3D model of an object (e.g., representative first object 112) depicted in video content 110 and to store the generated 3D model 132 of the first object 112 in the database 130. The database 130 may also include a library 134 of 3D models generated for a plurality of objects. The library 134 of 3D models may be categorized and organized based on visual properties of the objects, scene or background data surrounding the objects, other information descriptive of the objects, or a combination thereof”).
Regarding claim 7, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 1 as outlined above. Further, Astarabadi teaches that the personal data processing system according to claim 1, wherein at least one of the first device or the second device are real-time devices configured to process data in real-time (See Astarabadi: Fig. 1, and [0067], “Like the first method S100 described above, Blocks of the second method S200 can be executed by native or browser-based applications executing on a set of computing devices (e.g., smartphones, tablets, laptop computers) during a video call between two users in order: to compress a first video feed of a first user into a first lightweight (e.g., sub-kilobyte) feed of constellations of facial landmarks at a first device; and to reconstruct this first video feed at a second device by injecting this feed of facial landmark constellations and a first (pseudo-) unique face model of the first user into a synthetic face generator, which outputs a first stream of synthetic, photorealistic images of the first user that the second device then renders in near real-time. Simultaneously, the second device can compress a second video feed of the second user into a second lightweight constellation of facial landmarks; and the first device can reconstruct this second video feed by injecting this feed of facial landmark constellations and a second (pseudo-) unique face model of the second user into a synthetic face generator, which outputs a second stream of synthetic, photorealistic images of the second user that the first device then renders in near real-time”).
Regarding claim 8, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 1 as outlined above. Further, Astarabadi teaches that the personal data processing system according to claim 1, wherein the first device and the second device are connected via a communication network configured to transfer data including the image data and the descriptor from the first device to the second device (See Astarabadi: Fig. 1, and [0011], “In particular, rather than transmit and receive data-rich video feeds during a video call, a first device executing the first method S100 can instead extract facial landmark constellations from a video feed and transmit a feed of facial landmark constellations to a second device. The second device can then leverage a local model—such as a generic model or a model specific to the first user—to reconstruct a photorealistic representation of the first user's face and then render this photorealistic synthetic video feed in near real-time. The second user may thus experience the video call as though a color video was received from the first user's device but without necessitating a high-bandwidth, low-latency data connection between the first and second devices. The second and first devices can concurrently execute the same process in reverse to extract facial landmark constellations from a second video feed recorded at the second device, to transmit this second feed of facial landmark constellations to the first device, and to leverage a local model to reconstruct a photorealistic synthetic video feed of the second user. More specifically, by extracting facial landmark constellations from a high-definition video feed according to the method S100, the first device can compress this high-definition video feed by multiple orders of magnitude (e.g., by approximately 100 times). Transmission of a feed of facial landmark constellations—at the same frame rate as the original high-definition video (e.g., 24 frames per second)—from the first device to the second device during a video call may therefore require significantly less bandwidth than the original high-definition video (e.g., 10 kilobits per second rather than 1.5 Megabits per second). Therefore, the first method S100 can enable a high-quality video call with significantly less upload bandwidth to transmit a representation of a first video from the first device to a computer network and significantly less download bandwidth required to download this representation of the first video to the second device, and vice versa”).
Regarding claim 9, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 1 as outlined above. Further, Jung, Astarabadi, Lee, and Zavesky teach that a method of operating a personal data processing system, the method (See Jung: Fig. 24, and Col. 21 Lines 20-29, “FIG. 24 illustrates an exemplary embodiment of a system 1600. The system includes an imaging device 1610, an anonymization device 1650, and a broadcast device. The imaging device is operable to capture an image depicting a class of subjects 1605 that may have a temporal presence in an area 1603. The broadcast device is operable to display a human-perceivable indication of a selected anonymization policy to at least a portion of the area. The anonymization policy includes obscuring a property of the class of subjects depicted in the captured image”) comprising:
capturing image data including facial image data of a human being (See Jung: Fig. 3, and Col. 13 Lines 16-27, “In an embodiment, the imaging device 320 operable to acquire an image of a subject further includes a digital imaging device operable to acquire an image of a subject of the potential image subject(s) 305. In another embodiment, the imaging device operable to acquire an image of a subject further includes a digital camera operable to acquire an image of a subject. For example, the digital camera may include a lens 322, an imaging element 324, an analog to digital converter 326, and/or a processor 328. In a further embodiment, the imaging device operable to acquire an image of a subject further includes an imaging device operable to capture an image of a subject of the potential image subject(s) 305”),
the facial image data of a human being including a plurality of identification features each having a characteristic (See Jung: Fig. 24, and Col. 21 Lines 19-32, “In a further embodiment, the anonymization device further includes an anonymization device having a selector circuit operable to select an anonymization policy, where the anonymization policy includes obscuring at least one of a variable, a recognizable, a distinguishable, and/or a unique aspect of a property of the class of subjects depicted in the captured image. In another embodiment, the anonymization device further includes an anonymization device having a selector circuit operable to select an anonymization policy, where the anonymization policy includes obscuring at least one of a variable, a recognizable, a distinguishable, and/or a unique aspect of a property of each instance of the class of subjects depicted in the captured image”. Note that the unique aspect of the property of the subject may be the identification features of the face image) and a mimic expression feature having a characteristic (See Astarabadi: Figs. 2A-D, and [0179], “The second device can: transform the first face model for the first user and the first feed of facial landmark containers into a first synthetic face image feed that mimics the first user's face depicted in the first video feed; and render this first synthetic face image feed over the first background. Concurrently, the second device can: implement a generic face model (e.g., for a cartoon character, a cat, a dog, a mouse) to transform the third feed of facial landmark containers into a third synthetic face image feed that mimics a generic face (e.g., the cartoon character, the cat, the dog, the mouse); and render this third synthetic face image—adjacent the first synthetic face image feed—over the first background. In this implementation, once the first device (or the remote computer system) generates this new face model for the third face, the second device can download this new face model and transition to generating the third synthetic face image feed according to this new face model rather than the generic face model”. Note that mimic a generic face may be the mimic expressions for the facial image);
detecting the facial image data of the human being in the image data (See Jung: Figs. 3-5, and Col. 14 Lines 66-67 ~ Col. 15 Lines 1-18, “FIG. 5 illustrates an embodiment where the display 440B of the image capture device 420 indicating a "Green" color-based visual indication of the anonymization policy, corresponding to decreasing a fidelity of facial properties and license plate properties depicted in acquired images to be indistinguishable. The decreased fidelity of the illustrated embodiment includes a decreased fidelity of facial properties of human subjects 408A, 408B, and 408C implemented by obscuring portions of their faces and/or heads. The faces and/or heads of human subjects may be located in the image using techniques known to those skilled in the art, including artificial intelligence, and/or facial recognition techniques. FIG. 5 also illustrates a decreased fidelity of the identification number "XYZ 123" property of a license plate of a car subject 406 in the image that was implemented by an obscuring portion over the license identification number. In an embodiment, the image 401 may be used to study human and/or vehicle traffic patterns in the area 403 while preserving the anonymity of human subjects and/or registered owners of vehicles”. Note that using face recognition technique to recognize face in the image may be the face detection);
determining a descriptor associated with the facial image data of the human being and including information describing the characteristic of the mimic expression feature and the characteristic of each identification feature of a selectable subset of the plurality of identification features (See Lee: Fig. 2, and [0045], “FIG. 2 (a) shows an input image, FIG. 2 (b) shows a facial image detected by the face detection unit 110, FIG. 2 (c) shows a facial image that is normalized by the normalization unit 120, FIG. 2 (d) shows a facial image expressing key points extracted by the key point position setting unit 130, and FIG. 2 (e) shows a vector K extracted by the key point descriptor extraction unit 140, which may be expressed as K={k1, . . . , kM} (where k1 denotes a descriptor extracted from the first key point and kM denotes a descriptor extracted from the M-th key point). FIG. 2 (f) shows a case where a change in the facial image is detected by the matching unit 150 and the input facial image IN is recognized as the same face as a pre-stored facial image S1”. Note that the descriptors for the facial images are the facial image key point features aggregated in sets of vector, combining with the second art of Astarabadi, the descriptors may also include the mimic features of the facial images),
wherein the selectable subset of the plurality of identification features substantially does not allow a unique identification of the human being (See Jung: Fig. 3, and Col. 13 Lines 17-36, “In an embodiment, the anonymizer circuit 330 operable to generate an anonymized image that includes a decreased fidelity of a property of the subject of the acquired image in response to an anonymization policy further includes an anonymizer circuit operable to generate an anonymized image that includes at least one of a decreased resolution of a property, an obscuring of a property, a blackout of a property, and/or a removal of a property of the subject of the acquired image in response to an anonymization policy. In a further embodiment, the anonymizer circuit further includes an anonymizer circuit operable to generate an anonymized image that includes a decreased fidelity of an aspect, a property, and/or an attribute of the subject of the acquired image in response to an anonymization policy. In another embodiment, the anonymizer circuit further includes an anonymizer circuit operable to generate an anonymized image that includes a decreased fidelity of at least one of a face, a license plate, a label, and/or a recognizable property associated with the subject of the acquired image in response to an anonymization policy”. Note that the blackout or removal properties may be the identification features that does not allow unique identification of the human being);
providing an obfuscated image (See Jung: Fig. 3-5, and Col. 14 Lines 58-67 ~ Col. 15 Lines 1-18, “FIG. 5 illustrates an image 401 of the exemplary embodiment of an environment 400 of FIG. 4 with a decreased fidelity of a property of a subject. In an embodiment, the decreased fidelity may be implemented and/or practiced using systems, devices, apparatus, and/or methods disclosed herein. For example, the system 300 described in conjunction with FIG. 3 may be used to implement and/or practice the decreased fidelity of a property of a subject depicted in an image 405. For example, FIG. 5 illustrates an embodiment where the display 440B of the image capture device 420 indicating a "Green" color-based visual indication of the anonymization policy, corresponding to decreasing a fidelity of facial properties and license plate properties depicted in acquired images to be indistinguishable. The decreased fidelity of the illustrated embodiment includes a decreased fidelity of facial properties of human subjects 408A, 408B, and 408C implemented by obscuring portions of their faces and/or heads. The faces and/or heads of human subjects may be located in the image using techniques known to those skilled in the art, including artificial intelligence, and/or facial recognition techniques. FIG. 5 also illustrates a decreased fidelity of the identification number "XYZ 123" property of a license plate of a car subject 406 in the image that was implemented by an obscuring portion over the license identification number. In an embodiment, the image 401 may be used to study human and/or vehicle traffic patterns in the area 403 while preserving the anonymity of human subjects and/or registered owners of vehicles”),
wherein in the obfuscated image the facial image data of the human being is replaced with placeholder data (See Zavesky: Fig. 1, and [0011], “Further, the disclosed systems and methods may accommodate consumer-quality media (e.g., social media from mobile devices, video from an event, and user-generated video) as candidates for object replacement. For example, a consumer entertainment application may utilize the disclosed systems and methods for replacement of faces, bodies, clothing, etc., based on a consumer's personal photo collection. The disclosed systems and methods may also be utilized in security and surveillance applications to replace unknown or unapproved faces, products, or objects with generic placeholders. For example, an object representation database may include faces, products, and objects that have not been approved for release or that need to be prevented from display. When the disclosed system is executed, context rules may identify that the particular faces, products, or objects are to be replaced with placeholders. Thus, the system may automatically determine when the particular face, products, or objects should be replaced in an image or video”. Note that when the face image is not approved for use, and placeholder is displayed or placed in the face image position); and
performing, based on jointly processing the obfuscated image and the descriptor (See Jung: Fig. 3, and Col. 12 Lines 51-63, “In an embodiment, the anonymizer circuit 330 operable to generate an anonymized image that includes a decreased fidelity of a property of the subject of the acquired image in response to an anonymization policy further includes an anonymizer circuit operable to: generate an anonymized image that includes a decreased fidelity of a property of the subject of the acquired image in response to an anonymization policy; and restrict a dissemination of the acquired image. For example, a restriction of a dissemination of the acquired image may include blocking a dissemination of the image where the image does not include the decreased fidelity of a property of a subject of the acquired image in response to an anonymization policy. By way of further example, a restriction of a dissemination of the acquired image may include requiring a special permission before a dissemination of the image where the image does not include the decreased fidelity of a property of a subject of the acquired image in response to an anonymization policy”. Note that the anonymized image may be the obfuscated image, and the fidelity of the subject property may be the descriptor which may be the set of features of the facial images),
a replacement of the placeholder data of the obfuscated image with artificial facial image data corresponding to the descriptor to produce an anonymized image (See Zavesky: Figs. 3-5, and [0033], “The data provider 310 may also provide context rules 314 for the replacement process 300. For example, the data provider 310 may specify conditions (e.g., if, when, and how) for replacing a source object in the video data 312 with the replacement object 316. To illustrate, the data provider 310 may specify audio or music requirements, video semantic requirements, lighting requirements, texture requirements, other visual requirements, or any combination thereof. The context rules may be any combination of logical statements (e.g., AND, NOT, OR), statistical priors (e.g., choose object A 50% of the time and object B 50% of the time to replace a specific source object), strict geometric constraints (e.g., a team flag should only be substituted in the top third of an image frame of the video data 312), or any combination thereof. Other context rules may be provided from a controlling system that may have external knowledge of demographic information (e.g., viewership age), scheduled transmission time of video data 312 or video stream consumption (e.g. primetime TV broadcast), or any combination thereof. For example, during post-production, context rules 314 provided by a data provider (e.g., a movie director) may specify that a main character's face (e.g., Tom Cruise) should replace a specific stunt double's face in scenes where the movie director wants the final output video to include the main character performing a particular stunt (i.e., in specific video frames)”).
Regarding claim 11, Jung, Astarabadi, Lee, and Zavesky teach all the features with respect to claim 9 as outlined above. Further, Jung teaches that method according to claim 9, comprising:
storing the descriptor and the obfuscated image in a storage (See Jung: Fig. 24, and Col. 22 Lines 49-60, “For example, the anonymization policy may be applied to the class of subjects depicted in the captured image before the captured image is saved in a storage device, such as before an anonymized captured image 1642 is saved in a storage device 1640 of the imaging device 1610 and/or before an anonymized captured image 1694 is saved in a storage device 1690 of the anonymization device 1650. In another example, a captured image may be transitorily saved in a volatile memory (not shown)