DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The language in claims 6, 17, and 22-23 uses the format “at least one of A, B, and C” which triggers a claim interpretation of conjunctive (including A, B, and C) under SuperGuide Corp. v. DirecTV Enters., Inc., 358 F.3d 870 (Fed. Cir. 2004) if supported by the present specification; however upon examination of the specification Examiner fails to identify support for the conjunctive interpretation and therefore for the sake of examination the disjunctive (A, B, or C) interpretation will be used. If Applicant agrees with this interpretation, then Examiner requests a change in language from the conjunctive to the disjunctive; i.e. change format of “at least one of A, B, and C” to recite “at “least one of A, B, or C”). Appropriate correction is required.
Claim Objections
Claims 19-20 are objected to because of the following informalities: it appears from the lack of antecedent basis of the term “the system” from claim 1 from which both claims 19-20 depend from, that the proper dependence of claims 19-20 should be from claim 18 from which “a system” was introduced; this would yield proper antecedent basis. Appropriate correction is required.
Claim 25-26 are objected to because of the following informalities: the claim term “video” should recite “the video” for proper antecedent basis; further, the claim limitation in claim 25 reciting “analyzing video imagery contained with the video source results to determine a type of camera that captured the received video” should recite “analyzing video imagery contained within the video source results to determine a type of camera that captured the received video” for proper grammar; the claim limitation in claim 26 reciting “analyzing the video imagery contained with the video source results includes determining whether the video imagery reflects the requested changes” should recite “analyzing the video imagery contained within the video source results includes determining whether the video imagery reflects the requested changes” for proper grammar. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-18, 22-23, and 25-28 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No.: 12,413,865 (Derakhshani et al.) (hereinafter Derakhshani), in view of U.S. Patent Application Publication No.: 2024/0031178 (Rouse).
Regarding claim 1, Derakhshani teaches a method of verifying a provenance of video received from a video source, the method comprising: (Derakhshani, abstract; col. 1, lines 48-51; col. 2, lines 27-36; FIG. 1A-1B: “A technique for interrogating a camera pipeline is disclosed. The camera pipeline is issued a challenge command instructing a response by the camera pipeline. The challenge may include the camera pipeline capturing images at specified camera parameter values. The captured images are analyzed for features corresponding to images captured from a real camera pipeline.”; “In one implementation, the challenge comprises instructions for the camera pipeline to respond with a sequence of images captured utilizing a specified selection of camera parameter values. Features of the sequence of images are analyzed.” “As illustrated in FIG. 1A, FIG. 1B, and FIG. 1C a client computing device 102, such as a smartphone, includes a camera hardware layer 104. For example, as illustrated in FIG. 1A, an application 106 executing on the client device 102 may be responsible for interrogating the camera and making decisions whether to reject or verify the camera. The application 106 may report its results to an external authentication system 140, although it could alternatively take protective actions on the client computing device to prevent potential fraud.”;
PNG
media_image1.png
440
716
media_image1.png
Greyscale
PNG
media_image2.png
471
692
media_image2.png
Greyscale
);
receiving metadata from the video source, wherein the metadata purports that video received from the video source is captured in real time by a physical camera associated with a user device; issuing one or more challenges to the video source (Derakhshani, col. 3, lines 3-17; col. 6, lines 62-6; col. 7, lines 1-10; col. 8, lines 10-15; col. 4, lines 15-19; FIG. 2: “The response of the camera pipeline, to the instructions of a challenge, is evaluated to determine if the camera pipeline response is what would be expected from the output of a real camera pipeline that has not been bypassed, synthesized, or otherwise compromised. However, in some implementations, the interrogator could issue other challenges such as requesting metadata from the camera pipeline, requesting metadata from an operating system associated with the camera pipeline, or requesting metadata both, as an additional type of challenge to the identity of the camera pipeline. Additionally, in some implementations, a challenge-response protocol is implemented as one or more instructions for post-processing of images, such as modifying an image preview function to prompt a user to alter how they interact with the camera.”; “In one implementation, challenges are triggered based on triggering auto exposure (AE) by changing the scenery/illumination (e.g., via camera movement) and recording whether the reported frame metadata shows a difference in exposure settings (e.g., ISO or exposure time/frame rate.”; “In one implementation, the challenge is a camera integrity check. The interrogator may simply ask the operating system of the computing device to provide the camera hardware name, make, model, version, etc., and check if the reported metadata matches metadata provided by the camera hardware, or an unexpected value in the case of a virtual camera.”; “As illustrated in FIG. 2 a verification decision engine 202 interrogates a client camera pipeline 290. A challenge response interrogation module 204 includes a set of discrete challenges that the client camera pipeline 290 can be instructed to perform … In one implementation, the challenges are selected from camera parameter challenge tests 220 in which camera parameter values are changed … In one implementation, the digital artifacts (for synthetic or Deep Fake images) as a result of the forced motion can be used to detect injection-based attacks.”;
PNG
media_image3.png
638
1148
media_image3.png
Greyscale
);
receiving from the video source results generated in response to the one or more challenges (Derakhshani, col. 4, lines 40-46; FIG. 2: “A challenge response analysis module 205 analyzes the client camera pipeline responses. In some implementations, this may include generating a vote 214 to reject verifying the camera's integrity (e.g., to generate a rejection vote when there is strong evidence of a digital takeover).”; see client camera pipeline response received by verification decision engine 200);
inputting the video source results generated in response to the one or more challenges into a machine classifier, wherein the machine classifier is configured to distinguish between challenge results received from a physical camera and challenge results received from a virtual camera; using the machine classifier to issue a genuineness score indicating a likelihood that the video source results generated in response to the one or more challenges were received from a physical camera (Derakhshani, col. 4, lines 46-67; col. 5, lines 1-5; col. 5, lines 25-35; FIG. 5: “Each sub-challenge that changes a camera parameter value may have its own behavioral model. For example, the effect on camera images of changing a camera gain setting can be modeled. The captured images may be analyzed for image features that are expected to change according to a behavioral model for the camera for specific camera parameter values. An image feature analysis module 207 may be programmed to analyze a set of image features relevant to the sub-challenges and generate a score signal. For example, for an image captured at a high gain setting, the image analysis may identify image features consistent with the image being captured at a high gain setting. That is, models may be used to score features of images that the camera pipeline was supposed to have captured at specific camera parameter values. A fusion engine 208 may be provided to generate a combined score from the output signals of the models used for each of the discrete challenges. For example, if 20 different images are captured for 20 different camera parameter values, a combined score for the results of the 20 sub-challenges may be combined (fused) to generate a combined score indicative of the overall camera pipeline integrity. In some implementations, a machine learning model may be used to train a meta classifier 210 to combine results from the different sub-challenges.”; see FIG. 2 above; “FIG. 5 is a flow chart of a method in accordance with an implementation. In block 502, a sequence of camera parameter change commands is issued. In block 504 camera responses are monitored. In block 506, a weighted fused camera response is generated. For example, a meta classifier may be utilized. The meta classifier may, for example, be trained using machine learning techniques to learn rules for combining signals from the analysis of different challenges. In block 508, a confidence score is determined from the weighted fused camera response that is indicative of camera integrity.”;
PNG
media_image4.png
573
671
media_image4.png
Greyscale
;
the camera integrity indicated by confidence score is the likelihood that the “camera” that the sequence of images/video came from is an actual real physical camera vs. non-physical camera indicating an injection attack);
a threshold score (Derakhshani, col. 5, lines 15-16: “In one implementation the verification decision engine 202 has a decision threshold and/or a fusion rule that is determined by the requested security level.”); and
issuing a determination that the provenance of video received from the video source was either or captured by a physical camera, or not captured by a physical camera (col. 2, lines 60-67: “A real camera can be modelled as exhibiting specific types of image behavior in response to specific camera parameter values. There are a variety of differences between the response of a real camera in comparison to what would be received if the camera was compromised by camera hijacks, digital replays (including deep fakes), camera driver takeovers, or virtual cameras for injected imagery. Interrogating the camera pipeline in one or more challenge-response patterns provides a way to identify if the response corresponds to that expected from a real camera pipeline.”; “The response of the camera pipeline, to the instructions of a challenge, is evaluated to determine if the camera pipeline response is what would be expected from the output of a real camera pipeline that has not been bypassed, synthesized, or otherwise compromised.”).
Derakhshani fails to explicitly teach
when the genuineness score exceeds a threshold score, issuing a determination that the provenance of video received from the video source is a physical camera; and when the genuineness score is equal to or less than the threshold score, issuing a determination that the provenance of video received from the video source was not captured by a physical camera.
Rouse teaches
when the genuineness score exceeds a threshold score, issuing a determination that the provenance of video received from the video source is a physical camera; and when the genuineness score is equal to or less than the threshold score, issuing a determination that the provenance of video received from the video source was not captured by a physical camera (Rouse, para. [0049]; para. [0076]: “The application 110 may select a challenge subject and present a challenge prompt at [B]. A display device 102 of the user system 100, such as a screen, may present the challenge 112. In the illustrated example, the challenge 112 is a prompt to use the user system 100 to take a photograph of an object that is expected to be within the user's environment or otherwise near the user's location (e.g., a lamp). The challenge 112 further specifies that the photograph is to be taken within a threshold period of time (e.g., 2 minutes). In this example, the lamp is the subject of the challenge 112. As described in greater detail below, the specific object that is the subject of the challenge 112 may be randomly selected or pseudo-randomly selected from a set of objects or other challenge subjects that the application 110 is configured to recognize in photographs.”; “In another example, a response validation criterion may be a requirement that the timestamp from EXIF metadata for an image correspond to a time that is at least a threshold period of time after the time at which the challenged was prompted. If the timestamp from the EXIF metadata does not correspond to a time that is at least the threshold period of time after the time at which the challenge prompt was presented, then the image data or other challenge response data 300 was likely not generated by a genuine human user in response to the challenge prompt (e.g., it was generated too quickly to come from a human and therefore may have been automatically generated by malicious code) and the challenge response may fail to satisfy a response validation criterion.”;
PNG
media_image5.png
628
928
media_image5.png
Greyscale
;
a challenge is issued to the origin of the image/video source to take an image of a specific object to verify if the user taking the image as part of the challenge is a real person (i.e. using a physical camera); if the metadata of the time it takes for the image to be taken exceeds a minimum threshold time, then the user is real and is using a physical camera to take the image; if the time the image taken in response to the challenge issued was less than or equal to the minimum threshold time, then it was not taken by a real human with a real physical camera; the claim limitations “the provenance of video received from the video source was (not) captured by a physical camera” are equivalent under broadest reasonable interpretation to determining if the human user is real or not because if the human is verified as real, then a physical camera must have been used to participate in the challenge, while if a non-human user (malicious code) participated in the challenge then computer models (CGI), virtual machines, or emulators, are used, rather than physical cameras to take images/video).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to: 1) modify the step of issuing a determination that the provenance of video received from the video source was captured by a physical camera, as taught by Derakhshani, to include happening when the genuineness score exceeds a threshold score, as taught by Rouse, and 2) modify the step of issuing a determination that the provenance of video received from the video source was not captured by a physical camera, as taught by Derakhshani, to include happening when the genuineness score is equal to or less than the threshold score, as taught by Rouse.
The suggestion/motivation for doing so would have been for “preventing access to content or otherwise preventing performance of an operation; for example, if evaluation of the challenge response, challenge context data, or both indicates the challenge response is unlikely to have been generated by a genuine human user, then subsequent data provided by the user may be flagged as questionable or not flagged as verified; as another example, alternative information may be presented, such as false information to deceive or affect the performance of an automatic system that has generated the challenge response” (Rouse, para. [0086]), and that “data determined to potentially be associated with a non-human origin may be separated (e.g., quarantined) from data more likely to be associated with a human origin” (Rouse, para. [0091]); this improves security for all types of image-based verification over the internet.
Therefore, it would have been obvious to combine Derakhshani, with Rouse, to obtain the invention as specified in claim 1.
Regarding claim 2, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges comprise a first challenge and a second challenge, and the second challenge is based at least in part on video source results received from the first challenge (Derakhshani, col. 5, lines 60-65; col. 3, lines 18-40: “A non-exclusive set of examples of discrete challenge-response interrogations will now be discussed. While an individual camera parameter might be used in a single challenge, more generally, a sequence of challenges may be used, depending on which camera parameters are selectable for a given camera pipeline and the available APIs.”; “more generally, a complex challenge may include two or more discrete challenges. While interrogating the camera pipeline for a single camera parameter change may be useful, interrogating the camera pipeline for instructions corresponding to a sequence of different camera parameter values may provide more robust protection against different potential threats. For example, a sequence of images may be captured starting with an initial image I0 followed by a sequence of images each with different camera parameters value from 1 to N, e.g., I1 . . . . IN. It should be noted that many variations of a sequence of images are possible. As one example, a single camera parameter may have values varied in a sequence (e.g., low, medium, and high gain settings) to monitor a progression of image changes. Specific parameter values could be selected (e.g., a specific gain setting”; each camera parameter such as aperture setting, a gain setting, a shutter speed setting, a lighting setting, a white balance setting, and resolution setting, are each progressively challenged and once each challenge response (video source results) indicates a real physical camera, then an additional challenge is done until each camera parameter is verified; however, if one parameter indicates a virtual camera/injection attack, then the progressive challenges stop).
Regarding claim 3, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the video source results received in response to the one or more challenges depend on an order in which the one or more challenges were issued (Derakhshani, col. 5, lines 60-65; col. 3, lines 18-40; col. 8, lines 32-38: “A non-exclusive set of examples of discrete challenge-response interrogations will now be discussed. While an individual camera parameter might be used in a single challenge, more generally, a sequence of challenges may be used, depending on which camera parameters are selectable for a given camera pipeline and the available APIs.”; “more generally, a complex challenge may include two or more discrete challenges. While interrogating the camera pipeline for a single camera parameter change may be useful, interrogating the camera pipeline for instructions corresponding to a sequence of different camera parameter values may provide more robust protection against different potential threats. For example, a sequence of images may be captured starting with an initial image I0 followed by a sequence of images each with different camera parameters value from 1 to N, e.g., I1 . . . . IN. It should be noted that many variations of a sequence of images are possible. As one example, a single camera parameter may have values varied in a sequence (e.g., low, medium, and high gain settings) to monitor a progression of image changes. Specific parameter values could be selected (e.g., a specific gain setting”; each camera parameter such as aperture setting, a gain setting, a shutter speed setting, a lighting setting, a white balance setting, and resolution setting, are each progressively challenged and once each challenge response (video source results) indicates a real physical camera, then an additional challenge is done until each camera parameter is verified; however, if one parameter indicates a virtual camera/injection attack, then the progressive challenges stops; “It will be understood in the above challenges that in individual challenges that individual challenge may change one camera parameter value but hold other camera parameter values at a set of default values to simplify analysis of the resulting images”; therefore, there is an order to the camera parameters being challenged)
Regarding claim 5, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein video source results generated in response to the one or more challenges include information about a state of the video source following completion of the one or more challenges (Derakhshani, col. 6, lines 47-61: “In one implementation, the challenge is related to a lighting setting for a light source. Consider a camera with a light source, such as a torch, flash, or a backlit screen (these sources may emit visible light, or IR light in case of night vision or three-dimensional sensing). The light source may be activated to see the response of object(s) being imaged, with foreground being expected to show a greater degree of reflection change between flash on/off events. The photic response can be detected after the illumination challenge. Randomizing the onset of illumination emission can be employed as another security measure. A lack of such response (reflected lighting in the temporal vicinity of the illumination trigger) indicates a camera hijack attack or similar problem, (unless the environment is extremely bright and thus drowning the light source).”; other challenges include changing camera parameters with selectable settings (i.e., parameter values), such as an aperture setting, a gain setting, a shutter speed setting, a lighting setting, a white balance setting, a resolution setting, etc. and seeing their responses; each response has an indication of the state of the video source as being from a real physical camera or a virtual camera (malicious code) indicating a potential injection attack).
Regarding claim 6, Derakhshani, in view of Rouse, teaches the method of claim 5, wherein the information about the state of the video source includes at least one of a state of a flash toggle, a state of a white balance toggle, and a frame rate (Derakhshani, col. 6, lines 30-67; col. 7, lines 1-17: “In one implementation, the challenge is to change the shutter speed (also known as integration time). Changing the shutter speed changes image brightness in a similar fashion to gain (longer integration time increases image brightness similar to increasing the gain). However, increasing the integration time reduces image noise. In the case of camera and/or subject motion, decreased shutter speed induces a motion blur for the parts of the image that are in motion. The motion blur can be detected via a point spread function (PSF), the fast Fourier transform (FFT), and other motion blur detection techniques … The shutter speed (SS) also affects the maximum frame rate and caps it at 1/SS, so the effects of its increase are indirectly seen from a slowing of an otherwise high frame rate in a real camera system.”; “In one implementation, challenges are triggered based on triggering auto exposure (AE) by changing the scenery/illumination (e.g., via camera movement) and recording whether the reported frame metadata shows a difference in exposure settings (e.g., ISO or exposure time/frame rate. Dark scenery may increase exposure time and drop the frame rate on top of gain increases during AE).”; “In one implementation, the challenge is to change the white balance settings of the camera (e.g., from incandescent to fluorescent) and testing whether the expected change in color temperature/hue takes place in the resulting images. For example, in general the change of white balance should be reflected in the relative changes of R, G, and B histograms (e.g., from cooler/bluer to warmer/redder).”; Examiner is taking a disjunctive interpretation of this claim reciting “at least one of a state of a flash toggle, a state of a white balance toggle, or a frame rate”; see claim interpretation above).
Regarding claim 7, Derakhshani, in view of Rouse, teaches the method of claim 1.
Derakhshani, in view of Rouse, fails to teach
inputting to the machine classifier a time taken for the video source results to be generated in response to the one or more challenges.
Rouse further teaches
inputting to the machine classifier a time taken for the video source results to be generated in response to the one or more challenges (Rouse, para. [0060]; para. [0072]; para. [0011]: “In some embodiments, challenge response data may be required to be received within a maximum period of time. For example, the challenge response data may be required to be received no more than about 120 seconds from when the prompt for the challenge 112 is presented, no more than about 90 seconds from when the prompt is presented, or no more than about 60 seconds from when the prompt is presented. If the maximum period of time has passed since presentation of the prompt without receipt of a user input, the application 110 may determine at decision block 208 that the challenge response data has not been received.”; “In some embodiments, a challenge context model 324 may be a neural-network-based model or other model configured to evaluate challenge response metadata 312, environmental data 314, usage data 316, other data items, or some combination thereof. For example, response validation data 352 may be classification output generated by the model 324 that has been trained to classify the input data items as indicative of genuine human use in connection with responding to a challenge.”; “wherein the challenge context data represents a time at which the image data was generated.”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the method, as taught by Derakhshani, in view of Rouse, to include the step of inputting to the machine classifier a time taken for the video source results to be generated in response to the one or more challenges, as further taught by Rouse.
The suggestion/motivation for doing so would have been that by monitoring time it takes to respond to a challenge, it may be known if “it was generated too quickly to come from a human and therefore may have been automatically generated by malicious code” (Rouse, para. [0076]); which improves injection attack detection monitoring; this improves security.
Therefore, it would have been obvious to combine Derakhshani and Rouse, with Rouse further, to obtain the invention as specified in claim 7.
Regarding claim 8, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges include a request for video source metadata (Derakhshani, col. 3, lines 7-17; col. 8, lines 10-15: “However, in some implementations, the interrogator could issue other challenges such as requesting metadata from the camera pipeline, requesting metadata from an operating system associated with the camera pipeline, or requesting metadata both, as an additional type of challenge to the identity of the camera pipeline. Additionally, in some implementations, a challenge-response protocol is implemented as one or more instructions for post-processing of images, such as modifying an image preview function to prompt a user to alter how they interact with the camera”; “In one implementation, the challenge is a camera integrity check. The interrogator may simply ask the operating system of the computing device to provide the camera hardware name, make, model, version, etc., and check if the reported metadata matches metadata provided by the camera hardware, or an unexpected value in the case of a virtual camera.”).
Regarding claim 9, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges include a request for video imagery (Derakhshani, col. 2, lines 51-59: “In one implementation, a challenge-response interrogation of a camera pipeline is performed to ascertain the integrity of a camera pipeline. The interrogator (e.g., a software entity on a client device, or a remote server-based entity providing instructions via the Internet) issues one or more instructions that challenges the camera pipeline. The camera pipeline is given instructions to generate a response, such as to capture and provide to the interrogator images (image frames) taken at specific camera parameter values.”).
Regarding claim 10, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the video source results include video source metadata (Derakhshani, col. 3, lines 7-17; col. 8, lines 10-15: “However, in some implementations, the interrogator could issue other challenges such as requesting metadata from the camera pipeline, requesting metadata from an operating system associated with the camera pipeline, or requesting metadata both, as an additional type of challenge to the identity of the camera pipeline. Additionally, in some implementations, a challenge-response protocol is implemented as one or more instructions for post-processing of images, such as modifying an image preview function to prompt a user to alter how they interact with the camera”; “In one implementation, the challenge is a camera integrity check. The interrogator may simply ask the operating system of the computing device to provide the camera hardware name, make, model, version, etc., and check if the reported metadata matches metadata provided by the camera hardware, or an unexpected value in the case of a virtual camera.”) and
video imagery (Derakhshani, col. 2, lines 51-59: “In one implementation, a challenge-response interrogation of a camera pipeline is performed to ascertain the integrity of a camera pipeline. The interrogator (e.g., a software entity on a client device, or a remote server-based entity providing instructions via the Internet) issues one or more instructions that challenges the camera pipeline. The camera pipeline is given instructions to generate a response, such as to capture and provide to the interrogator images (image frames) taken at specific camera parameter values.”) and
the genuineness score is based at least in part on a comparison of the video source metadata with the received video imagery (Derakhshani, col. 6, lines 62-67; col. 7, lines 1-10; col. 4, lines 54-56: “In one implementation, challenges are triggered based on triggering auto exposure (AE) by changing the scenery/illumination (e.g., via camera movement) and recording whether the reported frame metadata shows a difference in exposure settings (e.g., ISO or exposure time/frame rate. Dark scenery may increase exposure time and drop the frame rate on top of gain increases during AE). The opposite is true, meaning that if an auto exposure lock command is sent to the camera, changing the scenery/illumination/subject composition should not change the exposure parameters of the incoming images while the exposure lock is in place. In another implementation, when the API orders the camera to auto-expose on a bright vs dark patch of the captured scene, the resulting exposure would go down/go up and thus make the incoming image overall darker/brighter, respectively.”; “An image feature analysis module 207 may be programmed to analyze a set of image features relevant to the sub-challenges and generate a score signal.”; the auto-exposure challenge response results compares video metadata to the change in autoexposure done during the challenge to the video (video imagery)).
Regarding claim 11, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges include a request to change camera resolution (Derakhshani, “In one implementation, the challenge is that the interrogating side asks the camera to change its resolution and the interrogator observes the effects on the receiving end to ascertain whether the camera is a real/non-simulated camera. In some implementations, the requested resolution is non-standard, where a real camera is expected not to honor such a non-standard resolution but a virtual camera may. In some implementations, a natively higher resolution camera (e.g., 1080p) is asked to capture images at a lower resolution (e.g., 480p). Then the camera is asked to render its output into an HTML Canvas at exactly double the resolution (2×) of the camera output using the nearest neighbor method for up-sampling. A real camera will show a corresponding blockiness as a result of the nearest neighbor up-sampling, whereas a virtual camera may not. One method to discover such a trend is to compare the 2× nearest neighbor downsampled followed by the 2× nearest neighbor up-sampled image with the original (e.g., by calculating the mean squared error over the whole image or a portion of it). A real camera capture should show no difference. In some implementations, one may use the expected delay for restarting a real camera at a new resolution when challenging the camera to change its resolution as a sign of the camera being real, whereas a virtual camera may not show such delay when forced into a new resolution. In certain cases, the attack can have a static image presented as a video, where having a high structural similarity index measure (SSIM) for two different resolutions can detect the camera injection attack. In one implementation, the SSIM can be measured across several frames to determine camera injection.
Regarding claim 12, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges include a request to use a flash (Derakhshani, col. 6, lines 47-61: “In one implementation. The challenge is related to a lighting setting for a light source. Consider a camera with a light source, such as a torch, flash, or a backlit screen (these sources may emit visible light, or IR light in case of night vision or three-dimensional sensing). The light source may be activated to see the response of object(s) being imaged, with foreground being expected to show a greater degree of reflection change between flash on/off events. The photic response can be detected after the illumination challenge. Randomizing the onset of illumination emission can be employed as another security measure. A lack of such response (reflected lighting in the temporal vicinity of the illumination trigger) indicates a camera hijack attack or similar problem, (unless the environment is extremely bright and thus drowning the light source)”).
Regarding claim 13, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the video source comprises a camera connected to an end-user device, and the machine classifier is implemented on the end-user device (Derakhshani, col. 3, lines 64-67; col. 4, lines 1-6; col. 2, lines 27-36; FIG. 1A-1B: “The camera pipeline may be included in a client computing device having a camera, such as a smartphone, a tablet device, a laptop computer, and a notebook computer, as a few examples. The client computing device may include … an image processing layer, a camera API, and a flash or other camera light source and light sensors.”; “As illustrated in FIG. 1A, FIG. 1B, and FIG. 1C a client computing device 102, such as a smartphone, includes a camera hardware layer 104. For example, as illustrated in FIG. 1A, an application 106 executing on the client device 102 may be responsible for interrogating the camera and making decisions whether to reject or verify the camera. The application 106 may report its results to an external authentication system 140, although it could alternatively take protective actions on the client computing device to prevent potential fraud.; see FIG. 1A-1B in the rejection of claim 1 above).
Regarding claim 14, Derakhshani, in view of Rouse, teaches the method of claim 1, further comprising sending the video source results to a server remote from the video source, wherein the machine classifier is implemented on the server (Derakhshani, col. 2, lines 27-40; FIG. 1B: “As illustrated in FIG. 1B, alternatively an external entity (e.g., a server) could interrogate the camera, such as through a Web API (e.g., a JavaScript API that a browser would provide for web page interactivity … Alternatively, an external server may use a Web API to access features of the camera pipeline available through the camera API”; see FIG. 1B in the rejection of claim 1 above).
Regarding claim 15, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the genuineness score is used to determine which subsequent action of a plurality of subsequent actions is taken by software executing on an end user device (Derakhshani, col. 2, lines 33-36: “The application 106 may report its results to an external authentication system 140, although it could alternatively take protective actions on the client computing device to prevent potential fraud.”).
Regarding claim 16, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the video source results include an indication as to whether the video source performed an action requested by the one or more challenges (Derakhshani, col. 6, lines 47-61: “In one implementation. The challenge is related to a lighting setting for a light source. Consider a camera with a light source, such as a torch, flash, or a backlit screen (these sources may emit visible light, or IR light in case of night vision or three-dimensional sensing). The light source may be activated to see the response of object(s) being imaged, with foreground being expected to show a greater degree of reflection change between flash on/off events. The photic response can be detected after the illumination challenge. Randomizing the onset of illumination emission can be employed as another security measure. A lack of such response (reflected lighting in the temporal vicinity of the illumination trigger) indicates a camera hijack attack or similar problem, (unless the environment is extremely bright and thus drowning the light source)”).
Regarding claim 17, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges request information about properties of a camera of the video source and the genuineness score depends at least in part on a comparison of the video source results with at least one of a camera device name and camera capabilities reported by metadata received from the video source (Derakhshani, col. 8, lines 10-15: “In one implementation, the challenge is a camera integrity check. The interrogator may simply ask the operating system of the computing device to provide the camera hardware name, make, model, version, etc., and check if the reported metadata matches metadata provided by the camera hardware, or an unexpected value in the case of a virtual camera.”; as discussed in the rejection of claim 1 above, each challenge is used to calculate the genuineness score of whether the video source comes from a physical camera or not).
Regarding claim 18, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges affect a computational load on a system associated with the video source and the video results include information indicating the computational load on the system (Derakhshani, col. 7, lines 47-56: “In one implementation, the challenge is that the interrogating side asks the camera to change its resolution and the interrogator observes the effects on the receiving end to ascertain whether the camera is a real/non-simulated camera. In some implementations, the requested resolution is non-standard, where a real camera is expected not to honor such a non-standard resolution but a virtual camera may. In some implementations, a natively higher resolution camera (e.g., 1080p) is asked to capture images at a lower resolution (e.g., 480p).”; as screen resolution increases/decreases during the challenge, the number of pixels that the GPU must process skyrockets/lowers, leading to a higher/lower load on the GPU (computational load)).
Regarding claim 22, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein video source results are used to determine whether or not the received video was captured by a camera of at least one of a specific model and a specific class of camera (Derakhshani, col. 8, lines 10-15: “In one implementation, the challenge is a camera integrity check. The interrogator may simply ask the operating system of the computing device to provide the camera hardware name, make, model, version, etc., and check if the reported metadata matches metadata provided by the camera hardware, or an unexpected value in the case of a virtual camera.”).
Regarding claim 23, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the video source results are used to determine at least one of a specific model and a class of a camera that captured the received video (Derakhshani, col. 8, lines 10-15: “In one implementation, the challenge is a camera integrity check. The interrogator may simply ask the operating system of the computing device to provide the camera hardware name, make, model, version, etc., and check if the reported metadata matches metadata provided by the camera hardware, or an unexpected value in the case of a virtual camera.”)
Regarding claim 25, Derakhshani, in view of Rouse, teaches the method of claim 1, wherein the one or more challenges request changes to video captured by the video source, and analyzing video imagery contained with the video source results to determine a type of camera that captured the received video (Derakhshani, col. 7, lines 35-46: “In one implementation, the challenge may occur over two or more frames for cameras in which the response doesn't immediately occur. For example, in some cameras the exposure change doesn't immediately take effect. This results in a multi-frame transitory regime evident from the image sequence brightness. Such gradual change of exposure can be also leveraged to detect a real camera. This may also be camera dependent. However, the camera type may be inferred from the platform type and release model of the computing device. For example, a specific model of the Android® Platform may have a particular camera type with camera dependent characteristics.”).
Regarding claim 26, Derakhshani, in view of Rouse, teaches the method of claim 25, wherein: the requested changes include at least one of changes in resolution, exposure setting and focus of imagery captured by the video source (Derakhshani, col. 7, lines 35-46; see rejection of claim 25 above discussing exposure change; Examiner is taking a disjunctive claim interpretation of the claim; see claim interpretation above); and
analyzing the video imagery contained with the video source results includes determining whether the video imagery reflects the requested changes (Derakhshani, col. 47-61: “In one implementation, the challenge is that the interrogating side asks the camera to change its resolution and the interrogator observes the effects on the receiving end to ascertain whether the camera is a real/non-simulated camera. In some implementations, the requested resolution is non-standard, where a real camera is expected not to honor such a non-standard resolution but a virtual camera may. In some implementations, a natively higher resolution camera (e.g., 1080p) is asked to capture images at a lower resolution (e.g., 480p). Then the camera is asked to render its output into an HTML Canvas at exactly double the resolution (2×) of the camera output using the nearest neighbor method for up-sampling. A real camera will show a corresponding blockiness as a result of the nearest neighbor up-sampling, whereas a virtual camera may not.”).
Regarding claim 27, Derakhshani teaches a computer program product comprising: a non-transitory computer-readable medium with computer-readable instructions encoded thereon, wherein the computer-readable instructions, when processed by a processing device instruct the processing device to perform a method of verifying a provenance of video received from a video source, the method comprising: (Derakhshani, col. 7, lines 35-46; col. 1, lines 32-39: “For the purposes of this description, a computer-usable or computer readable medium can be any non-transitory storage apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. A data processing system suitable for storing and/or executing program code may include at least one processor coupled directly or indirectly to memory elements through a system bus.”; “One implementation of a method includes interrogating a camera pipeline with instructions of a challenge requesting a response by the camera pipeline. The response of the camera pipeline is analyzed for features associated with a real camera pipeline response. An output is generated indicative of an integrity of the camera pipeline.”).
With regards to the remaining limitations of claim 27, they recite the functions of the process of claim 1, as a non-transitory computer-readable medium with computer-readable instructions. Thus, the analysis in rejecting claim 1 is equally applicable to the remaining limitations of claim 27.
Regarding claim 28, Derakhshani teaches a user device comprising: a memory for storing computer-readable instructions; and a processor connected to the memory, wherein the processor, when executing the computer-readable instructions, causes the user device to perform a method of verifying a provenance of video received from a video source, the method comprising: (Derakhshani, col. 3, lines 64-67; col. 4, lines 1-5; col. 2, lines 51-59: “The camera pipeline may be included in a client computing device having a camera, such as a smartphone, a tablet device, a laptop computer, and a notebook computer, as a few examples. The client computing device may include, for example, one or more processors, a memory, an operating system, local applications stored on a device memory, a user interface, and features related to the camera, such as a camera hardware layer, an image processing layer, a camera API, and a flash or other camera light source and light sensors.”; “In one implementation, a challenge-response interrogation of a camera pipeline is performed to ascertain the integrity of a camera pipeline. The interrogator (e.g., a software entity on a client device, or a remote server-based entity providing instructions via the Internet) issues one or more instructions that challenges the camera pipeline. The camera pipeline is given instructions to generate a response, such as to capture and provide to the interrogator images (image frames) taken at specific camera parameter values.”).
With regards to the remaining limitations of claim 28, they recite the functions of the process of claim 1, as an apparatus. Thus, the analysis in rejecting claim 1 is equally applicable to the remaining limitations of claim 28.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Derakhshani, in view of Rouse, and in further view of U.S. Patent Application Publication No.: 2020/0134148 (Pouria et al.) (hereinafter Pouria).
Regarding claim 19, Derakhshani, in view of Rouse, teaches the method of claim 1.
Derakhshani, in view of Rouse fails to teach
wherein the system associated with the video source includes a plurality of cameras, and the one or more challenges are directed independently to each camera.
Pouria teaches
wherein the system associated with the video source includes a plurality of cameras, and the one or more challenges are directed independently to each camera. (Pouria, para. [0046]; para. [0082]; FIG. 5A-5B: “FIG. 5a and FIG. 5b show two simulated screens that an application operating on the mobile electronic device 201 may show on display 204. The screens show two action instructions, each of which is a liveness test, overlaid on an image of the user 400 … The second action instruction relates to a motion challenge and requests the user 400 to look in a particular direction … The challenges are typically selected to provide complementary information.”; “The electronic device 201, 202 may also include one or more cameras 253 … In at least some embodiments, the electronic device 201, 202 includes a front facing camera 253 … A back facing camera may be used alternatively to, or in addition to, in some embodiments.”;
PNG
media_image6.png
434
684
media_image6.png
Greyscale
).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the system associated with the video source, as taught by Derakhshani, in view of Rouse, to include a plurality of cameras, and the one or more challenges are directed independently to each camera, as taught by Pouria.
The suggestion/motivation for doing so would have been that “one advantage of acquiring different data types is that environmental conditions affect different data types differently; therefore, acquiring different data types makes the method more robust, especially when it is operated in adverse environmental conditions … acquiring data from different data acquisition systems also means that any weaknesses or failures in one data acquisition system, or its related data, can be mitigated by another system, or data therefrom” (Pouria, para. [0014]); utilizing both a back and front facing cameras to respond to challenges for real camera verification improves security and prevents injection attacks.
Therefore, it would have been obvious to combine Derakhshani and Rouse, with Pouria, to obtain the invention as specified in claim 19.
Allowable Subject Matter
Claims 4, 20-21, and 24 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL ADAM SHARIFF whose telephone number is 571-272-9741. The examiner can normally be reached M-F 8:30-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached on 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ADAM SHARIFF/
Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672