DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
1. This action is in response to the amendment filed on February 12th, 2026. Claims 1, 8, 11, 16, 18, and 20 have been amended. Claims 1-20 are pending. Claims 1-20 remain rejected in the application. Applicant’s amendments to the claims have overcome each and every objection previously set forth in the non-final office action mailed November 12th, 2025.
Response to Arguments
2. Applicant’s arguments with respect to claim 1, and similarly claims 11 and 20, filed on 2/12/2026, with respect to the rejection under 35 U.S.C. 103 regarding that the prior art does not teach the limitation(s): “identifying, by the electronic device, one or more intrinsic properties comprising at least one of an energy, a force, a mass, a friction, or a pressure of the at least one of object, person, or background in each frame of plurality of frames based on the relationship among the at least one of object, person, or background and the corresponding pixel-motion information” and “identifying, by the electronic device, inconsistent motion based on a violation of laws of physics of the at least one of object, person, or background in at least one frame of the plurality of frames of the video based on the one or more intrinsic properties wherein the violation of the laws of physics comprises at least one of floating, interpenetration, or perpetual motion” have been fully considered, but are moot because of new grounds for rejection. Claim 1, and similarly claims 11 and 20, are now disclosed by Ramírez Vicente, Wang, and Conotter.
3. Regarding arguments to claims 2-10 and 12-19, they are dependent on independent claims 1 and 11 respectively. Applicant does not argue anything other than independent claim 1, and similarly claims 11 and 20. The limitations in those claims, in conjunction with combination, has previously been established and explained.
Claim Rejections - 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claims 1, 7, 9, 11, 17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ramírez Vicente et al. (US-2024/0281945-A1, hereinafter "Ramírez Vicente") in view of Wang et al. (CN-114565880-A, hereinafter "Wang"), and further in view of Conotter et al. (NPL: “Exposing Digital Forgeries in Ballistic Motion.” IEEE Transactions on Information Forensics and Security, vol. 7, no. 1, Feb. 2012, pp. 283–296, hereinafter "Conotter").
6. As per claim 1, Ramírez Vicente discloses: A method for detecting artificial intelligence (AI) generated content in a video, comprising:
obtaining, by an electronic device, the video comprising a plurality of frames; (Ramírez Vicente, Abstract, “A method of synthetic content detection in real-time, from a video input source providing images ...” and [0022], “The present invention can be integrated as an additional security layer within applications, devices, and other platforms, bolstering their defences against deepfake attacks.”)
identifying, by the electronic device, at least one of object, person, or background in each frame of the plurality of frames of the video; (Ramírez Vicente, [0037]-[0038], “The verification process described here is called “Head Pose verification” but goes beyond its definition as it is not only centered on the head, but it can be expanded to all parts of the human's body. The goal with the head pose verification is to offer a new approach to detect anomalies in real time, e.g., during a video call, to check that the person in front of the user(s) is 100% human.” and [0051], “The focal point allows the detection of a point/area/object at which the subject is looking.”)
identifying, by the electronic device, pixel-motion information of [[each]] pixel in each frame of the plurality of frames; (Ramírez Vicente, [0040], “In order to calculate the movements of the different captured 3D points, geometry and linear algebra techniques are used; in particular, the camera projection matrix, which relates the 3D points in space to their 2D projections in the image, is used. Using this matrix, the relative position and orientation of the camera in each frame of the video are calculated and then the motions of the different points are inferred.” and [0044], “In summary, the motions of the different captured 3D points can be calculated using a combination of translation and rotation, and these motions can be represented using homogeneous transformation matrices. These are the base calculations to detect anomalies in human motion.”)
identifying, by the electronic device, a relationship among the at least one of object, person, or background and the corresponding pixel-motion information in each frame of the plurality of frames; (Ramírez Vicente, [0081], “A set of position vectors (x, y, z) in real-time of a person's body part at different points in real-time is obtained (e.g. using Mediapipe solutions that can work with single images or a continuous stream of images and output body pose landmarks in image coordinates and in 3-dimensional world coordinates).”)
identifying, by the electronic device, one or more intrinsic properties [[comprising at least one of an energy, a force, a mass, a friction, or a pressure]] of the at least one of object, person, or background in each frame of plurality of frames based on the relationship among the at least one of object, person, or background and the corresponding pixel-motion information; (See Ramírez Vicente, [0046]-[0053] and [0081] below.)
identifying, by the electronic device, inconsistent motion [[based on a violation of laws of physics]] of the at least one of object, person, or background in at least one frame of the plurality of frames of the video based on the one or more intrinsic properties [[wherein the violation of the laws of physics comprises at least one of floating, interpenetration, or perpetual motion;]] and (Ramírez Vicente, [0046]-[0053], “Once all the information obtained in this process of vectorizations and 3D projection (1000) is stored in matrices, anomaly detection (1010) is performed as described below. In a preferred embodiment, the detected anomalies are the following: i. Static motion factors. It is common in deepfakes to always keep the gaze on the front and make as few movements as possible so that the mapping of the fake face is as perfect as possible and no cuts or artifacts are noticed. In a human conversation, making more natural and varied movements is more common. Too static movements of a body part have a speed that exceeds a predetermined first threshold. ii. Facial symmetry. The points extracted from the subject's face are calculated. If the deepfake process fails in the slightest, this symmetry is broken, resulting in an anomaly between the distance of the different points captured or a possible manipulation of the image. iii. Head abrupt turns. Head movements can also be captured and, taking into account the framerate, head movements that do not correspond to a natural movement of the head can be detected; i.e., jumps between one position and another which are too abrupt and not due to a too low framerate can also be detected. Too abrupt head turns have a speed that exceeds a predetermined second threshold. iv. Detect other parts of the body and their motion. The same which applies to the face and head can be applied to the torso and hands. For example, if the hands do not appear or no movement is detected, it can also indicate a deepfake. On the other hand, a torso that is too static or, on the contrary, with abrupt movements can also be taken as an indication of a deepfake. v. Checking with the focal point. The focal point allows the detection of a point/area/object at which the subject is looking. This can also indicate possible deepfakes if repetition in these values is detected, abrupt changes, etc. vi. Emotion detection. Emotions are reflected in facial expressions, and certain emotions have specific movement patterns on the face. If unusual movement patterns or expressions are detected that do not match the context of the image, this may be a sign that image manipulation is occurring. vii. Lip movement analysis. A common technique in detecting deepfakes in video conferences is lip movement analysis. This is achieved by tracking the 3D points corresponding to the lips, which are used to measure the synchronization between the lip movement and the sound of the voice. If a lack of synchronization or an unusual lip movement pattern is detected, this can mean that manipulation is occurring in the image.” and [0081], “A set of position vectors (x, y, z) in real-time of a person's body part at different points in real-time is obtained (e.g. using Mediapipe solutions that can work with single images or a continuous stream of images and output body pose landmarks in image coordinates and in 3-dimensional world coordinates).”)
displaying, by the electronic device, AI generated content in the at least one frame of the plurality of frames of the video based on the identified inconsistent motion of the at least one of object, person, or background in at least one frame of the plurality of frames of the video. (Ramírez Vicente, Fig. 6; Claim 1, “A computer-implemented method for detecting synthetic content in videos … and providing a result in real-time indicating whether a synthetic content is detected in the video, the result being based on the detected anomalies and each verified criterion.” and [0026]-[0029], “FIG. 3 shows a picture of a graphical user interface showing the different points that correspond to the person's face by identifying the eyes, mouth and face shape. FIG. 5 shows a block diagram of a system architecture for verifying the eyes blink, according to a possible embodiment of the present invention for detecting deepfake. … FIG. 6 shows a picture of a graphical user interface showing the results of the detection of eye blinks and calculation of the eye aspect ratio for a subject.” and [0045], “HEAD POSE VERIFICATION The basic scheme of operation of head pose verification (head position) is shown in FIG. 1. In this schematic diagram, a video content is input in real time, by the user using a graphical user interface (10), into the deepfake detector from different input sources … Once the 3D projection (1000) is obtained, the detection of one or more anomalies (1010) for head pose verification (e.g., static motion factors, facial symmetry, head abrupt turns, other parts of the body and their motion, the focal point, emotions and movements of the lips) is performed to output a final result (1100) indicating data about a deepfake based on the head pose verification.”)
7. Ramírez Vicente doesn't explicitly disclose but Wang discloses: [[identifying, by the electronic device, pixel-motion information of]] each [[pixel in each frame of the plurality of frames;]] (Wang, [0005], “In real video, the movement of objects is continuous and consistent between consecutive frames, implying the original temporal information of the video. In the case of fake videos, the face replacement is done frame by frame during the process of generating the fake video. The motion distortion and warping that occur during the replacement process inevitably lead to a lack of consistency. Therefore, extracting and identifying the differences between two adjacent frames in a video can help identify and verify fake videos. Optical flow tracing is a mature and reliable object motion tracking technology that can accurately predict object motion pixel by pixel, making it very suitable for detecting fake videos.”)
8. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Ramírez Vicente to include the disclosure of identifying pixel-motion information of each pixel in each frame, of Wang. The motivation for this modification could have been to allow for an entire image frame of a video to be tracked between frames. By doing so, this provides the ability to track all changes, including people, objects, and background, in a video to detect inconsistencies.
9. Ramírez Vicente in view of Wang doesn't explicitly disclose but Conotter discloses: [[identifying, by the electronic device, one or more intrinsic properties]] comprising at least one of an energy, a force, a mass, a friction, or a pressure [[of the at least one of object, person, or background in each frame of plurality of frames based on the relationship among the at least one of object, person, or background and the corresponding pixel-motion information;]] (Conotter, p. 283, ¶ 1, “In this paper, we describe a forensic technique that is tailored to determine if video of a purportedly ballistic motion, such as a ball being thrown (as in Fig. 1) or a person jumping through the air, or a motorcycle soaring off of a ramp, is consistent with the geometry and physics of a free-falling projectile.” and p. 283, ¶ 3, “We begin by describing a plausible, albeit somewhat simplified, physical model for the expected trajectory of a projectile motion, and a basic imaging model for a static or moving camera. We then describe a technique to determine if the image of the trajectory of a projectile motion is consistent with this physical model.”; Examiner’s note: Ballistic projectile motion utilizes kinetic energy.)
[[identifying, by the electronic device, inconsistent motion]] based on a violation of laws of physics (Conotter, Abstract, “We describe a geometric technique to detect physically implausible trajectories of objects in video sequences. This technique explicitly models the three-dimensional ballistic motion of objects in free-flight and the two-dimensional projection of the trajectory into the image plane of a static or moving camera. Deviations from this model provide evidence of manipulation.“ and p. 283, ¶ 1, “Increasingly sophisticated video editing and special effects software has made it possible to create forged video sequences that appear to contain realistic dynamic motion. ... In this paper, we describe a forensic technique that is tailored to determine if video of a purportedly ballistic motion, such as a ball being thrown (as in Fig. 1) or a person jumping through the air, or a motorcycle soaring off of a ramp, is consistent with the geometry and physics of a free-falling projectile.” and p. 288, ¶ 5, “Forensics: Compute the error between the two trajectory parametrizations (Section III-G). An error above a specified threshold is considered to be evidence of a fake.”) [[of the at least one of object, person, or background in at least one frame of the plurality of frames of the video based on the one or more intrinsic properties]] wherein the violation of the laws of physics comprises at least one of floating, interpenetration, or perpetual motion; [[and]] (Conotter, p. 283, ¶ 1, “Increasingly sophisticated video editing and special effects software has made it possible to create forged video sequences that appear to contain realistic dynamic motion. … These videos appear to show spectacular basketball shots, gravity-defying acrobatics, and the bone-crushing results of daredevil leaps and jumps. … In this paper, we describe a forensic technique that is tailored to determine if video of a purportedly ballistic motion, such as a ball being thrown (as in Fig. 1) or a person jumping through the air, or a motorcycle soaring off of a ramp, is consistent with the geometry and physics of a free-falling projectile.”; Examiner’s note: Conotter discloses a forensic technique to determine if “gravity-defying” style videos are real or fake. “Gravity-defying” is a type of “floating” outside the law of gravity.)
10. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Ramírez Vicente in view of Wang to include the disclosure of identifying intrinsic properties and violations of the laws of physics such as floating, interpenetration, or perpetual motion, of Conotter. The motivation for this modification could have been to help indicate motion or properties that break the laws of physics. This would help to point out inconsistencies and impossible scenarios in videos that are outside of natural behavior of the real-world. This can provide a significant piece of evidence that the video is either fake or synthetic in nature.
11. As per claim 7, Ramírez Vicente in view of Wang, and further in view of Conotter discloses: The method as claimed in claim 1, wherein the identifying, by the electronic device, the inconsistent motion of the at least one of object, person, or background in at least one frame for the plurality of frames of the video comprises:
identifying, by the electronic device, the at least one of object, person, or background in each frame of the plurality of frames being at least one of consistent or inconsistent based on the one or more intrinsic properties of the at least one of object, person, or background; (Ramírez Vicente, [0132], “The focal point where the person is looking at is calculated to be used as a base and variant to detect the abovementioned detection techniques for head pose verification. But if the detection is focused only on the focal point, the consistency between the movements of the eyes and the direction of the focal point can be monitored. In a normal situation, the focal point also changes coherently when the person's eyes (or nose) motion. If there are inconsistencies between the movement and the focal point, it could be a sign of a deepfake.” and [0161]-[0162], “Monitor consistency: Monitor the consistency of the closest matching emotions and real-time facial key points over time. Abrupt and inconsistent transitions between emotions or significant changes in facial key points without a corresponding shift in sentiment may indicate a deepfake.” and [0165]-[0166], “Check for inconsistencies: If emotions change abruptly without corresponding changes in facial key points, or if facial key points change significantly without changes in emotion, we could conclude that there is a possible deepfake.”)
localizing, by the electronic device, an inconsistent region in each frame of plurality of frames using a Convolutional Neural Network (CNN) model, based on a determination that each frame of the plurality of frames is inconsistent. (Wang, [0019], “Step 3: Based on the detection convolutional neural network, use optical flow tracing data to verify the fake video.” and [0005], “In real video, the movement of objects is continuous and consistent between consecutive frames, implying the original temporal information of the video. In the case of fake videos, the face replacement is done frame by frame during the process of generating the fake video. The motion distortion and warping that occur during the replacement process inevitably lead to a lack of consistency. Therefore, extracting and identifying the differences between two adjacent frames in a video can help identify and verify fake videos. Optical flow tracing is a mature and reliable object motion tracking technology that can accurately predict object motion pixel by pixel, making it very suitable for detecting fake videos.” and [0010], “1) This invention analyzes the motion and light characteristics of videos through optical flow tracing, thereby discovering inconsistencies generated during the forgery process and obtaining more accurate forgery video verification results.”)
12. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Ramírez Vicente in view of Conotter to include the disclosure of localizing an inconsistent region in each frame of plurality of frames using a Convolutional Neural Network (CNN) model, based on a determination that each frame of the plurality of frames is inconsistent, of Wang. The motivation for this modification could have been to use a neural network to help find frames with inconsistencies that a human might not have noticed. This would help to identify AI generated or fake video content.
13. As per claim 9, Ramírez Vicente in view of Wang , and further in view of Conotter discloses: The method as claimed in claim 1, further comprising:
localizing, by the electronic device, a spatial region in the at least one frame based on the inconsistent motion of the at least one of object, person, or background in the at least one frame of the video. (Ramírez Vicente, [0132], “The focal point where the person is looking at is calculated to be used as a base and variant to detect the abovementioned detection techniques for head pose verification. But if the detection is focused only on the focal point, the consistency between the movements of the eyes and the direction of the focal point can be monitored. In a normal situation, the focal point also changes coherently when the person's eyes (or nose) motion. If there are inconsistencies between the movement and the focal point, it could be a sign of a deepfake.” and [0177]-[0178], “Compare real-time lip features to expected lip movements: Compare the real-time lip features to the expected lip movements for the given speech sound or expression. The difference can be calculated using a distance metric, such as Euclidean distance, to measure the inconsistency between the expected and observed lip movements.” and [0181]-[0184], “Monitor consistency: Monitor the consistency of the real-time lip features and their relationship to speech or expressions over time. Inconsistent lip movements or significant changes in lip features without a corresponding change in speech or expressions may indicate a deepfake. Check for inconsistencies: If lip features change abruptly without corresponding changes in speech or expressions, or if speech or expressions change significantly without changes in lip features, a possible deepfake is concluded.”)
14. Claim 11 is similar in scope to claim 1 except for additional limitations that Ramírez Vicente in view of Wang , and further in view of Conotter discloses: An electronic device for detecting artificial intelligence (AI) generated content in a video, comprising:
one or more memories storing instructions;
one or more processors communicatively coupled to the memory;
wherein the one or more processors are configured to execute the instructions to: (Ramírez Vicente, [0033]-[0034], “The embodiments of the present invention can be implemented in a software application that can analyze any software application running in a personal computer (PC), mobile phone or any other smart device, which is capable of playing a video or directly a call from any videoconferencing platform. Videoconference calls are the ideal place to try to fool someone using deepfakes. The proposed fake detection method can obtain the image to be analyzed from a webcam or other camera or video capturing means. In a first analysis performed by at least one processor, one or more body parts of a subject (e.g., head, the whole face, eyes, mouth or lips, hands, torso, etc.) are detected.” and [0035], “In a possible implementation, the method can be programmed in Python in a Windows environment where a user interface is created to interact with the users, which provides simplicity when performing the detection operations.” and [0150], “To implement this analysis, it is mandatory to create a database of vector patterns that identify the person's mood. Generic patterns or templates for laughter, sadness, etc. are calculated. To do so, we must store these different states in matrices to check them later during detection.”)
15. Claim 17, which is similar in scope to claims 7 and 11, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 7.
16. Claim 19, which is similar in scope to claims 9 and 11, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 9.
17. Claim 20, which is similar in scope to claims 1 and 11, is thus rejected under the same rationale as described above.
18. Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Ramírez Vicente et al. (US-2024/0281945-A1, hereinafter "Ramírez Vicente") in view of Wang et al. (CN-114565880-A, hereinafter "Wang"), further in view of Conotter et al. (NPL: “Exposing Digital Forgeries in Ballistic Motion.” IEEE Transactions on Information Forensics and Security, vol. 7, no. 1, Feb. 2012, pp. 283–296, hereinafter "Conotter"), and further in view of Tang et al. (US-2022/0172518-A1, hereinafter "Tang").
19. As per claim 2, Ramírez Vicente in view of Wang, and further in view of Conotter discloses: The method as claimed in claim 1, wherein the identifying the at least one of object, person, or background in each frame of the plurality of frames of the video comprises: (see rejection of claim 1)
20. Ramírez Vicente in view of Wang, and further in view of Conotter doesn't explicitly disclose but Tang discloses: identifying, by the electronic device, one or more spatial semantics from each frame of the plurality of frames using a CNN model, wherein the one or more spatial semantics are captured as intermediate features for each frame of the plurality of frames; and (Tang, [0051], “In some embodiments, the human face in the image to be recognized may be alternatively detected using another human face detection algorithm, for example, a cascade convolutional neural network (Cascade CNN), a DenseBox, a human face detection network (Faceness-Net), a face region-convolutional neural network (Face R-CNN), or a multi-task convolutional neural network (MTCNN), to obtain the human face coordinate frame information.” and [0086], “According to the image recognition method in the embodiments of this application, the target object in the image to be recognized and the video to be recognized can be detected and recognized to determine whether the target object is real or fake ... when whether the target object is real or fake is recognized, a low-level feature including pixel and texture levels and a high-level feature including global semantic information are adopted, which includes detection at three different phases, to further improve the accuracy of image recognition ...” and [0064], “By extracting the global feature information, a contextual relationship between the human face and the background can be fully considered, and a difference between the human face region and the background can be used, to perform real or fake identification on the human face, and improve the accuracy of human face recognition.”)
identifying, by the electronic device, the at least one of object, person, or background in each frame of the plurality of frames of the video based on the one or more spatial semantics of each frame of the plurality of frames of the video. (Tang, [0067], “When the target object in the image to be recognized is determined to be real according to the feature information from low-level to high-level, the target object in the image to be recognized may be determined to be real, and the target information of “real face, safe” is outputted.” and [0086], “According to the image recognition method in the embodiments of this application, the target object in the image to be recognized and the video to be recognized can be detected and recognized to determine whether the target object is real or fake ... when whether the target object is real or fake is recognized, a low-level feature including pixel and texture levels and a high-level feature including global semantic information are adopted, which includes detection at three different phases, to further improve the accuracy of image recognition …” and [0064], “By extracting the global feature information, a contextual relationship between the human face and the background can be fully considered, and a difference between the human face region and the background can be used, to perform real or fake identification on the human face, and improve the accuracy of human face recognition.”)
21. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Ramírez Vicente in view of Wang, and further in view of Conotter to include the disclosure of identifying one or more spatial semantics from each frame of the plurality of frames using a CNN model, wherein the one or more spatial semantics are captured as intermediate features for each frame of the plurality of frames; and identifying the at least one of object, person, or background in each frame of the plurality of frames of the video based on the one or more spatial semantics of each frame of the plurality of frames of the video of, Tang. The motivation for this modification could have been to utilize the neural network to help define relationships between spatial features within a video frame. This would help to identify anomalies as well as other, real objects within the frame.
22. Claim 12, which is similar in scope to claims 2 and 11, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 2.
23. Claims 3-4 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Ramírez Vicente et al. (US-2024/0281945-A1, hereinafter "Ramírez Vicente") in view of Wang et al. (CN-114565880-A, hereinafter "Wang"), further in view of Conotter et al. (NPL: “Exposing Digital Forgeries in Ballistic Motion.” IEEE Transactions on Information Forensics and Security, vol. 7, no. 1, Feb. 2012, pp. 283–296, hereinafter "Conotter"), and further in view of Sabokrou et al. (NPL: "Real-time anomaly detection and localization in crowded scenes," 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Boston, MA, USA, 2015, pp. 56-62, hereinafter "Sabokrou").
24. As per claim 3, Ramírez Vicente in view of Wang, and further in view of Conotter discloses: The method as claimed in claim 1, wherein the identifying the pixel-motion information of each pixel in each frame of the plurality of frames comprises: (see rejection for claim 1)
25. Ramírez Vicente in view of Wang, and further in view of Conotter doesn't explicitly disclose but Sabokrou discloses: dividing, by the electronic device, each frame of the plurality of frames into base patches and centroidal patches, wherein the base patches and centroidal patches comprises the at least one of object, person, or background; (Sabokrou, Fig. 2; page 57, ¶ 5, “To represent each video, first each video is converted into a number of non-overlapping cubic patches; a sketch of this video representation is shown in Fig. 2. Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video.” and page 59, ¶ 7, page 60, ¶ 1, “This dataset includes two subsets, ped1 and ped2, that are from two different outdoor scenes. ... The dominant mobile objects in these scenes are pedestrians. Therefore, any object (e.g., a car, skateboarder, wheelchair, or bicycle) is considered as being an anomaly.”)
identifying, by the electronic device, a patch-wise trajectory of the at least one of object, person, or background in the base patches and centroidal patches of each frame of the plurality of frames; and (Sabokrou, page 57, ¶ 5, “Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video. Therefore, these anomaly patches should meet three conditions: 1. The similarity between the anomaly patches and their adjacent (i.e., defined by spatial changes) patches does not follow the same pattern as from normal patches to their adjacent patches. 2. It is most likely that the temporal changes of an anomaly patch would not follow the pattern in the temporal changes of normal patches. 3. It is obvious that the occurrence likelihood of an anomaly patch is less than that of normal patches.” and page 58, ¶ 4, “To describe each video patch, we use a set of local features. The similarity between each patch and its neighboring patches are calculated. As for the neighbors, we consider nine spatial neighboring patches and one temporal neighboring patch (the one right behind the patch of interest when arranged temporally), yielding to 10 neighbors for each single patch.”)
identifying, by the electronic device, the pixel-motion information of the at least one of object, person, or background across the plurality of frames of the video based on the patch-wise trajectory. (Sabokrou, page 59, ¶ 7, page 60, ¶ 1, “This dataset includes two subsets, ped1 and ped2, that are from two different outdoor scenes. ... The dominant mobile objects in these scenes are pedestrians. Therefore, any object (e.g., a car, skateboarder, wheelchair, or bicycle) is considered as being an anomaly.” and page 60, ¶ 2-3, "Frame level measure: If one pixel detects an anomaly then it is considered as being an anomaly. Pixel level measure: If at least 40 percent of anomaly ground truth pixels are covered by pixels detected by the algorithm, then the frame is considered to be an anomaly." and page 57, ¶ 5, “Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video. Therefore, these anomaly patches should meet three conditions: 1. The similarity between the anomaly patches and their adjacent (i.e., defined by spatial changes) patches does not follow the same pattern as from normal patches to their adjacent patches. 2. It is most likely that the temporal changes of an anomaly patch would not follow the pattern in the temporal changes of normal patches. 3. It is obvious that the occurrence likelihood of an anomaly patch is less than that of normal patches.”)
26. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Ramírez Vicente in view of Wang, and further in view of Conotter to include the disclosure of dividing each frame of the plurality of frames into base patches and centroidal patches, wherein the base patches and centroidal patches comprises the at least one of object, person, or background; identifying a patch-wise trajectory of the at least one of object, person, or background in the base patches and centroidal patches of each frame of the plurality of frames; and identifying the pixel-motion information of the at least one of object, person, or background across the plurality of frames of the video based on the patch-wise trajectory, of Sabokrou. The motivation for this modification could have been to divide up the video frames into patches in order to help track trajectory of items within the frame and be able to identify real content such as an object or person.
27. As per claim 4, Ramírez Vicente in view of Wang, further in view of Conotter, and further in view of Sabokrou discloses: The method as claimed in claim 3, wherein the identifying, by the electronic device, the patch-wise trajectory in the base patches and centroidal patches comprises:
identifying, by the electronic device, each of the pixels in the base patches and the centroidal patches of each frame of the plurality of frames; and (Sabokrou, page 60, ¶ 1-3, “We use two evaluation measures, one at frame level and the other at pixel level. In addition to these, we define a new measure for the accuracy of anomaly localization, called dual pixel level. These measures are defined as follows: Frame level measure: If one pixel detects an anomaly then it is considered as being an anomaly. Pixel level measure: If at least 40 percent of anomaly ground truth pixels are covered by pixels detected by the algorithm, then the frame is considered to be an anomaly.” page 60, ¶ 5, “Dual pixel level: In this measure, a frame is considered as being an anomaly if (1) it satisfies the anomaly condition at pixel level and (2) at least β percent (i.e., 10%) of the pixels detected as anomaly are covered by the anomaly ground truth. If, in addition to the anomaly region, irrelevant regions are also considered as being an anomaly, then this measure does not identify the frame as being positive. Figure 6 shows an example for the different measures of anomaly detection.” and page 57, ¶ 5, “Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video. Therefore, these anomaly patches should meet three conditions: 1. The similarity between the anomaly patches and their adjacent (i.e., defined by spatial changes) patches does not follow the same pattern as from normal patches to their adjacent patches. 2. It is most likely that the temporal changes of an anomaly patch would not follow the pattern in the temporal changes of normal patches. 3. It is obvious that the occurrence likelihood of an anomaly patch is less than that of normal patches.”)
obtaining, by the electronic device, the patch-wise trajectory by performing optical flow normalization in the base patches and the centroidal patches of each frame of the plurality of frames. (Sabokrou, page 57, ¶ 5, “Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video. Therefore, these anomaly patches should meet three conditions: 1. The similarity between the anomaly patches and their adjacent (i.e., defined by spatial changes) patches does not follow the same pattern as from normal patches to their adjacent patches. 2. It is most likely that the temporal changes of an anomaly patch would not follow the pattern in the temporal changes of normal patches. 3. It is obvious that the occurrence likelihood of an anomaly patch is less than that of normal patches.” and Wang, [0019], “Step 3: Based on the detection convolutional neural network, use optical flow tracing data to verify the fake video.” and Wang, [0005], “In real video, the movement of objects is continuous and consistent between consecutive frames, implying the original temporal information of the video. In the case of fake videos, the face replacement is done frame by frame during the process of generating the fake video. The motion distortion and warping that occur during the replacement process inevitably lead to a lack of consistency. Therefore, extracting and identifying the differences between two adjacent frames in a video can help identify and verify fake videos. Optical flow tracing is a mature and reliable object motion tracking technology that can accurately predict object motion pixel by pixel, making it very suitable for detecting fake videos.” and Wang, [0010], “1) This invention analyzes the motion and light characteristics of videos through optical flow tracing, thereby discovering inconsistencies generated during the forgery process and obtaining more accurate forgery video verification results.” and Wang, [0007], “The first and second 7×7 convolutional blocks each consist of a 7×7 convolutional layer, a normalization layer ...”)
28. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 3 of Ramírez Vicente in view of Wang, and further in view of Conotter to include the disclosure of identifying each of the pixels in the base patches and the centroidal patches of each frame of the plurality of frames; and obtaining the patch-wise trajectory by performing optical flow normalization in the base patches and the centroidal patches of each frame of the plurality of frames, of Sabokrou. The motivation for this modification could have been to keep track of the patches via optical flow and be able to identify motion on a pixel level to keep track of fine details within a video.
29. As per claim 13, Ramírez Vicente in view of Wang, further in view of Conotter, and further in view of Sabokrou discloses: The electronic device as claimed in claim 11, wherein to identify the pixel-motion information of each pixel in each frame of the plurality of frames, the one or more processors are further configured to execute the instructions to:
divide each frame of the plurality of frames into base patches and centroidal patches; (Sabokrou, Fig. 2; page 57, ¶ 5, “To represent each video, first each video is converted into a number of non-overlapping cubic patches; a sketch of this video representation is shown in Fig. 2. Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video.”)
identify a patch-wise trajectory of the at least one of object, person, or background in the base patches and centroidal patches, wherein the base patches and centroidal patches comprises at least one of object, person, or background; and (Sabokrou, page 57, ¶ 5, “Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video. Therefore, these anomaly patches should meet three conditions: 1. The similarity between the anomaly patches and their adjacent (i.e., defined by spatial changes) patches does not follow the same pattern as from normal patches to their adjacent patches. 2. It is most likely that the temporal changes of an anomaly patch would not follow the pattern in the temporal changes of normal patches. 3. It is obvious that the occurrence likelihood of an anomaly patch is less than that of normal patches.” and page 58, ¶ 4, “To describe each video patch, we use a set of local features. The similarity between each patch and its neighboring patches are calculated. As for the neighbors, we consider nine spatial neighboring patches and one temporal neighboring patch (the one right behind the patch of interest when arranged temporally), yielding to 10 neighbors for each single patch.” and page 59, ¶ 7, page 60, ¶ 1, “This dataset includes two subsets, ped1 and ped2, that are from two different outdoor scenes. ... The dominant mobile objects in these scenes are pedestrians. Therefore, any object (e.g., a car, skateboarder, wheelchair, or bicycle) is considered as being an anomaly.”)
identify the pixel-motion information of the at least one of object, person, or background across the plurality of frames of the video based on the patch-wise trajectory. (Sabokrou, page 59, ¶ 7, page 60, ¶ 1, “This dataset includes two subsets, ped1 and ped2, that are from two different outdoor scenes. ... The dominant mobile objects in these scenes are pedestrians. Therefore, any object (e.g., a car, skateboarder, wheelchair, or bicycle) is considered as being an anomaly.” and page 60, ¶ 2-3, "Frame level measure: If one pixel detects an anomaly then it is considered as being an anomaly. Pixel level measure: If at least 40 percent of anomaly ground truth pixels are covered by pixels detected by the algorithm, then the frame is considered to be an anomaly." and page 57, ¶ 5, “Generally, every video has one or a set of dominant events. Thus, one expects that normal patches have similar relations with their adjacent patches and a high likelihood of occurrence in the video. Therefore, these anomaly patches should meet three conditions: 1. The similarity between the anomaly patches and their adjacent (i.e., defined by spatial changes) patches does not follow the same pattern as from normal patches to their adjacent patches. 2. It is most likely that the temporal changes of an anomaly patch would not follow the pattern in the temporal changes of normal patches. 3. It is obvious that the occurrence likelihood of an anomaly patch is less than that of normal patches.”)
The motivation for this modification is the same as claim 3.
30. Claim 14, which is similar in scope to claims 4 and 11, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 4.
31. Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ramírez Vicente et al. (US-2024/0281945-A1, hereinafter "Ramírez Vicente") in view of Wang et al. (CN-114565880-A, hereinafter "Wang"), further in view of Conotter et al. (NPL: “Exposing Digital Forgeries in Ballistic Motion.” IEEE Transactions on Information Forensics and Security, vol. 7, no. 1, Feb. 2012, pp. 283–296, hereinafter "Conotter"), and further in view of Deng et al. (CN-112215180-A, hereinafter "Deng").
32. As per claim 5, Ramírez Vicente in view of Wang, and further in view of Conotter discloses: The method as claimed in claim 1, wherein the identifying the relationship among the at least one of object, person, or background and the corresponding pixel-motion information comprises: (see rejection for claim 1)
33. Ramírez Vicente in view of Wang, and further in view of Conotter doesn't explicitly disclose but Deng discloses: identifying, by the electronic device using an AI model, the relationship in a form of a fused feature map by fusing the information of the at least one of object, person, or background in each patch from each frame of the plurality of frames of the video and the pixel-motion information of the at least one of object, person, or background from the corresponding patch from each frame across the plurality of frames of the video. (Deng, [0215], “3D Convolutional Neural Networks (3D-CNNs) work by stacking multiple consecutive frames to form a cube. In other words, they combine local facial image sequences and global facial image sequences into cubes, and then use 3D convolutional kernels within the cubes to extract features. In this structure, each feature map in the convolutional layer is connected to multiple neighboring consecutive frames in the previous convolutional layer, thus capturing motion information.” and [0132], “In the process of determining the local image sequence of a face, it is necessary to determine the motion state of the local face region corresponding to the detection activity in the video frame to be detected, such as whether the mouth is open, whether the mouth is open to the maximum position, or whether the eyes are closed, whether the eyes are closed to a set position, etc. Therefore, in this embodiment of the application, in order to accurately determine the motion state of the local face region corresponding to the detection activity in the video frame to be detected, the motion state of the local face region corresponding to the detection activity in the video frame to be detected can be obtained by determining the position information of the set detection point of the detection target object.”)
34. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Ramírez Vicente in view of Wang, and further in view of Conotter to include the disclosure of Identifying the relationship in a form of a fused feature map by fusing the information of the at least one of object, person, or background in each patch from each frame of the plurality of frames of the video and the pixel-motion information of the at least one of object, person, or background from the corresponding patch from each frame across the plurality of frames of the video, of Deng. The motivation for this modification could have been to use a feature map to help identify spatial relationships between items within a video frame. This would help identify all movement within a frame so that any anomalies can be more likely to be identified.
35. Claim 15, which is similar in scope to claims 5 and 11, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 5.
36. Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ramírez Vicente et al. (US-2024/0281945-A1, hereinafter "Ramírez Vicente") in view of Wang et al. (CN-114565880-A, hereinafter "Wang"), further in view of Conotter et al. (NPL: “Exposing Digital Forgeries in Ballistic Motion.” IEEE Transactions on Information Forensics and Security, vol. 7, no. 1, Feb. 2012, pp. 283–296, hereinafter "Conotter"), further in view of Deng et al. (CN-112215180-A, hereinafter "Deng"), and further in view of Kraft (WO-2023/217681-A1).
37. As per claim 6, Ramírez Vicente in view of Wang, further in view of Conotter, and further in view of Deng discloses: The method as claimed in claim 1, wherein the identifying the one or more intrinsic properties of the at least one of object, person, or background based on the relationship among the at least one object, person, or background and the corresponding pixel-motion information comprises:
inputting, by the electronic device, a fused feature map to an encoder of AI model;
[[learning, by the electronic device, one or more latent vectors by training the encoder of AI model to predict the physical properties of the at least one of object, person, or background identified in the]] fused feature map, [[wherein the one or more latent vectors comprise at least one of an energy, a force, a mass, a friction, or a pressure of the at least one of object, person, or background;]]
reconstructing, by the electronic device, the fused feature map by the encoder of the AI model to generate a reconstructed feature map; and
[[identifying, by the electronic device, the one or more intrinsic properties of the at least one of object, person, or background in each frame of plurality of frames based on the]] reconstructed feature map [[and the latent vectors, wherein the one or more intrinsic properties comprises at least one of floating, penetration, perpetual motion, energy level, or angular distortions.]] (Deng, [0215], “3D Convolutional Neural Networks (3D-CNNs) work by stacking multiple consecutive frames to form a cube. In other words, they combine local facial image sequences and global facial image sequences into cubes, and then use 3D convolutional kernels within the cubes to extract features. In this structure, each feature map in the convolutional layer is connected to multiple neighboring consecutive frames in the previous convolutional layer, thus capturing motion information.” and [0198], “In one optional embodiment, since multi-layer feature extraction processing is performed on the global face image sequence and the local face image sequence respectively in this application embodiment, feature fusion can be performed after each layer of feature extraction processing, and after feature fusion, the fused feature extraction result can be subjected to the next layer of feature extraction processing.” and [0200], “In another optional embodiment, after at least one layer of feature extraction processing, the feature extraction results of the global face image sequence and the feature extraction results of the local face image sequence can be fused, and the fused feature extraction results can be subjected to a lower-level feature extraction process.”)
38. Ramírez Vicente in view of Wang, further in view of Conotter, and further in view of Deng doesn't explicitly disclose but Kraft discloses: learning, by the electronic device, one or more latent vectors by training the encoder of AI model to predict the physical properties of the at least one of object, person, or background identified in the [[fused feature map,]] wherein the one or more latent vectors comprise at least one of an energy, a force, a mass, a friction, or a pressure of the at least one of object, person, or background;
identifying, by the electronic device, the one or more intrinsic properties of the at least one of object, person, or background in each frame of plurality of frames based on the [[reconstructed feature map]] and the latent vectors, wherein the one or more intrinsic properties comprises at least one of floating, penetration, perpetual motion, energy level, or angular distortions. (Kraft, page 9, lines 16-24, “Next, a trajectory forecasting or prediction model is trained from scene flow and motion trajectories as automatically generated, and the scene flow and motion trajectories are used as ground truth data or training data. … In another embodiment, inputs are mapped to a latent feature space.” and page 3, lines 13-16, “It is possible to formulate the trajectory prediction problem as a classification and refinement problem over a set of initial trajectory proposals. Such an approach has the benefit that the proposed trajectories can be filtered based on semantic information and physics-based reasoning, which makes it interpretable.” and page 4, lines 20-24, “This approach enables us to extract motion information from the video data without requiring manual annotation, which is labor-intensive and time-consuming. By automatically generating training data, we can improve the scalability and efficiency of our motion forecasting models, enabling them to be used in various applications.” and page 5, lines 21-24, “It is possible to further utilize the method to identify independently moving objects in the scene. These usually correspond to other traffic participants, such as cars and pedestrians. Instead of warping consecutive video frames during the model training we can apply a similar warping to optical flow vectors at inference time.”)
39. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Ramírez Vicente in view of Wang, further in view of Conotter, and further in view of Deng to include the disclosure of learning one or more latent vectors by training the encoder of AI model to predict the physical properties of the at least one of object, person, or background identified in the fused feature map, wherein the one or more latent vectors comprise at least one of an energy, a force, a mass, a friction, or a pressure of the at least one of object, person, or background; identifying, the one or more intrinsic properties of the at least one of object, person, or background in each frame of plurality of frames based on the reconstructed feature map and the latent vectors, wherein the one or more intrinsic properties comprises at least one of floating, penetration, perpetual motion, energy level, or angular distortions, of Kraft. The motivation for this modification could have been to train an AI model to be able to identify features from a video related to physics, such as energy, force, mass, friction, and pressure. After training the AI, this analysis can then be used to detect anomalies that are outside the bounds of natural phenomena.
40. Claim 16, which is similar in scope to claims 6 and 11, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 6.
41. Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ramírez Vicente et al. (US-2024/0281945-A1, hereinafter "Ramírez Vicente") in view of Wang et al. (CN-114565880-A, hereinafter "Wang"), further in view of Conotter et al. (NPL: “Exposing Digital Forgeries in Ballistic Motion.” IEEE Transactions on Information Forensics and Security, vol. 7, no. 1, Feb. 2012, pp. 283–296, hereinafter "Conotter"), and further in view of Ciftci et al. (US-2021/0209388-A1, hereinafter "Ciftci").
42. As per claim 8, Ramírez Vicente in view of Wang, and further in view of Conotter discloses: The method as claimed in claim 1, wherein the detecting, by the electronic device, the inconsistent motion of the at least one of object, person, or background in at least one frame for the plurality of frames of the video comprises: (see rejection for claim 1)
43. Ramírez Vicente in view of Wang, and further in view of Conotter doesn't explicitly disclose but Ciftci discloses: classifying, by the electronic device, the at least one of object, person, or background in each frame of the plurality of frames being at least one of consistent or inconsistent based on the one or more intrinsic properties of the at least one of object, person, or background; (Ciftci, [0007], “Main features of the present technology include: formulations and experimental validations of signal transformations to exploit spatial coherence and temporal consistency of biological signals for both pairwise and general authenticity classification, a generalized and interpretable deep fake detector that operates in-the-wild, a novel biological signal map construction to train neural networks for authenticity classification, and a diverse dataset of portrait videos to create a test bed for fake content detection in the wild.” and [0113], “Fakecatcher, a novel approach to detect synthetic content in portrait videos is provided, as a preventive solution for the emerging threat of deep fakes. In other words, the present technology provides a deep fake detector. Detectors blindly utilizing deep learning are not effective in catching fake content, as generative models produce formidably realistic results. Biological signals hidden in portrait videos can be used as an implicit descriptor of authenticity, because they are neither spatially nor temporally preserved in fake content. Several signal transformations are formulated and engaged for the pairwise separation problem, achieving 99.39% accuracy. Those findings were used to formulate a generalized classifier for fake content, by analyzing proposed signal transformations and corresponding feature sets.”)
authenticating, by the electronic device, the at least one of object, person, or background in each frame of the plurality of frames using an AI model to reclassify each frame of the plurality of frames being at least one of consistent or inconsistent, based on a determination each frame of the plurality of frames is determined to be consistent. (Ciftci, [0147], “Based on the experiments, a conclusion was formed that “authenticity” (i) is observed both in time and frequency domains, (ii) is highly sensitive to small changes in motion, illumination, and compression if a single signal source is used, and (iii) can be discovered from coherence and consistency of multiple biological signals.” and [0283], “FakeCatcher, a fake portrait video detector based on biological signals is presented, and experimentally validated to demonstrate that spatial coherence and temporal consistency of such signals are not well preserved in GAN-erated content. … The approach for pairwise separation and authenticity classification is evaluated, of segments and videos, on Face Forensics [1] and newly introduced Deep Fakes datasets, achieving 99.39% pairwise separation accuracy, 96% constrained video classification accuracy, and 91.07% in the wild video classification accuracy. These results also verify that FakeCatcher detects fake content with high accuracy, independent of the generator, content, resolution, and quality of the video.” and [0275], “The ROI Alignment module extracts region of interest (ROI) from each frame of a video using facial landmarks …”)
44. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Ramírez Vicente in view of Wang, and further in view of Conotter to include the disclosure of classifying the at least one of object, person, or background in each frame of the plurality of frames being at least one of consistent or inconsistent based on the one or more intrinsic properties of the at least one of object, person, or background; authenticating the at least one of object, person, or background in each frame of the plurality of frames using an AI model to reclassify each frame of plurality of frames being at least one of consistent or inconsistent, based on a determination each frame of the plurality of frames is determined to be consistent, of Ciftci. The motivation for this modification could have been to distinguish different parts of a video frame by classifying those parts as either consistent or inconsistent. By doing so, this can be used to determine if the video has AI generated or fake content or if it can be authenticated as real.
45. Claim 18 is similar in scope to claim 8 except for additional limitations that Ramírez Vicente in view of Wang, further in view of Conotter, and further in view of Ciftci discloses: authenticate the at least one identified spatial context in each frame of the plurality of frames using an AI model to reclassify each frame of the plurality of frames being at least one of consistent or inconsistent, based on a determination each frame of the plurality of frames is determined to be consistent. (Ciftci, [0147], “Based on the experiments, a conclusion was formed that “authenticity” (i) is observed both in time and frequency domains, (ii) is highly sensitive to small changes in motion, illumination, and compression if a single signal source is used, and (iii) can be discovered from coherence and consistency of multiple biological signals.” and [0283], “FakeCatcher, a fake portrait video detector based on biological signals is presented, and experimentally validated to demonstrate that spatial coherence and temporal consistency of such signals are not well preserved in GAN-erated content. … The approach for pairwise separation and authenticity classification is evaluated, of segments and videos, on Face Forensics [1] and newly introduced Deep Fakes datasets, achieving 99.39% pairwise separation accuracy, 96% constrained video classification accuracy, and 91.07% in the wild video classification accuracy. These results also verify that FakeCatcher detects fake content with high accuracy, independent of the generator, content, resolution, and quality of the video.” and [0275], “The ROI Alignment module extracts region of interest (ROI) from each frame of a video using facial landmarks …”)
The motivation for this modification is the same as claim 8.
46. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Ramírez Vicente et al. (US-2024/0281945-A1, hereinafter "Ramírez Vicente") in view of Wang et al. (CN-114565880-A, hereinafter "Wang"), further in view of Conotter et al. (NPL: “Exposing Digital Forgeries in Ballistic Motion.” IEEE Transactions on Information Forensics and Security, vol. 7, no. 1, Feb. 2012, pp. 283–296, hereinafter "Conotter"), and further in view of Pourreza et al. (US-2022/0303568-A1, hereinafter "Pourreza").
47. As per claim 10, Ramírez Vicente in view of Wang, and further in view of Conotter discloses: The method as claimed in claim 1, further comprising localizing a patch in the at least one frame based on the inconsistent motion of the at least one of object, person, or background in the at least one frame of the video, the localizing the patch comprising: (see rejection for claim 10)
48. Ramírez Vicente in view of Wang, and further in view of Conotter doesn't explicitly disclose but Pourreza discloses: identifying, by the electronic device, one or more class activation maps by backtracking in intermediate convolution layers of the CNN model which caused decision of classification based on gradient between output layer of the CNN model and the convolved feature maps from the intermediate layers of the CNN model; and (Pourreza, [0004], “The optical flow identifies how different areas of the scene move between the reference frame and the input frame. In some aspects, the encoding device can generate the optical flow using a trained neural network.” and [0164], “The first layer of the CNN 900 is the convolutional hidden layer 922a. The convolutional hidden layer 922a analyzes the image data of the input layer 920. Each node of the convolutional hidden layer 922a is connected to a region of nodes (pixels) of the input image called a receptive field. The convolutional hidden layer 922a can be considered as one or more filters (each filter corresponding to a different activation or feature map) ... Each connection between a node and a receptive field for that node learns a weight and, in some cases, an overall bias such that each node learns to analyze its particular local receptive field in the input image.”)
localizing, by the electronic device, the patch in the at least one frame which activated signal for classifying as inconsistent based on the identified one or more class activation maps. (Pourreza, [0164], “The first layer of the CNN 900 is the convolutional hidden layer 922a. The convolutional hidden layer 922a analyzes the image data of the input layer 920. Each node of the convolutional hidden layer 922a is connected to a region of nodes (pixels) of the input image called a receptive field. The convolutional hidden layer 922a can be considered as one or more filters (each filter corresponding to a different activation or feature map) ... Each connection between a node and a receptive field for that node learns a weight and, in some cases, an overall bias such that each node learns to analyze its particular local receptive field in the input image.” and [0172], "Using the example above, the input layer includes 28×28 nodes encoding the pixel intensities of the input image, the convolutional hidden layer 922a includes 3×24×24 hidden feature nodes based on application of a 5×5 local receptive field (for the filters) to three activation maps ...")
49. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Ramírez Vicente in view of Wang, and further in view of Conotter to include the disclosure of identifying one or more class activation maps by backtracking in intermediate convolution layers of the CNN model which caused decision of classification based on gradient between output layer of the CNN model and the convolved feature maps from the intermediate layers of the CNN model; and localizing, by the electronic device, the patch in the at least one frame which activated signal for classifying as inconsistent based on the identified one or more class activation maps, of Pourreza. The motivation for this modification could have been to utilize the class activation maps with the neural network model to identify regions in a video frame that have anomalies. This could be used to help classify those regions as being inconsistent.
Conclusion
50. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
51. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW CLOTHIER/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights