DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/08/2025 has been considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed
invention is not identically disclosed as set forth in section 102 of this title, if the
differences between the claimed invention and the prior art are such that the claimed
invention as a whole would have been obvious before the effective filing date of the
claimed invention to a person having ordinary skill in the art to which the claimed
invention pertains. Patentability shall not be negated by the manner in which the
invention was made.
Claims 20, 29, 31, 34, 37, and 38 are rejected under 35 U.S.C. 103 as being unpatentable over RAKHSHANFAR (US 20170084007 A1), hereinafter referenced as RAKHSHANFAR in view of POSSOS (US 20180343448 A1), hereinafter referenced as POSSOS.
Regarding claim 20, RAKHSHANFAR teaches a method for detecting a dirty spot in a video (Figs. 3-5, Paragraph [0029] – RAKHSHANFAR discloses the systems and methods described herein detect and remove artifacts due to blocking and motion blur.), comprising:
obtaining a reference frame corresponding to a current image frame in the video (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses the method comprises the following processing steps: 1) time-domain filtering on current frame using motion-compensated previous and subsequent frames. See also paragraph [0064].),
wherein the reference frame comprises a front reference frame and a back reference frame (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses the method comprises the following processing steps: 1) time-domain filtering on current frame using motion-compensated previous and subsequent frames. Paragraph [0064] – RAKHSHANFAR further discloses the first step linearly averages reference frame and motion-compensated frames from prior and following times. To provide motion-compensated frames, motion estimation along reference frame and frames inside a predefined temporal window is accomplished and then a deblocking approach is applied on motion-compensated frames to reduce possible blocking artifacts from block-based motion estimation.),
the front reference frame is an image frame before the current image frame in the video (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses the method comprises the following processing steps: 1) time-domain filtering on current frame using motion-compensated previous and subsequent frames [wherein previous frames are the front reference frame]. See also paragraph [0064].),
and the back reference frame is an image frame after the current image frame in the video (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses the method comprises the following processing steps: 1) time-domain filtering on current frame using motion-compensated previous and subsequent frames [wherein subsequent frames are the back reference frame]. See also paragraph [0064].);
filtering the current image frame (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.),
Although RAKHSHANFAR further teaches the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames [wherein motion-compensated frames are filtered frames].);
RAKHSHANFAR fails to explicitly teach and determining whether there is the dirty spot in the current image frame according to a similarity of a corresponding one of the filtered image frames to the current image frame, a similarity of a corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and a similarity of a corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame.
However, POSSOS explicitly teaches and determining whether there is the dirty spot in the current image frame according to a similarity of a corresponding one of the filtered image frames to the current image frame (Fig. 2, Paragraph [0060] – POSSOS discloses adaptive motion compensated temporal filtering system 200 may use information from past and future frames to reject noise in current YUV frame 211 (e.g., the current picture). The resulting frame or image (e.g., denoised YUV frame 212) is a denoised version of the original (e.g., current YUV frame) with details preserved. For example, in contrast to spatial denoisers, AMCTF techniques discussed herein may discriminate details and texture from actual noise to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement as an encoder will not need to spend extra bits trying to preserve noise information.),
a similarity of a corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and a similarity of a corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame (Fig. 2, Paragraph [0060] – POSSOS discloses adaptive motion compensated temporal filtering system 200 may use information from past and future frames to reject noise in current YUV frame 211 (e.g., the current picture). The resulting frame or image (e.g., denoised YUV frame 212) is a denoised version of the original (e.g., current YUV frame) with details preserved. For example, in contrast to spatial denoisers, AMCTF techniques discussed herein may discriminate details and texture from actual noise to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement as an encoder will not need to spend extra bits trying to preserve noise information.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; with the teachings of POSSOS having and determining whether there is the dirty spot in the current image frame according to a similarity of a corresponding one of the filtered image frames to the current image frame, a similarity of a corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and a similarity of a corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein having and determining whether there is the dirty spot in the current image frame according to a similarity of a corresponding one of the filtered image frames to the current image frame, a similarity of a corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and a similarity of a corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video with improved compression efficiency, since both RAKHSHANFAR and POSSOS relate to video denoising techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and POSSOS relates to video processing and, in particular, the area of filtering of video content with the goal of reducing or eliminating noise from frames of video to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and POSSOS (US 20180343448 A1), Paragraph [0001, 0060].
Regarding claim 29, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
RAKHSHANFAR further teaches wherein before filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain the filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.),
the method further comprises: cutting the current image frame into a plurality of current image blocks, cutting the front reference frame into a plurality of front reference blocks, and cutting the back reference frame into a plurality of back reference blocks (Figs. 3-5, Paragraph [0070] – RAKHSHANFAR discloses the proposed time-space filter is summarized in FIG. 3. An overview of the computations executed by the computing system is presented in Algorithm 1 below. Algorithm 1: Mixed block-pixel based noise filter i) Estimate and compensate motion vectors in 2R (preceding, and subsequent) frames. ii) Compute the motion error probability of each non- overlapped blocks of L × L using (3). iii) Find the averaging weights for each pixel via (11). iv) Average the motion-compensated frames using (2). v) Restore the destructed structures due to motion blur via (18) and (19). vi) Filter spatially residual noise using pixel-level noise variance σ.sub.8.sup.2 computed in (20).);
the filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain the filtered image frames (Figs. 3-5, Paragraph [0070] – RAKHSHANFAR discloses the proposed time-space filter is summarized in FIG. 3. An overview of the computations executed by the computing system is presented in Algorithm 1 below. Algorithm 1: Mixed block-pixel based noise filter i) Estimate and compensate motion vectors in 2R (preceding, and subsequent) frames. ii) Compute the motion error probability of each non- overlapped blocks of L × L using (3). iii) Find the averaging weights for each pixel via (11). iv) Average the motion-compensated frames using (2). v) Restore the destructed structures due to motion blur via (18) and (19). vi) Filter spatially residual noise using pixel-level noise variance σ.sub.8.sup.2 computed in (20). See also Paragraph [0030].),
Although RAKHSHANFAR further teaches for each current image block, filtering the current image block, the front reference block corresponding to the current image block and the back reference block corresponding to the current image block by using a filtering algorithm, to obtain filtered image blocks (Figs. 3-5, Paragraph [0070] – RAKHSHANFAR discloses the proposed time-space filter is summarized in FIG. 3. An overview of the computations executed by the computing system is presented in Algorithm 1 below. Algorithm 1: Mixed block-pixel based noise filter i) Estimate and compensate motion vectors in 2R (preceding, and subsequent) frames. ii) Compute the motion error probability of each non- overlapped blocks of L × L using (3). iii) Find the averaging weights for each pixel via (11). iv) Average the motion-compensated frames using (2). v) Restore the destructed structures due to motion blur via (18) and (19). vi) Filter spatially residual noise using pixel-level noise variance σ.sub.8.sup.2 computed in (20).);
RAKHSHANFAR fails to explicitly teach determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the filtered image frames to the current image frame, the similarity of the corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame, comprises: and determining whether there is the dirty spot in the current image block according to a similarity of a corresponding one of the filtered image blocks to the current image block, a similarity of a corresponding one of the filtered image blocks to the front reference block and a similarity of a corresponding one of the filtered image blocks to the back reference block.
However, POSSOS explicitly teaches determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the filtered image frames to the current image frame (Fig. 2, Paragraph [0060] – POSSOS discloses adaptive motion compensated temporal filtering system 200 may use information from past and future frames to reject noise in current YUV frame 211 (e.g., the current picture). The resulting frame or image (e.g., denoised YUV frame 212) is a denoised version of the original (e.g., current YUV frame) with details preserved. For example, in contrast to spatial denoisers, AMCTF techniques discussed herein may discriminate details and texture from actual noise to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement as an encoder will not need to spend extra bits trying to preserve noise information.),
the similarity of the corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame (Fig. 2, Paragraph [0060] – POSSOS discloses adaptive motion compensated temporal filtering system 200 may use information from past and future frames to reject noise in current YUV frame 211 (e.g., the current picture). The resulting frame or image (e.g., denoised YUV frame 212) is a denoised version of the original (e.g., current YUV frame) with details preserved. For example, in contrast to spatial denoisers, AMCTF techniques discussed herein may discriminate details and texture from actual noise to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement as an encoder will not need to spend extra bits trying to preserve noise information.), comprises:
and determining whether there is the dirty spot in the current image block according to a similarity of a corresponding one of the filtered image blocks to the current image block (Fig. 2, Paragraph [0061] – POSSOS discloses adaptive motion compensated temporal filtering system 200 may properly use only references that are correlated and thereby may use blocks that have significant information to allow noise reduction. Paragraph [0062] – POSSOS DISCLOSES block motion estimation engine 202 may provide motion estimation for adaptive motion compensated temporal filter 200. Such motion estimation provides motion vectors that describe translational transformation between a frame (e.g., YUV frame 211) and a temporal reference frame (or multiple reference frames). Paragraph [0063] – POSSOS discloses motion compensated merging module 203 may use the motion vectors provided by block motion estimation engine 202 and/or additional information to measure similarity between a current block of YUV frame 211 and the blocks given by the motion vectors (e.g., reference block(s) from reference frame(s)).),
a similarity of a corresponding one of the filtered image blocks to the front reference block and a similarity of a corresponding one of the filtered image blocks to the back reference block (Fig. 2, Paragraph [0061] – POSSOS discloses adaptive motion compensated temporal filtering system 200 may properly use only references that are correlated and thereby may use blocks that have significant information to allow noise reduction. Paragraph [0062] – POSSOS DISCLOSES block motion estimation engine 202 may provide motion estimation for adaptive motion compensated temporal filter 200. Such motion estimation provides motion vectors that describe translational transformation between a frame (e.g., YUV frame 211) and a temporal reference frame (or multiple reference frames). Paragraph [0063] – POSSOS discloses motion compensated merging module 203 may use the motion vectors provided by block motion estimation engine 202 and/or additional information to measure similarity between a current block of YUV frame 211 and the blocks given by the motion vectors (e.g., reference block(s) from reference frame(s)).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of POSSOS having determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the filtered image frames to the current image frame, the similarity of the corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame, comprises: and determining whether there is the dirty spot in the current image block according to a similarity of a corresponding one of the filtered image blocks to the current image block, a similarity of a corresponding one of the filtered image blocks to the front reference block and a similarity of a corresponding one of the filtered image blocks to the back reference block.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein having determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the filtered image frames to the current image frame, the similarity of the corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame, comprises: and determining whether there is the dirty spot in the current image block according to a similarity of a corresponding one of the filtered image blocks to the current image block, a similarity of a corresponding one of the filtered image blocks to the front reference block and a similarity of a corresponding one of the filtered image blocks to the back reference block.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video with improved compression efficiency, since both RAKHSHANFAR and POSSOS relate to video denoising techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and POSSOS relates to video processing and, in particular, the area of filtering of video content with the goal of reducing or eliminating noise from frames of video to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and POSSOS (US 20180343448 A1), Paragraph [0001, 0060].
Regarding claim 31, RAKHSHANFAR in view of POSSOS teach the method according to claim 29,
RAKHSHANFAR fails to explicitly teach wherein two current image blocks closest to each other in a preset direction partially overlap, two front reference blocks closest to each other in the preset direction partially overlap, and two back reference blocks closest to each other in the preset direction partially overlap; and the preset direction comprises a horizontal axis direction or a vertical axis direction.
However, POSSOS explicitly teaches wherein two current image blocks closest to each other in a preset direction partially overlap (Fig. 9B, Paragraph [0087] – POSSOS discloses regions 912, 915 overlap between block 921 and block 922, regions 914, 915 overlap between block 921 and block 923, and so on. For example, region 915 is provided in each of blocks 921-924.),
two front reference blocks closest to each other in the preset direction partially overlap (Fig. 9B & 11, Paragraph [0094] – POSSOS discloses a reference block (e.g., a motion compensated block) for current frame block 1127 may be determined (e.g., using a motion vector that references the reference block from current frame block 1127) and three overlapping reference blocks may also be determined. See also Paragraph [0087].),
and two back reference blocks closest to each other in the preset direction partially overlap (Fig. 9B & 11, Paragraph [0094] – POSSOS discloses a reference block (e.g., a motion compensated block) for current frame block 1127 may be determined (e.g., using a motion vector that references the reference block from current frame block 1127) and three overlapping reference blocks may also be determined. See also Paragraph [0087].);
and the preset direction comprises a horizontal axis direction or a vertical axis direction (Fig. 9B, Paragraph [0087] – POSSOS discloses each block of blocks 921-924 may begin at half the width and/or height of the previous, neighboring block. For example, for a 16×16 setup, the first block (e.g., block 921) is located at (0,0), the block to the right of the first block (e.g., block 922) is located at (8,0), the block below the first block (e.g., block 923) is located at (0,8), and the block to the right of and below the first block (e.g., block 924) is located at (8,8).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of POSSOS having wherein two current image blocks closest to each other in a preset direction partially overlap, two front reference blocks closest to each other in the preset direction partially overlap, and two back reference blocks closest to each other in the preset direction partially overlap; and the preset direction comprises a horizontal axis direction or a vertical axis direction.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein two current image blocks closest to each other in a preset direction partially overlap, two front reference blocks closest to each other in the preset direction partially overlap, and two back reference blocks closest to each other in the preset direction partially overlap; and the preset direction comprises a horizontal axis direction or a vertical axis direction.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video with improved compression efficiency, since both RAKHSHANFAR and POSSOS relate to video denoising techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and POSSOS relates to video processing and, in particular, the area of filtering of video content with the goal of reducing or eliminating noise from frames of video to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and POSSOS (US 20180343448 A1), Paragraph [0001, 0060].
Regarding claim 34, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
RAKHSHANFAR further teaches wherein the filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain the filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames. See also Paragraph [0070], Table 1.), comprises:
filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame by using a filtering algorithm to obtain the filtered image frames (Figs. 3-5, Paragraph [0070] – RAKHSHANFAR discloses the proposed time-space filter is summarized in FIG. 3. An overview of the computations executed by the computing system is presented in Algorithm 1 below. Algorithm 1: Mixed block-pixel based noise filter i) Estimate and compensate motion vectors in 2R (preceding, and subsequent) frames. ii) Compute the motion error probability of each non- overlapped blocks of L × L using (3). iii) Find the averaging weights for each pixel via (11). iv) Average the motion-compensated frames using (2). v) Restore the destructed structures due to motion blur via (18) and (19). vi) Filter spatially residual noise using pixel-level noise variance σ.sub.8.sup.2 computed in (20).);
wherein the filtering algorithm comprises a time median filtering algorithm (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.),
and the time median filtering algorithm is configured for obtaining a median value of pixel values at the same position in a plurality frames of images (Figs. 3-5, Paragraph [0070] – RAKHSHANFAR discloses the proposed time-space filter is summarized in FIG. 3. An overview of the computations executed by the computing system is presented in Algorithm 1 below. Algorithm 1: Mixed block-pixel based noise filter i) Estimate and compensate motion vectors in 2R (preceding, and subsequent) frames. ii) Compute the motion error probability of each non- overlapped blocks of L × L using (3). iii) Find the averaging weights for each pixel via (11). iv) Average the motion-compensated frames using (2). v) Restore the destructed structures due to motion blur via (18) and (19). vi) Filter spatially residual noise using pixel-level noise variance σ.sub.8.sup.2 computed in (20).); or the filtering algorithm comprises a time mode filtering algorithm, and the time mode filtering algorithm is configured for obtaining same pixel values of a largest quantity from pixel values at the same position in a plurality frames of images.
Regarding claim 37, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
RAKHSHANFAR further teaches an electronic device (Fig. 2, Paragraph [0062] – RAKHSHANFAR discloses computing system or device 101 is a consumer electronics device with a body that houses components, such as a processor, memory and a camera device.),
comprising a processor and a memory (Fig. 2, Paragraph [0059] – RAKHSHANFAR discloses computing system or device 101 includes one or more processor devices 102 configured to execute the computations or instructions described herein. The computing system or device also includes memory 103 that stores the instructions and the image data.),
wherein the memory is configured to store programs executable by the processor (Fig. 2, Paragraph [0059] – RAKHSHANFAR discloses computing system or device 101 includes one or more processor devices 102 configured to execute the computations or instructions described herein. The computing system or device also includes memory 103 that stores the instructions and the image data.),
and the processor is configured to read the programs in the memory and perform steps of any one of the methods according to claim 20 (Fig. 2, Paragraph [0059] – RAKHSHANFAR discloses computing system or device 101 includes one or more processor devices 102 configured to execute the computations or instructions described herein. The computing system or device also includes memory 103 that stores the instructions and the image data.).
Regarding claim 38, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
RAKHSHANFAR further teaches a non-transitory computer storage medium, in which computer programs are stored (Fig. 2, Paragraph [0069] – RAKHSHANFAR discloses computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data, except transitory propagating signals per se. Any such computer storage media may be part of the computing system 101, or accessible or connectable thereto.),
wherein steps of any one of the methods according to claim 20 are implemented when the computer programs are executed by a processor (Fig. 2, Paragraph [0069] – RAKHSHANFAR discloses any module or component exemplified herein that executes instructions or operations may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape.).
Claims 21-24, 30, 32, 35, and 36 are rejected under 35 U.S.C. 103 as being unpatentable over RAKHSHANFAR (US 20170084007 A1), hereinafter referenced as RAKHSHANFAR in view of POSSOS (US 20180343448 A1), hereinafter referenced as POSSOS in further view of CARSON (US 8433143 B1), hereinafter referenced as CARSON.
Regarding claim 21, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
RAKHSHANFAR further teaches wherein the front reference frame and the current image frame are spaced by n image frames (Fig. 2, Paragraph [0064] – RAKHSHANFAR discloses to provide motion-compensated frames, motion estimation along reference frame and frames inside a predefined temporal window is accomplished and then a deblocking approach is applied on motion-compensated frames to reduce possible blocking artifacts from block-based motion estimation. Paragraph [0144] – RAKHSHANFAR further discloses temporal window R means that the computing system processed R previously and R for subsequent frames. In the example experiment, the value R=5 is used since it gives best quality-speed compromise; however, 0≦R≦5 can be selected depending on the factors: application, processing pipeline delay, and hardware limits.),
and n is an integer greater than or equal to 0 (Fig. 2, Paragraph [0144] – RAKHSHANFAR further discloses temporal window R means that the computing system processed R previously and R for subsequent frames. In the example experiment, the value R=5 is used since it gives best quality-speed compromise; however, 0≦R≦5 can be selected depending on the factors: application, processing pipeline delay, and hardware limits.);
and/or, the back reference frame and the current image frame are spaced by m image frames (Fig. 2, Paragraph [0064] – RAKHSHANFAR discloses to provide motion-compensated frames, motion estimation along reference frame and frames inside a predefined temporal window is accomplished and then a deblocking approach is applied on motion-compensated frames to reduce possible blocking artifacts from block-based motion estimation. Paragraph [0144] – RAKHSHANFAR further discloses temporal window R means that the computing system processed R previously and R for subsequent frames. In the example experiment, the value R=5 is used since it gives best quality-speed compromise; however, 0≦R≦5 can be selected depending on the factors: application, processing pipeline delay, and hardware limits.),
and m is an integer greater than or equal to 0 (Fig. 2, Paragraph [0144] – RAKHSHANFAR further discloses temporal window R means that the computing system processed R previously and R for subsequent frames. In the example experiment, the value R=5 is used since it gives best quality-speed compromise; however, 0≦R≦5 can be selected depending on the factors: application, processing pipeline delay, and hardware limits.).
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein n is determined based on a quantity of frames in which dirty spots occur in successive image frames, wherein m is determined based on a quantity of frames in which dirty spots occur in successive image frames,
However, CARSON explicitly teaches wherein n is determined based on a quantity of frames in which dirty spots occur in successive image frames (Fig. 1, Col. 7, Lines [56-61] – CARSON discloses frames F1/F3, F4/F6 and F7/F9 can be compared. This list is then analyzed to determine if there are spikes (changes above a threshold) in values. Upon finding three or more consecutive spikes in this list the associated frame can be tagged as potentially containing an artifact.),
wherein m is determined based on a quantity of frames in which dirty spots occur in successive image frames (Fig. 1, Col. 7, Lines [56-61] – CARSON discloses frames F1/F3, F4/F6 and F7/F9 can be compared. This list is then analyzed to determine if there are spikes (changes above a threshold) in values. Upon finding three or more consecutive spikes in this list the associated frame can be tagged as potentially containing an artifact.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of CARSON having wherein n is determined based on a quantity of frames in which dirty spots occur in successive image frames, wherein m is determined based on a quantity of frames in which dirty spots occur in successive image frames.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein having wherein n is determined based on a quantity of frames in which dirty spots occur in successive image frames, wherein m is determined based on a quantity of frames in which dirty spots occur in successive image frames.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Regarding claim 22, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
Although RAKHSHANFAR further teaches filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.)
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein the determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the image frames to the current image frame, the similarity of the corresponding one of the image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the image frames to the back reference frame corresponding to the current image frame, comprises: determining a maximum similarity from the similarity of the corresponding one of the image frames to the front reference frame and the similarity of the corresponding one of the image frames to the back reference frame; and determining whether there is the dirty spot in the current image frame according to a comparison result of the maximum similarity and the similarity of the corresponding one of the image frames to the current image frame.
However, CARSON explicitly teaches wherein the determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the image frames to the current image frame, the similarity of the corresponding one of the image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the image frames to the back reference frame corresponding to the current image frame (Fig. 13, Col. 10, Lines [54-64] – CARSON discloses analysis of FIG. 13 can be carried out in addition to, or in lieu of, the analysis of FIG. 12 and is carried out in a similar manner except on a window-by-window basis. For example, the system 130 can successively evaluate window pairs such as A1/C1, A4/C4, A7/C7 etc., and repeat these analyses with corresponding windows in frames A/B and B/C, with a view toward evaluating relative similarity between these window pairs. Such analysis can be used to detect scene changes at a greater resolution than in FIG. 12, as well as detecting the locations of artifacts.), comprises:
determining a maximum similarity from the similarity of the corresponding one of the image frames to the front reference frame and the similarity of the corresponding one of the image frames to the back reference frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.);
and determining whether there is the dirty spot in the current image frame according to a comparison result of the maximum similarity and the similarity of the corresponding one of the image frames to the current image frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of CARSON having wherein the determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the image frames to the current image frame, the similarity of the corresponding one of the image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the image frames to the back reference frame corresponding to the current image frame, comprises: determining a maximum similarity from the similarity of the corresponding one of the image frames to the front reference frame and the similarity of the corresponding one of the image frames to the back reference frame; and determining whether there is the dirty spot in the current image frame according to a comparison result of the maximum similarity and the similarity of the corresponding one of the image frames to the current image frame.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein the determining whether there is the dirty spot in the current image frame according to the similarity of the corresponding one of the filtered image frames to the current image frame, the similarity of the corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame, comprises: determining a maximum similarity from the similarity of the corresponding one of the filtered image frames to the front reference frame and the similarity of the corresponding one of the filtered image frames to the back reference frame; and determining whether there is the dirty spot in the current image frame according to a comparison result of the maximum similarity and the similarity of the corresponding one of the filtered image frames to the current image frame.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Regarding claim 23, RAKHSHANFAR and POSSOS in view of CARSON teach the method according to claim 22,
Although RAKHSHANFAR further teaches filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.)
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein the determining whether there is the dirty spot in the current image frame according to the comparison result of the maximum similarity and the similarity of the corresponding one of the image frames to the current image frame, comprises: in response to the similarity of the corresponding one of the image frames to the current image frame being less than the maximum similarity, determining there is the dirty spot in the current image frame; or in response to the similarity of the corresponding one of the image frames to the current image frame being greater than or equal to the maximum similarity, determining there is no dirty spot in the current image frame.
However, CARSON explicitly teaches wherein the determining whether there is the dirty spot in the current image frame according to the comparison result of the maximum similarity and the similarity of the corresponding one of the image frames to the current image frame, comprises: in response to the similarity of the corresponding one of the image frames to the current image frame being less than the maximum similarity, determining there is the dirty spot in the current image frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.);
or in response to the similarity of the corresponding one of the image frames to the current image frame being greater than or equal to the maximum similarity, determining there is no dirty spot in the current image frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of CARSON having wherein the determining whether there is the dirty spot in the current image frame according to the comparison result of the maximum similarity and the similarity of the corresponding one of the image frames to the current image frame, comprises: in response to the similarity of the corresponding one of the image frames to the current image frame being less than the maximum similarity, determining there is the dirty spot in the current image frame; or in response to the similarity of the corresponding one of the image frames to the current image frame being greater than or equal to the maximum similarity, determining there is no dirty spot in the current image frame.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein the determining whether there is the dirty spot in the current image frame according to the comparison result of the maximum similarity and the similarity of the corresponding one of the filtered image frames to the current image frame, comprises: in response to the similarity of the corresponding one of the filtered image frames to the current image frame being less than the maximum similarity, determining there is the dirty spot in the current image frame; or in response to the similarity of the corresponding one of the filtered image frames to the current image frame being greater than or equal to the maximum similarity, determining there is no dirty spot in the current image frame.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Regarding claim 24, RAKHSHANFAR and POSSOS in view of CARSON teach the method according to claim 22,
Although RAKHSHANFAR further teaches filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.)
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the image frames to each front reference frame and the similarity of the corresponding one of the image frames to the back reference frame; or in response to the back reference frame being a plurality of continuous image frames after the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the image frames to the front reference frame and the similarity of the corresponding one of the image frames to each back reference frame; or in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video, and the back reference frame being a plurality of continuous image frames after the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the image frames to each front reference frame and the similarity of the corresponding one of the image frames to each back reference frame.
However, CARSON explicitly teaches wherein in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video (Fig. 6, Col. 6, Lines [42-48] – CARSON discloses in some embodiments, groups of three frames are successively compared in a given scene. With reference to FIG. 6, this may include comparing frames F1 and F3, F1 and F2, and F2 and F3. If non-immediately successive frames F1 and F3 are closely related, then it follows that the comparisons F1/F2 and F2/F3 should also provide indications of closely related frames.),
determining the maximum similarity from the similarity of the corresponding one of the image frames to each front reference frame and the similarity of the corresponding one of the image frames to the back reference frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.);
or in response to the back reference frame being a plurality of continuous image frames after the current image frame in the video (Fig. 6, Col. 6, Lines [42-48] – CARSON discloses in some embodiments, groups of three frames are successively compared in a given scene. With reference to FIG. 6, this may include comparing frames F1 and F3, F1 and F2, and F2 and F3. If non-immediately successive frames F1 and F3 are closely related, then it follows that the comparisons F1/F2 and F2/F3 should also provide indications of closely related frames.),
determining the maximum similarity from the similarity of the corresponding one of the image frames to the front reference frame and the similarity of the corresponding one of the image frames to each back reference frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.);
or in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video (Fig. 6, Col. 6, Lines [42-48] – CARSON discloses in some embodiments, groups of three frames are successively compared in a given scene. With reference to FIG. 6, this may include comparing frames F1 and F3, F1 and F2, and F2 and F3. If non-immediately successive frames F1 and F3 are closely related, then it follows that the comparisons F1/F2 and F2/F3 should also provide indications of closely related frames.),
and the back reference frame being a plurality of continuous image frames after the current image frame in the video (Fig. 6, Col. 6, Lines [42-48] – CARSON discloses in some embodiments, groups of three frames are successively compared in a given scene. With reference to FIG. 6, this may include comparing frames F1 and F3, F1 and F2, and F2 and F3. If non-immediately successive frames F1 and F3 are closely related, then it follows that the comparisons F1/F2 and F2/F3 should also provide indications of closely related frames.),
determining the maximum similarity from the similarity of the corresponding one of the image frames to each front reference frame and the similarity of the corresponding one of the image frames to each back reference frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of CARSON having wherein in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the image frames to each front reference frame and the similarity of the corresponding one of the image frames to the back reference frame; or in response to the back reference frame being a plurality of continuous image frames after the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the image frames to the front reference frame and the similarity of the corresponding one of the image frames to each back reference frame; or in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video, and the back reference frame being a plurality of continuous image frames after the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the image frames to each front reference frame and the similarity of the corresponding one of the image frames to each back reference frame.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the filtered image frames to each front reference frame and the similarity of the corresponding one of the filtered image frames to the back reference frame; or in response to the back reference frame being a plurality of continuous image frames after the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the filtered image frames to the front reference frame and the similarity of the corresponding one of the filtered image frames to each back reference frame; or in response to the front reference frame being a plurality of continuous image frames before the current image frame in the video, and the back reference frame being a plurality of continuous image frames after the current image frame in the video, determining the maximum similarity from the similarity of the corresponding one of the filtered image frames to each front reference frame and the similarity of the corresponding one of the filtered image frames to each back reference frame.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Regarding claim 30, RAKHSHANFAR in view of POSSOS teach the method according to claim 29,
Although RAKHSHANFAR further teaches filtered image blocks (Figs. 3-5, Paragraph [0070] – RAKHSHANFAR discloses the proposed time-space filter is summarized in FIG. 3. An overview of the computations executed by the computing system is presented in Algorithm 1 below. Algorithm 1: Mixed block-pixel based noise filter i) Estimate and compensate motion vectors in 2R (preceding, and subsequent) frames. ii) Compute the motion error probability of each non- overlapped blocks of L × L using (3). iii) Find the averaging weights for each pixel via (11). iv) Average the motion-compensated frames using (2). v) Restore the destructed structures due to motion blur via (18) and (19). vi) Filter spatially residual noise using pixel-level noise variance σ.sub.8.sup.2 computed in (20).)
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein the determining whether there is the dirty spot in the current image block according to the similarity of the corresponding one of the image blocks to the current image block, the similarity of the corresponding one of the image blocks to the front reference block and the similarity of the corresponding one of the image blocks to the back reference block, comprises: determining a maximum similarity from the similarity of the corresponding one of the image blocks to the front reference block and the similarity of the corresponding one of the image blocks to the back reference block; and determining whether there is the dirty spot in the current image block according to a comparison result of the maximum similarity and the similarity of the corresponding one of the image blocks to the current image block.
However, CARSON explicitly teaches wherein the determining whether there is the dirty spot in the current image block according to the similarity of the corresponding one of the image blocks to the current image block, the similarity of the corresponding one of the image blocks to the front reference block and the similarity of the corresponding one of the image blocks to the back reference block (Fig. 13, Col. 10, Lines [54-64] – CARSON discloses analysis of FIG. 13 can be carried out in addition to, or in lieu of, the analysis of FIG. 12 and is carried out in a similar manner except on a window-by-window basis. For example, the system 130 can successively evaluate window pairs such as A1/C1, A4/C4, A7/C7 etc., and repeat these analyses with corresponding windows in frames A/B and B/C, with a view toward evaluating relative similarity between these window pairs. Such analysis can be used to detect scene changes at a greater resolution than in FIG. 12, as well as detecting the locations of artifacts.), comprises:
determining a maximum similarity from the similarity of the corresponding one of the image blocks to the front reference block and the similarity of the corresponding one of the image blocks to the back reference block (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.);
and determining whether there is the dirty spot in the current image block according to a comparison result of the maximum similarity and the similarity of the corresponding one of the image blocks to the current image block (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of CARSON having wherein the determining whether there is the dirty spot in the current image block according to the similarity of the corresponding one of the image blocks to the current image block, the similarity of the corresponding one of the image blocks to the front reference block and the similarity of the corresponding one of the image blocks to the back reference block, comprises: determining a maximum similarity from the similarity of the corresponding one of the image blocks to the front reference block and the similarity of the corresponding one of the image blocks to the back reference block; and determining whether there is the dirty spot in the current image block according to a comparison result of the maximum similarity and the similarity of the corresponding one of the image blocks to the current image block.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein the determining whether there is the dirty spot in the current image block according to the similarity of the corresponding one of the filtered image blocks to the current image block, the similarity of the corresponding one of the filtered image blocks to the front reference block and the similarity of the corresponding one of the filtered image blocks to the back reference block, comprises: determining a maximum similarity from the similarity of the corresponding one of the filtered image blocks to the front reference block and the similarity of the corresponding one of the filtered image blocks to the back reference block; and determining whether there is the dirty spot in the current image block according to a comparison result of the maximum similarity and the similarity of the corresponding one of the filtered image blocks to the current image block.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Regarding claim 32, RAKHSHANFAR in view of POSSOS teach the method according to claim 29,
RAKHSHANFAR fails to explicitly teach wherein the preset step size comprises a horizontal step size in a horizontal direction and a vertical step size in a vertical direction, the horizontal step size is less than or equal to a width of the preset window, and the vertical step size is less than or equal to a height of the preset window.
However, POSSOS explicitly teaches wherein the preset step size comprises a horizontal step size in a horizontal direction and a vertical step size in a vertical direction (Fig. 9B, Paragraph [0087] – POSSOS discloses each block of blocks 921-924 may begin at half the width and/or height of the previous, neighboring block. For example, for a 16×16 setup, the first block (e.g., block 921) is located at (0,0), the block to the right of the first block (e.g., block 922) is located at (8,0), the block below the first block (e.g., block 923) is located at (0,8), and the block to the right of and below the first block (e.g., block 924) is located at (8,8).),
the horizontal step size is less than or equal to a width of the preset window (Fig. 9B, Paragraph [0087] – POSSOS discloses each block of blocks 921-924 may begin at half the width and/or height of the previous, neighboring block. For example, for a 16×16 setup, the first block (e.g., block 921) is located at (0,0), the block to the right of the first block (e.g., block 922) is located at (8,0), the block below the first block (e.g., block 923) is located at (0,8), and the block to the right of and below the first block (e.g., block 924) is located at (8,8). Furthermore, the next block to the right (not shown) is located at (16,0), and likewise for the next block row.),
and the vertical step size is less than or equal to a height of the preset window (Fig. 9B, Paragraph [0087] – POSSOS discloses each block of blocks 921-924 may begin at half the width and/or height of the previous, neighboring block. For example, for a 16×16 setup, the first block (e.g., block 921) is located at (0,0), the block to the right of the first block (e.g., block 922) is located at (8,0), the block below the first block (e.g., block 923) is located at (0,8), and the block to the right of and below the first block (e.g., block 924) is located at (8,8). Furthermore, the next block to the right (not shown) is located at (16,0), and likewise for the next block row.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of POSSOS having wherein the preset step size comprises a horizontal step size in a horizontal direction and a vertical step size in a vertical direction, the horizontal step size is less than or equal to a width of the preset window, and the vertical step size is less than or equal to a height of the preset window.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein the preset step size comprises a horizontal step size in a horizontal direction and a vertical step size in a vertical direction, the horizontal step size is less than or equal to a width of the preset window, and the vertical step size is less than or equal to a height of the preset window.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video with improved compression efficiency, since both RAKHSHANFAR and POSSOS relate to video denoising techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and POSSOS relates to video processing and, in particular, the area of filtering of video content with the goal of reducing or eliminating noise from frames of video to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and POSSOS (US 20180343448 A1), Paragraph [0001, 0060].
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein the cutting the current image frame into the plurality of current image blocks, cutting the front reference frame into the plurality of front reference blocks, and cutting the back reference frame into the plurality of back reference blocks, comprises: traversing the current image frame, the back reference frame and the back reference frame respectively by using a preset window according to a preset step size, and cutting out the plurality of current image blocks of a size same as a size of the preset window, the plurality of front reference blocks of a size same as the size of the preset window and the plurality of back reference blocks of a size same as the size of the preset window;
However, CARSON explicitly teaches wherein the cutting the current image frame into the plurality of current image blocks, cutting the front reference frame into the plurality of front reference blocks, and cutting the back reference frame into the plurality of back reference blocks (Fig. 12-13, Col. 10, Lines [52-61] – CARSON discloses FIG. 12 illustrates windowed frame comparison analysis of the frames 200 (A-D) in FIG. 12. Each of the frames A-D are divided into portions, or windows 202. The analysis of FIG. 13 can be carried out in addition to, or in lieu of, the analysis of FIG. 12 and is carried out in a similar manner except on a window-by-window basis. For example, the system 130 can successively evaluate window pairs such as A1/C1, A4/C4, A7/C7 etc., and repeat these analyses with corresponding windows in frames A/B and B/C, with a view toward evaluating relative similarity between these window pairs.), comprises:
traversing the current image frame, the back reference frame and the back reference frame respectively by using a preset window according to a preset step size (Fig. 12-13, Col. 10, Lines [52-61] – CARSON discloses FIG. 12 illustrates windowed frame comparison analysis of the frames 200 (A-D) in FIG. 12. Each of the frames A-D are divided into portions, or windows 202. The analysis of FIG. 13 can be carried out in addition to, or in lieu of, the analysis of FIG. 12 and is carried out in a similar manner except on a window-by-window basis. For example, the system 130 can successively evaluate window pairs such as A1/C1, A4/C4, A7/C7 etc., and repeat these analyses with corresponding windows in frames A/B and B/C, with a view toward evaluating relative similarity between these window pairs.),
and cutting out the plurality of current image blocks of a size same as a size of the preset window (Fig. 12-13, Col. 10, Lines [52-61] – CARSON discloses FIG. 12 illustrates windowed frame comparison analysis of the frames 200 (A-D) in FIG. 12. Each of the frames A-D are divided into portions, or windows 202. The analysis of FIG. 13 can be carried out in addition to, or in lieu of, the analysis of FIG. 12 and is carried out in a similar manner except on a window-by-window basis. For example, the system 130 can successively evaluate window pairs such as A1/C1, A4/C4, A7/C7 etc., and repeat these analyses with corresponding windows in frames A/B and B/C, with a view toward evaluating relative similarity between these window pairs.),
the plurality of front reference blocks of a size same as the size of the preset window and the plurality of back reference blocks of a size same as the size of the preset window (Fig. 12-13, Col. 10, Lines [52-61] – CARSON discloses FIG. 12 illustrates windowed frame comparison analysis of the frames 200 (A-D) in FIG. 12. Each of the frames A-D are divided into portions, or windows 202. The analysis of FIG. 13 can be carried out in addition to, or in lieu of, the analysis of FIG. 12 and is carried out in a similar manner except on a window-by-window basis. For example, the system 130 can successively evaluate window pairs such as A1/C1, A4/C4, A7/C7 etc., and repeat these analyses with corresponding windows in frames A/B and B/C, with a view toward evaluating relative similarity between these window pairs.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of CARSON having wherein the cutting the current image frame into the plurality of current image blocks, cutting the front reference frame into the plurality of front reference blocks, and cutting the back reference frame into the plurality of back reference blocks, comprises: traversing the current image frame, the back reference frame and the back reference frame respectively by using a preset window according to a preset step size, and cutting out the plurality of current image blocks of a size same as a size of the preset window, the plurality of front reference blocks of a size same as the size of the preset window and the plurality of back reference blocks of a size same as the size of the preset window.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein the cutting the current image frame into the plurality of current image blocks, cutting the front reference frame into the plurality of front reference blocks, and cutting the back reference frame into the plurality of back reference blocks, comprises: traversing the current image frame, the back reference frame and the back reference frame respectively by using a preset window according to a preset step size, and cutting out the plurality of current image blocks of a size same as a size of the preset window, the plurality of front reference blocks of a size same as the size of the preset window and the plurality of back reference blocks of a size same as the size of the preset window.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Regarding claim 35, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
RAKHSHANFAR fails to explicitly teach wherein the similarity is configured for representing a correlation between pixels of different image frames, or a correlation between pixels of image blocks at a same position in different image frames;
However, POSSOS explicitly teaches wherein the similarity is configured for representing a correlation between pixels of different image frames, or a correlation between pixels of image blocks at a same position in different image frames (Fig. 2, Paragraph [0063] – POSSOS discloses motion compensated merging module 203 may use the motion vectors provided by block motion estimation engine 202 and/or additional information to measure similarity between a current block of YUV frame 211 and the blocks given by the motion vectors (e.g., reference block(s) from reference frame(s)). Based on the similarity and/or additional information, motion compensated merging module 203 may then determine a selected denoising technique for the current block. See also Paragraph [0122].);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of POSSOS having wherein the similarity is configured for representing a correlation between pixels of different image frames, or a correlation between pixels of image blocks at a same position in different image frames.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein the similarity is configured for representing a correlation between pixels of different image frames, or a correlation between pixels of image blocks at a same position in different image frames.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video with improved compression efficiency, since both RAKHSHANFAR and POSSOS relate to video denoising techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and POSSOS relates to video processing and, in particular, the area of filtering of video content with the goal of reducing or eliminating noise from frames of video to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and POSSOS (US 20180343448 A1), Paragraph [0001, 0060].
RAKHSHANFAR in view of POSSOS fail to explicitly teach the similarity comprises a structural similarity and/or an average structural similarity.
However, CARSON explicitly teaches the similarity comprises a structural similarity and/or an average structural similarity (Fig. 11, Col. 6, Lines [29-33] – CARSON discloses a variety of methods are known in the art to measure the degree of structural similarity between frames. These methods may include PSNR (peak signal-to-noise ratio), SSIM (structural similarity index measurement), JND (just noticeable difference), etc.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of CARSON having the similarity comprises a structural similarity and/or an average structural similarity.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein having the similarity comprises a structural similarity and/or an average structural similarity.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Regarding claim 36, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
RAKHSHANFAR further teaches a device for detecting a dirty spot in a video (Fig. 2, Paragraph [0059] – RAKHSHANFAR discloses a computing system is configured to perform the methods described herein. As shown in FIG. 2, an example computing system or device 101 includes one or more processor devices 102 configured to execute the computations or instructions described herein.),
comprising a processor (Fig. 2, #102 called processor, Paragraph [0059]) and a memory (Fig. 2, #103 called memory, Paragraph [0059]),
wherein the memory is configured to store programs executable by the processor (Fig. 2, Paragraph [0059] – RAKHSHANFAR discloses the computing system or device also includes memory 103 that stores the instructions and the image data.),
and the processor is configured to read the programs in the memory and perform following steps (Fig. 2, Paragraph [0059] – RAKHSHANFAR discloses one or more processor devices 102 configured to execute the computations or instructions described herein.):
Although RAKHSHANFAR further teaches wherein the time parameter represents a quantity of frames between the current image frame and the reference frame of the current image frame (Fig. 2, Paragraph [0064] – RAKHSHANFAR discloses to provide motion-compensated frames, motion estimation along reference frame and frames inside a predefined temporal window is accomplished and then a deblocking approach is applied on motion-compensated frames to reduce possible blocking artifacts from block-based motion estimation. Paragraph [0144] – RAKHSHANFAR further discloses temporal window R means that the computing system processed R previously and R for subsequent frames. In the example experiment, the value R=5 is used since it gives best quality-speed compromise; however, 0≦R≦5 can be selected depending on the factors: application, processing pipeline delay, and hardware limits.);
RAKHSHANFAR fails to explicitly teach in response to a user's input operation of at least one of a time parameter, a similarity parameter, a similarity threshold, a preset window size, or a preset step size, and detecting the dirty spot in an image frame in the video according to parameters corresponding to the input operation based on any of the methods according to claim 20;
However, POSSOS explicitly teaches in response to a user's input operation of at least one of a time parameter, a similarity parameter, a similarity threshold (Fig. 28, Paragraph [0157] – POSSOS discloses techniques discussed herein may provide adaptive motion compensated temporal filtering (AMCTF). In some embodiments, an AMCTF system may receive video content to be filtered and a filtering strength that is externally provided (e.g., selected by user or determined by a video noise measuring system or the like). For example, the AMCTF system may also take as input supplementary parameters (e.g., overlap motion estimation or not, number of references, subpel accuracy or not, and deblocking or not) that may provide a tradeoff between quality that can be afforded and speed performance that is desirable.), a preset window size, or a preset step size,
and detecting the dirty spot in an image frame in the video according to parameters corresponding to the input operation based on any of the methods according to claim 20 (Fig. 2, Paragraph [0060] – POSSOS discloses adaptive motion compensated temporal filtering system 200 may use information from past and future frames to reject noise in current YUV frame 211 (e.g., the current picture). The resulting frame or image (e.g., denoised YUV frame 212) is a denoised version of the original (e.g., current YUV frame) with details preserved.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of POSSOS having in response to a user's input operation of at least one of a time parameter, a similarity parameter, a similarity threshold, a preset window size, or a preset step size, and detecting the dirty spot in an image frame in the video according to parameters corresponding to the input operation based on any of the methods according to claim 20;
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein having in response to a user's input operation of at least one of a time parameter, a similarity parameter, a similarity threshold, a preset window size, or a preset step size, and detecting the dirty spot in an image frame in the video according to parameters corresponding to the input operation based on any of the methods according to claim 20;
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video with improved compression efficiency, since both RAKHSHANFAR and POSSOS relate to video denoising techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and POSSOS relates to video processing and, in particular, the area of filtering of video content with the goal of reducing or eliminating noise from frames of video to provide enhanced visual perception (e.g., a cleaner picture or frame) and compression efficiency improvement. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and POSSOS (US 20180343448 A1), Paragraph [0001, 0060].
RAKHSHANFAR in view of POSSOS fail to explicitly teach the similarity parameter represents a range of a local area that affects a structural similarity of pixels at the same position in two image frames when calculating an average structural similarity by using the structural similarity of pixels at the same position in two image frames.
However, CARSON explicitly teaches the similarity parameter represents a range of a local area that affects a structural similarity of pixels at the same position in two image frames when calculating an average structural similarity by using the structural similarity of pixels at the same position in two image frames (Fig. 11, Col. 3, Lines [15-21] – CARSON discloses a variety of similarity measurements can be generated to facilitate the inter-frame comparisons, including structural similarity (SSIM) measurement values, mean square error (MSE) measurement values, and/or peak signal to noise ratio (PSNR) measurement values. Difference values between successive pairs of similarity measurement values can further be monitored to detect the artifacts. Col. 8, Lines [4-7] – CARSON discloses the similarity index values (similarity measures) can be obtained in a variety of ways. In some embodiments, pixel values (e.g., luma values Y, etc.) can be used in the comparisons.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, filtering the current image frame, the front reference frame corresponding to the current image frame and the back reference frame corresponding to the current image frame to obtain filtered image frames; and determining whether there is the dirty spot in the current image frame with the teachings of CARSON having the similarity parameter represents a range of a local area that affects a structural similarity of pixels at the same position in two image frames when calculating an average structural similarity by using the structural similarity of pixels at the same position in two image frames.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein having the similarity parameter represents a range of a local area that affects a structural similarity of pixels at the same position in two image frames when calculating an average structural similarity by using the structural similarity of pixels at the same position in two image frames.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Claims 25 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over RAKHSHANFAR (US 20170084007 A1), hereinafter referenced as RAKHSHANFAR in view of POSSOS (US 20180343448 A1), hereinafter referenced as POSSOS in further view of FAN (US 20060115178 A1), hereinafter referenced as FAN.
Regarding claim 25, RAKHSHANFAR in view of POSSOS teach the method according to claim 20,
Although RAKHSHANFAR further teaches filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.)
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein in response to there being the dirty spot in the current image frame, the method further comprises: obtaining a binary image according to a comparison result of the similarity of the corresponding one of the filtered image frames to the current image frame and a similarity threshold, wherein the binary image is configured for representing a mask image of the dirty spot in the current image frame; and repairing the dirty spot in the current image frame according to the binary image and a reference frame corresponding to the current image frame.
However, FAN explicitly teaches wherein in response to there being the dirty spot in the current image frame (Fig. 4, Paragraph [0038] – FAN discloses verifications unit 418 determines that the candidate artifact is most likely an artifact caused by dust or scratches, for example, and stores information that identifies the region that encompasses the artifact region, e.g., region 602.), the method further comprises:
obtaining a binary image according to a comparison result of the similarity of the corresponding one of the filtered image frames to the current image frame and a similarity threshold (Fig. 4-5, Paragraph [0039] – FAN discloses after one or more artifacts are detected as just described, processing system 100 executes artifact removal module 114 to perform block matching with a previous frame (n-1) 402 and a next frame (n+1) 402 as indicated in a block 512. Artifact removal module 114 creates a binary mask of frame (n) 402 to indicate locations of artifact regions.),
wherein the binary image is configured for representing a mask image of the dirty spot in the current image frame (Fig. 4-5, Paragraph [0039] – FAN discloses artifact removal module 114 creates a binary mask of frame (n) 402 to indicate locations of artifact regions.);
and repairing the dirty spot in the current image frame according to the binary image and a reference frame corresponding to the current image frame (Figs. 4-5, Paragraph [0040] – FAN discloses processing system 100 executes artifact removal module 114 to replace each artifact region in frame (n) 402 with a corresponding block from either previous frame (n-1) 402 or next frame (n+1) 402 as indicated in a block 514.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of FAN having wherein in response to there being the dirty spot in the current image frame, the method further comprises: obtaining a binary image according to a comparison result of the similarity of the corresponding one of the filtered image frames to the current image frame and a similarity threshold, wherein the binary image is configured for representing a mask image of the dirty spot in the current image frame; and repairing the dirty spot in the current image frame according to the binary image and a reference frame corresponding to the current image frame.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein in response to there being the dirty spot in the current image frame, the method further comprises: obtaining a binary image according to a comparison result of the similarity of the corresponding one of the filtered image frames to the current image frame and a similarity threshold, wherein the binary image is configured for representing a mask image of the dirty spot in the current image frame; and repairing the dirty spot in the current image frame according to the binary image and a reference frame corresponding to the current image frame.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video to improve visual appearance, since both RAKHSHANFAR and FAN relate to video processing techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and FAN relates a system, method, and program product for generating an enhanced digital video is provided; as a result, visual appearance of a digital video may be enhanced. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and FAN (US 20060115178 A1), Paragraph [0012].
Regarding claim 33, RAKHSHANFAR in view of POSSOS teach the method according to claim 29,
Although RAKHSHANFAR further teaches filtered image blocks (Figs. 3-5, Paragraph [0070] – RAKHSHANFAR discloses the proposed time-space filter is summarized in FIG. 3. An overview of the computations executed by the computing system is presented in Algorithm 1 below. Algorithm 1: Mixed block-pixel based noise filter i) Estimate and compensate motion vectors in 2R (preceding, and subsequent) frames. ii) Compute the motion error probability of each non- overlapped blocks of L × L using (3). iii) Find the averaging weights for each pixel via (11). iv) Average the motion-compensated frames using (2). v) Restore the destructed structures due to motion blur via (18) and (19). vi) Filter spatially residual noise using pixel-level noise variance σ.sub.8.sup.2 computed in (20).)
RAKHSHANFAR in view of POSSOS fail to explicitly teach wherein in response to there being dirty spot in the current image block, the method further comprises: obtaining a binary image block according to a comparison result of the similarity of the corresponding one of the image blocks to the current image block and a similarity threshold, wherein the binary image block is configured for representing a mask image of the dirty spot in the current image block; repairing the dirty spot in the current image block according to the binary image block and a reference block corresponding to the current image block to obtain a repaired image block, wherein the reference block is determined according to an image block of a position same as a position of the current image block in the reference frame corresponding to the current image frame; and replacing the current image block with the repaired image block.
However, FAN explicitly teaches wherein in response to there being dirty spot in the current image block (Fig. 4, Paragraph [0038] – FAN discloses verifications unit 418 determines that the candidate artifact is most likely an artifact caused by dust or scratches, for example, and stores information that identifies the region that encompasses the artifact region, e.g., region 602.), the method further comprises:
obtaining a binary image block according to a comparison result of the similarity of the corresponding one of the image blocks to the current image block and a similarity threshold (Fig. 4-5, Paragraph [0039] – FAN discloses after one or more artifacts are detected as just described, processing system 100 executes artifact removal module 114 to perform block matching with a previous frame (n-1) 402 and a next frame (n+1) 402 as indicated in a block 512. Artifact removal module 114 creates a binary mask of frame (n) 402 to indicate locations of artifact regions.),
wherein the binary image block is configured for representing a mask image of the dirty spot in the current image block (Fig. 4-5, Paragraph [0039] – FAN discloses artifact removal module 114 creates a binary mask of frame (n) 402 to indicate locations of artifact regions.);
repairing the dirty spot in the current image block according to the binary image block and a reference block corresponding to the current image block to obtain a repaired image block (Figs. 4-5, Paragraph [0040] – FAN discloses processing system 100 executes artifact removal module 114 to replace each artifact region in frame (n) 402 with a corresponding block from either previous frame (n-1) 402 or next frame (n+1) 402 as indicated in a block 514.),
wherein the reference block is determined according to an image block of a position same as a position of the current image block in the reference frame corresponding to the current image frame (Fig. 4-5, Paragraph [0039] – FAN discloses after one or more artifacts are detected as just described, processing system 100 executes artifact removal module 114 to perform block matching with a previous frame (n-1) 402 and a next frame (n+1) 402 as indicated in a block 512. Artifact removal module 114 creates a binary mask of frame (n) 402 to indicate locations of artifact regions.);
and replacing the current image block with the repaired image block (Figs. 4-5, Paragraph [0040] – FAN discloses processing system 100 executes artifact removal module 114 to replace each artifact region in frame (n) 402 with a corresponding block from either previous frame (n-1) 402 or next frame (n+1) 402 as indicated in a block 514.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of FAN having wherein in response to there being dirty spot in the current image block, the method further comprises: obtaining a binary image block according to a comparison result of the similarity of the corresponding one of the image blocks to the current image block and a similarity threshold, wherein the binary image block is configured for representing a mask image of the dirty spot in the current image block; repairing the dirty spot in the current image block according to the binary image block and a reference block corresponding to the current image block to obtain a repaired image block, wherein the reference block is determined according to an image block of a position same as a position of the current image block in the reference frame corresponding to the current image frame; and replacing the current image block with the repaired image block.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein in response to there being dirty spot in the current image block, the method further comprises: obtaining a binary image block according to a comparison result of the similarity of the corresponding one of the filtered image blocks to the current image block and a similarity threshold, wherein the binary image block is configured for representing a mask image of the dirty spot in the current image block; repairing the dirty spot in the current image block according to the binary image block and a reference block corresponding to the current image block to obtain a repaired image block, wherein the reference block is determined according to an image block of a position same as a position of the current image block in the reference frame corresponding to the current image frame; and replacing the current image block with the repaired image block.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video to improve visual appearance, since both RAKHSHANFAR and FAN relate to video processing techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and FAN relates a system, method, and program product for generating an enhanced digital video is provided; as a result, visual appearance of a digital video may be enhanced. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and FAN (US 20060115178 A1), Paragraph [0012].
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over RAKHSHANFAR (US 20170084007 A1), hereinafter referenced as RAKHSHANFAR in view of POSSOS (US 20180343448 A1), hereinafter referenced as POSSOS in further view of FAN (US 20060115178 A1), hereinafter referenced as FAN in further view of NANJEGOWD (US 20220385870 A1), hereinafter referenced as NANJEGOWD.
Regarding claim 26, RAKHSHANFAR and POSSOS in view of FAN teach the method according to claim 25,
RAKHSHANFAR and POSSOS fail to explicitly teach wherein repairing the dirty spot in the current image frame according to the binary image and the reference frame corresponding to the current image frame, comprises: determining a first image matrix according to a product of the reference frame corresponding to the current image frame and the binary image,
However, FAN explicitly teaches wherein repairing the dirty spot in the current image frame according to the binary image and the reference frame corresponding to the current image frame (Figs. 4-5, Paragraph [0039] – FAN discloses artifact removal module 114 creates a binary mask of frame (n) 402 to indicate locations of artifact regions. Paragraph [0040] – FAN discloses processing system 100 executes artifact removal module 114 to replace each artifact region in frame (n) 402 with a corresponding block from either previous frame (n-1) 402 or next frame (n+1) 402 as indicated in a block 514.), comprises:
determining a first image matrix according to a product of the reference frame corresponding to the current image frame and the binary image (Figs. 4-5, Paragraph [0039] – FAN discloses artifact removal module 114 creates a binary mask of frame (n) 402 to indicate locations of artifact regions. Paragraph [0040] – FAN discloses processing system 100 executes artifact removal module 114 to replace each artifact region in frame (n) 402 with a corresponding block from either previous frame (n-1) 402 or next frame (n+1) 402 as indicated in a block 514.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of FAN having wherein repairing the dirty spot in the current image frame according to the binary image and the reference frame corresponding to the current image frame, comprises: determining a first image matrix according to a product of the reference frame corresponding to the current image frame and the binary image,
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein repairing the dirty spot in the current image frame according to the binary image and the reference frame corresponding to the current image frame, comprises: determining a first image matrix according to a product of the reference frame corresponding to the current image frame and the binary image,
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video to improve visual appearance, since both RAKHSHANFAR and FAN relate to video processing techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and FAN relates a system, method, and program product for generating an enhanced digital video is provided; as a result, visual appearance of a digital video may be enhanced. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and FAN (US 20060115178 A1), Paragraph [0012].
RAKHSHANFAR and POSSOS in view of FAN fail to explicitly teach inverting the binary image to obtain an inverted image; and determining a second image matrix according to a product of the current image frame and the inverted image; and determining an image frame after repairing the dirty spot in the current image frame according to the first image matrix and the second image matrix.
However, NANJEGOWD explicitly teaches inverting the binary image to obtain an inverted image (Fig. 2, Paragraph [0015] – NANJEGOWD discloses in act 210, the binary bit map 204 is inverted by forming its 1's complement (B′) 212, which is zero for all pixels within the glare region and one for all pixels outside the glare region.);
and determining a second image matrix according to a product of the current image frame and the inverted image (Fig. 2, Paragraph [0015] – NANJEGOWD discloses the Y plane 114 of the YUV image is multiplied, at 214, with the 1's complement 212 of the binary bit map 204. This operation generates another intermediate image (Y′) 216, in which pixel values corresponding to the glare region become zero.);
and determining an image frame after repairing the dirty spot in the current image frame according to the first image matrix and the second image matrix (Fig. 2, Paragraph [0015] – NANJEGOWD discloses the Y plane 114 of the YUV image is multiplied, at 214, with the 1's complement 212 of the binary bit map 204. This operation generates another intermediate image (Y′) 216, in which pixel values corresponding to the glare region become zero. The two intermediate images 208, 216 are added, at 218, to generate a single modified Y plane (Y″) 220 free of glare pixels, in which the glare pixels of the original Y plane 114 have been substituted by corresponding pixel values of the IR image 130.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of NANJEGOWD having inverting the binary image to obtain an inverted image; and determining a second image matrix according to a product of the current image frame and the inverted image; and determining an image frame after repairing the dirty spot in the current image frame according to the first image matrix and the second image matrix.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein having inverting the binary image to obtain an inverted image; and determining a second image matrix according to a product of the current image frame and the inverted image; and determining an image frame after repairing the dirty spot in the current image frame according to the first image matrix and the second image matrix.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video to improve visual quality, since both RAKHSHANFAR and NANJEGOWD relate to image/video processing techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and NANJEGOWD relates to computationally efficient implementations that facilitate glare removal in real time, e.g., within 30 ms or less; glare removal in accordance herewith can be applied to video streams, for instance, to improve the user experience during video calls or video recording under various lighting conditions, or to improve the video provided by dashboard and other vehicle camera systems. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and NANJEGOWD (US 20220385870 A1), Paragraph [0009].
Claim 27 is rejected under 35 U.S.C. 103 as being unpatentable over RAKHSHANFAR (US 20170084007 A1), hereinafter referenced as RAKHSHANFAR in view of POSSOS (US 20180343448 A1), hereinafter referenced as POSSOS in further view of FAN (US 20060115178 A1), hereinafter referenced as FAN in further view of CARSON (US 8433143 B1), hereinafter referenced as CARSON.
Regarding claim 27, RAKHSHANFAR and POSSOS in view of FAN teach the method according to claim 25,
Although RAKHSHANFAR further teaches filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.)
RAKHSHANFAR and POSSOS in view of FAN fail to explicitly teach wherein the reference frame corresponding to the current image frame comprises a reference frame corresponding to a maximum similarity of the similarity of the corresponding one of the image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the image frames to the back reference frame corresponding to the current image frame.
However, CARSON explicitly teaches wherein the reference frame corresponding to the current image frame comprises a reference frame corresponding to a maximum similarity of the similarity of the corresponding one of the image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the image frames to the back reference frame corresponding to the current image frame (Fig. 6, Col. 6, Lines [49-54] – CARSON discloses locating a potential visual disruption (e.g., a visual artifact) can be performed by noting similarity indexes in the F1/F2 and F2/F3 pairs that are well below the similarity index of the F1/F3 pair. In other words, if F1/F3 are closely related but F1/F2 and F2/F3 are not, there may be a visually detectable artifact (e.g., tear, dirt spot, etc.) in frame F2. See also Fig. 13, Col. 10, Lines [54-64].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of CARSON having wherein the reference frame corresponding to the current image frame comprises a reference frame corresponding to a maximum similarity of the similarity of the corresponding one of the image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the image frames to the back reference frame corresponding to the current image frame.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein the reference frame corresponding to the current image frame comprises a reference frame corresponding to a maximum similarity of the similarity of the corresponding one of the filtered image frames to the front reference frame corresponding to the current image frame and the similarity of the corresponding one of the filtered image frames to the back reference frame corresponding to the current image frame.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video, since both RAKHSHANFAR and CARSON relate to video artifact detection, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and CARSON relates to an apparatus and method for detecting human-visual artifacts in a video presentation; motion detection and rejection capabilities will improve the detection rate of actual artifacts and enhance the overall statistical validity of the similarity measurements. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and CARSON (US 8433143 B1), Col. 12, Lines [1-3].
Claim 28 is rejected under 35 U.S.C. 103 as being unpatentable over RAKHSHANFAR (US 20170084007 A1), hereinafter referenced as RAKHSHANFAR in view of POSSOS (US 20180343448 A1), hereinafter referenced as POSSOS in further view of FAN (US 20060115178 A1), hereinafter referenced as FAN in further view of NANJEGOWD (US 20220385870 A1), hereinafter referenced as NANJEGOWD in further view of YU (US 20190073752 A1), hereinafter referenced as YU.
Regarding claim 28, RAKHSHANFAR and POSSOS in view of FAN teach the method according to claim 25,
Although RAKHSHANFAR further teaches filtered image frames (Figs. 3-5, Paragraph [0030] – RAKHSHANFAR discloses time-domain filtering on current frame using motion-compensated previous and subsequent frames.)
RAKHSHANFAR and POSSOS in view of FAN fail to explicitly teach wherein after obtaining the binary image according to the comparison result of the similarity of the corresponding one of the image frames to the current image frame and the similarity threshold,
However, NANJEGOWD explicitly teaches wherein after obtaining the binary image according to the comparison result of the similarity of the corresponding one of the image frames to the current image frame and the similarity threshold (Fig. 2, Paragraph [0015] – NANJEGOWD discloses in act 210, the binary bit map 204 is inverted by forming its 1's complement (B′) 212, which is zero for all pixels within the glare region and one for all pixels outside the glare region. The Y plane 114 of the YUV image is multiplied, at 214, with the 1's complement 212 of the binary bit map 204. This operation generates another intermediate image (Y′) 216, in which pixel values corresponding to the glare region become zero. The two intermediate images 208, 216 are added, at 218, to generate a single modified Y plane (Y″) 220 free of glare pixels, in which the glare pixels of the original Y plane 114 have been substituted by corresponding pixel values of the IR image 130.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of NANJEGOWD having wherein after obtaining the binary image according to the comparison result of the similarity of the corresponding one of the image frames to the current image frame and the similarity threshold.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein after obtaining the binary image according to the comparison result of the similarity of the corresponding one of the filtered image frames to the current image frame and the similarity threshold.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video to improve visual quality, since both RAKHSHANFAR and NANJEGOWD relate to image/video processing techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and NANJEGOWD relates to computationally efficient implementations that facilitate glare removal in real time, e.g., within 30 ms or less; glare removal in accordance herewith can be applied to video streams, for instance, to improve the user experience during video calls or video recording under various lighting conditions, or to improve the video provided by dashboard and other vehicle camera systems. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and NANJEGOWD (US 20220385870 A1), Paragraph [0009].
RAKHSHANFAR and POSSOS in view of FAN in further view of NANJEGOWD fail to explicitly teach the method further comprises: denoising the binary image at least once to obtain a processed binary image to repair the dirty spot in the current image frame according to the processed binary image and the reference frame corresponding to the current image frame; wherein the denoising comprises a corrosion process and/or an expansion process.
However, YU explicitly teaches the method further comprises: denoising the binary image at least once to obtain a processed binary image to repair the dirty spot in the current image frame according to the processed binary image and the reference frame corresponding to the current image frame (Fig. 2, Paragraph [0041] – YU discloses erosion and dilation of image morphology can well denoise a binary image. The specific operation of erosion is: scanning each pixel in the image with a structural element (generally 3×3 size), and using each pixel in the structural element to perform an “AND” (&&) operation on the pixel it covers, if both are 1, then the pixel is 1, otherwise it is 0. See also Paragraph [0044].);
wherein the denoising comprises a corrosion process and/or an expansion process ((Fig. 2, Paragraph [0041] – YU discloses erosion and dilation of image morphology can well denoise a binary image. The function of erosion is to eliminate boundary points of the object, reduce the target, and eliminate noise points smaller than the structural elements. The effect of dilation is to merge all the background points that are in contact with the object into the object, increase the target and fill the holes, in the target.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of RAKHSHANFAR in view of POSSOS of having a method for detecting a dirty spot in a video, comprising: obtaining a reference frame corresponding to a current image frame in the video, wherein the reference frame comprises a front reference frame and a back reference frame, the front reference frame is an image frame before the current image frame in the video, and the back reference frame is an image frame after the current image frame in the video; with the teachings of NANJEGOWD having wherein after obtaining the binary image according to the comparison result of the similarity of the corresponding one of the image frames to the current image frame and the similarity threshold.
Wherein having RAKHSHANFAR’s method for detecting a dirty spot in a video wherein after obtaining the binary image according to the comparison result of the similarity of the corresponding one of the filtered image frames to the current image frame and the similarity threshold.
The motivation behind the modification would have been to obtain an enhanced method of reliably detecting and removing artifacts in a video to improve visual quality, since both RAKHSHANFAR and YU relate to image/video processing techniques, wherein RAKHSHANFAR generally relates to image and video noise analysis and specifically to the reduction of video noise; the systems and methods described herein give a solution for a color video denoising, handle both processed and white noise, integrate a spatial filter in order to remove residual noise, and detect and remove artifacts due to blocking and motion blur, and YU relates to the field of image segmentation technologies, and more particularly to a method and a device for removing a scanning bed from a computed tomography (CT) image; the application in image processing is mainly to use the basic operations of morphology to observe and process images to achieve the purpose of improving image quality.. Please see RAKHSHANFAR (US 20170084007 A1), Paragraph [0002, 00029], and YU (US 20190073752 A1), Paragraph [0041, 0064].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant’s disclosure.
MO et al. (US 20230396861 A1) - The present disclosure relates to a method and device for generating video preview content, a computer device and a storage medium. The method for generating video preview content includes parsing a video to be processed, to obtain all image frames of the video to be processed and generate a list of ordered image frames; processing the list of the ordered image frames by image recognition to filter out a slice header and a slice tail of the video; and generating preview content of the video based on a list of the filtered image frames.… Fig. 1, Abstract.
MAY et al. (US 20140232869 A1) - A vision system for a vehicle includes a camera disposed at a vehicle and having an image sensor and a lens. When the camera is disposed at the vehicle, the lens is exposed to the environment exterior the vehicle. The camera has an exterior field of view exterior the vehicle and is operable to capture image data representative of a scene occurring in the field of view of the camera. An image processor is operable to process multiple frames of image data captured by the camera, and wherein, responsive to processing multiple frames of captured image data, the image processor is operable to detect contaminants, such as water droplets or dirt, at the lens of the camera. Responsive to the image processor detecting contaminants at the lens of the camera, the vision system generates an alert and may trigger a function..… Fig. 1, Abstract.
KARCZEWICZ et al. (US 20190373258 A1) - A video encoder and video decoder may determine a set of adaptive loop filters, from among a plurality of sets of adaptive loop filters, on a per-block basis. Each set of adaptive loop filters may include filters from a previous picture, filters signaled for the current picture, and/or pre-trained filter. By varying the set of adaptive loop filters on a per-block basis, the adaptive loop filters available for each block of video data may be more adapted to local statistics of the video data.… Fig. 1, Abstract.
THEIS et al. (US 20160105591 A1) - Dirt or other non-steady defects are detected in a frame of a sequence of digitized image frames. No determination of motion vectors for motion compensation is required. Instead, absolute motion values for a plurality of pixels of the frame relative to a preceding frame and to a succeeding frame of the sequence are determined. Based on the assumption that motion is usually smooth in the sequence, temporal coherence violations between the frame and the preceding frame and between the frame and the succeeding frame are detected for the pixels, depending on the absolute motion values. Pixels of the plurality of pixels are determined as defective if corresponding temporal coherence violations are detected between the frame and the preceding frame and the succeeding frame.… Fig. 1, Abstract.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEZAWIT N SHIMELES whose telephone number is (571)272-7663. The examiner can normally be reached M-F 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEZAWIT NOLAWI SHIMELES/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights