Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/07/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted on 11/06/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 8-11, 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Pekkucuksen et al. (US 11889197 B1) referred to as Pekkucuksen hereinafter and further in view of Pekkucuksen et al. (US 11062436 B2) referred to as Pekkucuksen_2 hereinafter.
Regarding claim 1, Pekkucuksen teaches A method comprising: selecting, using at least one processing device of an electronic device, a reference frame and a non-reference frame from a plurality of image frames; (“The selected aligned input image frame 202 is used as a reference frame in subsequent processing operations, while one or more remaining aligned input image frames 202 can be treated as non-reference frames.” Pekkucuksen, col. 11, lines 39-42)
generating, using the at least one processing device, a reference edge map identifying edges in the reference frame and a non-reference edge map identifying edges in the non-reference frame; (“The downsampled image frames are provided to an edge score calculation function 404, which generally operates to generate scores or other indicators regarding the strengths of edges contained in the downsampled image frames. The edge score calculation function 404 can use any suitable technique to generates scores or other indicators of edge strengths in image frames.” Pekkucuksen, col. 16, lines 55-61)
generating, using the at least one processing device, a moving edge map based on one or more movements between one or more of the edges of the reference edge map and one or more of the edges of the non-reference edge map; (“A reference frame selection function 206 generally operates to process the aligned input image frames 202 in order to select one of the aligned input image frames 202 as a reference frame. The selected aligned input image frame 202 is used as a reference frame in subsequent processing operations, while one or more remaining aligned input image frames 202 can be treated as non-reference frames. The reference frame selection function 206 may use the techniques described below to select a reference frame, such as by selecting the reference frame based on blur level determinations and moving saturated region detections involving the aligned input image frames 202.” Pekkucuksen, col. 11, lines 36-47) and (“the edge strength filter can compare the pixel values in each set of two pixels that are opposite each other about the center pixel in the window. This results in a comparison of pixel values along a horizontal axis, a vertical axis, and both diagonal axes of the window. In this way, the edge strength filter can detect an edge, such as based on a large change in the luminance or other values of the compared pixels, regardless of whether the edge is horizontally, vertically, or diagonally disposed within the window.” Pekkucuksen, col. 16, lines 66-67, col. 17, lines 1-7)
generating, using the at least one processing device, a blend map based on the reference frame and the non-reference frame; (“When image frames captured during a multi-frame capture operation are blended, one of the image frames is typically selected as a reference frame, and portions of image contents in one or more non-reference frames can be selected for blending with, or for replacement of, portions of image contents in the reference frame.” Pekkucuksen, col. 6, lines 11-16)
modifying, using the at least one processing device, the blend map based on one or more indications of movement of corresponding pixels in the moving edge map to generate a modified blend map; (“The deghosting function 212 may be implemented in any suitable manner, such as by using a machine learning model that has been trained to reduce blur in images. An edge noise filtering function 214 can be used to filter the image data of the blended image in order to remove noise from object edges, which can help to provide cleaner edges to objects in the blended image. The edge noise filtering function 214 may be implemented in any suitable manner. A tone mapping function 216 can be used to adjust colors in the blended image, which can be useful or important in various applications, such as when generating high dynamic range (HDR) images.” Pekkucuksen, col. 12, lines 11-22)
However, Pekkucuksen does not teach and blending, using the at least one processing device, the reference frame and the non-reference frame based on the modified blend map to generate an output image.
Pekkucuksen_2 teaches and blending, using the at least one processing device, the reference frame and the non-reference frame based on the modified blend map to generate an output image. (“The blending operation 208 can compare the reference image frame to one or more non-reference image frames (which represent one or more longer-exposure image frames), possibly during multiple histogram matching operations if there are multiple non-reference image frames available. The blending operation 208 can also generate one or more motion maps based on the reference and non-reference image frames and blend the reference and non-reference image frames based on the histogram matching and the motion map(s).” Pekkucuksen_2, col. 10, lines 64-67, col. 11, lines 1-6)
Pekkucuksen and Pekkucuksen_2 are combinable because they are from the same field of endeavor, image processing and combining images.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Pekkucuksen in light of Pekkucuksen_2’s blending reference and non-reference frames. One would have been motivated to do so because it helps to improve or maximize noise reduction in the final image. (Pekkucuksen, col. 13, lines 37-38)
Regarding claim 2, Pekkucuksen teaches wherein generating the moving edge map comprises: filtering the edges of the reference edge map and the edges of the non-reference edge map that have a length below a threshold; and identifying the one or more movements based on a non-zero area of a difference between corresponding edges of the reference edge map and the non-reference edge map. (“The downsampled image frames are provided to an edge score calculation function 404, which generally operates to generate scores or other indicators regarding the strengths of edges contained in the downsampled image frames. The edge score calculation function 404 can use any suitable technique to generates scores or other indicators of edge strengths in image frames. In some embodiments, for instance, edge scores can be calculated using a 3×3 filter referred to as an edge strength filter (ESF) Here, a 3×3 window can be moved within each downsampled image frame, and pixel values within the window can be compared. For example, the edge strength filter can compare the pixel values in each set of two pixels that are opposite each other about the center pixel in the window. This results in a comparison of pixel values along a horizontal axis, a vertical axis, and both diagonal axes of the window. In this way, the edge strength filter can detect an edge, such as based on a large change in the luminance or other values of the compared pixels, regardless of whether the edge is horizontally, vertically, or diagonally disposed within the window. This allows the edge strength filter to detect changes in luminance or other values of pixels that indicate edges or textures in an image frame.” Pekkucuksen, col. 16, lines 55-67, col. 17, lines 1-10)
Regarding claim 3, Pekkucuksen teaches wherein modifying the blend map comprises: for each identified movement of a respective pixel in the moving edge map, multiplying a pixel value for the respective pixel in the blend map by an edge score in the moving edge map to generate a corresponding pixel value for the respective pixel in the modified blend map. (“A maximum and multiplication function 606 receives the saturation maps and the motion map. The “maximum” portion of the maximum and multiplication function 606 can identify, for each pixel location in the saturation maps, a maximum pixel value contained in any of the saturation maps at that pixel location. In other words, the maximum and multiplication function 606 here is selecting the maximum value contained in any of the saturation maps for each individual pixel location in the saturation maps. The “multiplication” portion of the maximum and multiplication function 606 can multiply the selected maximum value for each pixel location in the saturation maps and a corresponding pixel value at the same pixel location in the motion map. Repeating this across all pixel locations leads to the generation of a moving saturated region map, which represents a combination of locations identified as being saturated in any of the input image frames 202″ and locations identified as containing motion.” Pekkucuksen, col. 19, lines 53-67, col. 20, lines 1-3)
Regarding claim 4, Pekkucuksen does not teach wherein modifying the blend map further comprises: for each static pixel identified in the moving edge map, maintaining a pixel value for the static pixel in the modified blend map.
Pekkucuksen_2 teaches wherein modifying the blend map further comprises: for each static pixel identified in the moving edge map, maintaining a pixel value for the static pixel in the modified blend map. (“The output of the blending operation 326 is a blended image 328, which (ideally) has little or no motion blur as defined by the reference image frame 316 for any motion regions and larger image details and less noise as defined by the one or more non-reference image frames 312b for any stationary regions.” Pekkucuksen_2, col. 13, lines 58-63)
Pekkucuksen and Pekkucuksen_2 are combinable because they are from the same field of endeavor, image processing and combining images.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Pekkucuksen in light of Pekkucuksen_2’s static pixel. One would have been motivated to do so because it helps to improve or maximize noise reduction in the final image. (Pekkucuksen, col. 13, lines 37-38)
Regarding claim 8, refer to the explanation of claim 1.
Regarding claim 9, refer to the explanation of claim 2.
Regarding claim 10, refer to the explanation of claim 3.
Regarding claim 11, refer to the explanation of claim 4.
Regarding claim 15, Pekkucuksen teaches A non-transitory machine-readable medium containing instructions that when executed cause at least one processor to (“The non-transitory computer readable medium also contains instructions that when executed cause the at least one processor to” Pekkucuksen, col. 2, lines 21-23)
Regarding rest of claim 15, refer to the explanation of claim 1.
Regarding claim 16, refer to the explanation of claim 2.
Regarding claim 17, refer to the explanation of claims 3 and 4.
Allowable Subject Matter
Claims 5-7, 12-14, 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claims 5, 12, and 18, the combination of the closest prior arts does not teach “generating a second non-reference edge map identifying edges in the second non-reference frame; generating a second moving edge map based on at least one movement between at least one of the edges of the reference edge map and at least one of the edges of the second non-reference edge map; generating a second blend map based on the reference frame and the second non-reference frame; and modifying the second blend map based on at least one indication of movement of corresponding pixels in the second moving edge map to generate a second modified blend map; and wherein blending the reference frame and the non-reference frame comprises blending the reference frame, the non-reference frame weighted by the modified blend map, and the second non-reference frame weighted by the second modified blend map.”
Claims 6, 7, 13, 14, 19, and 20 are dependent upon claims 5, 12, and 18.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20150002704 A1 GHOST ARTIFACT DETECTION AND REMOVAL IN HDR IMAGE PROCESSING USING MULTI-SCALE NORMALIZED CROSS-CORRELATION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PARDIS SOHRABY whose telephone number is (571)270-0809. The examiner can normally be reached Monday - Friday 9 am till 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PARDIS SOHRABY/ Examiner, Art Unit 2664
/CHARLOTTE M BAKER/ Primary Examiner, Art Unit 2664
Prosecution Insights