DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 12/31/2025 have been entered into record. Claims 1-20 remain pending in the application.
Information Disclosure Statement
The information disclosure statements filed 08/26/2025 and 04/28/2026 have been considered by examiner, except where citations were struck through for failure to provide English translations of cited documents.
Response to Arguments
Applicant's arguments, see Remarks filed 12/31/2025, have been fully considered but they are not persuasive.
Regarding the double patenting rejection, Applicant argues that because the scopes of the claims of the present application have not yet been finally determined, Applicant reserves the right to argue that the claims are distinguishable over U.S. Pat. No. 11,995,156 or to file a terminal disclaimer at a later time.
This argument is in violation of the MPEP 804 which explicitly states “A complete response to a nonstatutory double patenting (NSDP) rejection is either a reply by applicant showing that the claims subject to the rejection are patentably distinct from the reference claims, or the filing of a terminal disclaimer in accordance with 37 CFR 1.321 in the pending application(s) with a reply to the Office action (see MPEP § 1490 for a discussion of terminal disclaimers). Such a response is required even when the nonstatutory double patenting rejection is provisional. As filing a terminal disclaimer, or filing a showing that the claims subject to the rejection are patentably distinct from the reference application’s claims, is necessary for further consideration of the rejection of the claims, such a filing should not be held in abeyance. Only compliance with objections or requirements as to form not necessary for further consideration of the claims may be held in abeyance until allowable subject matter is indicated.”
Double patenting rejections are not an objection or requirement as to form, and therefore the inclusion of this argument renders applicant’s response incomplete. The double patenting rejection has been updated below to reflect to the amendments to the claims. Any future responses failing to adequately respond to the double patenting rejection will be deemed Non-responsive and incomplete.
Regarding the rejection of Claims 1-20 under 35 U.S.C. 103 in view of Trejo and Pinhasov have been fully considered but are not persuasive. Applicant argues Trejo merely discloses computing a value of a noise filter based on a "weighted sum of the pixels in a neighborhood of the current pixel" but does not disclose computing a confidence weighted metric based on identifying "a maximum confidence value among a plurality of confidence values" as now recited in claims 1, 10, and 19. Applicant further argues Pinhasov merely discloses computing an "accumulated weight" based weighting neighboring pixels based on the confidence levels of the confidence map when computing a "weight sum" but does not appear to disclose or suggest that values in the "confidence map" are computed based on identifying "a maximum confidence value among one or more second portions of the image" as now recited in claims 1, 10 and 19.
Examiner respectfully disagrees. Pinhasov teaches wherein the weighted metric is a confidence weighted metric based on a confidence map that identifies a degree of confidence that includes in white in the confidence map 235 represent a high confidence level, such as a confidence level exceeding a high threshold percentage, such as 90%. Pixels illustrated in black in the confidence map 235 represent a low confidence level, such as a confidence level falling below a low threshold percentage, such as 10%. The confidence map 235 also includes six different shades of grey (other than black and white), each representing confidence levels falling into different ranges of confidence between the high threshold percentage and the low threshold percentage ([0062]). In this teaching, exceeding 90% is the maximum confidence value level, and each shade of black, white, and great appearing in the map is a different one of the plurality of confidence value levels. Pinhasov further discloses weighting the neighboring pixels differently in the confidence map based on their confidence levels, such that their contributions to the weighted sum are different ([0117]). Therefore, the confidence map in Pinhasov both illustrates and performs weighted calculations based on the maximum confidence levels of the neighboring pixels within the image, continuing to render obvious the amended limitations. For these reasons, the rejections are maintained below.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 11,995,156, as further detailed in the table provided below:
Instant Application
U.S. Patent No. 11,995,156
Claim 1. A method, comprising: for a first portion of an image, calculating a confidence weighted metric based on identifying a maximum confidence value among a plurality of confidence values corresponding to a plurality of second portions of the image and a range filter function; and processing the image based on the confidence weighted metric for the first portion of the image.
Claim 1. A method, comprising:
determining a category of a first pixel of an image, the first pixel having a plurality of neighboring pixels, each of the neighboring pixels having a category; and
processing the image based on the determined category,
the determining comprising:
calculating a confidence weighted metric for each of the neighboring pixels, the confidence weighted metric being based on a maximum confidence value among each of the neighboring pixels and a range filter function; and
determining the category of the first pixel based on the confidence weighted metric of each of the neighboring pixels and based on the category of one of the neighboring pixels.
Claim 2
Claim 1
Claim 3
Claim 1
Claim 4
Claim 2
Claim 5
Claim 3
Claim 6
Claim 4
Claim 7
Claim 5
Claim 8
Claim 6
Claim 9
Claim 8
Claim 10. A system comprising a processing circuit, the processing circuit being configured to: for a first portion of an image, calculate a confidence weighted metric based identifying a maximum confidence value among a plurality of confidence values corresponding to a plurality of second portions of the image and a range filter function; and process the image based on the confidence weighted metric for the first portion of the image.
Claim 10. A system comprising a processing circuit, the processing circuit being configured to:
determine a category of a first pixel of an image, the first pixel having a plurality of neighboring pixels, each of the neighboring pixels having a category; and
process the image based on the determined category,
the determining comprising:
calculating a confidence weighted metric for each of the neighboring pixels, the confidence weighted metric being based on a maximum confidence value for each of the neighboring pixels and a range filter function; and
determining the category of the first pixel based on the confidence weighted metric of each of the neighboring pixels and based on the category of one of the neighboring pixels.
Claim 11
Claim 10
Claim 12
Claim 10
Claim 13
Claim 11
Claim 14
Claim 12
Claim 15
Claim 13
Claim 16
Claim 14
Claim 17
Claim 15
Claim 18
Claim 17
Claim 19. A system comprising means for processing, the means for processing being configured to: for a first portion of an image, calculate a confidence weighted metric based on identifying a maximum confidence value among a plurality of confidence values corresponding to a plurality of second portions of the image and a range filter function; and process the image based on the confidence weighted metric for the first portion of the image.
Claim 18. A system comprising means for processing, the means for processing being configured to:
determine a category of a first pixel of an image, the first pixel having a plurality of neighboring pixels, each of the neighboring pixels having a category; and
process the image based on the determined category,
the determining comprising:
calculating a confidence weighted metric for each of the neighboring pixels, the confidence weighted metric being based on a maximum confidence value for each of the neighboring pixels and a range filter function; and
determining the category of the first pixel based on the confidence weighted metric of each of the neighboring pixels and based on the category of one of the neighboring pixels.
Claim 20
Claim 18
Although the conflicting claims are not identical, they are not patentably distinct from each other because the Patent claims include all the limitations of the instant application claims, respectively. The patent claims also include additional limitations. Hence, the instant application claims are generic to the species of invention covered by the respective patent claims. As such, the instant application claims are anticipated by the patent claims and are therefore not patentably distinct therefrom (See Eli Lilly and Co. v. Barr Laboratories Inc., 58 USPQ2D 1869, " a later genus claim limitation is anticipated by, and therefore not patentably distinct from, an earlier species claim", In re Goodman, 29 USPQ2d 2010, "Thus, the generic invention is 'anticipated' by the species of the patented invention" and the instant “application claims are generic to species of invention covered by the patent claim, and since without terminal disclaimer, extant species claim preclude issuance of generic application claims”).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Claim limitations in Claims 19-20 in this application that use the word “means” (or “step”), are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Trejo et al (US 2020/0211161), in view of Pinhasov et al (US 2022/0060619).
Regarding Claim 1, Trejo teaches a method, comprising: for a first portion of an image, calculating a weighted metric ([0134], for a pixel, the adaptive spatial noise filter adjusts the weights used in the filter not only based on the Euclidean distance of adjacent pixels from the pixel, but also based on differences in signal levels between the adjacent pixels and the current pixel and based on the noise component of the current pixel. Specifically, as explained more completely below, adaptive spatial noise filter 570 takes a weighted sum of the pixels in a neighborhood of the current pixel. The weights depend on the spatial distance and on the signal levels of the neighboring pixels as well as the noise component of the current pixel itself. Thus, the signal value at each pixel in a frame is replaced by a noise-based weighted average of signal values from nearby pixels)
based on a plurality of second portions of the image ([0022], adaptive noise filter is configured to determine a difference between the signal level of the first pixel and the signal level of each of a plurality of neighboring pixels. The adaptive spatial noise filter also uses a noise dependent signal level parameter, where the noise dependent signal level parameter is a function of the noise of the first pixel, [0075], When adaptive noise filter 210 is an adaptive spatial noise filter, a value of a current pixel at a location in a current frame of demosiaced and color transformed pixel data is compared with the values of neighboring pixels in the current frame. The percentage of the spatial change of current pixel that is passed through the filter, as an adaptively spatial noise filtered pixel, is weighted based on the spatial relationship of the current pixel to the neighboring pixels) and a range filter function ([0022], the adaptive spatial noise filter includes a distance filter and a signal level range filter. The signal level range filter is configured to filter the pixel based on a difference between a signal level of a pixel and each of the signal levels of the plurality of neighboring pixels and based on the noise dependent signal level parameter); and
processing the image based on the weighted metric for the first portion of the image ([0155-0158], COMBINE FILTERS process 607 multiplies a value at a location in the n by n block of the distance filter values by the value at the same location in the n by a block of range filter values for each location in the two blocks, and so generates a combined filter value for each location in a n by n block of combined filter values. COMBINE FILTERS process 607 transfers to NORMALIZE process 608, GENERATE PIXEL process 609 generates the adaptively spatial noise filtered pixel and writes the adaptively spatial noise filtered pixel to output frame of filtered pixel data 614, sometimes referred to as output frame of pixel data or filtered pixel output frame).
Trejo fails to teach the following, which in the same field of endeavor, Pinhasov teaches wherein the weighted metric is a confidence weighted metric ([0062], The confidence map 235 identifies a degree of confidence that the classification engine 220 has as to its classification of a given pixel in the category map 230. The classification engine 220 sends the category map 230 and the confidence map 235 to the ISP 240, [0117], Neighboring pixels with lower levels of confidence in the confidence map 235 can have lower weights in the neighbor weights 915 than neighboring pixels with higher levels of confidence in the confidence map 235, and can therefore contribute less to the weight sum. Neighboring pixels that are farther from the pixel(s) being upscaled can also have lower weights than neighboring pixels that are closer to the pixel(s) being upscaled, and can therefore contribute less to the weight sum. In other words, neighboring pixels that are closer to the pixel(s) being upscaled can have higher weights than neighboring pixels that are farther from the pixel(s) being upscaled. In operation 930, the category with the maximum weight is used for the upscaled pixel in the upscaled category map 950) based on identifying a maximum confidence value among a plurality of confidence values ([0062], Pixels illustrated in white in the confidence map 235 represent a high confidence level, such as a confidence level exceeding a high threshold percentage, such as 90%. Pixels illustrated in black in the confidence map 235 represent a low confidence level, such as a confidence level falling below a low threshold percentage, such as 10%. The confidence map 235 also includes six different shades of grey (other than black and white), each representing confidence levels falling into different ranges of confidence between the high threshold percentage and the low threshold percentage (~where exceeding 90% is the maximum confidence value, and each shade of black/white/grey is the plurality of confidence values)).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Regarding Claim 2, Trejo, as modified by Pinhasov, teaches the invention of Claim 1 above, where Pinhasov further teaches wherein the confidence weighted metric is based on a maximum confidence value ([0062], Pixels illustrated in white in the confidence map 235 represent a high confidence level, such as a confidence level exceeding a high threshold percentage, such as 90%. Pixels illustrated in black in the confidence map 235 represent a low confidence level, such as a confidence level falling below a low threshold percentage, such as 10%. The confidence map 235 also includes six different shades of grey (other than black and white), each representing confidence levels falling into different ranges of confidence between the high threshold percentage and the low threshold percentage (~where exceeding 90% is the maximum confidence value, and each shade of black/white/grey is the plurality of confidence values)) among each of the one or more second portions of the image ([0117], Neighboring pixels with lower levels of confidence in the confidence map 235 can have lower weights in the neighbor weights 915 than neighboring pixels with higher levels of confidence in the confidence map 235, and can therefore contribute less to the weight sum. Neighboring pixels that are farther from the pixel(s) being upscaled can also have lower weights than neighboring pixels that are closer to the pixel(s) being upscaled, and can therefore contribute less to the weight sum. In other words, neighboring pixels that are closer to the pixel(s) being upscaled can have higher weights than neighboring pixels that are farther from the pixel(s) being upscaled. In operation 930, the category with the maximum weight is used for the upscaled pixel in the upscaled category map 950).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Regarding Claim 3, Trejo, as modified by Pinhasov, teaches the invention of Claim 2 above, where Trejo further teaches wherein the one or more second portions of the image are neighboring portions of the first portion of the image ([0022], The adaptive spatial noise filter is configured to determine a difference between the signal level of the first pixel and the signal level of each of a plurality of neighboring pixels).
Regarding Claim 4, Trejo, as modified by Pinhasov, teaches the invention of Claim 1 above, where Trejo further teaches wherein the confidence weighted metric is further based on a spatial filter function ([0132], Adaptive spatial noise filter 570 uses a square block of pixels, e.g., a plurality of neighboring pixels, around a current pixel to determine how to filter the current pixel. The distance filter filters the current pixel based on weights determined from the Euclidean distance of adjacent pixels in the block from the current pixel. The signal range filter, sometimes called a range filter or an intensity filter, is an adaptive noise signal range filter based on the value of the pixel noise in frame imNoiseY corresponding to the pixel being filtered. Thus, the weights used in the range filter are based not only on differences in signal levels between the adjacent pixels in the block and the current pixel, but also on the noise component of the current pixel. The pixel noise in the pixel noise input frame at the same location as the location of current pixel in the input pixel frame is said to correspond to the current pixel and is referred to as the noise component of the current pixel).
Regarding Claim 5, Trejo, as modified by Pinhasov, teaches the invention of Claim 4 above, where Trejo further teaches wherein the one or more second portions of the image comprise a first neighboring portion and a second neighboring portion, and wherein the spatial filter function has a greater value for the first neighboring portion than for the second neighboring portion, the first portion being closer to the first neighboring portion than to the second neighboring portion ([0132-0134], adaptive spatial noise filter adjusts the weights used in the filter not only based on the Euclidean distance of adjacent pixels from the pixel, but also based on differences in signal levels between the adjacent pixels and the current pixel and based on the noise component of the current pixel. Specifically, as explained more completely below, adaptive spatial noise filter 570 takes a weighted sum of the pixels in a neighborhood of the current pixel. The weights depend on the spatial distance and on the signal levels of the neighboring pixels as well as the noise component of the current pixel itself. Thus, the signal value at each pixel in a frame is replaced by a noise-based weighted average of signal values from nearby pixels, [0075], The percentage of the spatial change of current pixel that is passed through the filter, as an adaptively spatial noise filtered pixel, is weighted based on the spatial relationship of the current pixel to the neighboring pixels; based on the signal level relationship of the current pixel to the signal level of neighboring pixels; and based on the noise component of the current pixel).
Regarding Claim 6, Trejo, as modified by Pinhasov, teaches the invention of Claim 5 above, where Trejo further teaches wherein: the spatial filter function is within 30% of (x2 – x) (y2 – y) / ((x2 – x1) (y2 – y1)), x1 and y1 are the coordinates of the first neighboring portion, x2 and y2 are the coordinates of the second neighboring portion, and x and y are the coordinates of the first portion ([0131], Ĩ(x)=(Distance Filter)*(Range Filter(Noise)), [0150-0152], DISTANCE FILTER process 604 generates a value of the spatial Gaussian function defined above for each pixel in the input block of pixels, i.e., for each pixel in the input block of pixels, and so generates a distance filter value for each location in an n by n block of distance filter values. DISTANCE FILTER process 604 transfers to a RANGE DIFFERENCE process 605, which evaluates I(y)−I(x) where x is the location of the center pixel in the block, I(x) is the value of the center pixel, I(y) is the value of the pixel at location y in the input block of pixels, and y ranges over the locations in the input block of pixels, and generates a range difference for each location in an n by n block of range differences, the adaptive spatial noise filter compares the signal level of the current pixel to the signal levels of a plurality of neighboring pixels and to an estimated pixel noise parameter by dividing the square of the absolute value of the difference between the signal level of the pixel at location y and the signal level of the pixel at location x by the square of signal level range parameter σ(Noise)r. Here, signal level range parameter σ(Noise)r is the estimated pixel noise parameter for this aspect. This comparison is used in outputting the adaptively spatial noise filtered pixel value).
Regarding Claim 7, Trejo, as modified by Pinhasov, teaches the invention of Claim 6 above, where Trejo further teaches wherein the spatial filter function is, for each of the one or more second portions of the image, within 30% of a Gaussian function of coordinate differences between the first portion and the one or more second portions of the image ([0131], Ĩ(x)=(Distance Filter)*(Range Filter(Noise)), [0150-0152], DISTANCE FILTER process 604 generates a value of the spatial Gaussian function defined above for each pixel in the input block of pixels, i.e., for each pixel in the input block of pixels, and so generates a distance filter value for each location in an n by n block of distance filter values. DISTANCE FILTER process 604 transfers to a RANGE DIFFERENCE process 605, which evaluates I(y)−I(x) where x is the location of the center pixel in the block, I(x) is the value of the center pixel, I(y) is the value of the pixel at location y in the input block of pixels, and y ranges over the locations in the input block of pixels, and generates a range difference for each location in an n by n block of range differences, the adaptive spatial noise filter compares the signal level of the current pixel to the signal levels of a plurality of neighboring pixels and to an estimated pixel noise parameter by dividing the square of the absolute value of the difference between the signal level of the pixel at location y and the signal level of the pixel at location x by the square of signal level range parameter σ(Noise)r. Here, signal level range parameter σ(Noise)r is the estimated pixel noise parameter for this aspect. This comparison is used in outputting the adaptively spatial noise filtered pixel value).
Regarding Claim 8, Trejo, as modified by Pinhasov, teaches the invention of Claim 1 above, where Trejo further teaches wherein the one or more second portions of the image comprise a first neighboring portion and a second neighboring portion, and wherein the range filter function has a greater value for the first neighboring portion than for the second neighboring portion, the first portion being closer, in intensity, to the first neighboring portion than to the second neighboring portion ([0132-0134], The distance filter filters the current pixel based on weights determined from the Euclidean distance of adjacent pixels in the block from the current pixel. The signal range filter, sometimes called a range filter or an intensity filter, is an adaptive noise signal range filter based on the value of the pixel noise in frame imNoiseY corresponding to the pixel being filtered. Thus, the weights used in the range filter are based not only on differences in signal levels between the adjacent pixels in the block and the current pixel, but also on the noise component of the current pixel. The pixel noise in the pixel noise input frame at the same location as the location of current pixel in the input pixel frame is said to correspond to the current pixel and is sometimes referred to as the noise component of the current pixel, adaptive spatial noise filter adjusts the weights used in the filter not only based on the Euclidean distance of adjacent pixels from the pixel, but also based on differences in signal levels between the adjacent pixels and the current pixel and based on the noise component of the current pixel. Specifically, as explained more completely below, adaptive spatial noise filter 570 takes a weighted sum of the pixels in a neighborhood of the current pixel. The weights depend on the spatial distance and on the signal levels of the neighboring pixels as well as the noise component of the current pixel itself. Thus, the signal value at each pixel in a frame is replaced by a noise-based weighted average of signal values from nearby pixels, [0075], The percentage of the spatial change of current pixel that is passed through the filter, as an adaptively spatial noise filtered pixel, is weighted based on the spatial relationship of the current pixel to the neighboring pixels; based on the signal level relationship of the current pixel to the signal level of neighboring pixels; and based on the noise component of the current pixel).
Regarding Claim 9, Trejo, as modified by Pinhasov, teaches the invention of Claim 1 above, where Pinhasov further teaches wherein the confidence weighted metric corresponds to a category of the first portion of the image, and wherein the processing of the image is further performed based on the category of the first portion of the image ([0061-0062], The classification engine 220 can generate a category map 230 and a confidence map 235 using the downscaled second copy of the raw image data 215. For example, the classification engine 220 can partition the downscaled second copy of the raw image data 215 into different image regions based on detection of different categories of objects within the different image regions in the downscaled second copy of the raw image data 215. The confidence map 235 identifies a degree of confidence that the classification engine 220 has as to its classification of a given pixel in the category map 230).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Regarding Claim 10, Trejo teaches a system comprising a processing circuit (Figs. 1-2), the processing circuit being configured to: for a first portion of an image, calculate a confidence weighted metric ([0134], for a pixel, the adaptive spatial noise filter adjusts the weights used in the filter not only based on the Euclidean distance of adjacent pixels from the pixel, but also based on differences in signal levels between the adjacent pixels and the current pixel and based on the noise component of the current pixel. Specifically, as explained more completely below, adaptive spatial noise filter 570 takes a weighted sum of the pixels in a neighborhood of the current pixel. The weights depend on the spatial distance and on the signal levels of the neighboring pixels as well as the noise component of the current pixel itself. Thus, the signal value at each pixel in a frame is replaced by a noise-based weighted average of signal values from nearby pixels) based on a plurality of second portions of the image ([0022], adaptive noise filter is configured to determine a difference between the signal level of the first pixel and the signal level of each of a plurality of neighboring pixels. The adaptive spatial noise filter also uses a noise dependent signal level parameter, where the noise dependent signal level parameter is a function of the noise of the first pixel, [0075], When adaptive noise filter 210 is an adaptive spatial noise filter, a value of a current pixel at a location in a current frame of demosiaced and color transformed pixel data is compared with the values of neighboring pixels in the current frame. The percentage of the spatial change of current pixel that is passed through the filter, as an adaptively spatial noise filtered pixel, is weighted based on the spatial relationship of the current pixel to the neighboring pixels) and a range filter function ([0022], the adaptive spatial noise filter includes a distance filter and a signal level range filter. The signal level range filter is configured to filter the pixel based on a difference between a signal level of a pixel and each of the signal levels of the plurality of neighboring pixels and based on the noise dependent signal level parameter); and
process the image based on the confidence weighted metric for the first portion of the image ([0155-0158], COMBINE FILTERS process 607 multiplies a value at a location in the n by n block of the distance filter values by the value at the same location in the n by a block of range filter values for each location in the two blocks, and so generates a combined filter value for each location in a n by n block of combined filter values. COMBINE FILTERS process 607 transfers to NORMALIZE process 608, GENERATE PIXEL process 609 generates the adaptively spatial noise filtered pixel and writes the adaptively spatial noise filtered pixel to output frame of filtered pixel data 614, sometimes referred to as output frame of pixel data or filtered pixel output frame).
Trejo fails to teach the following, which in the same field of endeavor, Pinhasov teaches wherein the weighted metric is a confidence weighted metric ([0062], The confidence map 235 identifies a degree of confidence that the classification engine 220 has as to its classification of a given pixel in the category map 230. The classification engine 220 sends the category map 230 and the confidence map 235 to the ISP 240, [0117], Neighboring pixels with lower levels of confidence in the confidence map 235 can have lower weights in the neighbor weights 915 than neighboring pixels with higher levels of confidence in the confidence map 235, and can therefore contribute less to the weight sum. Neighboring pixels that are farther from the pixel(s) being upscaled can also have lower weights than neighboring pixels that are closer to the pixel(s) being upscaled, and can therefore contribute less to the weight sum. In other words, neighboring pixels that are closer to the pixel(s) being upscaled can have higher weights than neighboring pixels that are farther from the pixel(s) being upscaled. In operation 930, the category with the maximum weight is used for the upscaled pixel in the upscaled category map 950) based on identifying a maximum confidence value among a plurality of confidence values ([0062], Pixels illustrated in white in the confidence map 235 represent a high confidence level, such as a confidence level exceeding a high threshold percentage, such as 90%. Pixels illustrated in black in the confidence map 235 represent a low confidence level, such as a confidence level falling below a low threshold percentage, such as 10%. The confidence map 235 also includes six different shades of grey (other than black and white), each representing confidence levels falling into different ranges of confidence between the high threshold percentage and the low threshold percentage (~where exceeding 90% is the maximum confidence value, and each shade of black/white/grey is the plurality of confidence values)).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Regarding Claim 11, Trejo, as modified by Pinhasov, teaches the invention of Claim 10 above, where Pinhasov further teaches wherein the confidence weighted metric is based on a maximum confidence value ([0062], Pixels illustrated in white in the confidence map 235 represent a high confidence level, such as a confidence level exceeding a high threshold percentage, such as 90%. Pixels illustrated in black in the confidence map 235 represent a low confidence level, such as a confidence level falling below a low threshold percentage, such as 10%. The confidence map 235 also includes six different shades of grey (other than black and white), each representing confidence levels falling into different ranges of confidence between the high threshold percentage and the low threshold percentage (~where exceeding 90% is the maximum confidence value, and each shade of black/white/grey is the plurality of confidence values)) among each of the one or more second portions of the image ([0117], Neighboring pixels with lower levels of confidence in the confidence map 235 can have lower weights in the neighbor weights 915 than neighboring pixels with higher levels of confidence in the confidence map 235, and can therefore contribute less to the weight sum. Neighboring pixels that are farther from the pixel(s) being upscaled can also have lower weights than neighboring pixels that are closer to the pixel(s) being upscaled, and can therefore contribute less to the weight sum. In other words, neighboring pixels that are closer to the pixel(s) being upscaled can have higher weights than neighboring pixels that are farther from the pixel(s) being upscaled. In operation 930, the category with the maximum weight is used for the upscaled pixel in the upscaled category map 950).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Regarding Claim 12, Trejo, as modified by Pinhasov, teaches the invention of Claim 11 above, where Trejo further teaches wherein the one or more second portions of the image are neighboring portions of the first portion of the image ([0022], The adaptive spatial noise filter is configured to determine a difference between the signal level of the first pixel and the signal level of each of a plurality of neighboring pixels).
Regarding Claim 13, Trejo, as modified by Pinhasov, teaches the invention of Claim 10 above, where Trejo further teaches wherein the confidence weighted metric is further based on a spatial filter function ([0132], Adaptive spatial noise filter 570 uses a square block of pixels, e.g., a plurality of neighboring pixels, around a current pixel to determine how to filter the current pixel. The distance filter filters the current pixel based on weights determined from the Euclidean distance of adjacent pixels in the block from the current pixel. The signal range filter, sometimes called a range filter or an intensity filter, is an adaptive noise signal range filter based on the value of the pixel noise in frame imNoiseY corresponding to the pixel being filtered. Thus, the weights used in the range filter are based not only on differences in signal levels between the adjacent pixels in the block and the current pixel, but also on the noise component of the current pixel. The pixel noise in the pixel noise input frame at the same location as the location of current pixel in the input pixel frame is said to correspond to the current pixel and is referred to as the noise component of the current pixel).
Regarding Claim 14, Trejo, as modified by Pinhasov, teaches the invention of Claim 13 above, where Trejo further teaches wherein the one or more second portions of the image comprise a first neighboring portion and a second neighboring portion, and wherein the spatial filter function has a greater value for the first neighboring portion than for the second neighboring portion, the first portion being closer to the first neighboring portion than to the second neighboring portion ([0132-0134], adaptive spatial noise filter adjusts the weights used in the filter not only based on the Euclidean distance of adjacent pixels from the pixel, but also based on differences in signal levels between the adjacent pixels and the current pixel and based on the noise component of the current pixel. Specifically, as explained more completely below, adaptive spatial noise filter 570 takes a weighted sum of the pixels in a neighborhood of the current pixel. The weights depend on the spatial distance and on the signal levels of the neighboring pixels as well as the noise component of the current pixel itself. Thus, the signal value at each pixel in a frame is replaced by a noise-based weighted average of signal values from nearby pixels, [0075], The percentage of the spatial change of current pixel that is passed through the filter, as an adaptively spatial noise filtered pixel, is weighted based on the spatial relationship of the current pixel to the neighboring pixels; based on the signal level relationship of the current pixel to the signal level of neighboring pixels; and based on the noise component of the current pixel).
Regarding Claim 15, Trejo, as modified by Pinhasov, teaches the invention of Claim 14 above, where Trejo further teaches wherein: the spatial filter function is within 30% of (x2 – x) (y2 – y) / ((x2 – x1) (y2 – y1)), x1 and y1 are the coordinates of the first neighboring portion, x2 and y2 are the coordinates of the second neighboring portion, and x and y are the coordinates of the first portion ([0131], Ĩ(x)=(Distance Filter)*(Range Filter(Noise)), [0150-0152], DISTANCE FILTER process 604 generates a value of the spatial Gaussian function defined above for each pixel in the input block of pixels, i.e., for each pixel in the input block of pixels, and so generates a distance filter value for each location in an n by n block of distance filter values. DISTANCE FILTER process 604 transfers to a RANGE DIFFERENCE process 605, which evaluates I(y)−I(x) where x is the location of the center pixel in the block, I(x) is the value of the center pixel, I(y) is the value of the pixel at location y in the input block of pixels, and y ranges over the locations in the input block of pixels, and generates a range difference for each location in an n by n block of range differences, the adaptive spatial noise filter compares the signal level of the current pixel to the signal levels of a plurality of neighboring pixels and to an estimated pixel noise parameter by dividing the square of the absolute value of the difference between the signal level of the pixel at location y and the signal level of the pixel at location x by the square of signal level range parameter σ(Noise)r. Here, signal level range parameter σ(Noise)r is the estimated pixel noise parameter for this aspect. This comparison is used in outputting the adaptively spatial noise filtered pixel value).
Regarding Claim 16, Trejo, as modified by Pinhasov, teaches the invention of Claim 15 above, where Trejo further teaches wherein the spatial filter function is, for each of the one or more second portions of the image, within 30% of a Gaussian function of coordinate differences between the first portion and the one or more second portions of the image ([0131], Ĩ(x)=(Distance Filter)*(Range Filter(Noise)), [0150-0152], DISTANCE FILTER process 604 generates a value of the spatial Gaussian function defined above for each pixel in the input block of pixels, i.e., for each pixel in the input block of pixels, and so generates a distance filter value for each location in an n by n block of distance filter values. DISTANCE FILTER process 604 transfers to a RANGE DIFFERENCE process 605, which evaluates I(y)−I(x) where x is the location of the center pixel in the block, I(x) is the value of the center pixel, I(y) is the value of the pixel at location y in the input block of pixels, and y ranges over the locations in the input block of pixels, and generates a range difference for each location in an n by n block of range differences, the adaptive spatial noise filter compares the signal level of the current pixel to the signal levels of a plurality of neighboring pixels and to an estimated pixel noise parameter by dividing the square of the absolute value of the difference between the signal level of the pixel at location y and the signal level of the pixel at location x by the square of signal level range parameter σ(Noise)r. Here, signal level range parameter σ(Noise)r is the estimated pixel noise parameter for this aspect. This comparison is used in outputting the adaptively spatial noise filtered pixel value).
Regarding Claim 17, Trejo, as modified by Pinhasov, teaches the invention of Claim 10 above, where Trejo further teaches wherein the one or more second portions of the image comprise a first neighboring portion and a second neighboring portion, and wherein the range filter function has a greater value for the first neighboring portion than for the second neighboring portion, the first portion being closer, in intensity, to the first neighboring portion than to the second neighboring portion ([0132-0134], The distance filter filters the current pixel based on weights determined from the Euclidean distance of adjacent pixels in the block from the current pixel. The signal range filter, sometimes called a range filter or an intensity filter, is an adaptive noise signal range filter based on the value of the pixel noise in frame imNoiseY corresponding to the pixel being filtered. Thus, the weights used in the range filter are based not only on differences in signal levels between the adjacent pixels in the block and the current pixel, but also on the noise component of the current pixel. The pixel noise in the pixel noise input frame at the same location as the location of current pixel in the input pixel frame is said to correspond to the current pixel and is sometimes referred to as the noise component of the current pixel, adaptive spatial noise filter adjusts the weights used in the filter not only based on the Euclidean distance of adjacent pixels from the pixel, but also based on differences in signal levels between the adjacent pixels and the current pixel and based on the noise component of the current pixel. Specifically, as explained more completely below, adaptive spatial noise filter 570 takes a weighted sum of the pixels in a neighborhood of the current pixel. The weights depend on the spatial distance and on the signal levels of the neighboring pixels as well as the noise component of the current pixel itself. Thus, the signal value at each pixel in a frame is replaced by a noise-based weighted average of signal values from nearby pixels, [0075], The percentage of the spatial change of current pixel that is passed through the filter, as an adaptively spatial noise filtered pixel, is weighted based on the spatial relationship of the current pixel to the neighboring pixels; based on the signal level relationship of the current pixel to the signal level of neighboring pixels; and based on the noise component of the current pixel).
Regarding Claim 18, Trejo, as modified by Pinhasov, teaches the invention of Claim 10 above, where Pinhasov further teaches wherein the confidence weighted metric corresponds to a category of the first portion of the image, and wherein the processing circuit is further configured to process the image based on the category of the first portion of the image ([0061-0062], The classification engine 220 can generate a category map 230 and a confidence map 235 using the downscaled second copy of the raw image data 215. For example, the classification engine 220 can partition the downscaled second copy of the raw image data 215 into different image regions based on detection of different categories of objects within the different image regions in the downscaled second copy of the raw image data 215. The confidence map 235 identifies a degree of confidence that the classification engine 220 has as to its classification of a given pixel in the category map 230).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Regarding Claim 19, Trejo teaches a system comprising means for processing, the means for processing being configured to: for a first portion of an image, calculate a confidence weighted metric ([0134], for a pixel, the adaptive spatial noise filter adjusts the weights used in the filter not only based on the Euclidean distance of adjacent pixels from the pixel, but also based on differences in signal levels between the adjacent pixels and the current pixel and based on the noise component of the current pixel. Specifically, as explained more completely below, adaptive spatial noise filter 570 takes a weighted sum of the pixels in a neighborhood of the current pixel. The weights depend on the spatial distance and on the signal levels of the neighboring pixels as well as the noise component of the current pixel itself. Thus, the signal value at each pixel in a frame is replaced by a noise-based weighted average of signal values from nearby pixels) based on a plurality of second portions of the image ([0022], adaptive noise filter is configured to determine a difference between the signal level of the first pixel and the signal level of each of a plurality of neighboring pixels. The adaptive spatial noise filter also uses a noise dependent signal level parameter, where the noise dependent signal level parameter is a function of the noise of the first pixel, [0075], When adaptive noise filter 210 is an adaptive spatial noise filter, a value of a current pixel at a location in a current frame of demosiaced and color transformed pixel data is compared with the values of neighboring pixels in the current frame. The percentage of the spatial change of current pixel that is passed through the filter, as an adaptively spatial noise filtered pixel, is weighted based on the spatial relationship of the current pixel to the neighboring pixels) and a range filter function ([0022], the adaptive spatial noise filter includes a distance filter and a signal level range filter. The signal level range filter is configured to filter the pixel based on a difference between a signal level of a pixel and each of the signal levels of the plurality of neighboring pixels and based on the noise dependent signal level parameter); and
process the image based on the confidence weighted metric for the first portion of the image ([0155-0158], COMBINE FILTERS process 607 multiplies a value at a location in the n by n block of the distance filter values by the value at the same location in the n by a block of range filter values for each location in the two blocks, and so generates a combined filter value for each location in a n by n block of combined filter values. COMBINE FILTERS process 607 transfers to NORMALIZE process 608, GENERATE PIXEL process 609 generates the adaptively spatial noise filtered pixel and writes the adaptively spatial noise filtered pixel to output frame of filtered pixel data 614, sometimes referred to as output frame of pixel data or filtered pixel output frame).
Trejo fails to teach the following, which in the same field of endeavor, Pinhasov teaches wherein the weighted metric is a confidence weighted metric ([0062], The confidence map 235 identifies a degree of confidence that the classification engine 220 has as to its classification of a given pixel in the category map 230. The classification engine 220 sends the category map 230 and the confidence map 235 to the ISP 240, [0117], Neighboring pixels with lower levels of confidence in the confidence map 235 can have lower weights in the neighbor weights 915 than neighboring pixels with higher levels of confidence in the confidence map 235, and can therefore contribute less to the weight sum. Neighboring pixels that are farther from the pixel(s) being upscaled can also have lower weights than neighboring pixels that are closer to the pixel(s) being upscaled, and can therefore contribute less to the weight sum. In other words, neighboring pixels that are closer to the pixel(s) being upscaled can have higher weights than neighboring pixels that are farther from the pixel(s) being upscaled. In operation 930, the category with the maximum weight is used for the upscaled pixel in the upscaled category map 950) based on identifying a maximum confidence value among a plurality of confidence values ([0062], Pixels illustrated in white in the confidence map 235 represent a high confidence level, such as a confidence level exceeding a high threshold percentage, such as 90%. Pixels illustrated in black in the confidence map 235 represent a low confidence level, such as a confidence level falling below a low threshold percentage, such as 10%. The confidence map 235 also includes six different shades of grey (other than black and white), each representing confidence levels falling into different ranges of confidence between the high threshold percentage and the low threshold percentage (~where exceeding 90% is the maximum confidence value, and each shade of black/white/grey is the plurality of confidence values)).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Regarding Claim 20, Trejo, as modified by Pinhasov, teaches the invention of Claim 19 above, where Pinhasov further teaches wherein the confidence weighted metric is based on a maximum confidence value ([0062], Pixels illustrated in white in the confidence map 235 represent a high confidence level, such as a confidence level exceeding a high threshold percentage, such as 90%. Pixels illustrated in black in the confidence map 235 represent a low confidence level, such as a confidence level falling below a low threshold percentage, such as 10%. The confidence map 235 also includes six different shades of grey (other than black and white), each representing confidence levels falling into different ranges of confidence between the high threshold percentage and the low threshold percentage (~where exceeding 90% is the maximum confidence value, and each shade of black/white/grey is the plurality of confidence values)) among each of the one or more second portions of the image ([0117], Neighboring pixels with lower levels of confidence in the confidence map 235 can have lower weights in the neighbor weights 915 than neighboring pixels with higher levels of confidence in the confidence map 235, and can therefore contribute less to the weight sum. Neighboring pixels that are farther from the pixel(s) being upscaled can also have lower weights than neighboring pixels that are closer to the pixel(s) being upscaled, and can therefore contribute less to the weight sum. In other words, neighboring pixels that are closer to the pixel(s) being upscaled can have higher weights than neighboring pixels that are farther from the pixel(s) being upscaled. In operation 930, the category with the maximum weight is used for the upscaled pixel in the upscaled category map 950).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of a category map and a confidence score, as taught in Pinhasov, in the weighted metrics of Trejo, in order to better apply different image processing technologies to different portions of the image, enhancing overall accuracy.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Smirnov et al (US 2022/0044372) teaches In some embodiments, the confidence values of the pixels are used to determine how the image fusion circuit (e.g., the image fusion processor 434 illustrated in FIG. 4) fuses images. The image fusion circuit uses the confidence values to assign a weight to each pixel when performing image fusion. For example, when fusing a first pixel from an image corresponding to several other images fused together, and a second pixel from an image that has not been fused with any other images, the first pixel may be assigned a greater weight relative to the second pixel, as it already reflects the pixel data of multiple other images. The confidence value of the pixels may also be used to determine an amount of noise reduction to be performed on the pixels of the image (e.g., by the noise reduction circuit 442 illustrated in FIG. 4). For example, a higher confidence value indicates a lower standard deviation, and as such less noise reduction is needed to be applied to the pixels of the image. ([0082])
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARGARET G WEBB whose telephone number is (571)270-7803. The examiner can normally be reached M-F 9:00-6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Appiah can be reached at (571) 272-7904. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARGARET G WEBB/Primary Examiner, Art Unit 2641
Prosecution Insights