DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
2. This communication is in response to the application filled on 11/28/2024.
3. Claims 1-14 are pending.
4. Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Information Disclosure Statement
5. The information disclosure statements (IDS) submitted on 01/15/2025 and 02/03/2026 have been considered by the examiner.
Specification
6. The abstract of the disclosure is objected to because it contains an erroneous reference to Figure 2 with no context. See ln. 15. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
7. The disclosure is objected to because of the following informalities:
In pg. 4, ln. 23-24 recites “…two methods 10a, 10b, 10c…”, consider correcting to “…three methods 10a, 10b, and 10c…”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1-4, 6-12, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over “Realization of the Contrast Limited Adaptive Histogram Equalization (CLAHE) for Real-Time Image Enhancement” to Reza (hereinafter Reza), and further in view of “A General-Purpose Dehazing Algorithm based on Local Contrast Enhancement Approaches” to Sun et al. (hereinafter Sun).
10. Regarding Claim 1, Reza discloses a method for processing an image comprising image data, the image data comprising pixel intensity values, said pixel intensity values being associated with respective pixel locations, the method comprising ([pg. 35, col. 2, par. 1, ln. 1-11] “The main idea in adaptive histogram equalization is to find the mapping for each pixel based on its local (neighborhood) grayscale distribution. In this method which independently developed in [5, 6], and [7], the contrast enhancement mapping applied to a particular pixel is a function of the intensity values immediately surrounding the pixel. The number of times that this calculation should be repeated is the same as the number of pixels in the image. This rises to an extensive computation requirement, which even with some modification, cannot be utilized for real-time image enhancement.”, [pg. 36, col. 1, par. 3, ln. 17] “In some cases, when grayscale distribution is highly localized, it might not be desirable to transform very low-contrast images by full histogram equalization. In these cases, the mapping curve may include segments with high slopes, meaning that two very close grayscales might be mapped to significantly different grayscales. This issue is resolved by limiting the contrast that is allowed through histogram equalization. Combination of this contrast limiting approach with the aforementioned adaptive histogram equalization results in what is referred to as Contrast Limited Adaptive Histogram Equalization (CLAHE) discussed in[1] and summarized in [2]. Complete formulation of this approach is briefly reviewed in the next section. How ever, before starting the next section we shall provide a brief review of some of implementation issues related to adaptive histogram equalization”, [pg. 36, Figure 1], [pg. 38, Figure 2-4]):
for a plurality of zones of the image ([pg. 36, Figure 1], [pg. 36, col. 2, 2. Contrast Limited Adaptive Histogram Equalization (CLAHE), par. 1, ln. 1-16] “Contrast limited adaptive histogram equalization has produced good results on medical images. This method is formulated based on dividing the image to several non-overlapping regions of almost equal sizes. For 512 × 512 images, to achieve good statistical estimation, the number of regions is generally selected to be equal to 64 by equally dividing the image by 8 in each direction. One example of such division is shown in Fig. 1. This partition results in three different groups of regions. One group, which consists only of four regions, is the class of corner regions (CR). The second group, which consists of 24 regions, is the class of boarder regions (BR). All regions on the image boarder, excluding the corner regions, belong to this class. The last group, which consists of all the remaining 36 regions, is called the class of inner regions (IR).”), determining, based on a plurality of pixel intensity values in the respective zone of the image, a value of a characteristic pertaining to the plurality of pixel intensity values ([pg. 36, col. 2, 2. Contrast Limited Adaptive Histogram Equalization (CLAHE), par. 2, ln. 1 to pg. 37, col. 2, par. 2, ln. 14] “In this approach, first, histogram of each region is calculated. Then, based on a desired limit for contrast expansion, a clip limit for clipping histograms is obtained. Next, each histogram is redistributed in such a way that its height does not go beyond the clip limit. Finally, cumulative distribution functions, CDF [4], of the resultant contrast limited histograms are determined for grayscale mapping. In the CLAHE technique, pixels are mapped by linearly combining the results from the mappings of the four nearest regions… Calculation of histogram for each region is straight forward. In this case, for each grayscale, number of pixels with that grayscale in the region is counted… This function is in general a rough estimate of the grayscale density function. Histogram equalization is obtained by using an estimate of the CDF. If numbers of pixels and grayscales, in each region, are respectively M and N, and if
h
i
,
j
(
n
)
, for
n
=
0,1
,
2
,
…
,
N
-
1
, is the histogram of
(
i
,
j
)
region, then an estimate of the corresponding CDF, properly scaled by
(
N
-
1
)
for greyscale mapping, is
f
i
,
j
n
=
(
N
-
1
)
M
∙
∑
k
=
0
n
h
i
,
j
k
;
n
=
1,2
,
3
,
…
,
N
-
1
(1) This function can be used to convert the given grayscale density function, approximately, to a uniform density function. This procedure is referred to as histogram equalization. The problem with this approach is that the region contrast is increased to its maximum. In order to limit the contrast to a desired level, the maximum slope of (1) is limited to a desired maximum slope. One approach in limiting the maximum slope is to use a clip limit
β
to clip all histograms… Modification of the original histogram, based on the desired limit in change of image contrast, proceeds by limiting the maximum number of counts, for each grayscale, to
β
… For each region, the grayscale mapping is obtained by using (1) on its modified histogram.”);
performing at least a spatial filtering process on data representative of the values of the characteristic for the plurality of zones, to obtain filtered values of an image characteristic at respective locations ([pg. 38, Figure 2-4], [pg. 37, col. 2, 2.2. Combination of Mapping Functions, par. 1, ln. 1 to pg. 39, col. 1, par. 2, ln. 13] “For regions in IR group each quadrant of the region is mapped based on the mappings of its four nearest neighboring regions. For example, with reference to Fig. 2(a), a given pixel in quadrant 1 of
(
i
,
j
)
region is mapped based on its vertical and horizontal distances from centers of
i
,
j
,
i
,
j
-
1
,
i
-
1
,
j
and
(
i
-
1
,
j
-
1
)
regions. These distances are shown in Fig.2(b). If the mapping function for pixels in
(
i
,
j
)
region is represented by
f
i
,
j
(
∙
)
, then the new value of pixel p in the first quadrant of
(
i
,
j
)
region, based on the specified distances is
p
n
e
w
=
s
r
+
s
y
x
+
y
f
i
-
1
,
j
-
1
p
o
l
d
+
x
x
+
y
f
i
,
j
-
1
(
p
o
l
d
)
+
r
r
+
s
(
y
x
+
y
f
i
-
1
,
j
p
o
l
d
+
x
x
+
y
f
i
,
j
(
p
o
l
d
)
)
(3)… Similar mappings can be derived for pixels in other quadrants of
i
,
j
region. For regions in BR group, the neighborhood structure is different. One such case is shown in Fig. 3(a). In this case, the neighborhood structure for pixels in quadrants 1 or 3 is the same as that of regions in the IR group. However, the neighborhood structure for pixels in quadrant 2 or 4 is different. For example, the structure for quadrant 2 is shown in Fig. 3(b). In this case, the new grayscale for pixel p is obtained by
p
n
e
w
=
s
r
+
s
f
i
,
j
-
1
p
o
l
d
+
r
r
+
s
f
i
,
j
(
p
o
l
d
)
(4)… Similar mapping can be derived for pixels in quad rant 4 of
i
,
j
region. Note that the
i
,
j
region shown in Fig. 3(a) is from right hand side boarder of the image. All right hand side regions in BR group are similar. The regions in the BR group that are from left hand side, top, and bottom boarders behave similarly. In those cases, the two quadrants, which require mapping similar to (4), are not the same as those given in Fig. 3(a). For regions in CR group, different quadrants have different characteristics. One typical region in that group, the top-left corner, is depicted in Fig. 4. With reference to the discussions on IR and BR groups, quadrant 4 has neighborhood structure similar to those of IR regions and quadrants 2 and 3 have neighborhood structures similar to the two side quadrants of BR regions. Quadrant 1, in this group, is unique and has no contact with other regions. Mapping function for pixels in this quadrant is the same as the regional mapping with no consideration of other regions. In this case, the mapping would be
p
n
e
w
=
f
i
,
j
(
p
o
l
d
)
. The other three CR regions behave in a similar way. In summary, histogram of each region is calculated and then modified by using Algorithm 1 based on the desired clip factor. Grayscale mapping of each region is obtained by applying (1) to the modified histogram of that region. The final image is built by mapping each pixel based on the Contrast Limited Adaptive Histogram Equalized mappings of regions whose centers are in the four nearest, two nearest, or one nearest neighbor(s) of that pixel. The way that these mappings are combined depends on the vertical and horizontal distances of the pixel from those nearest centers. The equations for these combinations are similar or the same as Eqs. (3), (4), and (5).”); and
interpolating from the filtered values to determine a local value of the image characteristic at a said pixel location ([pg. 36, col. 1, par. 2, ln. 1-11] “An alternative to the fully adaptive algorithm is to approximate the mapping required for each pixel by choosing a small number of contextual regions within the image and calculating the mapping for a given pixel as a bilinear interpolation of the mappings derived from nearby contextual regions. In other words, the mapping for a pixel is obtained by using a weighted-sum of the mappings of its four nearest regions. The weights are calculated based on the proximity of the pixel to the centers of the four nearest regions. A full description of this method may be found in [7] and [8].”, [pg. 38, Figure 2-4], [pg. 37, col. 2, 2.2. Combination of Mapping Functions, par. 1, ln. 1 to pg. 39, col. 1, par. 2, ln. 13]),
wherein the determining and/or the interpolating is performed using fixed function circuitry ([pg. 39, col. 1, 3. A Hardware Realization System, par. 1, ln. 1 to col. 2, par. 2, ln. 23] “Application of CLAHE algorithm on a single video frame demands extensive computation time. Approximate computation time for a n-bit
N
×
N
image frame, which is divided into
N
64
×
[
N
64
]
regions of size
64
×
64
, on a general-purpose computer is estimated at about
(
2
n
-
9
+
2
7
)
∙
N
2
processing clock cycles. For a 16-bit
512
×
512
image this amounts to 64 Mega cycles. For an 8-bit
512
×
512
this value is reduces to its lower limit of 32 Mega cycles. If we assume that a dedicated special purpose computer is solely working on CLAHE algorithm for real-time processing, the processor needs to work at least at the rate of 2 GHz. Although this may appear that it is not out of reach with today’s technology but it is pushing the limit and will be very costly. Our objective in this hardware implementation is to use available low cost FPGA technology (or VLSI implementation for high volume production) for application of CLAHE algorithm on real-time image sequences (videos). The application in mind is a real time medical X-ray imaging system, which uses continuous low-level exposure video until the region of interest is identified for the final high-level exposure still imaging. The proposed architecture will meet our objectives due to the fact that we can use combination of parallel processing along with pipelining to achieve significant efficiency in calculation time. Also application of look up tables and use of local memory results in efficient operation of computational tasks like division and multiplication. Advantages of the proposed approach over general-purpose processor, multiprocessor, or parallel implementation is that the end product of the FPGA or VLSI design is on a single chip and is suitable for special purpose medical imaging system without significantly affecting the price of the system for this added functionality. In the proposed design it is also desirable to limit the intermediate memory requirement so that they can all be utilized on the same VLSI or FPGA chip. Whenever possible, independent calculations are carried out in parallel and serial calculations are done in a pipelined structure. The objective is to minimize the overall latency for this operation. In the following presentation, a bottom-up approach is used in the sense that individual processing or computational units are first discussed and then their integration is presented. For simplicity of the discussion and better presentation of the main idea it is assumed that the image is 512 × 512 and it is divided into 8×8 (64) square regions of size 64 × 64. For other sizes, similar general approach will be applicable and only implementation details are different. For example for different image sizes the size of regions on the borders (on the right border and bottom border) might be different than 64×64 and also the total number of regions would be different. The boarder regions are allowed to be smaller or larger than 64×64 such that to avoid impractical small size regions. For each region we only use the existing pixels in that region to calculate and properly normalize the corresponding histogram. For the clarity of this presentation, details of the control unit and special treatment of boarder regions with different sizes are not discussed and left to be dealt with during the final hardware implementation. Also many issues are left flexible for the hardware designer to adjust and finalize at final stages of the implementation.”, [pg. 40, Figure 5], [pg. 41, Fig. 6], [pg. 42, Fig. 7 and 8], [pg. 42, Fig. 9 and 10]), {and the spatial filtering process is performed using a programmable processor}.
Reza does not specifically disclose wherein the spatial filtering process is performed by a programable processor, though the examiner notes that general-purpose programmable processors/computers are disclosed in Reza ([pg. 39, col. 1, 3. A Hardware Realization System, par. 1, ln. 1 to col. 2, par. 2, ln. 23]).
However, Sun specifically discloses wherein a spatial filtering processes is performed using a programmable processor ([pg. 1, Abstract, par. 1, ln. 1-17] “Dehazing is in the image processing and computer vision communities, the task of enhancing the image taken in foggy conditions. To better understand this type of algorithm, we present in this document a dehazing method which is suitable for several local contrast adjustment algorithms. We base it on two filters. The first filter is built with a step of normalization with some other statistical tricks while the last represents the local contrast improvement algorithm. Thus, it can work on both CPU and GPU for real-time applications. We hope that our approach will open the door to new ideas in the community. Other advantages of our method are first that it does not need to be trained, then it does not need additional optimization processing. Furthermore, it can be used as a pre-treatment or post-processing step in many vision tasks. In addition, that it does not need to convert the problem into a physical interpretation, and finally that it is very fast. This family of defogging algorithms is fairly simple, but it shows promising results compared to state-of-the-art algorithms based not only on a visual assessment but also on objective criteria.”, [pg. 5, 3 Proposed Approach, par, 1, ln. 1 to pg. 6, par. 3, ln. 4] “Dehazing problems can be solved with various techniques. One common way of doing this is to use Koschmieder physical model:
I
x
=
J
x
t
x
+
A
1
-
t
x
(
1
)
Where x is a pixel (single-pixel or not). The hazy image is the sum of the scene’s radiance J(x) and the atmospheric light A, weighted by a transmission factor t(x). The A is the airlight scattered by an object located at infinity with respect to the observer. Here, we use another way to solve the problem. Our method relies on local contrast improvement techniques, it does not require any post-processing or any optimization procedure to function. Our dehazing algorithm has been tested on some local contrast enhancement scheme, and it clearly shows satisfying results. Thus, it represents a family of dehazing algorithms that deals with local contrast filtering algorithms such as [15], [35] ace [13], [41], stress [22] or clahe [48]. It is a very fast algorithm when the local contrast subroutine is also fast. It can enhance both dense and non dense images taken in hazy or foggy conditions. However, it appears to be more suitable for non-dense conditions. This simple and novel algorithm is composed of two steps of filtering. The front-end filter that uses a simple normalization step combined with a statistical trick, and it tends to darken and homogenize the haze density over the entire image. Let xi denote a given sample in the image, xmin and xmax are respectively the minimum and the maximum sample in the entire image. The initial filter is computed in two stages as follows:
f
i
n
i
t
x
i
=
x
i
-
x
m
i
n
x
m
a
x
-
x
m
i
n
(2) Before applying the second filter, we further extend the previous filter to the more general filters as the following:
f
g
x
i
=
f
i
n
i
t
x
i
-
λ
(
x
i
)
|
|
f
i
n
i
t
x
i
|
|
(3) Where the function λ is either a constant ∈ [0,1] or any other specified function. In our settings, we have tested λ = 0.35 and the inverted intensity function of the first filter, that is:
λ
x
i
=
1
-
f
i
n
i
t
x
i
(4) One can notice here that the inverted intensity function is also ∈ [0,1]. . represents the distance of a given pixel over the three chromatic channels. We have only investigated Euclidean norm in this paper, but the other norm may be interesting also. We experimentally notice that when Equation 4 is used in Equation 8, then it can serve as a dehazing algorithm directly. Because its appearance might look too dark for some images, the contrast enhancement techniques or a gamma correction algorithm may be necessary to have the final output. Empirically, we notice that the general form of Equation 2 is as the following:
f
i
n
i
t
x
i
=
α
x
i
-
x
m
i
n
x
m
a
x
-
x
m
i
n
+
β
(5) α and β are two real numbers. In the experiment presented here, we have set α =1.0 and β =0.0. The back-end and final filter is an image local contrast enhancement scheme that takes the output of the front-end filter and it can be expressed as follows:
f
f
i
n
a
l
x
i
=
f
L
C
E
(
f
g
(
x
i
)
)
(6) Where
f
L
C
E
represents a local contrast enhancement algorithm such as clahe. We have empirically checked that
f
L
C
E
functions share a local contrast property (see supplementary documents for more details on this). One can notice that the initial and the final filter algorithms work on global and local contrast schemes respectively, and the algorithm does not require an explicit segmentation as well. This family of algorithm can appear to be useful even in some challenging cases such as dense haze removal problem. We have use two main settings in our research. The first one (λ is fixed) works with homogeneous haze. The second (λ is dynamic) not only fits homogeneous haze, but meets also non-homogeneous haze. This stage dehazing can be further augmented with a haze physical constraint procedure to tackle white balancing issue by using soft matting [26], [20] to refine the transmission in the model. We refer the reader to the supplementary materials for more on this.”, [pg. 14, par. 1, ln. 1-10] “In the main paper, we define two filters that help us to carry out the dehazing task in our procedure. The first filter is defined as follows:
f
i
n
i
t
x
i
=
x
i
-
x
m
i
n
x
m
a
x
-
x
m
i
n
” (7)… We then further extend this filter to the more general filter as the following
f
i
n
i
t
x
i
-
λ
(
x
i
)
|
|
f
i
n
i
t
x
i
|
|
(8) The back end and final filter is an image local contrast enhancement scheme that takes the output of the front-end filter and it can be expressed as follows:
f
f
i
n
a
l
x
i
=
f
L
C
E
(
f
g
(
x
i
)
)
(9) Consequently, our solution consists, roughly speaking, in applying a normalization scheme to the entire image by blackening pixels, then using a technique of enhancement of the low-contrast generated to improve the image in order to obtain the final haze-free image”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would specifically recognize Reza and Sun as within the same field of image contrast enhancement using CLAHE, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, in that by incorporating a general-purpose spatial filtering algorithm performed by a programmable processor as taught in Sun, you effectively expand the number of tasks that the contrast improvement algorithms such as CLAHE may be able to improve ([pg. 1, Abstract, par. 1, ln. 1-17]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Reza with the general-purpose spatial filtering on a programmable processor in Sun through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would have applied the general-purpose spatial filtering on a programmable processor of Sun such that the spatial filtering as performed in Reza is performed using the filtered values as taught in equation (6) of Sun.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 1.
11. Regarding Claim 2, a combination of Reza and Sun teaches the method of claim 1. Reza discloses wherein the fixed function circuitry comprises hardware circuitry to form part of an integrated circuit ([pg. 39, col. 1, 3. A Hardware Realization System, par. 1, ln. 1 to col. 2, par. 2, ln. 23], [pg. 40, Figure 5], [pg. 41, Fig. 6], [pg. 42, Fig. 7 and 8], [pg. 42, Fig. 9 and 10]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 2.
12. Regarding Claim 3, a combination of Reza and Sun teaches the method of claim 1. Reza discloses wherein the method is performed by a plurality of instructions forming an instruction set ([pg. 40, Algorithm 1], [pg. 41, Algorithm 2]), and wherein the method may be performed on a general-purpose computer ([pg. 39, col. 1, 3. A Hardware Realization System, par. 1, ln. 1 to col. 2, par. 2, ln. 23]), but Reza does not specifically disclose the programmable processor is configured to decode the instruction, since Reza is designed as a fixed hardware configuration as opposed to a programable processor.
However, Sun discloses general-purpose spatial filtering on a programmable processor ([pg. 1, Abstract, par. 1, ln. 1-17], [pg. 5, 3 Proposed Approach, par, 1, ln. 1 to pg. 6, par. 3, ln. 4]). The motivation to combine remains analogous to claim 1. One of ordinary skill in the art, in combining the programmable processor of Sun with the method of Reza, would recognize that Sun would be implemented using an analogous plurality of instructions forming an instruction set as disclosed in Reza. Specifically, one of ordinary skill in the art, before the effective filing date of the claimed invention, would recognize the CPU and GPU of Sun to be programable processors, and that such programmable processors necessarily involve decoding a plurality of instructions from an instruction set. In the case of Sun, this instruction set would be in a programing language/code implemented analogous to pseudo-code as provided in Reza. Specifically, all widely used programming languages (e.g., Java, Python, C, JavaScript, etc.) effectively involve decoding of the language into assembly code, followed by decoding of the assembly into binary, which is subsequently executed on the programmable processor. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Reza with the general-purpose spatial filtering on a programmable processor in Sun through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have implemented the filtering on the programable processor of Sun analogous to the pseudo-code plurality of instructions forming an instruction set as taught in Reza using a programing language which would be decoded at the processor during runtime.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 3
13. Regarding Claim 4, a combination of Reza and Sun teaches the method of claim 1. Reza discloses wherein performing the spatial filtering process comprises: reading, {by the processor}, from a memory ([pg. 42, Fig. 10], see RAM), a {programmable} algorithm; and executing, {by the processor}, the {programmable} algorithm using the data representative of the values of the characteristic pertaining to the pluralities of pixel intensity values, to obtain the filtered values ([pg. 39, col. 1, 3. A Hardware Realization System, par. 1, ln. 1 to col. 2, par. 2, ln. 23], [pg. 40, Figure 5], [pg. 41, Fig. 6], [pg. 42, Fig. 7 and 8], [pg. 42, Fig. 9 and 10]). Reza does not specifically disclose a programable processor, and likewise, does not specifically disclose a programable algorithm, though one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize the pseudo-code of Reza to be analogous to a programable algorithm implementation.
However, Sun specifically discloses a programable processor. Arguments analogous to claim 3 are likewise applicable to claim 4 with regard to the programmable algorithm in view of the programmable processor of Sun. The motivation to combine remains analogous to claim 1 and 3. One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Reza with the general-purpose spatial filtering on a programmable processor in Sun through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have implemented the filtering on the programable processor of Sun analogous to the pseudo-code as taught in Reza using a programing language, which would have rendered the filtering as a “programable algorithmic” (i.e., implementing the pseudo-code filtering as a programmable algorithm).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 4.
14. Regarding Claim 6, a combination of Reza and Sun teaches the method of claim 1. Reza discloses interpolating from the local values to determine local values of the image characteristic at a plurality of said pixel locations ([pg. 36, col. 1, par. 2, ln. 1-11], [pg. 38, Figure 2-4], [pg. 37, col. 2, 2.2. Combination of Mapping Functions, par. 1, ln. 1 to pg. 39, col. 1, par. 2, ln. 13]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 6.
15. Regarding Claim 7, a combination of Reza and Sun teaches the method of claim 1. Reza discloses wherein the interpolating comprises bilinear interpolation using four of the filtered values ([pg. 36, col. 1, par. 2, ln. 1-11], [pg. 38, Figure 2-4], [pg. 37, col. 2, 2.2. Combination of Mapping Functions, par. 1, ln. 1 to pg. 39, col. 1, par. 2, ln. 13]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 7.
16. Regarding Claim 8, a combination of Reza and Sun teaches the method of claim 1. Reza discloses wherein the pluralities of pixel intensity values in the respective zones are disjoint sets of pixel intensity values ([pg. 36, Figure 1], [pg. 36, col. 2, 2. Contrast Limited Adaptive Histogram Equalization (CLAHE), par. 1, ln. 1-16], [pg. 36, col. 2, 2. Contrast Limited Adaptive Histogram Equalization (CLAHE), par. 2, ln. 1 to pg. 37, col. 2, par. 2, ln. 14]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 8.
17. Regarding Claim 9, a combination of Reza and Sun teaches the method of claim 1. Reza further discloses wherein determining the values of the characteristic pertaining to the pluralities of pixel intensity values comprises: receiving the pixel intensity values in a given sequence; and processing the pixel intensity values in the given sequence to determine the values of the characteristic ([pg. 39, col. 2, 3.1. Calculation of the Clipped Histogram, par. 1, ln. 1 to pg. 40, col. 2, par. 1, ln. 8] “It is assumed that calculations of histograms and their original clipping, for at least eight regions in a row, are computed in parallel. One engine for redistribution of excess counts in the clipped histograms and estimation of the regional gray scale mapping functions, if properly optimized, might be sufficient for all eight regions in a given row. As pixels from each row of the image are accessed, each engine (dedicated to a given region), simultaneously calculates the regional histogram, limits the bin-counts to β (the given clip limit), and finds the excess counts. Pseudo codes for this operation is shown in Algorithm 2 and block diagram of its hardware realization is shown in Fig. 5.”, [pg. 40, Algorithm 1], [pg. 41, Algorithm 2]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 9.
18. Regarding Claim 10, a combination of Reza and Sun teaches the method of claim 1. Reza discloses wherein the image characteristic comprises contrast. ([pg. 36, col. 2, 2. Contrast Limited Adaptive Histogram Equalization (CLAHE), par. 2, ln. 1 to pg. 37, col. 2, par. 2, ln. 14] “…the problem with this approach is that the region contrast is increased to its maximum. In order to limit the contrast to a desired level, the maximum slope of (1) is limited to a desired maximum slope. One approach in limiting the maximum slope is to use a clip limit
β
to clip all histograms. This clip limit can be related to what is referred to as clip factor,
α
in percent, as follows
β
=
M
N
(
1
+
α
100
s
m
a
x
-
1
)
(2) In this case, for clip factor of zero percent, α = 0, the clip limit becomes exactly equal to (M/N), which results into an identity mapping by evenly distributing all regional pixels into all possible grayscales. No change in pixel values will occur in this case. The maximum clip limit, achieved for α = 100, will go to the maximum of
(
s
m
a
x
∙
M
N
)
. This means, the maximum allowable slope is
s
m
a
x
. Normally
s
m
a
x
is set to four for still X-ray images. However, for any other application, it is recommended to obtain a good choice for
s
m
a
x
by experiment. As clip factor α is changing between zero to hundred, the maximum slope, in each mapping, is changing between one to
s
m
a
x
…”). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 10.
19. Regarding Claim 11, a combination of Reza and Sun teaches the method of claim 10. Reza further discloses applying contrast stretching to a target pixel intensity value based on the local value of the contrast at the said pixel location ([pg. 36, col. 2, 2. Contrast Limited Adaptive Histogram Equalization (CLAHE), par. 2, ln. 1 to pg. 37, col. 2, par. 2, ln. 14]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 11.
20. Regarding Claim 12, a combination of Reza and Sun teaches the method of claim 1. Reza further discloses wherein both the determining and the interpolating are performed using fixed function circuitry ([pg. 39, col. 1, 3. A Hardware Realization System, par. 1, ln. 1 to col. 2, par. 2, ln. 23], [pg. 40, Figure 5], [pg. 41, Fig. 6], [pg. 42, Fig. 7 and 8], [pg. 42, Fig. 9 and 10]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 12.
21. Regarding Claim 14, the claim language is analogous to claim 1 with the exception of “A system comprising fixed function circuitry and a programmable processor, the system being configured to perform a method”. Reza teaches a system comprising fixed function circuitry configured to perform the method ([pg. 35, Abstract, par. 1, ln. 8-9] “In this paper, a system level realization of CLAHE is proposed, which is suitable for VLSI or FPGA implementation.”), but does not specifically disclose a programable processor performing the filtering. However, Sun discloses a programmable processor to perform filtering analogous to claim 1 ([pg. 1, Abstract, par. 1, ln. 1-17], [pg. 5, 3 Proposed Approach, par, 1, ln. 1 to pg. 6, par. 3, ln. 4]). The motivation to combine remains analogous to claim 1. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the system of Reza with the general-purpose spatial filtering on a programmable processor of Sun to obtain the invention as specified in claim 14.
22. Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over “Realization of the Contrast Limited Adaptive Histogram Equalization (CLAHE) for Real-Time Image Enhancement” to Reza, in view of “A General-Purpose Dehazing Algorithm based on Local Contrast Enhancement Approaches” to Sun, and further in view of U.S. Publication No. 2021/0073957 to Slabaugh et al. (hereinafter Slabaugh).
23. Regarding Claim 5, a combination of Reza and Sun teaches the method of claim 1. Reza and Sun do not specifically disclose wherein the processor comprises a neural processing unit, and performing the spatial filtering process comprises performing, by the neural processing unit: reading, from a memory, data representing a neural network; and applying the neural network to the values of the characteristic pertaining to the pluralities of pixel intensity values, to obtain the filtered values.
However, Slabaugh specifically discloses wherein the processor comprises a neural processing unit ([par. 0055, ln. 1-16] “The denoising stage can be implemented as a convolutional neural network (CNN). In one non-limiting embodiment, the RAW data passed into the RAW denoiser module 20 is an image formed using a color filter array (CFA) that captures light of specific colors at each pixel, for example, using the well-known Bayer pattern. FIG. 3 (a) shows the standard Bayer pattern colour filter array on the sensor. This pattern has a recurring 2×2 mosaic that is tiled across the image. At each pixel, either a red 30, green 31 or blue color 32 is acquired. An image captured in this format is said to be mosaiced. In FIG. 3(b), the mosaiced image is packed into four colour channels representing the R, G1, G2, and B colours, 33, 34, 35 and 36 respectively. In the packed form, the spatial resolution of each colour channel is half the original mosaicked image resolution.”, [par. 0058, ln. 1-15] “There are many traditional approaches to denoising. One simple method for denoising involves local averaging using filters like a box or Gaussian filter. These methods achieve denoising through low-pass filtering, which will suppress high frequencies in the image, including noise. While effective at reducing noise, these filters also blur edges, which are also high frequencies, and therefore local averaging produces blurry results. Methods have been proposed for improved noise reduction while preserving important detail like edges in the image, including techniques such as anisotropic diffusion, bilateral filtering, and non-local means.”, [par. 0065, ln. 1-13] “The simplest traditional approach to demosaicing is to interpolate to find the missing values, for example, using bilinear interpolation. This will produce a valid demosaiced result and is effective for low frequencies in the image. However, at higher frequencies, bilinear interpolation often produces artifacts such as spurious colours and zippering along edges, as the interpolation is guided only by spatial location, but not by image content. To achieve better results, methods with increasing sophistication to perform content-aware interpolation have been proposed. Although considerable progress has been made, the best demosaicing methods still produce artifacts in high frequency image regions.”, [par. 0066, ln. 1-7] “The AISP deep learning approach learns how to best demosaic an image based on its content, but informed by training pairs. Here, the training pairs each consist of a RAW image, and its demosaiced RGB version. One can easily create training pairs. Given an RGB image, one can sample it using the Bayer pattern of FIG. 3(a) to produce a mosaiced version.”), and performing the spatial filtering process comprises performing, by the neural processing unit: reading, from a memory, data representing a neural network ([par. 0078, ln. 10-17] “Each physical device implementing an entity comprises a processor and a memory. The devices may also comprise a transceiver for transmitting and receiving data to and from the transceiver 5 of camera 1. The memory stores in a non-transient way code that is executable by the processor to implement the respective entity in the manner described herein.”); and applying the neural network to the values of the characteristic pertaining to the pluralities of pixel intensity values, to obtain the filtered values ([par. 0055, ln. 1-16], [par. 0058, ln. 1-15], [par. 0065, ln. 1-13], [par. 0066, ln. 1-7]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Reza, Sun, and Slabaugh as within the same filed of color control using image intensity and filtering, and as analogous to the claimed invention. The motivation to combine is disclosed in Slabaugh, wherein it allows for learned the best patterns and can reduce noise in the resulting image ([par. 0058, ln. 1-15], [par. 0065, ln. 1-13], [par. 0066, ln. 1-7]). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Reza with the general-purpose spatial filtering on a programmable processor in Sun, and further combined the method of the combination of Reza and Sun with the neural network filtering as taught in Slabaugh through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would have applied the method of the combination of Reza and Sun with the neural network filtering as taught in Slabaugh being performed as the filtering function
f
g
as taught in equation (6) of Sun.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor in Sun and the neural network filtering as taught in Slabaugh to obtain the invention as specified in claim 5.
24. Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over “Realization of the Contrast Limited Adaptive Histogram Equalization (CLAHE) for Real-Time Image Enhancement” to Reza, in view of “A General-Purpose Dehazing Algorithm based on Local Contrast Enhancement Approaches” to Sun, and further in view of U.S. Publication No. 2019/0188857 to Rivard et al. (hereinafter Rivard).
25. Regarding Claim 13, a combination of Reza and Sun teaches the method of claim 1. Reza discloses wherein determining the values of the characteristic pertaining to the pluralities of pixel intensity values is performed using fixed function circuitry ([pg. 39, col. 1, 3. A Hardware Realization System, par. 1, ln. 1 to col. 2, par. 2, ln. 23], [pg. 40, Figure 5], [pg. 41, Fig. 6], [pg. 42, Fig. 7 and 8], [pg. 42, Fig. 9 and 10]), {and the method comprises determining, based on the values of the characteristic, an exposure value for use in capturing a subsequent image}. Reza does not specifically disclose determining based on the values of the characteristic, an exposure value for use in capturing a subsequent image. Likewise, Sun does not specifically disclose determining based on the values of the characteristic, an exposure value for use in capturing a subsequent image.
However, Rivard specifically discloses determining based on the values of the characteristic, an exposure value for use in capturing a subsequent image ([par. 0067, ln. 1-18] “…a camera module 330 is configured to store exposure parameters for sampling each image associated with an image stack. For example, in one embodiment, when directed to sample a photographic scene, the camera module 330 may sample a set of images comprising the image stack according to stored exposure parameters. A software module comprising programming instructions executing within a processor complex 310 may generate and store the exposure parameters prior to directing the camera module 330 to sample the image stack. In other embodiments, the camera module 330 may be used to meter an image or an image stack, and the software module comprising programming instructions executing within a processor complex 310 may generate and store metering parameters prior to directing the camera module 330 to capture the image. Of course, the camera module 330 may be used in any manner in combination with the processor complex 310.”, [par. 0068, ln. 1-14] “…exposure parameters associated with images comprising the image stack may be stored within an exposure parameter data structure that includes exposure parameters for one or more images. In another embodiment, a camera interface unit (not shown in FIG. 3B) within the processor complex 310 may be configured to read exposure parameters from the exposure parameter data structure and to transmit associated exposure parameters to the camera module 330 in preparation of sampling a photographic scene. After the camera module 330 is configured according to the exposure parameters, the camera interface may direct the camera module 330 to sample the photographic scene; the camera module 330 may then generate a corresponding image stack.”, [par. 0082, ln. 1-34] “…camera interface unit 386 may be configured to accumulate statistics while receiving image data from camera module 330. In particular, the camera interface unit 386 may accumulate exposure statistics for a given image while receiving image data for the image through interconnect 334. Exposure statistics may include, without limitation, one or more of an intensity histogram, a count of over-exposed pixels, a count of under-exposed pixels, an intensity-weighted sum of pixel intensity, or any combination thereof. The camera interface unit 386 may present the exposure statistics as memory-mapped storage locations within a physical or virtual address space defined by a processor, such as one or more of CPU cores 370, within processor complex 310… exposure statistics reside in storage circuits that are mapped into a memory-mapped register space, which may be accessed through the interconnect 334… the exposure statistics are transmitted in conjunction with transmitting pixel data for a captured image… the exposure statistics for a given image may be transmitted as in-line data, following transmission of pixel intensity data for the captured image. Exposure statistics may be calculated, stored, or cached within the camera interface unit 386… an image sensor controller within camera module 330 may be configured to accumulate the exposure statistics and transmit the exposure statistics to processor complex 310, such as by way of camera interface unit 386… the exposure statistics are accumulated within the camera module 330 and transmitted to the camera interface unit 386, either in conjunction with transmitting image data to the camera interface unit 386, or separately from transmitting image data.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize Reza, Sun, and Rivard as within the same field of image color processing using pixel intensity and histograms, and as analogous to the claimed invention. The motivation to combine would have been obvious to one of ordinary skill in the art, and is disclosed in Rivard, wherein it allows for active improvement of subsequent image quality ([par. 0003, ln. 1-6] “Current digital photographic systems use histogram equalization and adaptive histogram equalization (including contrast-limited techniques) to improve perceived image detail and quality by adjusting contrast within digital images.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined the method of Reza with the general-purpose spatial filtering on a programmable processor in Sun, and further combined the method of the combination of Reza and Sun with the exposure determination and subsequent image exposure control as taught in Rivard through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results. Specifically, one of ordinary skill in the art would have used the characteristics determined in the method of the combination of Reza and Sun to further determine the exposure as taught in Rivard and use the exposure to control subsequent image acquisition as also taught in Rivard.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Reza with the general-purpose spatial filtering on a programmable processor in Sun and the exposure determination and subsequent image exposure control of Rivard to obtain the invention as specified in claim 13.
Conclusion
26. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669
/CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights