DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Response to Amendment
Applicant’s remarks, filed 07/07/2026, regarding the objections made to the abstract and claims and the rejections under 35 U.S.C. 101 and 35 U.S.C. 112(a)/(b) submitted in the non-final office action dated 04/07/2026 are withdrawn due to the amendments made to the abstract and claims.
Response to Arguments
Applicant’s arguments see remarks, filed 07/07/2026, with respect to the claims 1-7 have been fully considered but are moot because the arguments do not apply to the current combinations of references being used in the current rejection.
Claim Objections
Claim 7 is objected to because of the following informalities:
In claim 7, line 16, the term “is in a saturated state” should be changed to “is in a saturated state;” in order to avoid typographical issue.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim 5 recites limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claim 5; recites the limitation, “the sensing device is configured to…..” [Line 5].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claim 5;
(i) “sensing device” (Fig. 8, #63 called sensing device with #91 processing unit. Paragraph [0078]-The sensing device 63 includes a processing unit 91 and an advanced driver-assistance system (ADAS) 92. The processing unit 91 includes a recognition signal processing unit 93 and a recognition DNN 94. The image data and the saturation flag output from the signal processing unit 11 are supplied to the sensing device 63, the recognition signal processing unit 93 performs recognition signal processing on the image data with reference to the saturation flag, and the recognition DNN 94 performs recognition deep learning with respect to the image data with reference to the saturation flag. Then, the image data to which the recognition signal processing has been applied and the recognition result recognized by the recognition deep learning with respect to the image data are supplied to the ADAS 92, and the ADAS 92 performs vehicle control. The sensing device is illustrated in Fig. 8, as #63 called sensing device with #91 processing unit, thus have sufficient structure or material wherein is a processor.).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 6, and 7 along with their dependent claims are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, because the specification, while being enabling for the blending ratio calculation unit 21 includes a first tap generation unit 31, a luminance generation unit 32, a maximum value selection unit 33, a saturation determination unit 34, a first selector 35, a calculation unit 36, a second selector 37, a second tap generation unit 38, a low-pass filter processing unit 39, and a third selector 40. The first tap generation unit 31 generates, from large- pixel RAW signals sequentially supplied to the blending ratio calculation unit 21, pixel values of the large-pixel RAW signals of a 3-by-3 tap with a pixel of interest being the center that is a target pixel for which the blending ratio a is to be calculated, and supplies the generated pixel values to the luminance generation unit 32 and the maximum value selection unit 33. The calculation unit 36 calculates the blending ratio a in accordance with the reference signal BASE supplied from the first selector 35, and supplies the blending ratio a to the second selector 37. That is, in a case where the reference signal BASE is equal to or lower than the lower limit threshold, the calculation unit 36 calculates 0 as the blending ratio a. In step S11, the calculation unit 36 calculates a blending ratio a in accordance with the reference signal BASE supplied from the first selector 35, and supplies the blending ratio a to the second selector 37, as described in Paragraph [0045-0046, 0051 and 0061] in the specification, does not reasonably provide enablement for explicitly disclosing the claim language “calculate, based on the comparison of the reference signal with the threshold, a value of a first blending ratio for a blending process, select, based on the saturation determination value, an output value of the first blending ratio, output the selected output value of the first blending ratio”, as claimed in claims 1, 6, and 7.
The specification does not enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to use the invention commensurate in scope with these claims. The claimed subject matter, not taught by the specification is calculate, based on the comparison of the reference signal with the threshold, a value of a first blending ratio for a blending process, select, based on the saturation determination value, an output value of the first blending ratio, output the selected output value of the first blending ratio.
Claims 3-4 along with their dependent claims are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, because the specification, while being enabling for the calculation unit 36 calculates the blending ratio a in accordance with the reference signal BASE supplied from the first selector 35, and supplies the blending ratio a to the second selector 37. That is, in a case where the reference signal BASE is equal to or lower than the lower limit threshold, the calculation unit 36 calculates 0 as the blending ratio a. The second selector 37 selects the blending ratio a on the basis of the saturation determination value SATU supplied from the saturation determination unit 34 and supplies the blending ratio a to the second tap generation unit 38. For example, in a case where the saturation determination value SATU is not 1, that is, in a case where the saturation determination value SATU is 0, the second selector 37 outputs the blending ratio a supplied from the calculation unit 36 as it is. On the other hand, in a case where the saturation determination value SATU is 1, the second selector 37 outputs 1 as the blending ratio a. That is, in a case where the saturation determination value SATU is 1, it is determined that the large pixel is saturated, and the blending ratio a is forcibly replaced with 1. The second tap generation unit 38 generates, from the blending ratios a sequentially supplied from the second selector 37, blending ratios a of the 3-by-3 tap with the pixel of interest being the center, and supplies the blending ratios a to the low-pass filter processing unit 39 and the third selector 40, as described in Paragraph [0051-0053] in the specification, does not reasonably provide enablement for explicitly disclosing the claim language “the plurality of blending ratios includes a value of a second blending ratio of the pixel of interest”, as claimed in claim 3, and “output the value of the second blending ratio as is in a case where the value of the second blending ratio is 1; obtain a value of a low-pass filtered second blending ratio of the pixel of interest based on the application of the low-pass filter to the plurality of blending ratios; and output the value of the low-pass filtered second blending ratio in a case where the value of the second blending ratio is different from 1”, as claimed in claim 4.
The specification does not enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to use the invention commensurate in scope with these claims. The claimed subject matter, not taught by the specification is “the plurality of blending ratios includes a value of a second blending ratio of the pixel of interest” and “output the value of the second blending ratio as is in a case where the value of the second blending ratio is 1; obtain a value of a low-pass filtered second blending ratio of the pixel of interest based on the application of the low-pass filter to the plurality of blending ratios; and output the value of the low-pass filtered second blending ratio in a case where the value of the second blending ratio is different from 1”.
The office respectfully requests the Applicant to indicate where in the specification teaches the limitation in claims 1, 3-4, and 6-7 or amend in order to overcome the rejection under 35 U.S.C. 112(a.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over KIMURA (US 20080106647 A1), hereinafter referenced as KIMURA, in view of MATSUZAKI (US 20090147326 A1), hereinafter referenced as MATSUZAKI, and further in view of ZHANG et al. (US 20200154083 A1), hereinafter referenced as ZHANG.
Regarding claim 1, KIMURA explicitly teaches a signal processing device, comprising (Fig. 1. Paragraph [0093]-KIMURA discloses FIG. 1 is a block diagram showing the noise-region detector 18 in detail.):
circuitry configured to: (Fig. 2, illustrates circuitry. Paragraph [0085-0086]-KIMURA discloses A subtracting circuit 3 subtracts the delayed video signal S3 from the input video signal S1 to generate a noise signal component S4. A signal-level correcting circuit 4 corrects the signal level of the noise signal component S4 to generate a correction signal S5. A subtracting circuit 5 subtracts the correction signal 35 from the input video signal S1 to generate the output-video signal 32. A measurement unit 6 measures the signal level of a noise signal component in the input video signal S1 to output a measurement result S7. The signal-level correcting circuit 4 changes the signal level of the correction signal S5 according to the measurement result S7.):
compare a reference signal with a threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh (wherein the reference video signal is comprised of a reference signal).),
wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh (wherein the reference video signal is comprised of a large-pixel RAW signal).);
wherein the blending process blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest (Fig. 41. Paragraph [0225]-KIMURA discloses a blending unit 161 adds together the noise components S1N and S16N output from the non-linear-characteristics processor 160, using weights based, on the blending ratio BR, and thereby outputs a noise signal S19 (wherein noise signal S19 is the result of blending processing of a small-pixel RAW signal with a large-pixel RAW signal and wherein S1N is a component of pixels S1P which are the small-pixel RAW signal and S16N is a component of pixels S16P which are the large-pixel RAW signal).);
compare the pixel value of the large-pixel RAW signal with a saturation determination threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh, and outputs saturation flags S1PAveF and S11PAveF indicating the possibility of occurrence of clipping of pixel values of the input video signal S1 and the reference video signal S11 in each region due to the effect of a dynamic range (wherein the pixel values in S1 and S11 are the pixel values compared to the threshold and SminTh and SmaxTh are saturation determination thresholds).);
output a saturation determination value based on the comparison of the pixel value of the large-pixel RAW signal with the saturation determination threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh, and outputs saturation flags S1PAveF and S11PAveF indicating the possibility of occurrence of clipping of pixel values of the input video signal S1 and the reference video signal S11 in each region due to the effect of a dynamic range (wherein the pixel values in S1 and S11 are the pixel values compared to the threshold and SminTh and SmaxTh are saturation determination thresholds).),
wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state (Fig. 9. Paragraph [0096]-KIMURA discloses when S1PAve<SminTh or S1PAve> SmaxTh is satisfied, the saturated-region detector 22 turns on the saturation flag S1PAveF for the input video signal S1. Similarly, when S11PAve<SminTh or S11PAve>SmaxTh is satisfied, the saturated-region detector 22 turns on the saturation flag S11PAveF for the reference video signal S11 (wherein a saturation flag indicates a saturated state).);
wherein the output value is one of the calculated value of the first blending ratio or 1 (Fig. 41. Paragraph [0222]-KIMURA discloses when the value of the noise confidence S1C or S16C checked against the first and second thresholds DRTh1 and DRTh2 is less than or equal to the first threshold DRTh1 or greater than or equal to the second threshold DRTh2, the noise-extraction blending-ratio setting unit 158 sets 0 or 1 as the blending ratio BR so that only noise components of the input video signal S1 or the difference signal S16 are selected. On the other hand, when the value of the noise confidence S1C or S16C is between the first and second thresholds DRTh1 and DRTh2 the noise-extraction blending-ratio setting unit 158 sets the value of the blending ratio BR by linear interpolation using the value of the noise confidence S1C or S16C (wherein the blending ratio set using linear interpolation is the first blending ratio and wherein the saturation determination value is noise confidence).); and
output the selected output value of the first blending ratio (Fig. 41. Paragraph [0222]-KIMURA discloses when the value of the noise confidence S1C or S16C checked against the first and second thresholds DRTh1 and DRTh2 is less than or equal to the first threshold DRTh1 or greater than or equal to the second threshold DRTh2, the noise-extraction blending-ratio setting unit 158 sets 0 or 1 as the blending ratio BR so that only noise components of the input video signal S1 or the difference signal S16 are selected. On the other hand, when the value of the noise confidence S1C or S16C is between the first and second thresholds DRTh1 and DRTh2 the noise-extraction blending-ratio setting unit 158 sets the value of the blending ratio BR by linear interpolation using the value of the noise confidence S1C or S16C (wherein the blending ratio set using linear interpolation is the first blending ratio and wherein the saturation determination value is noise confidence).).
KIMURA fails to explicitly teach calculate, based on the comparison of the reference signal with the threshold, a value of a first blending ratio for a blending process.
However, MATSUZAKI explicitly teaches calculate, based on the comparison of the reference signal with the threshold (Fig. 10. Paragraph [0067]-MATSUZAKI discloses in Step S1004, it is determined whether or not the values are greater than the set threshold. Here, in the case where either of the two input signal values is greater than the threshold, a determination of "greater" is made. Of course, the process may also make a determination of "greater" in the case where both values are greater than the threshold.), a value of a first blending ratio for a blending process (Fig. 10. Paragraph [0067-0069]-MATSUZAKI discloses in the case where the result of the determination indicates that the values are not greater than the threshold, the procedure advances to Step S1007, where the two signal values are blended at a set ratio and monochrome image data is generated. Next, in Step S1008, the results are stored in the 2D-to-1D conversion LUT 1002. Here, the blend ratio may be any ratio, but the present example uses a ratio of "3:1" as the blend ratio. For example, the present example has a ratio of "G component:B component=3:1". Meanwhile, if the result of the determination made in Step S1004 above is greater than the threshold, the procedure advances to Step S1005, where the sizes of the two signal values are compared. After this, in Step S1006, the ratio of the signal value that is lower is increased, and blending is carried out.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA of a signal processing device, comprising: circuitry configured to: compare a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending process blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; compare the pixel value of the large-pixel RAW signal with a saturation determination threshold; output a saturation determination value based on the comparison of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and output the selected output value of the first blending ratio with the teachings of MATSUZAKI of calculate, based on the comparison of the reference signal with the threshold, a value of a first blending ratio for a blending process.
Wherein having KIMURA’s imaging signal processing device/method having calculate, based on the comparison of the reference signal with the threshold, a value of a first blending ratio for a blending process.
The motivation behind the modification would have been to obtain an imaging signal processing device/method that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and MATSUZAKI relate to blending images and processing image signals, wherein KIMURA the accuracy of noise-level measurement can be improved, while MATSUZAKI to enable the reproduction of highlight portions of colors when performing color conversion processing on color image data and outputting monochrome image data. Please see KIMURA (US 20080106647 A1), Paragraph [0020], and MATSUZAKI (US 20090147326 A1), Paragraph [0012].
KIMURA in view of MATSUZAKI fail to explicitly teach select, based on the saturation determination value, an output value of the first blending ratio.
However, ZHANG explicitly teaches select, based on the saturation determination value, an output value of the first blending ratio (Fig. 4. Paragraph [0184]-ZHANG discloses the control unit 55 (55a) sets the blend ratio on the basis of blend ratio determination elements such as the dispersion change amount, how frequent pixels become saturated from an unsaturated state by matrix conversion, how frequent the dispersion change amount of a pixel exceeds a threshold, and the area and position of an image area including image pixels saturated from an unsaturated state by matrix conversion (wherein how frequent pixels become saturated is the saturation determination value).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA in view of MATSUZAKI of a signal processing device, comprising: circuitry configured to: compare a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending process blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; compare the pixel value of the large-pixel RAW signal with a saturation determination threshold; output a saturation determination value based on the comparison of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and output the selected output value of the first blending ratio with the teachings of ZHANG of select, based on the saturation determination value, an output value of the first blending ratio.
Wherein having KIMURA’s imaging signal processing device/method having select, based on the saturation determination value, an output value of the first blending ratio.
The motivation behind the modification would have been to obtain an imaging signal processing device/method that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and ZHANG relate to processing and blending image signals, wherein KIMURA the accuracy of noise-level measurement can be improved, while ZHANG if the color reproduction processing unit 33a performs a correction operation to reduce unnaturalness, the post processing load can be reduced when performing post processing of the image signal generated by the imaging device 20 (20a). Please see KIMURA (US 20080106647 A1), Paragraph [0020], and ZHANG et al. (US 20200154083 A1), Paragraph [0187].
Regarding claim 6, KIMURA explicitly teaches a signal processing method comprising (Fig. 1. Paragraph [0093]-KIMURA discloses FIG. 1 is a block diagram showing the noise-region detector 18 in detail.):
comparing a reference signal with a threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh (wherein the reference video signal is comprised of a reference signal).),
wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh (wherein the reference video signal is comprised of a large-pixel RAW signal).);
wherein the blending processing blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest (Fig. 41. Paragraph [0225]-KIMURA discloses a blending unit 161 adds together the noise components S1N and S16N output from the non-linear-characteristics processor 160, using weights based, on the blending ratio BR, and thereby outputs a noise signal S19 (wherein noise signal S19 is the result of blending processing of a small-pixel RAW signal with a large-pixel RAW signal and wherein S1N is a component of pixels S1P which are the small-pixel RAW signal and S16N is a component of pixels S16P which are the large-pixel RAW signal).);
comparing the pixel value of the large-pixel RAW signal with a saturation determination threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh, and outputs saturation flags S1PAveF and S11PAveF indicating the possibility of occurrence of clipping of pixel values of the input video signal S1 and the reference video signal S11 in each region due to the effect of a dynamic range (wherein the pixel values in S1 and S11 are the pixel values compared to the threshold and SminTh and SmaxTh are saturation determination thresholds).);
outputting a saturation determination value based on the comparing of the pixel value of the large-pixel RAW signal with the saturation determination threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh, and outputs saturation flags S1PAveF and S11PAveF indicating the possibility of occurrence of clipping of pixel values of the input video signal S1 and the reference video signal S11 in each region due to the effect of a dynamic range (wherein the pixel values in S1 and S11 are the pixel values compared to the threshold and SminTh and SmaxTh are saturation determination thresholds).),
wherein the saturation determination value indicates whether the large- pixel RAW signal of the pixel of interest is in a saturated state (Fig. 9. Paragraph [0096]-KIMURA discloses when S1PAve<SminTh or S1PAve> SmaxTh is satisfied, the saturated-region detector 22 turns on the saturation flag S1PAveF for the input video signal S1. Similarly, when S11PAve<SminTh or S11PAve>SmaxTh is satisfied, the saturated-region detector 22 turns on the saturation flag S11PAveF for the reference video signal S11 (wherein a saturation flag indicates a saturated state).);
wherein the output value is one of the calculated value of the first blending ratio or 1 (Fig. 41. Paragraph [0222]-KIMURA discloses when the value of the noise confidence S1C or S16C checked against the first and second thresholds DRTh1 and DRTh2 is less than or equal to the first threshold DRTh1 or greater than or equal to the second threshold DRTh2, the noise-extraction blending-ratio setting unit 158 sets 0 or 1 as the blending ratio BR so that only noise components of the input video signal S1 or the difference signal S16 are selected. On the other hand, when the value of the noise confidence S1C or S16C is between the first and second thresholds DRTh1 and DRTh2 the noise-extraction blending-ratio setting unit 158 sets the value of the blending ratio BR by linear interpolation using the value of the noise confidence S1C or S16C (wherein the blending ratio set using linear interpolation is the first blending ratio and wherein the saturation determination value is noise confidence).); and
outputting the selected output value of the first blending ratio (Fig. 41. Paragraph [0222]-KIMURA discloses when the value of the noise confidence S1C or S16C checked against the first and second thresholds DRTh1 and DRTh2 is less than or equal to the first threshold DRTh1 or greater than or equal to the second threshold DRTh2, the noise-extraction blending-ratio setting unit 158 sets 0 or 1 as the blending ratio BR so that only noise components of the input video signal S1 or the difference signal S16 are selected. On the other hand, when the value of the noise confidence S1C or S16C is between the first and second thresholds DRTh1 and DRTh2 the noise-extraction blending-ratio setting unit 158 sets the value of the blending ratio BR by linear interpolation using the value of the noise confidence S1C or S16C (wherein the blending ratio set using linear interpolation is the first blending ratio and wherein the saturation determination value is noise confidence).).
KIMURA fails to explicitly teach calculating, based on the comparing of the reference signal with the threshold, a value of a first blending ratio for blending processing.
However, MATSUZAKI explicitly teaches calculating, based on the comparing of the reference signal with the threshold (Fig. 10. Paragraph [0067]-MATSUZAKI discloses in Step S1004, it is determined whether or not the values are greater than the set threshold. Here, in the case where either of the two input signal values is greater than the threshold, a determination of "greater" is made. Of course, the process may also make a determination of "greater" in the case where both values are greater than the threshold.), a value of a first blending ratio for blending processing (Fig. 10. Paragraph [0067-0069]-MATSUZAKI discloses in the case where the result of the determination indicates that the values are not greater than the threshold, the procedure advances to Step S1007, where the two signal values are blended at a set ratio and monochrome image data is generated. Next, in Step S1008, the results are stored in the 2D-to-1D conversion LUT 1002. Here, the blend ratio may be any ratio, but the present example uses a ratio of "3:1" as the blend ratio. For example, the present example has a ratio of "G component:B component=3:1". Meanwhile, if the result of the determination made in Step S1004 above is greater than the threshold, the procedure advances to Step S1005, where the sizes of the two signal values are compared. After this, in Step S1006, the ratio of the signal value that is lower is increased, and blending is carried out.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA of a signal processing method comprising: comparing a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending processing blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; comparing the pixel value of the large-pixel RAW signal with a saturation determination threshold; outputting a saturation determination value based on the comparing of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large- pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and outputting the selected output value of the first blending ratio with the teachings of MATSUZAKI of calculating, based on the comparing of the reference signal with the threshold, a value of a first blending ratio for blending processing.
Wherein having KIMURA’s imaging signal processing device/method having calculating, based on the comparing of the reference signal with the threshold, a value of a first blending ratio for blending processing.
The motivation behind the modification would have been to obtain an imaging signal processing device/method that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and MATSUZAKI relate to blending images and processing image signals, wherein KIMURA the accuracy of noise-level measurement can be improved, while MATSUZAKI to enable the reproduction of highlight portions of colors when performing color conversion processing on color image data and outputting monochrome image data. Please see KIMURA (US 20080106647 A1), Paragraph [0020], and MATSUZAKI (US 20090147326 A1), Paragraph [0012].
KIMURA in view of MATSUZAKI fail to explicitly teach selecting, based on the saturation determination value, an output value of the first blending ratio.
However, ZHANG explicitly teaches selecting, based on the saturation determination value, an output value of the first blending ratio (Fig. 4. Paragraph [0148]-ZHANG discloses the control unit 55 (55a) sets the blend ratio on the basis of blend ratio determination elements such as the dispersion change amount, how frequent pixels become saturated from an unsaturated state by matrix conversion, how frequent the dispersion change amount of a pixel exceeds a threshold, and the area and position of an image area including image pixels saturated from an unsaturated state by matrix conversion (wherein how frequent pixels become saturated is the saturation determination value).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA in view of MATSUZAKI of a signal processing method comprising: comparing a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending processing blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; comparing the pixel value of the large-pixel RAW signal with a saturation determination threshold; outputting a saturation determination value based on the comparing of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large- pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and outputting the selected output value of the first blending ratio with the teachings of ZHANG of selecting, based on the saturation determination value, an output value of the first blending ratio.
Wherein having KIMURA’s imaging signal processing device/method having selecting, based on the saturation determination value, an output value of the first blending ratio.
The motivation behind the modification would have been to obtain an imaging signal processing device/method that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and ZHANG relate to processing and blending image signals, wherein KIMURA the accuracy of noise-level measurement can be improved, while ZHANG if the color reproduction processing unit 33a performs a correction operation to reduce unnaturalness, the post processing load can be reduced when performing post processing of the image signal generated by the imaging device 20 (20a). Please see KIMURA (US 20080106647 A1), Paragraph [0020], and ZHANG et al. (US 20200154083 A1), Paragraph [0187].
Regarding claim 7, KIMURA explicitly teaches a non-transitory computer readable medium having stored thereon, computer executable instructions (Paragraph [0272]-KIMURA discloses the program may be provided as recorded on a recording medium, such as an optical disc, a magnetic disc, or a memory card. Furthermore, the program may be provided by downloading via a network, such as the Internet.),
comparing a reference signal with a threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh (wherein the reference video signal is comprised of a reference signal).),
wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh (wherein the reference video signal is comprised of a large-pixel RAW signal).);
wherein the blending processing blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest (Fig. 41. Paragraph [0225]-KIMURA discloses a blending unit 161 adds together the noise components S1N and S16N output from the non-linear-characteristics processor 160, using weights based, on the blending ratio BR, and thereby outputs a noise signal S19 (wherein noise signal S19 is the result of blending processing of a small-pixel RAW signal with a large-pixel RAW signal and wherein S1N is a component of pixels S1P which are the small-pixel RAW signal and S16N is a component of pixels S16P which are the large-pixel RAW signal).);
comparing the pixel value of the large-pixel RAW signal with a saturation determination threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh, and outputs saturation flags S1PAveF and S11PAveF indicating the possibility of occurrence of clipping of pixel values of the input video signal S1 and the reference video signal S11 in each region due to the effect of a dynamic range (wherein the pixel values in S1 and S11 are the pixel values compared to the threshold and SminTh and SmaxTh are saturation determination thresholds).);
outputting a saturation determination value based on the comparing of the pixel value of the large-pixel RAW signal with the saturation determination threshold (Fig. 9. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh, and outputs saturation flags S1PAveF and S11PAveF indicating the possibility of occurrence of clipping of pixel values of the input video signal S1 and the reference video signal S11 in each region due to the effect of a dynamic range (wherein the pixel values in S1 and S11 are the pixel values compared to the threshold and SminTh and SmaxTh are saturation determination thresholds).),
wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state (Fig. 9. Paragraph [0096]-KIMURA discloses when S1PAve< SminTh or S1PAve> SmaxTh is satisfied, the saturated-region detector 22 turns on the saturation flag S1PAveF for the input video signal S1. Similarly, when S11PAve< SminTh or S11PAve> SmaxTh is satisfied, the saturated-region detector 22 turns on the saturation flag S11PAveF for the reference video signal S11 (wherein a saturation flag indicates a saturated state).)
wherein the output value is one of the calculated value of the first blending ratio or 1 (Fig. 41. Paragraph [0222]-KIMURA discloses when the value of the noise confidence S1C or S16C checked against the first and second thresholds DRTh1 and DRTh2 is less than or equal to the first threshold DRTh1 or greater than or equal to the second threshold DRTh2, the noise-extraction blending-ratio setting unit 158 sets 0 or 1 as the blending ratio BR so that only noise components of the input video signal S1 or the difference signal S16 are selected. On the other hand, when the value of the noise confidence S1C or S16C is between the first and second thresholds DRTh1 and DRTh2 the noise-extraction blending-ratio setting unit 158 sets the value of the blending ratio BR by linear interpolation using the value of the noise confidence S1C or S16C (wherein the blending ratio set using linear interpolation is the first blending ratio and wherein the saturation determination value is noise confidence).); and
outputting the selected output value of the first blending ratio (Fig. 41. Paragraph [0222]-KIMURA discloses when the value of the noise confidence S1C or S16C checked against the first and second thresholds DRTh1 and DRTh2 is less than or equal to the first threshold DRTh1 or greater than or equal to the second threshold DRTh2, the noise-extraction blending-ratio setting unit 158 sets 0 or 1 as the blending ratio BR so that only noise components of the input video signal S1 or the difference signal S16 are selected. On the other hand, when the value of the noise confidence S1C or S16C is between the first and second thresholds DRTh1 and DRTh2 the noise-extraction blending-ratio setting unit 158 sets the value of the blending ratio BR by linear interpolation using the value of the noise confidence S1C or S16C (wherein the blending ratio set using linear interpolation is the first blending ratio and wherein the saturation determination value is noise confidence).).
Although KIMURA explicitly teaches a non-transitory computer readable medium having stored thereon, computer executable instructions, KIMURA fails to explicitly teach a non-transitory computer readable medium having stored thereon, computer executable instructions which when executed by a computer, cause the computer to execute operations, the operations comprising: calculating, based on the comparing of the reference signal with the threshold, a value of a first blending ratio for blending processing.
However, MATSUZAKI explicitly teaches a non-transitory computer readable medium having stored thereon, computer executable instructions which when executed by a computer, cause the computer to execute operations, the operations comprising (Fig. 2. Paragraph [0118]-MATSUZAKI discloses it goes without saying that the object of the present invention can also be achieved by supplying, to a system or apparatus, a storage medium in which the program code for software that realizes the functions of the aforementioned embodiments has been stored, and causing a computer (CPU or MPU) of the system or apparatus to read out and execute the program code stored in the storage medium.):
calculating, based on the comparing of the reference signal with the threshold (Fig. 10. Paragraph [0067]-MATSUZAKI discloses in Step S1004, it is determined whether or not the values are greater than the set threshold. Here, in the case where either of the two input signal values is greater than the threshold, a determination of "greater" is made. Of course, the process may also make a determination of "greater" in the case where both values are greater than the threshold.), a value of a first blending ratio for blending processing (Fig. 10. Paragraph [0067-0069]-MATSUZAKI discloses in the case where the result of the determination indicates that the values are not greater than the threshold, the procedure advances to Step S1007, where the two signal values are blended at a set ratio and monochrome image data is generated. Next, in Step S1008, the results are stored in the 2D-to-1D conversion LUT 1002. Here, the blend ratio may be any ratio, but the present example uses a ratio of "3:1" as the blend ratio. For example, the present example has a ratio of "G component:B component=3:1". Meanwhile, if the result of the determination made in Step S1004 above is greater than the threshold, the procedure advances to Step S1005, where the sizes of the two signal values are compared. After this, in Step S1006, the ratio of the signal value that is lower is increased, and blending is carried out.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA of a non-transitory computer readable medium having stored thereon, computer executable instructions, comparing a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending processing blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; comparing the pixel value of the large-pixel RAW signal with a saturation determination threshold; outputting a saturation determination value based on the comparing of the pixel value of the large-pixel RAW signal with the saturation determination, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state wherein the output value is one of the calculated value of the first blending ratio or 1; and outputting the selected output value of the first blending ratio with the teachings of MATSUZAKI of a non-transitory computer readable medium having stored thereon, computer executable instructions which when executed by a computer, cause the computer to execute operations, the operations comprising: calculating, based on the comparing of the reference signal with the threshold, a value of a first blending ratio for blending processing.
Wherein having KIMURA’s imaging signal processing device/method having a non-transitory computer readable medium having stored thereon, computer executable instructions which when executed by a computer, cause the computer to execute operations, the operations comprising: calculating, based on the comparing of the reference signal with the threshold, a value of a first blending ratio for blending processing.
The motivation behind the modification would have been to obtain an imaging signal processing device/method that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and MATSUZAKI relate to blending images and processing image signals, wherein KIMURA the accuracy of noise-level measurement can be improved, while MATSUZAKI to enable the reproduction of highlight portions of colors when performing color conversion processing on color image data and outputting monochrome image data. Please see KIMURA (US 20080106647 A1), Paragraph [0020], and MATSUZAKI (US 20090147326 A1), Paragraph [0012].
KIMURA in view of MATSUZAKI fail to explicitly teach selecting, based on the saturation determination value, an output value of the first blending ratio.
However, ZHANG explicitly teaches selecting, based on the saturation determination value, an output value of the first blending ratio (Fig. 4. Paragraph [0148]-ZHANG discloses the control unit 55 (55a) sets the blend ratio on the basis of blend ratio determination elements such as the dispersion change amount, how frequent pixels become saturated from an unsaturated state by matrix conversion, how frequent the dispersion change amount of a pixel exceeds a threshold, and the area and position of an image area including image pixels saturated from an unsaturated state by matrix conversion (wherein how frequent pixels become saturated is the saturation determination value).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA in view of MATSUZAKI of a non-transitory computer readable medium having stored thereon, computer executable instructions, comparing a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending processing blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; comparing the pixel value of the large-pixel RAW signal with a saturation determination threshold; outputting a saturation determination value based on the comparing of the pixel value of the large-pixel RAW signal with the saturation determination, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state wherein the output value is one of the calculated value of the first blending ratio or 1; and outputting the selected output value of the first blending ratio with the teachings of ZHANG of selecting, based on the saturation determination value, an output value of the first blending ratio.
Wherein having KIMURA’s imaging signal processing device/method having selecting, based on the saturation determination value, an output value of the first blending ratio.
The motivation behind the modification would have been to obtain an imaging signal processing device/method that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and ZHANG relate to processing and blending image signals, wherein KIMURA the accuracy of noise-level measurement can be improved, while ZHANG if the color reproduction processing unit 33a performs a correction operation to reduce unnaturalness, the post processing load can be reduced when performing post processing of the image signal generated by the imaging device 20 (20a). Please see KIMURA (US 20080106647 A1), Paragraph [0020], and ZHANG et al. (US 20200154083 A1), Paragraph [0187].
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over KIMURA (US 20080106647 A1), hereinafter referenced as KIMURA, in view of MATSUZAKI (US 20090147326 A1), hereinafter referenced as MATSUZAKI, and further in view of ZHANG et al. (US 20200154083 A1), hereinafter referenced as ZHANG, and further in view of OGASAHARA (US 20130100311 A1), hereinafter referenced as OGASAHARA.
Regarding claim 2, KIMURA in view of MATSUZAKI and further in view of ZHANG explicitly teach the signal processing device according to claim 1,
KIMURA in view of MATSUZAKI and further in view of ZHANG fail to explicitly teach wherein the circuitry is further configured to generate the reference signal, as a luminance value, based on pixel values of a plurality of pixels, the plurality of pixels includes the pixel of interest, and the pixel of interest is at a center of the plurality of pixels.
However, OGASAHARA explicitly teaches wherein the circuitry is further configured to generate the reference signal (Fig. 3. Paragraph [0061]-OGASAHARA discloses the imaging processing circuit 20 temporarily stores, for example, the monochrome image data 51 and the first luminance information 55 in the frame memory 26 (wherein the stored luminance information is the reference signal).), as a luminance value (Fig. 7, #42 called luminance information generating section. Paragraph [0059]-OGASAHARA discloses the luminance information generating section 42 outputs the luminance value detected by the G pixel and the luminance value of the G component calculated for the R pixel and the B pixel as the first luminance information 55. The solid-state imaging device 12 adopts the G component for the first luminance information 55 because the information on the luminance can be obtained the most for the G component out of each component of R, G, and B.), based on pixel values of a plurality of pixels (Figs. 8-9. Paragraph [0057-0058]-OGASAHARA discloses the luminance information generating section 42 references the luminance value detected by the four G pixels contained in a 3.times.3 pixel block having the R pixel as a center, as illustrated in FIG. 8, for example, in the calculation of the luminance value of the G component for the R pixel. The luminance information generating section 42 calculates the luminance value of the G component in the R pixel through the following equation, for example. In the equation, the terms "G1", "G2", "G3", and "G4" respectively represent the luminance values of the G components detected by the G pixels (G1, G2, G3, and G4) illustrated in FIG. 8. (luminance value of G component in R pixel)=(G1+G2+G3+G4)/4),
the plurality of pixels includes the pixel of interest (Figs. 8-9, illustrate the plurality of pixels and the pixel of interest. Paragraph [0057-0058]-OGASAHARA discloses the luminance information generating section 42 references the luminance value detected by the four G pixels contained in a 3.times.3 pixel block having the R pixel as a center, as illustrated in FIG. 8, for example, in the calculation of the luminance value of the G component for the R pixel. The luminance information generating section 42 calculates the luminance value of the G component in the R pixel through the following equation, for example. In the equation, the terms "G1", "G2", "G3", and "G4" respectively represent the luminance values of the G components detected by the G pixels (G1, G2, G3, and G4) illustrated in FIG. 8. (luminance value of G component in R pixel)=(G1+G2+G3+G4)/4 (wherein the R or G pixel in the center is the pixel of interest and the surrounding pixels and the center pixel are the plurality of pixels).), and
the pixel of interest is at a center of the plurality of pixels (Figs. 8-9, illustrate the plurality of pixels and the pixel of interest. Paragraph [0057-0058]-OGASAHARA discloses the luminance information generating section 42 references the luminance value detected by the four G pixels contained in a 3.times.3 pixel block having the R pixel as a center, as illustrated in FIG. 8, for example, in the calculation of the luminance value of the G component for the R pixel. The luminance information generating section 42 calculates the luminance value of the G component in the R pixel through the following equation, for example. In the equation, the terms "G1", "G2", "G3", and "G4" respectively represent the luminance values of the G components detected by the G pixels (G1, G2, G3, and G4) illustrated in FIG. 8. (luminance value of G component in R pixel)=(G1+G2+G3+G4)/4 (wherein the R or G pixel in the center is the pixel of interest and the surrounding pixels and the center pixel are the plurality of pixels).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA in view of MATSUZAKI and further in view of ZHANG of a signal processing device, comprising: circuitry configured to: compare a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending process blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; compare the pixel value of the large-pixel RAW signal with a saturation determination threshold; output a saturation determination value based on the comparison of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and output the selected output value of the first blending ratio with the teachings of OGASAHARA of wherein the circuitry is further configured to generate the reference signal, as a luminance value, based on pixel values of a plurality of pixels, the plurality of pixels includes the pixel of interest, and the pixel of interest is at a center of the plurality of pixels.
Wherein having KIMURA’s imaging signal processing device/method having wherein the circuitry is further configured to generate the reference signal, as a luminance value, based on pixel values of a plurality of pixels, the plurality of pixels includes the pixel of interest, and the pixel of interest is at a center of the plurality of pixels.
The motivation behind the modification would have been to obtain signal processing device that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and OGASAHARA relate to signal processing and image analysis, wherein KIMURA the accuracy of noise-level measurement can be improved, while OGASAHARA reduces the color noise by the smoothing process of the color information. Please see KIMURA (US 20080106647 A1), Paragraph [0020], and OGASAHARA (US 20130100311 A1), Paragraph [0070].
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over KIMURA (US 20080106647 A1), hereinafter referenced as KIMURA, in view of MATSUZAKI (US 20090147326 A1), hereinafter referenced as MATSUZAKI, and further in view of ZHANG et al. (US 20200154083 A1), hereinafter referenced as ZHANG, and further in view of LUO et al. (US 20100157072 A1), hereinafter referenced as LUO.
Regarding claim 3, KIMURA in view of MATSUZAKI and further in view of ZHANG explicitly teach the signal processing device according to claim 1,
KIMURA further explicitly teaches wherein the circuitry is further configured to (Fig. 2, illustrates circuitry. Paragraph [0085-0086]-KIMURA discloses A subtracting circuit 3 subtracts the delayed video signal S3 from the input video signal S1 to generate a noise signal component S4. A signal-level correcting circuit 4 corrects the signal level of the noise signal component S4 to generate a correction signal S5. A subtracting circuit 5 subtracts the correction signal 35 from the input video signal S1 to generate the output-video signal 32. A measurement unit 6 measures the signal level of a noise signal component in the input video signal S1 to output a measurement result S7. The signal-level correcting circuit 4 changes the signal level of the correction signal S5 according to the measurement result S7.):
KIMURA in view of MATSUZAKI and further in view of ZHANG fail to explicitly teach generate, based on the output value, a plurality of blending ratios of a tap, wherein the tap is associated with the pixel of interest, the pixel of interest is at a center of a plurality of pixels, and the plurality of blending ratios includes a value of a second blending ratio of the pixel of interest; and apply a low-pass filter to the plurality of blending ratios of the tap.
However, LUO explicitly teaches generate, based on the output value, a plurality of blending ratios of a tap (Fig. 11, illustrates a tap. Paragraph [0228-0229]-LUO discloses 9 pixels (3 rows by 3 columns) centered on a pixel 251 as the target to be adjusted are selected, and the pixel value of the center pixel 251 is set to a value calculated using the following formula: Pixel value=1/9(.SIGMA.Pv.sub.i) where Pv.sub.i is the pixel value of each of the 9 pixels (3 rows by 3 columns) centered on the pixel 251 as the target to be adjusted. The blend ratios corresponding to the pixel values calculated through the process is used as the final blend ratios (wherein the blending ratio calculated through the process is the blending ratio of a tap and wherein the output value is one of the intermediary blending ratios).),
wherein the tap is associated with the pixel of interest (Fig. 11, illustrates a tap with a pixel of interest indicated by pixel #251. Paragraph [0228-0229]-LUO discloses 9 pixels (3 rows by 3 columns) centered on a pixel 251 as the target to be adjusted are selected, and the pixel value of the center pixel 251 is set to a value calculated using the following formula: Pixel value=1/9(.SIGMA.Pv.sub.i) where Pv.sub.i is the pixel value of each of the 9 pixels (3 rows by 3 columns) centered on the pixel 251 as the target to be adjusted. The blend ratios corresponding to the pixel values calculated through the process is used as the final blend ratios.),
the pixel of interest is at a center of a plurality of pixels (Fig. 11, illustrates a tap with a pixel of interest in the center indicated by pixel #251. Paragraph [0228-0229]-LUO discloses 9 pixels (3 rows by 3 columns) centered on a pixel 251 as the target to be adjusted are selected, and the pixel value of the center pixel 251 is set to a value calculated using the following formula: Pixel value=1/9(.SIGMA.Pv.sub.i) where Pv.sub.i is the pixel value of each of the 9 pixels (3 rows by 3 columns) centered on the pixel 251 as the target to be adjusted. The blend ratios corresponding to the pixel values calculated through the process is used as the final blend ratios.), and
the plurality of blending ratios includes a value of a second blending ratio of the pixel of interest (Fig. 11. Paragraph [0227]-LUO discloses the boundary processing section 204 executes a blend ratio adjustment process to eliminate the unnaturalness of the boundary portion between pixels with different blend ratios. For example, a pixel value smoothening process that employs a low-pass filter (LPF) is executed on the pixel values calculated in accordance with the blend ratios determined by the blend ratio determination section 203 to adopt the blend ratios calculated through the pixel value smoothening process (wherein the low-pass filtered blend ratio is a second blending ratio).); and
apply a low-pass filter to the plurality of blending ratios of the tap (Fig. 11. Paragraph [0227]-LUO discloses a pixel value smoothening process that employs a low-pass filter (LPF) is executed on the pixel values calculated in accordance with the blend ratios determined by the blend ratio determination section 203 to adopt the blend ratios calculated through the pixel value smoothening process (wherein the blend ratios determined by blend ratio determination section 203 are applied based on the pixels as illustrated in Fig. 11).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA in view of MATSUZAKI and further in view of ZHANG of a signal processing device, comprising: circuitry configured to: compare a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending process blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; compare the pixel value of the large-pixel RAW signal with a saturation determination threshold; output a saturation determination value based on the comparison of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and output the selected output value of the first blending ratio with the teachings of LUO of generate, based on the output value, a plurality of blending ratios of a tap, wherein the tap is associated with the pixel of interest, the pixel of interest is at a center of a plurality of pixels, and the plurality of blending ratios includes a value of a second blending ratio of the pixel of interest; and apply a low-pass filter to the plurality of blending ratios of the tap.
Wherein having KIMURA’s imaging signal processing device/method having generate, based on the output value, a plurality of blending ratios of a tap, wherein the tap is associated with the pixel of interest, the pixel of interest is at a center of a plurality of pixels, and the plurality of blending ratios includes a value of a second blending ratio of the pixel of interest; and apply a low-pass filter to the plurality of blending ratios of the tap.
The motivation behind the modification would have been to obtain a signal processing device that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and LUO relate to signal processing and image analysis, wherein KIMURA the accuracy of noise-level measurement can be improved, while LUO it is desirable to provide an image processing apparatus, an image processing method, and a program that synthesize a plurality of images employing an optimum motion vector to generate a high-quality noise-reduced image. Please see KIMURA (US 20080106647 A1), Paragraph [0020], and LUO et al. (US 20100157072 A1), Paragraph [0010].
Regarding claim 4, KIMURA in view of MATSUZAKI and further in view of ZHANG and further in view of LUO explicitly teach the signal processing device according to claim 3,
KIMURA further explicitly teaches wherein the circuitry is further configured to (Fig. 2, illustrates circuitry. Paragraph [0085-0086]-KIMURA discloses A subtracting circuit 3 subtracts the delayed video signal S3 from the input video signal S1 to generate a noise signal component S4. A signal-level correcting circuit 4 corrects the signal level of the noise signal component S4 to generate a correction signal S5. A subtracting circuit 5 subtracts the correction signal 35 from the input video signal S1 to generate the output-video signal 32. A measurement unit 6 measures the signal level of a noise signal component in the input video signal S1 to output a measurement result S7. The signal-level correcting circuit 4 changes the signal level of the correction signal S5 according to the measurement result S7.):
KIMURA in view of MATSUZAKI and further in view of ZHANG fail to explicitly teach output the value of the second blending ratio as is in a case where the value of the second blending ratio is 1; obtain a value of a low-pass filtered second blending ratio of the pixel of interest based on the application of the low-pass filter to the plurality of blending ratios; and output the value of the low-pass filtered second blending ratio in a case where the value of the second blending ratio is different from 1.
However, LUO explicitly teaches output the value of the second blending ratio as is in a case where the value of the second blending ratio is 1 (Fig. 11. Paragraph [0225]-LUO discloses when the GMC image blend ratio is 1.0, the pixel value of the relevant pixel is set to the pixel value of the GMC image (wherein set to the pixel value of the GMC image is the second blending ratio as is).);
obtain a value of a low-pass filtered second blending ratio of the pixel of interest based on the application of the low-pass filter to the plurality of blending ratios (Fig. 11. Paragraph [0227]-LUO discloses the boundary processing section 204 executes a blend ratio adjustment process to eliminate the unnaturalness of the boundary portion between pixels with different blend ratios. For example, a pixel value smoothening process that employs a low-pass filter (LPF) is executed on the pixel values calculated in accordance with the blend ratios determined by the blend ratio determination section 203 to adopt the blend ratios calculated through the pixel value smoothening process (wherein the low-pass filtered blend ratio is a low-pass filtered second blending ratio).); and
output the value of the low-pass filtered second blending ratio (Fig. 11. Paragraph [0227]-LUO discloses the boundary processing section 204 executes a blend ratio adjustment process to eliminate the unnaturalness of the boundary portion between pixels with different blend ratios. For example, a pixel value smoothening process that employs a low-pass filter (LPF) is executed on the pixel values calculated in accordance with the blend ratios determined by the blend ratio determination section 203 to adopt the blend ratios calculated through the pixel value smoothening process (wherein the low-pass filtered blend ratio is a low-pass filtered second blending ratio).) in a case where the value of the second blending ratio is different from 1 (Fig. 11. Paragraph [0225]-LUO discloses when the GMC image blend ratio is 0.5, the pixel value of the relevant pixel is set to an intermediate value between the pixel value of the LMC image and the pixel value of the GMC image.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA in view of MATSUZAKI and further in view of ZHANG of a signal processing device, comprising: circuitry configured to: compare a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending process blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; compare the pixel value of the large-pixel RAW signal with a saturation determination threshold; output a saturation determination value based on the comparison of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and output the selected output value of the first blending ratio with the teachings of LUO of output the value of the second blending ratio as is in a case where the value of the second blending ratio is 1; obtain a value of a low-pass filtered second blending ratio of the pixel of interest based on the application of the low-pass filter to the plurality of blending ratios; and output the value of the low-pass filtered second blending ratio in a case where the value of the second blending ratio is different from 1.
Wherein having KIMURA’s imaging signal processing device/method having output the value of the second blending ratio as is in a case where the value of the second blending ratio is 1; obtain a value of a low-pass filtered second blending ratio of the pixel of interest based on the application of the low-pass filter to the plurality of blending ratios; and output the value of the low-pass filtered second blending ratio in a case where the value of the second blending ratio is different from 1.
The motivation behind the modification would have been to obtain a signal processing device that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and LUO relate to signal processing and image analysis, wherein KIMURA the accuracy of noise-level measurement can be improved, while LUO it is desirable to provide an image processing apparatus, an image processing method, and a program that synthesize a plurality of images employing an optimum motion vector to generate a high-quality noise-reduced image. Please see KIMURA (US 20080106647 A1), Paragraph [0020], and LUO et al. (US 20100157072 A1), Paragraph [0010].
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over KIMURA (US 20080106647 A1), hereinafter referenced as KIMURA, in view of MATSUZAKI (US 20090147326 A1), hereinafter referenced as MATSUZAKI, and further in view of ZHANG et al. (US 20200154083 A1), hereinafter referenced as ZHANG, and further in view of GALLO et al. (US 20150125091 A1), hereinafter referenced as GALLO.
Regarding claim 5, KIMURA in view of MATSUZAKI and further in view of ZHANG explicitly teach the signal processing device according to claim 1,
KIMURA further explicitly teaches wherein the circuitry is further configured to (Fig. 2, illustrates circuitry. Paragraph [0085-0086]-KIMURA discloses A subtracting circuit 3 subtracts the delayed video signal S3 from the input video signal S1 to generate a noise signal component S4. A signal-level correcting circuit 4 corrects the signal level of the noise signal component S4 to generate a correction signal S5. A subtracting circuit 5 subtracts the correction signal 35 from the input video signal S1 to generate the output-video signal 32. A measurement unit 6 measures the signal level of a noise signal component in the input video signal S1 to output a measurement result S7. The signal-level correcting circuit 4 changes the signal level of the correction signal S5 according to the measurement result S7.):
output the saturation determination value, as a saturation flag (Fig. 1. Paragraph [0096]-KIMURA discloses a saturated-region detector 22 checks the average values S1PAve and S11PAve of pixel values in the input video signal S1 and the reference video signal S11, calculated by the pixel-region feature extractor 21, against predetermined thresholds SminTh and SmaxTh, and outputs saturation flags S1PAveF and S11PAveF indicating the possibility of occurrence of clipping of pixel values of the input video signal S1 and the reference video signal S11 in each region due to the effect of a dynamic range.), to a sensing device (Fig. 7. Paragraph [0101]-KIMURA discloses In the measurement-validity checker 24, a measurement-prohibited-region checker 27 executes a logic operation on the saturation flags S1PAveF and S11PAveF and the noise-free-region detection flags S1NFF, S11NFF, By and S16NFF for each of the input video signal S1, the reference video signal S11, and the difference signal S16 to output measurement-prohibited-region checking flags S1NF, S11NF, and S16NF indicating whether the relevant regions are unsuitable for noise measurement (wherein the measurement-validity checker 24 is a sensing device).),
KIMURA in view of MATSUZAKI and further in view of ZHANG fail to explicitly teach execute the blending process on an image with a wide dynamic range; and wherein the sensing device is configured to execute recognition processing based on the image with the wide dynamic range.
However, GALLO explicitly teaches execute the blending process on an image (Fig. 4. Paragraph [0033]-GALLO discloses once each image in the HDR image stack is registered to the reference image 410, the HDR image may be created by blending three or more images in the HDR image stack.) with a wide dynamic range (Fig. 4. Paragraph [0032]-GALLO discloses FIG. 4 illustrates two images in an HDR image stack, in accordance with one embodiment. The HDR image stack includes a plurality of images including at least a reference image 410 and a source image 420. For example, the HDR image stack may include five images captured with different exposure settings, with the third captured image designated as the reference image 410.); and
wherein the sensing device (Fig. 2. Paragraph [0018]-GALLO discloses FIG. 2 illustrates a device 200 configured to perform fast feature recognition in two-dimensional images, in accordance with one embodiment. Further in paragraph [0048]-GALLO discloses the FNRR algorithm 500 may be implemented by any type of processor coupled to a memory storing the reference image 410 and the source image 420. In one embodiment, the FNRR algorithm 500 may be implemented, at least in part, by the GPU 214 of the device 200. In other embodiments, the FNRR algorithm 500 may be implemented by the CPU 212 or the IPP 250 of the device 200.) is configured to execute recognition processing based on the image with the wide dynamic range (Fig. 5A-5F. Paragraph [0034]-GALLO discloses FIGS. 5A through 5F illustrate a conceptual diagram of a fast, non-rigid registration (FNRR) algorithm 500 for HDR image stacks, in accordance with one embodiment.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of KIMURA in view of MATSUZAKI and further in view of ZHANG of a signal processing device, comprising: circuitry configured to: compare a reference signal with a threshold, wherein the reference signal is based on a large-pixel RAW signal of a pixel of interest; wherein the blending process blends a pixel value of a small-pixel RAW signal of the pixel of interest with a pixel value of the large-pixel RAW signal of the pixel of interest; compare the pixel value of the large-pixel RAW signal with a saturation determination threshold; output a saturation determination value based on the comparison of the pixel value of the large-pixel RAW signal with the saturation determination threshold, wherein the saturation determination value indicates whether the large-pixel RAW signal of the pixel of interest is in a saturated state; wherein the output value is one of the calculated value of the first blending ratio or 1; and output the selected output value of the first blending ratio with the teachings of GALLO of execute the blending process on an image with a wide dynamic range; and wherein the sensing device is configured to execute recognition processing based on the image with the wide dynamic range.
Wherein having KIMURA’s imaging signal processing device/method having execute the blending process on an image with a wide dynamic range; and wherein the sensing device is configured to execute recognition processing based on the image with the wide dynamic range.
The motivation behind the modification would have been to obtain an imaging signal processing device/method that enhances the accuracy of detecting and reducing noise and the operational speed at which the device works. Since both KIMURA and GALLO relate to processing and blending image signals, wherein KIMURA the accuracy of noise-level measurement can be improved, while GALLO some applications require HDR registration and blending to be performed at interactive frame rates (such as when viewing HDR video in real-time) while not sacrificing the quality of the product. Thus, there is a need for addressing these issues and/or other issues associated with the prior art. Please see KIMURA (US 20080106647 A1), Paragraph [0020], and GALLO et al. (US 20150125091 A1), Paragraph [0004].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant’s disclosure.
KOIZUMI et al. (US 20180084175 A1) – The present technology relates to an image capture device, a method of image capture, a signal processing device, a method of signal processing, and a program capable of capturing reliably a blinking image capture target. A predetermined image capture timing of an image capture unit is controlled so that image capture is performed N times during an exposure time in which at least a part of a turn-on period in which an image capture target is on overlaps with an exposure time of image capture of at least one time at a timing of dividing a blinking cycle in which the image capture target to be captured by the image capture unit configured to perform image capture at the image capture timing blinks into N equal parts where N represents more than one. The present technology is applicable to, in one example, a camera unit or the like used to capture an image…Abstract, Fig. 20.
IIJIMA et al. (US 20200175660 A1) - An image processor includes: an image sensor outputting a short exposure image and a long exposure image; a sensor controller that, when brightness of the subject changes, controls first exposure sensitivity to cause the short exposure image to have first brightness and controls second exposure sensitivity to cause the long exposure image to have second brightness; a motion blending ratio calculator calculating a motion blending ratio based on a motion amount of the subject; a motion-adapted image synthesizer generating a motion-adapted image by synthesizing a corrected short exposure image and the long exposure image based on the motion blending ratio; and an HDR image synthesizer generating an HDR image by synthesizing the motion-adapted image and the short exposure image together. When the subject becomes darker, the sensor controller controls the first and second exposure sensitivities to cause the first sensor gain to be at most the second sensor gain…Abstract, Fig. 1.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ETHAN N WOLFSON whose telephone number is (571)272-1898. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ETHAN N WOLFSON/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights