DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed August 3rd, 2026 has been entered. Claims 1-8 and 10-21 remain pending in the application. Claims 1, 16, 19 and 20 have been amended. Claim 9 has been canceled. Claim 21 is newly added.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Kuramoto (US 10441150 B2) in view of Yamashita (US 20140180004 A1).
Regarding Claim 1, Kuramoto teaches a method for color correcting images performed by a computing system (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 36-41: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image. A color converter performs color conversion of the first color image signal into a second color image signal, and adjusts a setting of the color conversion according to the set drive value of the light source apparatus”),
The method comprising:
Receiving, at a first time: first image data from an imaging device of a medical imaging system captured during a medical procedure (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”; Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”. Notes: an endoscope is a medical instrument. Given that the endoscope is utilized to image a body cavity, it is performed during a medical procedure); and
A first value of an operating parameter of a light source of the medical imaging system, wherein a color of light emitted from the light source shifts based on values of the operating parameter (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light);
Determining a first color shift based on the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”);
Applying a color correction function to the first image data to compensate for the first color shift to generate first color corrected image data (Column 2, lines 42-46: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value”. Notes: Drive value is a light specific value related to required current for producing certain wavelengths of light);
Providing, to a display associated with the computing system, the first color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Kuramoto does not explicitly teach receiving second image data a second time different from the first time, a second value of an operating parameter of a light source, where a color of light emitted by the light source shifts based on the values of the operating parameter, determining a second color shift based on the second value, applying a color correction function based on the second color shift, and displaying the second color corrected image.
However, the method of operating the medical imaging system of Kuramoto implicitly receives, at a second time different from the first time: second image data from an imaging device of a medical imaging system captured during a medical procedure (Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”; Column 10, lines 16-19: “The video signal generator 66 converts the RGB image signals inputted from the first or second image processing device 62 or 64 into video signals displayable on the monitor display panel 18”. Notes: Kuramoto clearly receives second image data at a second time given the presence of an imaging sensor that converts multiple image signals into video signals, which inherently includes multiple images corresponding with separate image data); and
A second value of an operating parameter of the light source, wherein the second value is different from the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light. Since current can be adjusted, a second value different from the first value can be obtained);
Determining a second color shift based on the second value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: a second color shift occurs with a wavelength shift based on the second value, obtained from second image data. As noted previously, the display of a video signal from image signals necessitates multiple instances of image data, from which color shifts resulting from changes to wavelength can occur, and therefore, color shifts are necessarily performed on the current value associated with an instance of image data, consistent with the video signal outputting color corrected images);
Applying a color correction function to the second image data to compensate for the second color shift to generate second color corrected image data (Column 2, lines 42-46: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value”; Column 3, lines 27-32: “Preferably, furthermore, a table memory stores the first color image signal and the second color image signal associated with the first color image signal, the table memory being accessed for the color conversion by the color converter. The table memory is associated with each one of levels of a drive value of the light source apparatus”. Notes: Drive value is a light specific value related to required current for producing certain wavelengths of light. The different wavelengths produced correspond with different levels of the drive value of the light source, and hence any changes of light wavelength capable from the light source has an associated value utilized for color conversion, which is performed according to the matrix operation as mentioned above);
Providing, to a display associated with the computing system, the second color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”);
It is common in the art to obtain multiple images of an object under different lighting; in the context of the medical instrument of Kuramoto, current adjusting the color of the light source is established, and as a result, one would be motivated to obtain image data regarding an object under a light source at a different current from an initial image data and initial current for the purpose of color adjusting the second image under a different color of light for viewing; this motivation is implicit in Kuramoto, considering the matrix coefficients for color converting images are dependent on the current of the light source, which is adjustable.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that the color correction method of Kumamoto can be applied to different images under different lighting conditions; Doing so would yield the predictable result of viewing images under different lighting conditions that have been color corrected for viewing in an applicable manner.
Kuramoto does not teach determining a difference between the first value and the second value satisfies a predefined threshold value; and in response to the difference between the first value and the second value satisfying the predefined threshold value: determining a second color shift based on the second value; applying the color correction function to the second image data to compensate for the second color shift to generate second color corrected image data; and providing, to the display, the second color corrected image for display.
However, Yamashita teaches a difference between the first value and the second value satisfies a predefined threshold value (Paragraph [0042]: “As shown in FIG. 2, the light adjusting circuit 14 as the light source control section controls brightness of the illumination light emitted from the white LED 17 by varying the pulse width of the PWM signal in every image pickup period of the CCD 9, that is, by adjusting driving time (ON/OFF time) of the white LED 17. In a case of performing observation with a normal light amount, the pulse width of the PWM signal is within a range wider than a first threshold TH1 and narrower than a second threshold TH2. In a case of a pulse P1 having the pulse width of the PWM signal wider than the first threshold TH1 and narrower than the second threshold TH2, a spectrum of the illumination light emitted from the white LED 17 in one image pickup period is a spectrum SP1 shown by the solid line in FIG. 3”; Paragraph [0045]: “As described above, when the pulse width of the PWM signal becomes not more than the first threshold TH1 or not less than the second threshold TH2, the color balance of the illumination light is lost, and therefore the endoscope apparatus 1 performs the color correction processing with respect to the video signal in the video signal processing section 23 of the processor 3”. Notes: pulse width of a light source is an operating parameter of a light source. A first and second operating parameter value (pulse width) is apparent as described by Paragraph [0045], which states “when the pulse width of the PWM signal becomes not more than the first threshold TH1 or not less than the second threshold TH2”, where for the pulse width to become more than the higher threshold or less than the lower threshold (second operating parameter value at a second time), an initial pulse width (first operating parameter value at a first time) must exist, contextually related to Paragraph [0042] describing an initial threshold range. Note that a difference between the first and second value is defined by a first value existing within a range described by Paragraph [0042], and a second value described by being outside that range in Paragraph [0045]); and
in response to the difference between the first value and the second value satisfying the predefined threshold value (Paragraph [0045]: “As described above, when the pulse width of the PWM signal becomes not more than the first threshold TH1 or not less than the second threshold TH2”. Notes: A difference between a first and second value can satisfy the predefined threshold value, as is evident in Paragraph [0042] and Paragraph [0045] when describing the predetermined lower and upper threshold value): determining a second color shift based on the second value (Paragraph [0045]: “As described above, when the pulse width of the PWM signal becomes not more than the first threshold TH1 or not less than the second threshold TH2 the color balance of the illumination light is lost, and therefore the endoscope apparatus 1 performs the color correction processing with respect to the video signal in the video signal processing section 23 of the processor 3”. Notes: a color shift is determined to occur considering color correction is performed); applying the color correction function to the second image data to compensate for the second color shift to generate second color corrected image data (Paragraph [0045]: “and therefore the endoscope apparatus 1 performs the color correction processing with respect to the video signal in the video signal processing section 23 of the processor 3”; Paragraph [0048]: “The set value information of the pulse width of the PWM signal inputted from the light adjusting circuit 14 is inputted to the color correction parameter deriving section 31. The color correction parameter deriving section 31 stores functions showing relation between the pulse width of the PWM signal and correction parameters. When the set value information of the pulse width of the PWM signal is inputted to the color correction parameter deriving section 31, the color correction parameter deriving section 31 derives correction parameters in accordance with the set value of the pulse width, i.e. the driving time of the white LED 17, on the basis of the functions”); and providing, to the display, the second color corrected image for display (Paragraph [0038]: “The video signal processing section 23 as a signal conversion section performs predetermined video signal processing such as gamma correction and enhance processing with respect to the digital signal from the A/D converter 22, the digital signal being converted into the video signal for display on the monitor 4. Further, the video signal processing section 23 outputs the video signal obtained by performing color correction processing, as described later, based on the set value information of the pulse width of the PWM signal from the light adjusting circuit 14, to the D/A converter 24”).
Kuramoto and Yamashita are considered analogous in the art with respect to the color correction of images in accordance with operating parameters of light sources in medical imaging. A common motivation in the art is to prevent overcorrection, or otherwise unnecessary correction to images based on light change; thresholds are commonly utilized to batch values and generalize changes, where doing so reduces and simplifies systems in cases where overcorrection is to be avoided, or where extreme precision or accuracy does not outweigh the cost of more computation without the generalization through thresholds. While Kuramoto’s operating parameter is current and Yamashita’s operating parameter is pulse width, both operating parameters are understood in the art to have an effect on the light source resulting in color correction functions being applied to correct images captured under the light source. A person having ordinary skill in the art before the effective filing date of the claimed invention would find it obvious that thresholding for operating parameters in general for light sources with regards to color correction of color shifts in the light resulting from changes in the operating parameters over time can be applied similarly, and is further supported by the fact that both Kuramoto and Yamashita teach color correction functions that map their respective operational parameter values to color correction parameters used to perform the color correction.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method for color correcting images with respect to values of an operational parameter obtained at different times of Kuramoto with the determination of a difference between values of an operational parameter obtained at different times satisfying a preset threshold, and subsequent determination of a second color shift and resultant color correction and display of an image of Yamashita; Doing so would yield the predictable result of a method for color correcting images depending on whether a difference in operational parameter values at different times indicates that color correction should be performed.
Regarding Claim 2, the method of Claim 1 is rejected over Kuramoto as modified.
Kuramoto as modified teaches characterizing a plurality of color shifts in light emitted by one or more light sources of one or more medical imaging systems, including at least the light source of the medical imaging system, across a plurality of values of the operating parameter to obtain and store color shift data (Kuramoto, Column 2, lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”; Kuramoto, Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”; Kuramoto, Column 6, lines 17-32: “The light source controller 21 controls an emitted light amount of each of the LEDs 20a-20d by applying predetermined current values (corresponding to a drive value of the LED in the invention) to the V-LED 20a, the B-LED 20b, the G-LED 20c and the R-LED 20d. The current values to be applied to the V-LED 20a, the B-LED 20b and the G-LED 20c are determined in accordance with a target light amount signal outputted from a light amount acquisition device 54 of the processing apparatus 16. The current value to be applied to the R-LED 20d is determined based on the light amount of the red light R measured by the measurement sensor 25, in addition to the target light amount signal. Note that, in this embodiment, the current value “c” to be applied to each of the LEDs 20a-20d is represented by 10 bits, that is, a value of 0-1023”; Kuramoto, Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”).
Regarding Claim 3, the method of Claim 2 is rejected over Kuramoto as modified.
Kuramoto as modified teaches determining the first color shift, comprising:
Identifying first color shift corresponding to the first value within the stored color shift data (Kuramoto, Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction),
The first color shift data indicating a measured change in pixel intensity values of one or more image data components based on the first color shift of the light emitted by the light source when the operating parameter has the first value (Kuramoto, Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: the matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image).
Regarding Claim 4, the method of Claim 3 is rejected over Kuramoto as modified.
Kuramoto as modified teaches applying the color correction function to the first image data, which comprises determining one or more first compensation coefficients of the color correction function based on the first color shift data to compensate for the measured change in the pixel intensity values of the one or more image data components (Kuramoto, Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”; Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Regarding Claim 5, the method of Claim 3 is rejected over Kuramoto as modified.
Kuramoto as modified teaches the first color shift data further indicating a timing associated with the measured change in the pixel intensity values of one or more image data components when the operating parameter has the first value (Kuramoto, Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed), and
Applying the color correction function to the first image data comprises: applying the color correction function to the first image data based on the timing (Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Regarding Claim 6, the method of Claim 2 is rejected over Kuramoto as modified.
Kuramoto as modified teaches determining the second color shift, comprising:
Identifying the second color shift corresponding to the second value within the stored color shift data (Kuramoto, Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected),
The second color shift data indicating a measured change in pixel intensity values of one or more image data components based on the second color shift of the light emitted by the light source when the operating parameter has the second value (Kuramoto, Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: the matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Regarding Claim 7, the method of Claim 6 is rejected over Kuramoto as modified.
Kuramoto as modified teaches applying the color correction function to the second image data, which comprises determining one or more second compensation coefficients of the color correction function based on the second color shift data to compensate for the measured change in the pixel intensity values of the one or more image data components (Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Regarding Claim 8, the method of Claim 6 is rejected over Kuramoto as modified.
Kuramoto as modified teaches the second color shift data further indicating a timing associated with the measured change in the pixel intensity values of one or more image data components when the operating parameter has the second value (Kuramoto, Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected), and
Applying the color correction function to the second image data comprises: applying the color correction function to the second image data based on the timing (Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Regarding Claim 13, the method of Claim 1 is rejected over Kuramoto as modified.
Kuramoto as modified teaches applying the color correction function to the first image data, which comprises:
Determining a first timing associated with the first color shift, the first timing including a measured time period from a detection of the operating parameter of the light source operating at the first value to an observation of the first color shift (Kuramoto, Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed); and
Applying the color correction function to the first image data based on the first timing (Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Regarding Claim 14, the method of Claim 1 is rejected over Kuramoto as modified.
Kuramoto as modified teaches applying the color correction function to the second image data, which comprises:
Determining a second timing associated with the second color shift, the second timing including a measured time period from a detection of the operating parameter of the light source operating at the second value to an observation of the second color shift (Kuramoto, Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected); and
Applying the color correction function to the second image data based on the second timing (Kuramoto, Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Claims 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kuramoto (US 10441150 B2) in view of D’Alfonso (US 20140052004 A1).
Regarding Claim 16, Kuramoto teaches A computing system for color correcting images that is communicatively connectable to a medical imaging system (Column 4, lines 57-62: “As illustrated in FIG. 1, an endoscope system 10 according to a first embodiment has an endoscope 12, a light source apparatus 14, a processing apparatus 16, a monitor display panel 18 and a console unit 19. The endoscope 12 is connected optically to the light source apparatus 14, and electrically to the processing apparatus 16”), the computing system comprising:
a data store storing color shift data obtained from a characterization of a plurality of color shifts in light emitted by one or more light sources of one or more medical imaging systems, including a light source of the medical imaging system, across a plurality of values of an operating parameter of the one or more light sources (Column 5, lines 32-34: “Note that an external storage (not shown) may be connected to the processing apparatus 16 to store the image information and the like”; Column 4, lines 57-62: “As illustrated in FIG. 1, an endoscope system 10 according to a first embodiment has an endoscope 12, a light source apparatus 14, a processing apparatus 16, a monitor display panel 18 and a console unit 19. The endoscope 12 is connected optically to the light source apparatus 14, and electrically to the processing apparatus 16”);
at least one memory storing instructions (Column 5, lines 32-34: “Note that an external storage (not shown) may be connected to the processing apparatus 16 to store the image information and the like”); and
one or more processors, including an image processor (Column 7, lines 42-47: “The processing apparatus 16 includes a receiver 53, the light amount acquisition device 54, a digital signal processor (DSP) 56, a noise canceller 58, a two-way switching device 60, a first image processing device 62 for normal light, a second image processing device 64 for special light, and a video signal generator 66”), wherein execution of the instructions by the one or more processors, causes the computing system to perform operations, including:
receiving, at a first time:
first image data from an imaging device of the medical imaging system captured during a medical procedure (Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”. Notes: endoscope system is a medical instrument, and considering its use involves illuminating an object in a body cavity, it is used during medical procedures); and
a first value of an operating parameter of the light source of the medical imaging system (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light);
identifying, from the color shift data stored in the data store, first color shift data corresponding to the first value, the first color shift data indicating a measured change in pixel intensity values of one or more image components based on a first color shift of the light emitted by the light source when the operating parameter has the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”; Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction. The matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image);
determining one or more first compensation coefficients of a color correction function based on the first color shift data to compensate for the measured change in the pixel intensity values of the one or more image components (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”);
applying the color correction function, based on the one or more first compensation coefficients, to the first image data to generate first color corrected image data (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”);
providing, to a display associated with the computing system, the first color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”);
receiving, at a second time different from the first time:
second image data from the imaging device captured during the medical procedure (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”. Notes: endoscope system is a medical instrument, and considering its use involves illuminating an object in a body cavity, it is used during medical procedures. As previously established, Kuramoto’s method is inherently receives a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected); and
a second value of the operating parameter of the light source, wherein the second value is different from the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light. Since current can be adjusted, a second value different from the first value can be obtained. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected);
identifying, from the color shift data stored in the data store, second color shift data corresponding to the second value, the second color shift data indicating a measured change in pixel intensity values of one or more image components based on a second color shift of the light emitted by the light source when the operating parameter has the second value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”; Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction. The matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected);
determining one or more second compensation coefficients of the color correction function based on the second color shift data to compensate for the measured change in the pixel intensity values of the one or more image components (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected);
applying the color correction function, based on the one or more second compensation coefficients, to the second image data to compensate for the second color shift to generate second color corrected image data (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected); and
providing, to the display, the second color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Kuramoto does not teach that the color shift data further includes information indicating a plurality of time periods measured from a detection of the operating parameter operating at the plurality of values to an observation of the plurality of color shifts, the plurality of time periods representative of a delay in timing of the plurality of color shifts in response to a change of the operating parameter to the plurality of values, and does not teach that for one or more of the first image data or the second image data, the color correction function is applied to the one or more of the first image data or the second image data based on a corresponding time period, of the plurality of time periods, the corresponding time period being representative of the delay in timing of a corresponding color shift, of the first color shift or the second color shift, in response to the operating parameter changing to a corresponding value, of the first value or the second value, to account for the delay.
However, D’Alfonso teaches color shift data further including information indicating a plurality of time periods measured from a detection of the operating parameter operating at the plurality of values to an observation of the plurality of color shifts, the plurality of time periods representative of a delay in timing of the plurality of color shifts in response to a change of the operating parameter to the plurality of values (Paragraph [0025]: “FIG. 11 is a graph showing the blue gain necessary to maintain white balance in relation to changing drive current over time in an example system according to an embodiment of the present invention”; Paragraph [0054]: “To correct color temperature shift, the processor FPGA 58 can compensate by adjusting the color balance of the video signal in the camera image processing system. There is a relationship between the LED temperature and the amount of color temperature shift. As shown in FIG. 9, as the temperature of the LED chip decreases, the color temperature of light emitted from the LED tends to change from bluish (high color temperature) to reddish (low color temperature). As LED power is changed by adjusting the drive signal, the processor FPGA can adjust color balance to hold the output image color temperature stable over the expected operating range”; Paragraph [0056]: “The pulse signal to the light source may change every frame and the color balance adjustments in the video processing can also be made to change every frame to match the expected color shift. FIG. 11 shows the blue correction necessary over time in an example system wherein the drive signal to the LED was changed from being constant (on for the full frame) to being pulsed at a width equal to an exposure period occupying about 3% of the frame time. The change in LED drive signal occurs after frame 9. As shown in FIG. 11, the color correction needed to compensate typically lags somewhat in time behind changes in drive power (duty cycle). The color correction needed typically exhibits a first order or simple low pass filter response. The time lag may be factored into the color correction to keep the final camera image color stable”. Notes: the time lag for a color correction pertaining to a color shift is described as a first order response, which establishes a plurality of time periods for each existing color correction for a color shift), and
that for one or more of the first image data or the second image data (Paragraph [0056]: “The pulse signal to the light source may change every frame and the color balance adjustments in the video processing can also be made to change every frame to match the expected color shift”), the color correction function is applied to the one or more of the first image data or the second image data based on a corresponding time period, of the plurality of time periods (Paragraph [0056]: “The pulse signal to the light source may change every frame and the color balance adjustments in the video processing can also be made to change every frame to match the expected color shift. FIG. 11 shows the blue correction necessary over time in an example system wherein the drive signal to the LED was changed from being constant (on for the full frame) to being pulsed at a width equal to an exposure period occupying about 3% of the frame time. The change in LED drive signal occurs after frame 9. As shown in FIG. 11, the color correction needed to compensate typically lags somewhat in time behind changes in drive power (duty cycle). The color correction needed typically exhibits a first order or simple low pass filter response. The time lag may be factored into the color correction to keep the final camera image color stable”. Notes: the time lag for a color correction pertaining to a color shift is described as a first order response, which establishes a plurality of time periods for each existing color correction for a color shift),
the corresponding time period being representative of the delay in timing of a corresponding color shift, of the first color shift or the second color shift (Paragraph [0025]: “FIG. 11 is a graph showing the blue gain necessary to maintain white balance in relation to changing drive current over time in an example system according to an embodiment of the present invention”; Paragraph [0056]: “The color correction needed typically exhibits a first order or simple low pass filter response. The time lag may be factored into the color correction to keep the final camera image color stable”; See Figure 11),
in response to the operating parameter changing to a corresponding value, of the first value or the second value, to account for the delay (Paragraph [0056]: “The pulse signal to the light source may change every frame and the color balance adjustments in the video processing can also be made to change every frame to match the expected color shift. FIG. 11 shows the blue correction necessary over time in an example system wherein the drive signal to the LED was changed from being constant (on for the full frame) to being pulsed at a width equal to an exposure period occupying about 3% of the frame time. The change in LED drive signal occurs after frame 9”).
Kuramoto and D’Alfonso are considered analogous in the art with respect to the changing of operating parameters of a light source for an imaging system and respective color correction of images obtained for the imaging system. The time delay of a color shift after a change in an operating parameter for a light source is known in the art, as is evident in D’Alfonso; One would be motivated to account for this delay to prevent over correction resulting in instability in the corrected image, as is stated in Alfonso. While Kuramoto’s operating parameter is current and D’Alfonso’s operating parameter is pulse width, both operating parameters are understood in the art to have an effect on the light source resulting in a color shift with subsequent color correction being applied to correct images captured under the light source. A person having ordinary skill in the art before the effective filing date of the claimed invention would find it obvious that a time lag with respect to a observed color shift resulting from a change in an operating parameter value would be observable for a change in either operating parameter value, as the end result is a color shift.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the color correcting method in accordance with a change in operating parameters of Kuramoto with the incorporation of information regarding a time delay for a color shift in color correction of an image; Doing so would yield the predictable result of a color correction method accounting for a time delay resulting from a color shift of a light source.
Regarding Claim 18, the computing system of Claim 16 is rejected over Kuramoto as modified.
Kuramoto as modified teaches an operating parameter being a current of the light source or temperature of the light source (Kuramoto, Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Kuramoto (US 10441150 B2) in view of D’Alfonso (US 20140052004 A1), and in further view of Yamashita (US 20140180004 A1)
Regarding Claim 17, the computing system of Claim 16 is rejected over Kuramoto as modified.
Kuramoto as modified does not teach determining a difference between the second value and the first value is above a predefined threshold.
However, Yamashita teaches determining a difference between a second value and a first value is above a predefined threshold (Paragraph [0042]: “As shown in FIG. 2, the light adjusting circuit 14 as the light source control section controls brightness of the illumination light emitted from the white LED 17 by varying the pulse width of the PWM signal in every image pickup period of the CCD 9, that is, by adjusting driving time (ON/OFF time) of the white LED 17. In a case of performing observation with a normal light amount, the pulse width of the PWM signal is within a range wider than a first threshold TH1 and narrower than a second threshold TH2. In a case of a pulse P1 having the pulse width of the PWM signal wider than the first threshold TH1 and narrower than the second threshold TH2, a spectrum of the illumination light emitted from the white LED 17 in one image pickup period is a spectrum SP1 shown by the solid line in FIG. 3”; Paragraph [0045]: “As described above, when the pulse width of the PWM signal becomes not more than the first threshold TH1 or not less than the second threshold TH2, the color balance of the illumination light is lost, and therefore the endoscope apparatus 1 performs the color correction processing with respect to the video signal in the video signal processing section 23 of the processor 3”. Notes: pulse width of a light source is an operating parameter of a light source. A first and second operating parameter value (pulse width) is apparent as described by Paragraph [0045], which states “when the pulse width of the PWM signal becomes not more than the first threshold TH1 or not less than the second threshold TH2”, where for the pulse width to become more than the higher threshold or less than the lower threshold (second operating parameter value at a second time), an initial pulse width (first operating parameter value at a first time) must exist, contextually related to Paragraph [0042] describing an initial threshold range. Note that a difference between the first and second value is defined by a first value existing within a range described by Paragraph [0042], and a second value described by being outside that range in Paragraph [0045]).
Kuramoto as modified and Yamashita are considered analogous in the art with respect to the color correction of images in accordance with operating parameters of light sources in medical imaging. A common motivation in the art is to prevent overcorrection, or otherwise unnecessary correction to images based on light change; thresholds are commonly utilized to batch values and generalize changes, where doing so reduces and simplifies systems in cases where overcorrection is to be avoided, or where extreme precision or accuracy does not outweigh the cost of more computation without the generalization through thresholds. While Kuramoto’s operating parameter is current and Yamashita’s operating parameter is pulse width, both operating parameters are understood in the art to have an effect on the light source resulting in color correction functions being applied to correct images captured under the light source. A person having ordinary skill in the art before the effective filing date of the claimed invention would find it obvious that thresholding for operating parameters in general for light sources with regards to color correction of color shifts in the light resulting from changes in the operating parameters over time can be applied similarly, and is further supported by the fact that both Kuramoto and Yamashita teach color correction functions that map their respective operational parameter values to color correction parameters used to perform the color correction.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method for color correcting images with respect to values of an operational parameter obtained at different times of Kuramoto as modified with the determination of a difference between values of an operational parameter obtained at different times satisfying a preset threshold, and subsequent determination of a second color shift and resultant color correction and display of an image of Yamashita; Doing so would yield the predictable result of a method for color correcting images depending on whether a difference in operational parameter values at different times indicates that color correction should be performed
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Kuramoto (US 10441150 B2) in view of Yamashita (US 20140180004 A1), and in further view of Matsura (Japanese Patent No. 4787032).
Regarding Claim 15, the method of Claim 1 is rejected over Kuramoto as modified.
Kuramoto as modified teaches an operating parameter that is a first operating parameter (Kuramoto, Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light),
A second operating parameter of the light source (Kuramoto, Column 1, lines 29-38: “It is known that a light amount of the semiconductor light source varies with temperature variation, aging degradation, and the like. In the case of using a combination of a plurality of color semiconductor light sources, the ratio in the emitted light amount between the plural color semiconductor light sources is necessarily set at a predetermined value even at any brightness. However, variation in the emitted light amount of any color of the semiconductor light sources due to the temperature variation or the like makes the ratio go out of the set value to create a change in a color tone”), and
Determining the first operating parameter has a more dominant effect on the color of the light emitted from the light source than the second operating parameter (Kuramoto, Column 6, lines 49-58: “According to this embodiment, as described above, the light amount of the red light R is monitored using the measurement sensor 25, and the emitted light amount of the R-LED 20d is feedback controlled based on the result of monitoring. Thus, even assuming that the emitted light amount of the R-LED 20d varies due to a temperature drift (wavelength shift) or aging degradation, adjustment of the current value to be applied to the R-LED 20d corrects the variation, and therefore, the emitted light amount of the R-LED 20d is always maintained at the target value”. Notes: controlling the current corrects for variation resulting from temperature change, and hence is more dominant than temperature)
Kuramoto as modified does not teach receiving, at the first time, a first value of a second operating parameter of the light source in addition to the first value of the first operating parameter.
However, Matsura teaches determining a first value of a second operating parameter of the light source (Paragraph [0034]: “Therefore, the reference brightness serving as the reference of the brightness of the LED light source 33 is determined in advance, and the gain is calculated on the basis of the value obtained by dividing the current brightness obtained on the basis of the detected temperature of the LED light source 33 by the reference brightness”; Paragraph [0035]: “The gain thus obtained is multiplied by the luminance signal component of the image signal in the post-stage signal processing circuit 45. The coefficient of the temperature-to-luminance characteristic of the LED for obtaining the luminance of the LED light source 33 at that temperature from the temperature of the LED light source 33 is stored in the ROM 47, and is sent to the processor controller 46 as necessary”. Notes: temperature reads on second operating parameter).
Kuramoto as modified and Matsura are considered analogous in the art with respect to methods of color correction of images derived using medical systems with a light source, specifically endoscopes. A common motivation in the art is to control for all relevant parameters that can affect the color change resulting from the use of a light source on an endoscope, as is evident in Kuramoto (Kuramoto, Column 1, lines 29-32: “It is known that a light amount of the semiconductor light source varies with temperature variation, aging degradation, and the like”).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method for color correction utilizing a current value of Kuramoto as modified with the determining of a temperature value of Matsura for the purpose of color correction in medical imaging; Doing so would yield the predictable result of a medical imaging system with two operating parameters with determined values.
Response to Arguments
Applicant’s arguments, see Applicant Remarks, pg. 15-17, filed August 3rd, 2026, with respect to the rejection of Claim 1 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection, necessitated by amendment, is made in view of Kuramoto (US 10441150 B2) in view of Yamashita (US 20140180004 A1).
Applicant asserts that Kuramoto fails to teach “determining a difference between the first value and the second value satisfies a predefined threshold value”, and subsequent application of the color correction function and display of the second corrected image “in response to the difference between the first value and the second value satisfying the predefined threshold value”, as recited in amended independent Claim 1.
The Examiner agrees that Kuramoto fails to teach the aforementioned limitations as recited in amended independent Claim 1.
However, Yamashita teaches “determining a difference between the first value and the second value satisfies a predefined threshold value”, and subsequent application of the color correction function and display of the second corrected image “in response to the difference between the first value and the second value satisfying the predefined threshold value”. Refer to the rejection of amended independent Claim 1 for more details and reasons for combination.
Dependent claims of amended independent Claim 1 are not considered allowable based on the allowability of amended independent Claim 1.
Applicant’s arguments, see Applicant Remarks, pg. 17-19, filed August 3rd, 2026, with respect to the rejection of Claim 16 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection, necessitated by amendment, is made in view of Kuramoto (US 10441150 B2) in view of D’Alfonso (US 20140052004 A1).
Applicant asserts that Kuramoto fails to teach “wherein, for one or more of the first image data or the second image data, the color correction function is applied to the one or more of the first image data or the second image data based on a corresponding time period, of the plurality of time periods, the corresponding time period being representative of the delay in timing of a corresponding color shift, of the first color shift or the second color shift, in response to the operating parameter changing to a corresponding value, of the first value or the second value, to account for the delay”, as recited in amended independent Claim 16.
The Examiner agrees that Kuramoto fails to teach the aforementioned limitations as recited in amended independent Claim 16.
However, D’Alfonso teaches “wherein, for one or more of the first image data or the second image data, the color correction function is applied to the one or more of the first image data or the second image data based on a corresponding time period, of the plurality of time periods, the corresponding time period being representative of the delay in timing of a corresponding color shift, of the first color shift or the second color shift, in response to the operating parameter changing to a corresponding value, of the first value or the second value, to account for the delay”. Refer to the rejection of amended independent Claim 16 for more details and reasons for combination.
Dependent claims of amended independent Claim 16 are not considered allowable based on the allowability of amended independent Claim 16.
Applicant’s arguments, see Applicant Remarks, pg 19-20, filed August 3rd, 2026, with respect to independent Claim 19 have been fully considered and are persuasive. The rejection of independent Claim 19 has been withdrawn.
With regards to the Applicant’s request to rejoin withdrawn Claims 10-12, Claims 10-12 are only eligible to be rejoined on condition of the allowance of their respective generic claim. Amended independent Claim 1 is not allowable, therefore, Claims 10-12, being dependent or ultimately being dependent on Claim 1, are ineligible to be rejoined.
Allowable Subject Matter
Claims 19-21 are allowed.
Independent Claim 19 is deemed allowable based on the limitation reciting “wherein, at one or more of the first time or the second time, one or more of a first value or a second value of a second operating parameter of the light source, different from the first operating parameter and also causing the color of light emitted from the light source to shift, is received, the second operating parameter being the other of the current of the light source or the temperature of the light source and wherein the first operating parameter is selected for use as the basis for determining one or more of the first color shift or the second color shift based on a comparison between a first effect of the first operating parameter on the color and a second effect of the second operating parameter on the color, when the first operating parameter and the second operating parameter are operating at respective values, that determines the first operating parameter has a more dominant effect on the color than the second operating parameter”. Specifically, determining whether current or temperature as an operating parameter is to be utilized as a basis for determining color shift, depending on whether the current or temperature operating parameter has a more dominant effect on color, is not apparent in prior art, nor would a person having ordinary skill in the art before the effective filing date of the claimed invention find doing so obvious.
Claims 20-21, being dependent or ultimately dependent on Claim 19, are allowed based on the allowability of independent Claim 19.
Notable art close to the claimed invention is Kuramoto (US 10441150 B2).
While determining whether light or temperature having more of an effect on color is known in the art (Kuramoto, Column 6, lines 49-58: “According to this embodiment, as described above, the light amount of the red light R is monitored using the measurement sensor 25, and the emitted light amount of the R-LED 20d is feedback controlled based on the result of monitoring. Thus, even assuming that the emitted light amount of the R-LED 20d varies due to a temperature drift (wavelength shift) or aging degradation, adjustment of the current value to be applied to the R-LED 20d corrects the variation, and therefore, the emitted light amount of the R-LED 20d is always maintained at the target value”. Notes: controlling the current corrects for variation resulting from temperature change, and hence is more dominant than temperature), explicitly choosing either current or temperature as an operating parameter to form a basis on which to determine color shift based on a dominance of the current over the temperature, or vice versa, regarding an effect on color, is not known in prior art. Furthermore, a person having ordinary skill in the art before the effective filing date of the claimed invention would not find it obvious to do so.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
