DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Claims 18-20 along with generic claims 1-9 and 13-17 in the reply filed on March 3rd, 2026 is acknowledged. Claims 10-12 are subsequently considered withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-9, 13-14, and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kuramoto (US 10441150 B2).
Regarding Claim 1, Kuramoto teaches a method for color correcting images performed by a computing system (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 36-41: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image. A color converter performs color conversion of the first color image signal into a second color image signal, and adjusts a setting of the color conversion according to the set drive value of the light source apparatus”),
The method comprising:
Receiving, at a first time: first image data from an imaging device of a medical imaging system captured during a medical procedure (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”; Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”. Notes: an endoscope is a medical instrument. Given that the endoscope is utilized to image a body cavity, it is performed during a medical procedure); and
A first value of an operating parameter of a light source of the medical imaging system, wherein a color of light emitted from the light source shifts based on values of the operating parameter (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light);
Determining a first color shift based on the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”);
Applying a color correction function to the first image data to compensate for the first color shift to generate first color corrected image data (Column 2, lines 42-46: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value”. Notes: Drive value is a light specific value related to required current for producing certain wavelengths of light);
Providing, to a display associated with the computing system, the first color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Kuramoto does not explicitly teach receiving second image data a second time different from the first time, a second value of an operating parameter of a light source, where a color of light emitted by the light source shifts based on the values of the operating parameter, determining a second color shift based on the second value, applying a color correction function based on the second color shift, and displaying the second color corrected image.
However, the method of operating the medical imaging system of Kuramoto implicitly receives, at a second time different from the first time: second image data from an imaging device of a medical imaging system captured during a medical procedure (Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”; Column 10, lines 16-19: “The video signal generator 66 converts the RGB image signals inputted from the first or second image processing device 62 or 64 into video signals displayable on the monitor display panel 18”. Notes: Kuramoto clearly receives second image data at a second time given the presence of an imaging sensor that converts multiple image signals into video signals, which inherently includes multiple images corresponding with separate image data); and
A second value of an operating parameter of the light source, wherein the second value is different from the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light. Since current can be adjusted, a second value different from the first value can be obtained);
Determining a second color shift based on the second value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: a second color shift occurs with a wavelength shift based on the second value, obtained from second image data. As noted previously, the display of a video signal from image signals necessitates multiple instances of image data, from which color shifts resulting from changes to wavelength can occur, and therefore, color shifts are necessarily performed on the current value associated with an instance of image data, consistent with the video signal outputting color corrected images);
Applying a color correction function to the second image data to compensate for the second color shift to generate second color corrected image data (Column 2, lines 42-46: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value”; Column 3, lines 27-32: “Preferably, furthermore, a table memory stores the first color image signal and the second color image signal associated with the first color image signal, the table memory being accessed for the color conversion by the color converter. The table memory is associated with each one of levels of a drive value of the light source apparatus”. Notes: Drive value is a light specific value related to required current for producing certain wavelengths of light. The different wavelengths produced correspond with different levels of the drive value of the light source, and hence any changes of light wavelength capable from the light source has an associated value utilized for color conversion, which is performed according to the matrix operation as mentioned above);
Providing, to a display associated with the computing system, the second color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”);
It is common in the art to obtain multiple images of an object under different lighting; in the context of the medical instrument of Kuramoto, current adjusting the color of the light source is established, and as a result, one would be motivated to obtain image data regarding an object under a light source at a different current from an initial image data and initial current for the purpose of color adjusting the second image under a different color of light for viewing; this motivation is implicit in Kuramoto, considering the matrix coefficients for color converting images are dependent on the current of the light source, which is adjustable.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that the color correction method of Kumamoto can be applied to different images under different lighting conditions; Doing so would yield the predictable result of viewing images under different lighting conditions that have been color corrected for viewing in an applicable manner.
Regarding Claim 2, the method of Claim 1 is rejected over Kuramoto.
Kuramoto teaches characterizing a plurality of color shifts in light emitted by one or more light sources of one or more medical imaging systems, including at least the light source of the medical imaging system, across a plurality of values of the operating parameter to obtain and store color shift data (Column 2, lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”; Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”; Column 6, lines 17-32: “The light source controller 21 controls an emitted light amount of each of the LEDs 20a-20d by applying predetermined current values (corresponding to a drive value of the LED in the invention) to the V-LED 20a, the B-LED 20b, the G-LED 20c and the R-LED 20d. The current values to be applied to the V-LED 20a, the B-LED 20b and the G-LED 20c are determined in accordance with a target light amount signal outputted from a light amount acquisition device 54 of the processing apparatus 16. The current value to be applied to the R-LED 20d is determined based on the light amount of the red light R measured by the measurement sensor 25, in addition to the target light amount signal. Note that, in this embodiment, the current value “c” to be applied to each of the LEDs 20a-20d is represented by 10 bits, that is, a value of 0-1023”; Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”).
Regarding Claim 3, the method of Claim 2 is rejected over Kuramoto.
Kuramoto teaches determining the first color shift, comprising:
Identifying first color shift corresponding to the first value within the stored color shift data (Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction),
The first color shift data indicating a measured change in pixel intensity values of one or more image data components based on the first color shift of the light emitted by the light source when the operating parameter has the first value (Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: the matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image).
Regarding Claim 4, the method of Claim 3 is rejected over Kuramoto.
Kuramoto teaches applying the color correction function to the first image data, which comprises determining one or more first compensation coefficients of the color correction function based on the first color shift data to compensate for the measured change in the pixel intensity values of the one or more image data components (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”; Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Regarding Claim 5, the method of Claim 3 is rejected over Kuramoto.
Kuramoto teaches the first color shift data further indicating a timing associated with the measured change in the pixel intensity values of one or more image data components when the operating parameter has the first value (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed), and
Applying the color correction function to the first image data comprises: applying the color correction function to the first image data based on the timing (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Regarding Claim 6, the method of Claim 2 is rejected over Kuramoto.
Kuramoto teaches determining the second color shift, comprising:
Identifying the second color shift corresponding to the second value within the stored color shift data (Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected),
The second color shift data indicating a measured change in pixel intensity values of one or more image data components based on the second color shift of the light emitted by the light source when the operating parameter has the second value (Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: the matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Regarding Claim 7, the method of Claim 6 is rejected over Kuramoto.
Kuramoto teaches applying the color correction function to the second image data, which comprises determining one or more second compensation coefficients of the color correction function based on the second color shift data to compensate for the measured change in the pixel intensity values of the one or more image data components (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Regarding Claim 8, the method of Claim 6 is rejected over Kuramoto.
Kuramoto teaches the second color shift data further indicating a timing associated with the measured change in the pixel intensity values of one or more image data components when the operating parameter has the second value (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected), and
Applying the color correction function to the second image data comprises: applying the color correction function to the second image data based on the timing (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Regarding Claim 9, the method of Claim 1 is rejected over Kuramoto.
Kuramoto teaches determining a difference between the second value and the first value (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”; Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected. Each unique current value has an associated matrix coefficient from which color correction is performed. Therefore, determining a difference between two different current values is inherent).
Kuramoto does not explicitly teach determining a difference between the second and the first value is above a predefined threshold value.
However, increasing current implicitly defines a predefined threshold value for current change at the smallest unit specific to the degree of control available for detecting current change.
It is obvious that for a change in current to be registered, a predefined threshold value for observing change must exist.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that a predefined threshold value with regards to a first and second value of current with regards to a light source is inherently incorporated into the method of Kuramoto; Doing so would yield the predictable result of establishing a change in current with regards to the sensitivity of control available for the light source.
Regarding Claim 13, the method of Claim 1 is rejected over Kuramoto.
Kuramoto teaches applying the color correction function to the first image data, which comprises:
Determining a first timing associated with the first color shift, the first timing including a measured time period from a detection of the operating parameter of the light source operating at the first value to an observation of the first color shift (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed); and
Applying the color correction function to the first image data based on the first timing (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”).
Regarding Claim 14, the method of Claim 1 is rejected over Kuramoto.
Kuramoto teaches applying the color correction function to the second image data, which comprises:
Determining a second timing associated with the second color shift, the second timing including a measured time period from a detection of the operating parameter of the light source operating at the second value to an observation of the second color shift (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: a time is associated with each encountered current value when the current is changed. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected); and
Applying the color correction function to the second image data based on the second timing (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Regarding Claim 16, Kuramoto teaches A computing system for color correcting images that is communicatively connectable to a medical imaging system (Column 4, lines 57-62: “As illustrated in FIG. 1, an endoscope system 10 according to a first embodiment has an endoscope 12, a light source apparatus 14, a processing apparatus 16, a monitor display panel 18 and a console unit 19. The endoscope 12 is connected optically to the light source apparatus 14, and electrically to the processing apparatus 16”), the computing system comprising:
a data store storing color shift data obtained from a characterization of a plurality of color shifts in light emitted by one or more light sources of one or more medical imaging systems, including a light source of the medical imaging system, across a plurality of values of an operating parameter of the one or more light sources (Column 5, lines 32-34: “Note that an external storage (not shown) may be connected to the processing apparatus 16 to store the image information and the like”; Column 4, lines 57-62: “As illustrated in FIG. 1, an endoscope system 10 according to a first embodiment has an endoscope 12, a light source apparatus 14, a processing apparatus 16, a monitor display panel 18 and a console unit 19. The endoscope 12 is connected optically to the light source apparatus 14, and electrically to the processing apparatus 16”);
at least one memory storing instructions (Column 5, lines 32-34: “Note that an external storage (not shown) may be connected to the processing apparatus 16 to store the image information and the like”); and
one or more processors, including an image processor (Column 7, lines 42-47: “The processing apparatus 16 includes a receiver 53, the light amount acquisition device 54, a digital signal processor (DSP) 56, a noise canceller 58, a two-way switching device 60, a first image processing device 62 for normal light, a second image processing device 64 for special light, and a video signal generator 66”), wherein execution of the instructions by the one or more processors, causes the computing system to perform operations, including:
receiving, at a first time:
first image data from an imaging device of the medical imaging system captured during a medical procedure (Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”. Notes: endoscope system is a medical instrument, and considering its use involves illuminating an object in a body cavity, it is used during medical procedures); and
a first value of an operating parameter of the light source of the medical imaging system (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light);
identifying, from the color shift data stored in the data store, first color shift data corresponding to the first value, the first color shift data indicating a measured change in pixel intensity values of one or more image components based on a first color shift of the light emitted by the light source when the operating parameter has the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”; Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction. The matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image);
determining one or more first compensation coefficients of a color correction function based on the first color shift data to compensate for the measured change in the pixel intensity values of the one or more image components (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”);
applying the color correction function, based on the one or more first compensation coefficients, to the first image data to generate first color corrected image data (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”);
providing, to a display associated with the computing system, the first color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”);
receiving, at a second time different from the first time:
second image data from the imaging device captured during the medical procedure (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”. Notes: endoscope system is a medical instrument, and considering its use involves illuminating an object in a body cavity, it is used during medical procedures. As previously established, Kuramoto’s method is inherently receives a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected); and
a second value of the operating parameter of the light source, wherein the second value is different from the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light. Since current can be adjusted, a second value different from the first value can be obtained. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected);
identifying, from the color shift data stored in the data store, second color shift data corresponding to the second value, the second color shift data indicating a measured change in pixel intensity values of one or more image components based on a second color shift of the light emitted by the light source when the operating parameter has the second value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”; Column 2, lines 42-51: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value. Preferably, furthermore, a table memory stores a drive value of the light source apparatus and the first matrix coefficient associated with the drive value, the table memory being accessed for the matrix operation by the color converter”; Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: Stored current values (drive values) correspond with the stored matrix coefficient used for color correction. The matrix coefficients are used in conjunction with the input image signals to color correct the image signals such that they are viewed under normal light after correction; Therefore, each stored matrix coefficient corresponding with a stored current is indicative of a measured change in pixel intensity values of the color corrected output image derived from input image. As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected);
determining one or more second compensation coefficients of the color correction function based on the second color shift data to compensate for the measured change in the pixel intensity values of the one or more image components (Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected);
applying the color correction function, based on the one or more second compensation coefficients, to the second image data to compensate for the second color shift to generate second color corrected image data (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected); and
providing, to the display, the second color corrected image data for display (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected).
Claim 17, being similar in scope to Claim 9, is rejected under the same rationale.
Regarding Claim 18, the computing system of Claim 16 is rejected over Kuramoto.
Kuramoto teaches an operating parameter being a current of the light source or temperature of the light source (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light).
Regarding Claim 19, Kuramoto teaches a method for color correcting images performed by a computing system, the method comprising:
receiving, at a first time:
first image data from an imaging device of a medical imaging system captured during a medical procedure (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”; Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”. Notes: an endoscope is a medical instrument. Given that the endoscope is utilized to image a body cavity, it is performed during a medical procedure); and
a first value of an operating parameter of a light source of the medical imaging system, wherein a color of light emitted from the light source shifts based on values of the operating parameter, wherein the operating parameter is one of a current of the light source or a temperature of the light source (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light),
determining a first color shift based on the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: a color shift is determined in conjunction with the change in current, which results in a change in color tone);
applying a color correction function to the first image data to compensate for the first color shift to generate first color corrected image data (Column 2, lines 42-46: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value”. Notes: Drive value is a light specific value related to required current for producing certain wavelengths of light);
providing, to a display associated with the computing system, the first color corrected image data for display (Column 10, lines 56-59: “The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18”); and
receiving, at a second time different from the first time: second image data from the imaging device captured during the medical procedure (Column 2, Lines 24-28: “In view of the foregoing problems, an object of the present invention is to provide an endoscope system that can correct a change in a color tone associated with a wavelength shift of a semiconductor light source and the like, and a method for operating the endoscope system”; Column 2, Lines 29-32: “In order to achieve the above and other objects and advantages of this invention, an endoscope system includes a semiconductor light source apparatus for emitting light to illuminate an object in a body cavity”; Column 2, Lines 36-37: “An image sensor images the object illuminated with the light to obtain a first color image signal of a color image”; Column 10, lines 16-19: “The video signal generator 66 converts the RGB image signals inputted from the first or second image processing device 62 or 64 into video signals displayable on the monitor display panel 18”. Notes: Kuramoto clearly receives second image data at a second time given the presence of an imaging sensor that converts multiple image signals into video signals, which inherently includes multiple images corresponding with separate image data. As previously noted, an endoscope is a medical instrument. Given that the endoscope is utilized to image a body cavity, it is performed during a medical procedure); and
A second value of the operating parameter of the light source, wherein the second value is different from the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light. Since current can be adjusted, a second value different from the first value can be obtained);
determining a difference between the second value and the first value is above a predefined threshold value (as noted previously in the rejection of Claim 9, increasing current implicitly defines a predefined threshold value for current change at the smallest unit specific to the degree of control available for detecting current change); and
based on the difference being above the predefined threshold value: determining a second color shift based on the first value (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: a change in current (as identified in being above a predefined threshold value) allows a color shift to be performed based on a change in the first value (current));
applying the color correction function to the second image data to compensate for the second color shift to generate second color corrected image data (Column 2, lines 42-46: “Preferably, the color conversion includes a matrix operation of converting the first color image signal into the second color image signal according to a first matrix coefficient, and the first matrix coefficient is changeable according to the set drive value”; Column 3, lines 27-32: “Preferably, furthermore, a table memory stores the first color image signal and the second color image signal associated with the first color image signal, the table memory being accessed for the color conversion by the color converter. The table memory is associated with each one of levels of a drive value of the light source apparatus”. Notes: Drive value is a light specific value related to required current for producing certain wavelengths of light. The different wavelengths produced correspond with different levels of the drive value of the light source, and hence any changes of light wavelength capable from the light source has an associated value utilized for color conversion, which is performed according to the matrix operation as mentioned above); and
providing, to the display, the second color corrected image data for display (Column 10, lines 56-59: “The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18”).
Claim 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kuramoto (US 10441150 B2) in view of Matsura (Japanese Patent No. 4787032).
Regarding Claim 15, the method of Claim 1 is rejected over Kuramoto.
Kuramoto teaches an operating parameter that is a first operating parameter (Column 9, lines 41-56: “Note that the peak wavelength of the R-LED shifts to the long wavelength side with an increase in the light intensity, in other words, with an increase in the current value “c” (see FIGS. 18 and 20), and therefore, the first matrix coefficient Mij_c is so determined as to correct a change in a color tone associated with the wavelength shift. In imaging with dye of the crystal violet, for example, redness in the image becomes strong as the light intensity of the R-LED increases. To correct the redness, the first matrix coefficients M00_c, M01_c and M02_c are so determined as to lower the color-converted R image signal in a case where the current value “c” exceeds a predetermined value. For example, M00_c by which the R image signal is multiplied may be decreased, or M01_c and M02_c by which the G image signal and the B image signal are multiplied, respectively, may be increased”. Notes: current is considered an operating parameter due to its control over light),
A second operating parameter of the light source (Column 1, lines 29-38: “It is known that a light amount of the semiconductor light source varies with temperature variation, aging degradation, and the like. In the case of using a combination of a plurality of color semiconductor light sources, the ratio in the emitted light amount between the plural color semiconductor light sources is necessarily set at a predetermined value even at any brightness. However, variation in the emitted light amount of any color of the semiconductor light sources due to the temperature variation or the like makes the ratio go out of the set value to create a change in a color tone”), and
Determining the first operating parameter has a more dominant effect on the color of the light emitted from the light source than the second operating parameter (Column 6, lines 49-58: “According to this embodiment, as described above, the light amount of the red light R is monitored using the measurement sensor 25, and the emitted light amount of the R-LED 20d is feedback controlled based on the result of monitoring. Thus, even assuming that the emitted light amount of the R-LED 20d varies due to a temperature drift (wavelength shift) or aging degradation, adjustment of the current value to be applied to the R-LED 20d corrects the variation, and therefore, the emitted light amount of the R-LED 20d is always maintained at the target value”. Notes: controlling the current corrects for variation resulting from temperature change, and hence is more dominant than temperature)
Kuramoto does not teach receiving, at the first time, a first value of a second operating parameter of the light source in addition to the first value of the first operating parameter.
However, Matsura teaches determining a first value of a second operating parameter of the light source (Paragraph [0034]: “Therefore, the reference brightness serving as the reference of the brightness of the LED light source 33 is determined in advance, and the gain is calculated on the basis of the value obtained by dividing the current brightness obtained on the basis of the detected temperature of the LED light source 33 by the reference brightness”; Paragraph [0035]: “The gain thus obtained is multiplied by the luminance signal component of the image signal in the post-stage signal processing circuit 45. The coefficient of the temperature-to-luminance characteristic of the LED for obtaining the luminance of the LED light source 33 at that temperature from the temperature of the LED light source 33 is stored in the ROM 47, and is sent to the processor controller 46 as necessary”. Notes: temperature reads on second operating parameter).
Kuramoto and Matsura are considered analogous in the art with respect to methods of color correction of images derived using medical systems with a light source, specifically endoscopes. A common motivation in the art is to control for all relevant parameters that can affect the color change resulting from the use of a light source on an endoscope, as is evident in Kuramoto (Column 1, lines 29-32: “It is known that a light amount of the semiconductor light source varies with temperature variation, aging degradation, and the like”).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method for color correction utilizing a current value of Kuramoto with the determining of a temperature value of Matsura for the purpose of color correction in medical imaging; Doing so would yield the predictable result of a medical imaging system with two operating parameters with determined values.
Regarding Claim 20, the method of Claim 19 is rejected over Kuramoto.
Kuramoto teaches determining a difference between the second value and the first value of an operating parameter, where the operating parameter is current of the light source (Column 10, lines 48-61: “The current value “c” of the R-LED 20d outputted from the light source controller 21 is inputted to the 3×3 matrix circuit 80 of the normal color converter 68. In the 3×3 matrix circuit 80, the current value “c” is inputted to each LUT_Mij. Each LUT_Mij outputs the first matrix coefficient Mij_c corresponding to the inputted current value “c”. The RGB image signals are subjected to the matrix operation based on the outputted first matrix coefficient Mij_c. Thus, the color-converted RGB image signals are obtained. The normal light image is produced from the color-converted RGB image signals, and displayed on the monitor display panel 18. In the normal light image, a change in a color tone associated with the wavelength shift and aging degradation of the R-LED 20d is prevented”; Column 9, lines 26-40: “Note that the correlation between the current value “c” of the R-LED 20d and the first matrix coefficient Mij stored in each LUT_Mij is obtained by initial measurement at the time of shipping the endoscope system 10 and determined as follows. First, a minimum current value “Cmin” is applied to the R-LED 20d to emit the red light R, and the object of interest is imaged under irradiation with the red light R to output the RGB image signals. A first matrix coefficient Mij_0 is determined based on the outputted RGB image signals and target RGB image signals. The determined first matrix coefficient Mij_0 is stored to the LUT_Mij. Then, the current value “c” to be applied to the R-LED 20d is gradually increased. At each time that the current value “c” is increased, a first matrix coefficient Mij_p (p is an integer of 1-1023) is calculated and stored to the LUT_Mij”. Notes: As previously established, Kuramoto’s method is capable of receiving a second image to be color corrected, with an identical method of color correction as is performed to the first image to be color corrected. Each unique current value has an associated matrix coefficient from which color correction is performed. Therefore, determining a difference between two different current values is inherent).
Kuramoto does not explicitly teach a predefined threshold value.
However, increasing current implicitly defines a predefined threshold value for current change at the smallest unit specific to the degree of control available for detecting current change.
It is obvious that for a change in current to be registered, a predefined threshold value for observing change must exist.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that a predefined threshold value with regards to a first and second value of current with regards to a light source is inherently incorporated into the method of Kuramoto; Doing so would yield the predictable result of establishing a change in current with regards to the sensitivity of control available for the light source.
Kuramoto also does not teach a second predefined threshold value associated with the temperature of the light source.
However, Matsura teaches determining a first value of a second operating parameter of the light source (Paragraph [0034]: “Therefore, the reference brightness serving as the reference of the brightness of the LED light source 33 is determined in advance, and the gain is calculated on the basis of the value obtained by dividing the current brightness obtained on the basis of the detected temperature of the LED light source 33 by the reference brightness”; Paragraph [0035]: “The gain thus obtained is multiplied by the luminance signal component of the image signal in the post-stage signal processing circuit 45. The coefficient of the temperature-to-luminance characteristic of the LED for obtaining the luminance of the LED light source 33 at that temperature from the temperature of the LED light source 33 is stored in the ROM 47, and is sent to the processor controller 46 as necessary”).
Kuramoto and Matsura are considered analogous in the art with respect to methods of color correction of images derived using medical systems with a light source, specifically endoscopes. A common motivation in the art is to control for all relevant parameters that can affect the color change resulting from the use of a light source on an endoscope, as is evident in Kuramoto (Column 1, lines 29-32: “It is known that a light amount of the semiconductor light source varies with temperature variation, aging degradation, and the like”).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method for color correction utilizing a current value of Kuramoto with the determining of a temperature value of Matsura for the purpose of color correction in medical imaging; Doing so would yield the predictable result of a medical imaging system with two operating parameters with determined values.
Kuramoto as modified by Matsura does not teach a second predefined threshold regarding the temperature of the light source.
However, the same reasoning as a threshold for current values being implicit in Kuramoto can be applied to Matsura regarding the temperature threshold.
Lastly, Kuramoto as modified does not teach that the first predefined threshold value associated with the current of the light source is less than that of a second predefined threshold value associated with the temperature of the light source.
However, the broadest reasonable interpretation of the limitation is that the numerical value of the threshold value of the current of the light source is less than that of the threshold value of the temperature of the light source, since there is not an apparent conversion between a unit of current with a unit of temperature; It is obvious in the art that units of measure may be changed to bigger or smaller units (for instance, a meter is a fraction of a kilometer, and a centimeter is a fraction of a meter). In an arbitrary comparison of units, the units may be scaled such that the value associated with the current in a particular unit of size is smaller than that of the value associated with the current in a particular unit of size.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that units of measure of the current and temperature thresholds of Kuramoto as modified can be scaled up or down, such that the current threshold is numerically less than that of the temperature threshold; Doing so would yield the predictable result of the current threshold being less than that of the temperature threshold.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights