DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 8/27/2025 has been entered.
Response to Arguments
Applicant’s arguments with respect to the claim(s) have been considered but are not persuasive.
Applicant argues:
PNG
media_image1.png
176
636
media_image1.png
Greyscale
Examiner’s Response:
The Examiner disagrees. Hiroshi discloses staining a region with a dye such as “methylene blue”. Based on the IHb value (IHb = log10 (R/G), a bleeding region is automatically determined. The rejection below modifies Hiroshi with Pethania. Pethania discloses selective color correction, where a user manually selects a region in which to modify the color. When the user selects the region, they are guided by the colors that they see on the screen. As shown below, The automatic selection is replaced with the manual selection. Thus Hiroshi in view of Pethania discloses the claimed invention.
Applicant argues:
PNG
media_image2.png
112
642
media_image2.png
Greyscale
PNG
media_image3.png
236
638
media_image3.png
Greyscale
Examiner’s Response:
The Examiner would like to note the critical first sentence in Paragraph 5, which states ”However, in the conventional emphasizing process …” It appears paragraph 5 is discussing the Prior Art, not the invention of Hiroshi. Even if this were not the case, the claims do not require false colors or natural colors as Applicant is arguing.
The Examiner agrees Hiroshi does not disclose a user is to select a specfic image area…. But the rejection uses a combination of Hiroshi and Pethania.
Applicant argues:
PNG
media_image4.png
278
630
media_image4.png
Greyscale
Examiner’s Response:
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Regarding Applicant’s arguments concerning Yamada, they are not persuasive. Nevertheless, the rejection has been withdrawn in light of the Hiroshi reference.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-19 and 22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “wherein the user defines for which color values of the color images the color transformation is to be applied by manually selecting the image area, from which the determined color value is calculated and/or wherein the user can adapt the color transformation to be applied to the color images by manually selecting the selected image area guided by a characteristic hue of a different dye.” The Examiner is unable to find support for this limitation. In the response, Applicant has not shown where support can be found for this limitation.
Claims 2-19 and 22 are rejected as dependent upon a rejected claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hiroshi (JPH10210324) in view of Pethania (PGPub 2004/0036695)
Hiroshi discloses 1. (Currently Amended) An image acquisition method for use in chromoendoscopy compnsmg:
capturing a sequence of color images with an image sensor of a medical image acquisition system, and subjecting at least one of the color images to a color transformation to generate a desired modified representation of the at least one color image on a monitor, (Hiroshi, paragraph 2, “Images of these electronic endoscopes can be recorded as digital images, and various image processing’s are actively performed for the purpose of improving diagnostic performance.”)
wherein the color transformation cornprises:
an image area in one of the color images of the sequence is selected (Hiroshi, pg. 8, 1st paragraph “In the region where the value is extremely high and a stain that is complementary to the hemoglobin dye such as methylene blue is scattered, the calculated IHb value is negative or extremely low. Then, it is possible to determine the bleeding site and the area where the stain is sprayed. Also, in the portion where halation occurs, IHb ≒ 0 because the R and G signals are saturated in white. Therefore, a halation portion can be detected.”)
Hiroshi discloses detecting a bleeding site based on Dye Color (Last 2 paragraphs of page 7, “The IHb value calculated for each pixel is input to the effective area determination unit 61, and detects a bleeding site, a region where a stain is sprayed, and the like based on the IHb value”) and enhances the color of the selected area (pg. 8 last 4 paragraphs, “Since the weighting coefficient determination circuit 36 calculates the weighting coefficient by referring to the IHb value for each pixel by the above-described method, the intensity of an area that is not suitable for enhancement, such as a bleeding part or an area where a stain is sprayed, is determined. Can be automatically suppressed.”). See additionally pg. 10 paragraph 3, “According to the above conversion, the input image signal is output as it is in the area (normal mucous membrane) where the IHb value is the average value, and the area (such as a lesion) having a difference from the average value is emphasized IHb value. It is converted to an image signal having a value. That is, the region showing the IHb value larger than the average value is emphasized red, and the region showing the IHb value smaller than the average value is emphasized white. In the converted image, a region with a large amount of blood is emphasized as having more blood, and a region with a small amount of blood is emphasized as having less blood. Therefore, it is possible to obtain an enhanced image with a natural tone. ”
Hiroshi discloses an automated segmentation and does not expressly disclose “and, wherein the image area is selected by a user of the medical image acquisition system.”
Pethania discloses “and, wherein the image area is selected by a user of the medical image acquisition system.” (Pethania, paragraph 169, “[0169] Rather than select a specific colour as in the selective colour correction technique, a user may identify a region of an image containing several colours, and the editing technique applied, across the complete image, to all of the colours contained within the selected region.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to replace the automated segmentation of Hiroshi with the manual segmentation of Pethania
The suggestion/motivation for doing so would have been to allow user control of what they see.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hiroshi with Pethania to obtain the invention as specified above.
Hiroshi in view of Pethania discloses “determining a color value of the selected image area, the color value corresponding to the characteristic hue of the dye; and” (Hiroshi, pg. 8, 1st paragraph “In the region where the value is extremely high and a stain that is complementary to the hemoglobin dye such as methylene blue is scattered, the calculated IHb value is negative or extremely low. Then, it is possible to determine the bleeding site and the area where the stain is sprayed. Also, in the portion where halation occurs, IHb ≒ 0 because the R and G signals are saturated in white. Therefore, a halation portion can be detected.”)
applying the color transformation to the at least one of the color images on the basis of the determined color value such that respective color distances between the determined color value of the selected image area and respective other single color values within non-selected image areas of the at least one color image which deviate from the determined color value are increased from before the color transformation to after the color transformation. (Hiroshi, pg. 10, paragraph 4, “According to the above conversion, the input image signal is output as it is in the area (normal mucous membrane) where the IHb value is the average value, and the area (such as a lesion) having a difference from the average value is emphasized IHb value. It is converted to an image signal having a value. That is, the region showing the IHb value larger than the average value is emphasized red, and the region showing the IHb value smaller than the average value is emphasized white. In the converted image, a region with a large amount of blood is emphasized as having more blood, and a region with a small amount of blood is emphasized as having less blood. Therefore, it is possible to obtain an enhanced image with a natural tone.”)
wherein the user defines for which color values of the color images the color transformation is to be applied by manually selecting the image area, from which the determined color value is calculated (As shown above the combination of Hiroshi and Pethania discloses this limitation. Hiroshi in view of Pethania discloses manual selection of an area to select colors from the selected region) and/or wherein the user can adapt the color transformation to be applied to the color
images by manually selecting the selected image area guided by a characteristic hue of a different dye. (Hiroshi, page 3 paragraph 7, “A brightness average value calculation unit 7 for calculating an average value of brightness values in the effective area, and an enhancement level conversion unit 8 for determining an enhancement level for each image based on the enhancement level set by the user and the average brightness value. Have been.”, where the user sets the enhancement level)
Hiroshi in view of Pethania discloses 2. An image acquisition method in accordance with claim 1, wherein the color transformation results in an increase in a color saturation value distance and /or a hue value distance and /or a color brightness value distance, in each case based on a comparison between the selected image area and the non-selected image areas; and wherein the color distance is increased by adapting hue values and /or color saturation values and /or color brightness values of the selected image area and /or of the remaining non-selected image areas, taking into consideration the determined color value in each case. (whereby changing the color changes the color distance )
Hiroshi in view of Pethania discloses 3. An image acquisition method in accordance with claim 1, wherein the image area is selected using a characteristic hue, and wherein the characteristic hue is generated by means of a dye with which tissue can be stained or that is a natural hue of a tissue, in particular a malignant tissue. (see combination Hiroshi and Pethania in claim 1; where the user selects a region based on the image of the dyed tissue sample; in other words since the images are dyed, any selection by a user would be guided by the characteristic hue of a dye used )
Hiroshi in view of Pethania discloses 4. An image acquisition method in accordance with claim 1, wherein the image area to be selected by the user or already selected is displayed to the user on the monitor. (see combination Hiroshi and Pethania in claim 1; where the user selects a region based on the image of the dyed tissue sample)
Hiroshi in view of Pethania discloses 5. An image acquisition method in accordance with claim 1, further comprising: automatically selecting additional image areas of the at least one color image, which have a determined color value; applying a color transformation to the automatically selected additional image areas to increase color distance; and wherein the automatically selected, additional image areas have additional image pixels with color values that differ from the determined color value. (Hiroshi, “red”; Hiroshi selects all regions which have the color red and performs color transformation such that the color distance is increased.)
Hiroshi in view of Pethania discloses 6. An image acquisition method in accordance with claim 1, wherein the determined color value is determined using a statistical value calculated from image pixels of the selected image area, in particular an average; and wherein, to this end, RGB values of these image pixels are processed.” (Hiroshi, pg. 3, paragraph 7, “A brightness average value calculation unit 7 for calculating an average value of brightness values in the effective area, and an enhancement level conversion unit 8 for determining an enhancement level for each image based on the enhancement level set by the user and the average brightness value. Have been.”)
Hiroshi in view of Pethania discloses 8. An image acquisition method in accordance with claim 1, wherein the color distance is increased by color values of the selected image area and of the automatically selected additional image areas, and /or by adapting lowering, color values of the remaining non- selected image areas of the at least one color image, which do not have the determined color value and/or lie outside of a color similarity space of the determined color value. (whereby changing the color changes the color distance )
Hiroshi in view of Pethania discloses 9. An image acquisition method in accordance with claim 1, wherein the color transformation preserves respective brightness values and /or color values of the selected image areas and /or of the non-selected image areas to enable a detailed representation of image information; or the color transformation preserves relevant relative differences in color values of the selected image areas and /or of the non-selected image areas. (Hiroshi, last paragraph of pg. 8 – 1st paragraph of pg. 9, “This is performed using a ROM or the like in which predetermined conversion characteristics are stored as in the case of the weighting coefficient determination circuit 36, and the emphasis level set by the user according to the input IHb average value is set to 1 It is determined to be a value between 00% and 30%.”, where the emphasis level can be set to 0 thus maintaining color)
Yamada in view of Hiroshi in view of Pethania discloses 15. An image acquisition method in accordance with claim 1, wherein the color value to be determined for the selected image area is first automatically pre-determined by the endoscopy system by means of a statistical analysis of color values (Hiroshi, pg. 3, paragraph 7, “A brightness average value calculation unit 7 for calculating an average value of brightness values in the effective area, and an enhancement level conversion unit 8 for determining an enhancement level for each image based on the enhancement level set by the user and the average brightness value. Have been.”)
Hiroshi in view of Pethania discloses 17. An image acquisition method in accordance with claim 1, wherein a user selects the image area manually on the monitor such that the color value of the selected image area is determined, and /or the user is presented with a fixed target area for computer-assisted selection of the image area. (See claim 1 above)
Yamada in view of Hiroshi in view of Pethania discloses 18. An image acquisition method in accordance with claim 1, wherein the color transformation is applied successively to multiple color images of the sequence, in a manner wherein a color distance between the determined color value and non-selected image areas of the respective transformed color images is increased. (Hiroshi, pg 8, 3rd paragraph “The calculation result is reset every frame.”, Thus the combination of Hiroshi with Pethania would teach making this modification for a sequence of frames)
Hiroshi in view of Pethania discloses 19. A medical image acquisition system comprising:
an image sensor; a camera control unit; and wherein the image acquisition system includes a controller (Hiroshi relates to modifying endoscopic images; Page 1, paragraph 2 “2. Description of the Related Art In recent years, electronic endoscopes using a solid-state imaging device such as a CCD as an imaging means have become widespread. Images of these electronic endoscopes can be recorded as digital images, and various image processings are actively performed for the purpose of improving diagnostic performance.”), which is configured to perform, in combination with an external monitor, an image acquisition method in accordance with claim 1;(see claim 1)
and wherein the controller is set up for performing the color transformation with which a color distance between a selected image area of a color image captured with the image acquisition system and non-selected image areas of this color image is increased. (see claim 1)
wherein the image acquisition system is configured such that the image area can be manually selected by a user on the monitor. (see claim 1)
Hiroshi in view of Pethania discloses 22. (New) An image acquisition method in accordance with claim 1, wherein a color similarity space is determined around the color value determined for the selected image area and for image pixels of the at least one color image which lie within the color similarity space, color saturation values and/or hue values are raised and/or for image pixels of the at least one color image which lie outside of the color similarity space, color saturation values and/or hue values are lowered. (see claim 1 above)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hiroshi in view of Pethania in view of Kim in further view of Mitamura (2018/0069988)
Hiroshi in view of Pethania discloses 7. An image acquisition method in accordance with claim 1, wherein in the color transformation an output signal, in particular a raw data signal, of the image sensoris converted into a signal in a hue-based color space (Hiroshi, pg. 3, 2nd paragraph “As shown in FIG. 1, an image processing apparatus 1 according to the present embodiment is an image processing apparatus for performing contrast enhancement on an input image. An emphasis level determining circuit 2 for determining an emphasis level for each image based on the emphasis level set by the above; a weighting coefficient determining circuit 3 for weighting the degree of emphasis for each pixel from the luminance value of the input image; The emphasis level determination circuit 2 and the weighting coefficient determination circuit 3”),; and
But does not expressly disclose “wherein the determined color value of the selected image area is a hue value, in particular averaged over image pixels of the selected image area; and wherein a saturation value of the selected image area is determined using the signal in the hue-based color space”
Kim discloses “wherein the determined color value of the selected image area is a hue value, in particular averaged over image pixels of the selected image area; and wherein a saturation value of the selected image area is determined using the signal in the hue-based color space”
(Kim, Fig. 1 &2, Col. 4.30-4.36, “The hue, light, chrominance elements with respect to the average color of the image region are obtained, and then the color of each pixel is changed into hue(H_i), light(L_i), chrominance(Ch_i) elements with respect to each pixel Pi of all pixels, and the distance(hereinafter color difference) in a color coordinate space is obtained between the values (H_i, L_i, Ch_i) and the average values (H_av, L_av, Ch_av), and the color differences are summed up to the last pixel.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to determine the average of selected region as shown by Kim to use as the target color value of HIroshi
The suggestion/motivation for doing so would have been to allow all the selected pixels have the same luminance.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Hiroshi discloses a hue based system, but does not expressly disclose “HSV”
Mitamura discloses “HSV” (Mitamura, paragraph 3)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to replace the hue based color system of Yamada of HSV as shown by Mitamura
The suggestion/motivation for doing so would have been it is commonly used in endoscopy systems.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hiroshi with Kim and Mitamura to obtain the invention as specified in claim 7.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hiroshi in view of Pethania in view of Weiss (PGPub 2009/0201310) .
Hiroshi in view of Pethania discloses 16. An image acquisition method in accordance with claim 15, But does not expressly disclose “wherein the user subsequently confirms or discards the pre-determined color value via the monitor, and / or re-adjusts a presented color value, and / or a presented saturation value adaptation, with the aid of a color value scale and /or saturation value scale shown graphically on the monitor as an overlay of an additional fine adjustment scale for fine adjustment of the color and /or saturation values.”
Weiss discloses “wherein the user subsequently confirms or discards the pre-determined color value via the monitor, and / or re-adjusts a presented color value, and / or a presented saturation value adaptation, with the aid of a color value scale and /or saturation value scale shown graphically on the monitor as an overlay of an additional fine adjustment scale for fine adjustment of the color and /or saturation values.” (Weiss, Fig. 12,
PNG
media_image5.png
468
358
media_image5.png
Greyscale
)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use the GUI of Weiss as the means to change values in the system of Hiroshi.
The suggestion/motivation for doing so would have been it is as an easy way to modify values.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Yamada in view of Kim with Weiss to obtain the invention as specified in claim 16.
Allowable Subject Matter
Claims 20-21 are allowed.
Claims 10-13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GANDHI THIRUGNANAM whose telephone number is (571)270-3261. The examiner can normally be reached M-F 8:30-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GANDHI THIRUGNANAM/Primary Examiner, Art Unit 2672