DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/26/26 has been entered.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 21,25-44 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph and second paragraph, as failing to comply with the written description and definiteness requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 21 recites “determining, from a plurality of wavelengths in a visible spectrum, a first wavelength at which oxygenated hemoglobin and deoxygenated hemoglobin have a greatest difference in light absorption from one another and the deoxygenated hemoglobin has a lower light absorption than the oxygenated hemoglobin;
determining, from the plurality of wavelengths in the visible spectrum, a second wavelength at which the oxygenated hemoglobin and the deoxygenated hemoglobin have the greatest difference in light absorption from one another and the deoxygenated hemoglobin has a higher light absorption than the oxygenated hemoglobin;”
Claim 21 recites active determining steps using the greatest difference, yet the specification and dependent claims treat this as constant values (490,630 and 640) derived from Molar Extinction Coefficient data. (See paragraphs 27-28). Therefore the Specification does not appear to actually determine them at best it teaches selecting them. As the Examiner stated in the previous office action, the Examiner has calculated these values. Here is a table for the both sets of data.
Wavelength
HB02
HB
HB02-HB
Wavelength
HB02
HB
HB02-HB
468
34870.8
17025.6
17845.2
436
132820
547040
-414220
456
49496
30698.8
18797.2
434
165332
552160
-386828
466
37073.2
18142.4
18930.8
438
119140
501560
-382420
462
41320
20891.2
20428.8
432
214120
552160
-338040
394
212060
191540
20520
440
102580
413280
-310700
464
39807.2
19260.8
20546.4
430
246072
528600
-282528
460
44480
23388.8
21091.2
442
92780
363240
-270460
458
47496
25886.4
21609.6
428
283112
500840
-217728
396
231612
202124
29488
444
81444
282724
-201280
398
248404
212712
35692
446
76324
237224
-160900
400
266232
223296
42936
426
326032
481840
-155808
402
284224
236188
48036
448
67044
173320
-106276
404
308716
253368
55348
424
376236
461200
-84964
420
480360
407560
72800
450
62816
103292
-40476
406
354208
270548
83660
380
109564
145232
-35668
418
515520
385680
129840
382
116968
145232
-28264
408
422320
287356
134964
384
125420
148668
-23248
416
521880
363848
158032
560
32613.2
53788
-21174.8
410
466840
303956
162884
558
33456
54164
-20708
412
500200
321344
178856
556
34476.8
54540
-20063.2
414
524280
342596
181684
For reference the values of 490,630 and 640 have the following values:
490
23684.4
16684
7000.4
630
610
5148.8
-4538.8
640
442
4345.2
-3903.2
As it can plainly be seen (https://omlc.org/spectra/hemoglobin/summary.html), Applicant’s claim and Specification fail to clearly set bounds of the claim limitations and the Specification at best provides conflicting definitions for critical elements of the claim therefore also lacks written description support. And it is unclear if claim 21 is determining the greatest difference as the examiner has shown above(wavelengths of 468nm and 436nm) or merely selecting 490,630,640 nm wavelengths.
The Examiner notes that the scale of “MOTOR EXTINCTION COEFFICIENT”(SP??) in Fig. 5A is not a constant scale, but a logarithmic scale. Even considering log scale the wavelengths selected would be 412nm followed by 578nm and on the other side 700nm followed by 660 nm.
Claim 33 and 39 are rejected under similar grounds as claim 21.
Claims 25-44 are rejected as dependent on a rejected claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 21,25, 30, 33,39, 42-43 is/are rejected under 35 U.S.C. 103 as being unpatentable over Furuho (2019/0110669) in view of Morita (2012/0220840)
Furuho discloses 21. (Currently Amended) A computer-implemented method for enhancing images, the method comprising:
receiving a spectrum image of a target area of a patient, the target area including a plurality of blood vessels, wherein one or more of the plurality of blood vessels are bleeding vessels causing pooling blood to be viewed in the spectrum image; (Furuho,” [0075] At Step S211, the imaging unit 24 performs a particular light read process of reading, from the image sensor 242, an image signal from only the pixels having sensitivity to the particular light. Specifically, as illustrated in FIG. 8, a narrow band light of NBI is irradiated as the particular light, because the R pixel does not have sensitivity to the narrow band light, the imaging unit 24 reads image signals from only the Gr pixel, the Gb pixel, and the B pixel that have sensitivity to the narrow band light and outputs the image signals to the A/D converter 251. In this case, the imaging unit 24 resets the pixel value of the R pixel.”, Furuho discloses DRI imaging. DRI imaging main purpose if to understand blow flow and bleeding, as evidenced by https://pmc.ncbi.nlm.nih.gov/articles/PMC5283170/pdf/10-1055-s-0042-119393.pdf
PNG
media_image1.png
252
338
media_image1.png
Greyscale
)
determining, from a plurality of wavelengths in a visible spectrum, a first wavelength at which oxygenated hemoglobin and deoxygenated hemoglobin have a greatest difference in light absorption from one another and the deoxygenated hemoglobin has a lower light absorption than the oxygenated hemoglobin; (Furuho, paragraph 40, “[0040] The color filter 241 passes light of a predetermined wavelength band. Specifically, the color filter 241 is constituted by a plurality of filters each of which passes light in a red wavelength band, light in a green wavelength band, and light in a blue wavelength band (for example, red: 600 nm to 700 nm, green: 500 nm to 600 nm, and blue: 400 nm to 500 nm).”; “[0088] Furthermore, in the embodiments according to the present disclosure, the particular light is exemplified to include light in a blue narrow band (for example, 390 nm to 445 nm) and/or light in a green narrow band (for example, 530 nm to 550 nm); however, the embodiment is not limited to this. When the capsule endoscope performs a fluorescence observation (autofluorescence imaging: AFI), light including light in a wavelength band (for example, 390 nm to 470 nm) that is used for the autofluorescence observation irradiating a fluorescent material and light in a wavelength band (for example 540 nm to 560 nm) that is absorbed by hemoglobin in the blood may also be used as the particular light. Furthermore, when the capsule endoscope performs a narrow band light observation (dual red imaging: DRI) that observes a mucous membrane of the digestive tract and a lower layer of a mucous membrane, the particular light may include light and additional light both of which are in a red narrow band (for example, the light of 600 nm and the additional light of 630 nm). Furthermore, when the capsule endoscope performs an infrared light observation (Infra-Red Imaging: IRI), the particular light may include an infrared light and additional infrared light both of which are in an infrared wavelength band (for example, 790 nm to 820 nm and 905 nm to 970 nm). In these cases, a color filter suitable for observation may be arranged on the light-receiving surface of the image sensor.”)
determining, from the plurality of wavelengths in the visible spectrum, a second wavelength at which the oxygenated hemoglobin and the deoxygenated hemoglobin have the greatest difference in light absorption from one another and the deoxygenated hemoglobin has a higher light absorption than the oxygenated hemoglobin; ; (Furuho, “[0088] Furthermore, in the embodiments according to the present disclosure, the particular light is exemplified to include light in a blue narrow band (for example, 390 nm to 445 nm) and/or light in a green narrow band (for example, 530 nm to 550 nm); however, the embodiment is not limited to this. When the capsule endoscope performs a fluorescence observation (autofluorescence imaging: AFI), light including light in a wavelength band (for example, 390 nm to 470 nm) that is used for the autofluorescence observation irradiating a fluorescent material and light in a wavelength band (for example 540 nm to 560 nm) that is absorbed by hemoglobin in the blood may also be used as the particular light. Furthermore, when the capsule endoscope performs a narrow band light observation (dual red imaging: DRI) that observes a mucous membrane of the digestive tract and a lower layer of a mucous membrane, the particular light may include light and additional light both of which are in a red narrow band (for example, the light of 600 nm and the additional light of 630 nm). Furthermore, when the capsule endoscope performs an infrared light observation (Infra-Red Imaging: IRI), the particular light may include an infrared light and additional infrared light both of which are in an infrared wavelength band (for example, 790 nm to 820 nm and 905 nm to 970 nm). In these cases, a color filter suitable for observation may be arranged on the light-receiving surface of the image sensor.”)
generating a pre-enhanced image based on a first filter and a second filter applied to the spectrum image, wherein the first filter enhances light absorption at [[a]]the first wavelength, wherein the second filter enhances light absorption at [[a]] the second wavelength , and wherein the first filter and the second filter provide image differentiation of the bleeding vessels from the pooling blood in the pre-enhanced image; (Furuho, paragraph 88, “In these cases, a color filter suitable for observation may be arranged on the light-receiving surface of the image sensor.”)
Furuho does not expressly disclose “generating an enhanced contrast image by applying a contrast enhancement technique to the pre-enhanced image; and”
Morita discloses “generating an enhanced contrast image by applying a contrast enhancement technique to the pre-enhanced image; and “ (Morita, “[0360] A color image that includes R, G, and B channels is generated from the G2' image and the B2' image in the same manner as in the first embodiment. For example, a color image is generated by inputting the G2 image to the R channel, and inputting the B2 image to the G channel and the B channel. The special light image generation section 331 performs a white balance process, a grayscale transformation process, and the like on the generated color image, and outputs the resulting color image as a narrow-band light image. The special light image storage section 332 stores the special light image output from the special light image generation section 331.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to perform white balancing of Morita to generate the bleed point observation image of Furuho.
The suggestion/motivation for doing so would have been a better quality output image can be achieved.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results
generating a final colorized image by applying one or more spatial filters to the enhanced contrast image. (Furuho, paragraph 47, “gradation conversion process”)
Furuho in view of Morita discloses 25. (Previously Presented) The computer-implemented method of claim 21, wherein the first filter is a 490 nanometer (nm) filter, and wherein the second filter is a filter in a range of 630 nm to 640 nm. (see claim 21, blue: 400 nm to 500 nm and the light of 600 nm and the additional light of 630 nm)
Furuho in view of Morita discloses 30. (Previously Presented) The computer-implemented method of claim 21, wherein the final colorized image represents a recolorization of the enhanced contrast image.(see claim 21)
Claim 33 is rejected under similar grounds as claim 21.
Claim 39 is rejected under similar grounds as claim 21.
Furuho in view of Morita discloses 42. (New) The computer-implemented method of claim 21, further comprising: determining, from the plurality of wavelengths in the visible spectrum, a third wavelength, different from the second wavelength, at which the deoxygenated hemoglobin has the higher light absorption than the oxygenated hemoglobin; and generating a pre-enhanced image further based on a third filter applied to the spectrum image, wherein the third filter enhances light absorption at the third wavelength. (see claim 21, 600 nm and 630 nm)
Furuho in view of Morita discloses 43. (New) The computer-implemented method of claim 42, wherein the third wavelength is determined relative to the second wavelength based on light absorption slope characteristics for the deoxygenated hemoglobin in a wavelength range between the second wavelength and the third wavelength. (see claim 21, 600 nm and 630 nm)
Claim(s) 27, 34, 35AND 40 is/are rejected under 35 U.S.C. 103 as being unpatentable over Furuho in view of Morita in further view of Kutsuma (PGPub 2020/0260940)
Furuho in view of Morita discloses 27. (Previously Presented) The computer-implemented method of claim 21,
But does not expressly teach “wherein generating the enhanced contrast image comprises: applying a trained classifier to the pre-enhanced image, wherein the trained classifier is configured to classify the bleeding vessels from the pooling blood; and applying a first artificial color to the bleeding vessels classified by the trained classifier and a second artificial color different from the first artificial color to the pooling blood classified by the trained classifier.”
Kutsuma discloses “wherein generating the enhanced contrast image comprises: applying a trained classifier to the pre-enhanced image, wherein the trained classifier is configured to classify the bleeding vessels from the pooling blood; and applying a first artificial color to the bleeding vessels classified by the trained classifier and a second artificial color different from the first artificial color to the pooling blood classified by the trained classifier.” .(Kutsuma, Fig. 2, #52, paragraph 79-81, where the color conversion is controlled by the matrix C, which has been pretrained, such that “ the image signals Rin, Gin, and Bin can be respectively assigned to the output channels in the monitor 15, and a color tone of a color of blood, a color of a bleeding point, or the like to be displayed can be set to a desired color tone” )
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use Katsuma’s color conversion matrix to enhance the image of Furuho
The suggestion/motivation for doing so would have been to clearly see where the bleeding point is located..
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Furuho in view of Morita and Katsuma to obtain the invention as specified in claim 27.
Claim 34 is rejected under similar grounds as claim 21.
Claim 35 is rejected under similar grounds as claim 27.
Claim 40 is rejected under similar grounds as claim 27.
Claim(s) 28,31,32,36 38 is/are rejected under 35 U.S.C. 103 as being unpatentable over Furuho in view of Morita in further view of Guissin (PGPub 2015/0379712)
Furuho in view of Morita discloses 28. (Previously Presented) The computer-implemented method of claim 21, wherein generating the enhanced contrast image comprises:
But does not expressly disclose “applying a histogram contrast enhancement (HCE) algorithm to the pre- enhanced image.”
Guissin discloses “applying a histogram contrast enhancement (HCE) algorithm to the pre- enhanced image.” (Guissin, paragraph 159, “[0159] “PRP” indicates a pre-processing function applied to I: such as, for example, noise reduction and/or histogram stretching and/or color conversions (e.g. RGB to Lab; YcrCb to RGB; RGB to CYMK or CYMK to Lab or converse conversions);”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to apply the HCE algorithm as shown by Gussin to the pre-enhanced image of Furuho in view of Morita
The suggestion/motivation for doing so would have been to [“0197] An enhancing filter, used, for example, during regular viewing, serves to enhance a digital image in a manner compatible with the human visual system.”
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Furuho with Morita and Guissin to obtain the invention as specified in claim 28.
Furuho in view of Morita discloses 31. (New) The computer-implemented method of claim 21, further comprising: prior to generating the enhanced contrast image, analyzing the pre-enhanced image to identify an area of interest, wherein the area of interest includes at least a portion of the target area including the bleeding vessels. (Guissin, “[0260] In another exemplary embodiment of the invention camera units 720 and/or 730 are initially operated in enhance mode 122 at a first (relatively low) resolution. Optionally, once a region of interest is defined, camera units 720 and/or 730 are aimed at the region of interest and camera optics are zoomed in to provide a higher resolution image which can be enhanced 120 in analyze mode 124 and/or detect mode 126. Optionally, additional screening in enhance mode is subsequently conducted to identify additional regions of interest.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to select a region of interest as shown by Gussin on the Furuho in view of Morita
The suggestion/motivation for doing so would have been to show the user where bleeding is occurs.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Furuho with Morita and Guissin to obtain the invention as specified in claim 31.
Furuho in view of Morita in view of Guissin discloses 32. (New) The computer-implemented method of claim 31, wherein the contrast enhancement technique is applied to the pre-enhanced image to provide further image differentiations within the area of interest beyond the bleeding vessels from the pooling blood. (see claim 21, where the contrast enhancement provides further image differentiation between the blood vessels and pooling blood.)
Claim 36 is rejected under similar grounds as claim 28.
Claim 38 is rejected under similar grounds as claim 31.
Claim(s) 29 and 37 is/are rejected under 35 U.S.C. 103 as being unpatentable over Furuho in view of Morita in further view of Horn (2006/0183993).
Furuho in view of Morita discloses 29. (New) The computer-implemented method of claim 21, wherein applying the one or more spatial filters comprises:
Kutsuma discloses preprocessing(Kutsuma, “[0084] The post-processing unit 53 performs post-processing such as various types of enhancement processing and size conversion processing, and outputs an image signal, which has been subjected to various types of processing, to the monitor 15.”) ,but does not expressly disclose “applying one or more of an averaging filter, a smoothing filter, a zero-padding filter, a symmetrical filter, a circular filter, a low pass filter, or a high pass filter.”
Horn discloses “applying one or more of an averaging filter, a smoothing filter, a zero-padding filter, a symmetrical filter, a circular filter, a low pass filter, or a high pass filter.” (Horn, “[0093] In accordance with embodiments of the invention, median filtering, median smoothing, and/or other suitable methods of filtering, smoothing or enhancing may be performed on localization signals, localization data, localization graphs, motility data, or images or visual representations corresponding to localization data. In some embodiments, the filtering, smoothing or enhancement may be performed substantially in real time, e.g., upon reception of localization signals and while the signal source 100 may be in-vivo. In alternate embodiments, the filtering, smoothing or enhancement may be performed at a later period of time, e.g., during post-processing of previously-collected localization data.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to preprocess the image of Kutsuma in view of Morita using the method shown by Horn.
The suggestion/motivation for doing so would have been provide a better quality image.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Kutsuma with Morita and Horn to obtain the invention as specified in claim 31.
Claim 37 is rejected under similar grounds as claim 29.
No Prior Art reads on the specific combination of wavelengths recited in claims 26 and 44. Olympus’s RDI (formerly called DRI) technology uses the wavelengths of Green(~540), Amber (~600) and Red (~630) to improve visibility of bleeding points
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GANDHI THIRUGNANAM whose telephone number is (571)270-3261. The examiner can normally be reached M-F 8:30-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GANDHI THIRUGNANAM/Primary Examiner, Art Unit 2672