Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
RESPONSE TO AMENDMENT
Based on applicants’ amendment, filed on 11/4/2025, see pages 2 through 10 of remark, with respect to amended claims 1-5, 10 and 12, have been fully considered, and upon further consideration, they are moot in view of the new ground (s) of rejection as necessitated by applicant’s amendment is made.
Contrary to the applicant’s assertion, limitations in the amended claim, that Kubota does not disclose either “outputting the second reporting process (sound output) along with the first reporting process (information superimposition) or not outputting the second reporting process based on whether the region of interest is continuously detected”.
The Examiner has thoroughly reviewed Applicant's argument and respectfully wants to point out, regarding claim 1, Kubota reference discloses:
a) page 1, paragraphs, [0008] and [0025], the present invention is a non-transitory computer-readable recording medium storing a program to be executed by a computer, the computer-readable storage medium storing an endoscope image processing program for causing the computer to execute sequentially acquiring a plurality of observation images obtained by picking up images of an object with an endoscope, detecting from the observation images a lesion area (region of interest), which is an observation target of the endoscope, judging a level of an oversight risk which is a risk that an operator may overlook the lesion area, on the basis of the observation images, controlling notification methods of detection of the lesion area on the basis of the level of the oversight risk, notifying the operator of detection of the lesion area on a basis of control of the notification methods using a display apparatus including a first image region and a second image region that are for displaying a marker image indicating the lesion area, the second image region being smaller than the first image region, displaying the marker image only in the second image region in a case where a level of the oversight risk relating to the lesion area is a first level and displaying the marker image in both the first image region and the second image region in a case where a level of the oversight risk relating to the lesion area is a second level higher than the first level. The monitor is a display apparatus at which the endoscope image is to be displayed. Further, the monitor includes a speaker 5a which outputs voice (sound). Note that the monitor 5 also has a function as a notification unit.
b) (see above, also page 3, paragraphs, [0036-0037] the support information generation unit includes a lesion detection, unit, an oversight risk analysis unit, a notification control unit 35, and an “image superimposing “unit. The lesion detection unit 33 detects a lesion area included in the frames of the generated image “sequentially output” (first report), from the image input unit 31. The lesion detection unit 33 detects a lesion area (region of interest), from the generated image, for example, by performing processing of applying an image discriminator which has acquired a function of being capable of discriminating a “polyp” image in advance through a learning method such as “deep learning”, to the generated image. Note that the lesion area may be detected using other methods as well as the above-described learning method.
c) Also, page 7, paragraphs, [0085-0087] finally, the notification control unit 35 generates support information and makes a notification on the basis of the notification method determined in step S4 (S5). More specifically, the notification control unit 35 generates a marker image G2 in accordance with the level of the oversight risk as the support information and “outputs the marker image” to the image “superimposing” (first report). The image superimposing unit 36 outputs an endoscope image in which the marker image G2 input from the notification control unit 35 is superimposed on the observation image G1 input from the image input unit, to the monitor and causes the endoscope image to be displayed. FIG. 6 and FIG. 7 are views illustrating an example of the endoscope “image displayed” on the monitor. In other words, FIG. 6 and FIG. 7 illustrate endoscope images in which the support information is superimposed, FIG. 6 illustrates an example where the oversight risk is low, and FIG. 7 illustrates an example where the oversight risk is high. As illustrated in FIG. 6 and FIG. 7, the observation image G1 to which the marker image G2 is added is displayed in a display region D1 on a display screen 51A of the monitor 5. As illustrated in FIG. 6, in a case where the oversight risk is low, the marker image G2 having a size enclosing the circumference of a lesion area L1 is superimposed on the observation image G1. On the other hand, as illustrated in FIG. 7, in a case where the oversight risk is high, the marker image G2 having a size enclosing a periphery portion of the observation image G1 is superimposed on the observation image G1);
d) also, page 7, paragraphs, [0094-0095] the notification means selection unit 35A selects means for making a notification that the support information is displayed on the monitor 5. The notification means which can be selected includes a notification using “voice from a speaker” (second report), 5a (hereinafter, referred to as a voice notification) (second report), a notification using vibration of a portion such as the operation portion 10 grasped by the surgeon, or the like, in addition to the above-described display of an image on the monitor 5 (hereinafter, referred to as an image notification). The notification target risk setting unit 35B sets an oversight risk for which a notification is to be controlled. The oversight risk which can be set is a risk for which the level of the risk can be determined on the basis of the analysis result of the oversight risk analysis unit 34. Setting items include, for example, (A) the number of detected lesion areas Ln, (B) a degree of difficulty in finding the lesion area Ln, (C) a size of the lesion area Ln, (D) a type of the lesion area Ln, (E) detection reliability of the lesion area Ln, and (F) an elapsed time period since the lesion area Ln has been detected. The notification target risk setting unit 35B selects an item to be set as the notification target risk among these setting items.
E) Also, page 9, paragraph, [0116] Settings are made such that in a case where the oversight risk is low, the volume of the voice is lowered or tone is lowered, and in a case where the oversight risk is high, the volume of the voice is increased or tone is made higher. Alternatively, settings may be made such that a voice notification is not made (unnecessary sound output), in a case where the oversight risk is low and a “voice notification is “made only” in a case where the oversight risk is high” (not execute the second reporting, if risk is low).
F) finally, page 10, paragraphs, [0127-0128] in a case where an image notification and a vibration notification are made in combination, the notifications are made from a detection start time point regardless of a level of the oversight risk. On the other hand, in a case where a voice notification and a vibration notification are made in combination, a vibration notification is not made while the oversight risk is low, and the notification is made from a time point at which the oversight risk becomes high. Note that an alert period may be set in accordance with the oversight risk or may be set at a predetermined set period. In a case where the alert period is set at the predetermined set period, an alert is stopped if a period exceeds the set period even if the lesion area Ln is continuously detected in a case where an image notification and a vibration notification are made in combination. In a case where a voice notification and a vibration notification are made in combination, the notifications are continuously made while the lesion area Ln is “continuously detected”. Procedure of “executing” a diagnosis support function to be performed using the endoscope system configured as described above is similar to the diagnosis support procedure in the first embodiment illustrated in FIG. 5).
DETAILED ACTION
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(e), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 1-5 and 8-13 are rejected under 35 U.S.C. 102(a)(2) based upon a public use or sale or other public availability of the invention as being anticipated by Kubota et al (Pub. No.: U.S. 2021/0274999 A1).
Regarding claim 1, Kubota discloses a medical image processing apparatus comprising a processor, the processor being configured to execute; an image acquisition process of acquiring a time-series medical image; execute a region-of-interest detection process of detecting a region of interest from the acquired medical image (see abstract, a processor of an endoscope system detects from the observation images a lesion area which is an observation target of the endoscope, judges a level of an oversight risk which is a risk that an operator may overlook the lesion area, on the basis of the observation images, and controls notification methods of detection of the lesion area on the basis of the level of the oversight risk. The processor displays the marker image indicating the lesion area only in a second image region of the display apparatus when a level of the oversight risk relating to the lesion area is a first level and displays the marker image in both a first image region and the second image region which is smaller than the first image region when a level of the oversight risk relating to the lesion area is a second level higher than the first level. Also see page 1, paragraph, [0006] an endoscope system according to one aspect of the present invention includes a processor including hardware; and a display apparatus, the processor being configured to sequentially (time series), input a plurality of observation images obtained by picking up images of an object with an endoscope, detect from the observation images a lesion area which is an observation target (region of interest), of the endoscope);
execute a display control process of causing a display apparatus to display the medical image; execute a first reporting process of, in response to the region of interest being detected in the region-of-interest detection process, causing the display apparatus to superimposed and display information about the detected region of interest on the medical image (see above, also page 3, paragraphs, [0036-0037] the support information generation unit 32 includes a lesion detection, unit 33, an oversight risk analysis unit 34, a notification control unit 35, and an image superimposing unit 26. The lesion detection unit 33 detects a lesion area included in the frames of the generated image sequentially output from the image input unit 31. The lesion detection unit 33 detects a lesion area (region of interest), from the generated image, for example, by performing processing of applying an image discriminator which has acquired a function of being capable of discriminating a “polyp” image in advance through a learning method such as “deep learning”, to the generated image. Note that the lesion area may be detected using other methods as well as the above-described learning method);
determine, while the first reporting process is being executed, whether the detection is a continuous detection; and in a case where the detection is determined to be the continuous detection, execute a (see page 7, paragraphs, [0085-0087] finally, the notification control unit 35 generates support information and makes a notification on the basis of the notification method determined in step S4 (S5). More specifically, the notification control unit 35 generates a marker image G2 in accordance with the level of the oversight risk as the support information and outputs the marker image G2 to the image “superimposing” unit 36. The image superimposing unit 36 outputs an endoscope image in which the marker image G2 input from the notification control unit 35 is superimposed on the observation image G1 input from the image input unit 31, to the monitor 5 and causes the endoscope image to be displayed. FIG. 6 and FIG. 7 are views illustrating an example of the endoscope “image displayed” on the monitor 5. In other words, FIG. 6 and FIG. 7 illustrate endoscope images in which the support information is superimposed, FIG. 6 illustrates an example where the oversight risk is low, and FIG. 7 illustrates an example where the oversight risk is high. As illustrated in FIG. 6 and FIG. 7, the observation image G1 to which the marker image G2 is added is displayed in a display region D1 on a display screen 51A of the monitor 5. As illustrated in FIG. 6, in a case where the oversight risk is low, the marker image G2 having a size enclosing the circumference of a lesion area L1 is superimposed on the observation image G1. On the other hand, as illustrated in FIG. 7, in a case where the oversight risk is high, the marker image G2 having a size enclosing a periphery portion of the observation image G1 is superimposed on the observation image G1);
and a second reporting process of, in response to the region of interest being detected in the region-of-interest detection process, outputting a sound from a sound output apparatus along with the first reporting process and in a case where the detection is determined not to be the continuous detection, not execute the second reporting process (see above, also page 7, paragraphs, [0094-0095] the notification means selection unit 35A selects means for making a notification that the support information is displayed on the monitor 5. The notification means which can be selected includes a notification using “voice from a speaker” 5a (hereinafter, referred to as a voice notification), a notification using vibration of a portion such as the operation portion 10 grasped by the surgeon (hereinafter, referred to as a vibration notification), or the like, in addition to the above-described display of an image on the monitor 5 (hereinafter, referred to as an image notification). The notification target risk setting unit 35B sets an oversight risk for which a notification is to be controlled. The oversight risk which can be set is a risk for which the level of the risk can be determined on the basis of the analysis result of the oversight risk analysis unit 34. Setting items include, for example, (A) the number of detected lesion areas Ln, (B) a degree of difficulty in finding the lesion area Ln, (C) a size of the lesion area Ln, (D) a type of the lesion area Ln, (E) detection reliability of the lesion area Ln, and (F) an elapsed time period since the lesion area Ln has been detected. The notification target risk setting unit 35B selects an item to be set as the notification target risk among these setting items. Also, page 9, paragraph, [0116] Settings are made such that in a case where the oversight risk is low, the volume of the voice is lowered or tone is lowered, and in a case where the oversight risk is high, the volume of the voice is increased or tone is made higher. Alternatively, settings may be made such that a voice notification is not made (unnecessary sound output), in a case where the oversight risk is low and a “voice notification is made only in a case where the oversight risk is high” (not execute the second reporting.
Finally, page 10, paragraphs, [0127-0128] in a case where an image notification and a vibration notification are made in combination, the notifications are made from a detection start time point regardless of a level of the oversight risk. On the other hand, in a case where a voice notification and a vibration notification are made in combination, a vibration notification is not made while the oversight risk is low, and the notification is made from a time point at which the oversight risk becomes high. Note that an alert period may be set in accordance with the oversight risk or may be set at a predetermined set period. In a case where the alert period is set at the predetermined set period, an alert is stopped if a period exceeds the set period even if the lesion area Ln is continuously detected in a case where an image notification and a vibration notification are made in combination. In a case where a voice notification and a vibration notification are made in combination, the notifications are continuously made while the lesion area Ln is continuously detected. Procedure of “executing” a diagnosis support function to be performed using the endoscope system configured as described above is similar to the diagnosis support procedure in the first embodiment illustrated in FIG. 5).
Regarding claim 2, Kubota discloses the medical image processing apparatus according to claim 1, wherein the processor is configured to execute the second reporting process in response to the region of interest being detected after a first period elapses from when the region of interest is detected in the region-of-interest detection process, in a case where the detection is determined to be the continuous detection, and is configured not to execute the second reporting process in response to the region of interest not being detected after the first period elapses in a case where the detection is determined not to be the continuous detection (see claim 1, also pages 8-9, paragraphs, [0104] and [0113-0114], the present item is an oversight risk determined on the basis of an elapsed time period since the lesion area Ln has been detected, which is a time period during which the lesion area Ln is continuously detected since the lesion area Ln has been detected in the observation image G1. The elapsed time period since the lesion area Ln has been detected is measured at a detection period measurement unit 35B1. In a case where the elapsed time period since the lesion area Ln has been detected is shorter than a threshold (for example, five seconds) set in advance, it is determined that the oversight risk is low. On the other hand, in a case where the elapsed time period since the lesion area Ln has been detected is equal to or greater than the threshold set in advance, it is determined that the oversight risk is high. A display start timing and a display period of the marker image G2 are made different in accordance with the oversight risk. For example, the marker image G2 is not displayed in a stage at which the elapsed time period since the lesion area Ln has been detected is shorter than a threshold and the oversight risk is low, and display of the marker image is started at a timing at which the elapsed time period since the lesion area Ln has been detected exceeds the threshold and the oversight risk becomes high. Further, for example, in a case where the elapsed time period since the lesion area Ln has been detected is long, the display period of the marker image G2 is set longer. An example of setting content of the voice notification setting unit 35C2 will be described next).
Regarding claim 3, Kubota discloses the medical image processing apparatus according to claim 1, wherein the processor is configured to, in the first reporting process, cause the display apparatus to superimposed and display of the information in accordance with a position of the region of interest in the medical image (see claim 1, also page 7, paragraphs, [0085-0087] finally, the notification control unit 35 generates support information and makes a notification on the basis of the notification method determined in step S4 (S5). More specifically, the notification control unit 35 generates a marker image G2 in accordance with the level of the oversight risk as the support information and outputs the marker image G2 to the image “superimposing” unit 36. The image superimposing unit 36 outputs an endoscope image in which the marker image G2 input from the notification control unit 35 is superimposed on the observation image G1 input from the image input unit 31, to the monitor 5 and causes the endoscope image to be displayed.
Also, page 8, paragraphs, [0096-0097] The present item is an oversight risk determined on the basis of the number of lesion areas Ln existing in the observation image G1. The number of lesion areas Ln existing in the observation image G1 is detected at a lesion number analysis unit 34A5 of the oversight risk analysis unit 34. The lesion number analysis unit 34A5 determines the oversight risk on the basis of the number of lesion areas Ln detected in the observation image G1. In other words, in a case where the number of lesion areas Ln is larger than a threshold (for example, two) set in advance, it is determined that the oversight risk is low. On the other hand, in a case where the number of lesion areas Ln is equal to or less than the threshold set in advance, it is determined that the oversight risk is high. The present item is an oversight risk determined while oversight risks based on the shape, the position, the size, and the like, of the lesion area Ln are comprehensively taken into account. More specifically, the degree of difficulty in finding the lesion area Ln is judged using analysis results or risk determination results at the lesion shape analysis unit 34A4, the lesion position analysis unit 34A2 and the lesion size analysis unit 34A1 of the oversight risk analysis unit 34. For example, in a case where the lesion area Ln has a protruding shape, the lesion area Ln is located near the center of the observation image G1 or the size of the lesion area Ln is large, it is judged that the degree of difficulty in finding the lesion area Ln is small. Further, for example, in a case where the lesion area Ln has a flat shape, the lesion area Ln is located near the periphery portion of the observation image G1 or the size of the lesion area Ln is small, it is judged that the degree of difficulty in finding the lesion area Ln is large. In a case where the degree of difficulty in finding the lesion area Ln is small, it is determined that the oversight risk is low. On the other hand, in a case where the degree of difficulty in finding the lesion area Ln is large, it is determined that the oversight risk is high.
Finally, page 9, paragraph, [0106] a notification method using an image notification is set in an image notification setting unit 35C1. Further, a notification method using a voice notification and a notification method using a vibration notification are respectively set in a voice notification setting unit 35C2 and in a vibration notification setting unit 35C3. An example of setting content at each notification setting unit will be described below).
Regarding claim 4, Kubota discloses the medical image processing apparatus according to claim 1, wherein the processor is configured to execute a number-of-detections calculation process of calculating the number of consecutive detections for the region of interest detected in the region-of-interest detection process, and in a case where the detection is determined to be the continuous detection, execute the second reporting process in response to the number of consecutive detections exceeding a predetermined number (see page 5, paragraphs, [0063-0067] a block in which the lesion area Ln exists is specified from these “nine blocks” 1A to 3C and output as the position of the lesion area (region of interest), Ln. Note that in a case where the lesion area Ln exists across a plurality of blocks, a block which has the largest area in which the lesion area Ln exists is specified as a block in which the lesion area Ln exists. Note that a method for specifying the block in which the lesion area Ln exists is not limited to the above-described method, but other methods such as a method in which a pixel located at the center of the lesion area Ln exists is specified as the block in which the lesion area Ln exists. Further, the number of blocks generated by dividing the observation image G1 is not limited to nine blocks, but may be, for example, 2×2=4 blocks, or 4×4=16 blocks. Note that the position of the lesion area Ln may be calculated as a distance from a central pixel position of the observation image G1 instead of being calculated as the above-described block position. In this case, in a case where the calculated distance is greater than a threshold set in advance, it is determined that the oversight risk is high. On the other hand, in a case where the calculated distance is smaller than the threshold set in advance, it is determined that the oversight risk is low. The lesion density analysis unit 34A3 extracts density values (luminance values) of respective pixels included in the lesion area Ln, obtains an average value of the density values and sets the average value as a density value of the lesion area Ln. Note that other statistics such as a mode value may be used in calculation of the density value, instead of using the average. In a case where the calculated density value is greater than a threshold (for example, a density value of normal mucosa), it is determined that the oversight risk is low. On the other hand, in a case where the estimated density value of the lesion area Ln is smaller than the threshold, it is determined that the oversight risk is high. Note that a value registered in advance or a value of a normal mucosa portion in the observation image G1 in which the lesion area Ln exists may be used as the density value which becomes a criterion for determination. The notification target risk setting unit 35B sets an oversight risk for which a notification is to be controlled. The oversight risk which can be set is a risk for which the level of the risk can be determined on the basis of the analysis result of the oversight risk analysis unit 34. Setting items include, for example, (A) the number of detected lesion areas Ln, (B) a degree of difficulty in finding the lesion area Ln, (C) a size of the lesion area Ln, (D) a type of the lesion area Ln, (E) detection reliability of the lesion area Ln, and (F) an elapsed time period since the lesion area Ln has been detected. The notification target risk setting unit 35B selects an item to be set as the notification target risk among these setting items.
Also, page 7, paragraphs, [0096-0097] the present item is an oversight risk determined on the basis of the number of lesion areas Ln existing in the observation image G1. The number of lesion areas Ln existing in the observation image G1 is detected at a lesion number analysis unit 34A5 of the oversight risk analysis unit 34. The lesion number analysis unit 34A5 determines the oversight risk on the basis of the number of lesion areas Ln detected in the observation image G1. In other words, in a case where the number of lesion areas Ln is larger than a threshold (for example, two) set in advance, it is determined that the oversight risk is low. On the other hand, in a case where the number of lesion areas Ln is equal to or less than the threshold set in advance, it is determined that the oversight risk is high).
Regarding claim 5, Kubota discloses the medical image processing apparatus according to claim 4, wherein the processor is further configured to execute: a feature quantity hold process of holding a feature quantity of the detected region of interest; and execute an identity determination process of comparing a feature quantity of a first region of interest detected from a medical image captured at a first time with the held feature quantity of a second region of interest detected from a second medical image captured at a second time that is a time before the first time, thereby determining identity between the first region of interest and the second region of interest, wherein the processor is configured to, in the number-of-detections calculation process, calculate the number of consecutive detections for the first region of interest in accordance with a determination result obtained in the identity determination process (see claim 1, also page 4, paragraphs, [0055-0059] while the method based on the image has been described here, the image pickup distance may be calculated on the basis of a ranging sensor, or the like, as other methods. As described above, after a distance between the endoscope 2 and the lesion area Ln is estimated, the lesion size analysis unit provides a threshold smaller than the image pickup distance and a threshold greater than the image pickup distance for the image pickup distance of pixels around the lesion and extracts a region in an image pickup distance band in which the lesion is located through threshold processing. The lesion size analysis unit calculates a degree of circularity of the region and, in a case where the degree of circularity is greater than a predetermined value, detects the region as a lumen. Finally, the lesion size analysis unit “compares” the lumen with the lesion area and estimates the size of the lesion area. More specifically, the lesion size analysis unit 34A1 estimates an actual size of the lesion by calculating a ratio of a length of the lesion with respect to a circumferential length of the detected lumen. Note that it is also possible to improve accuracy of size estimation by setting a circumferential length of the lumen of each organ site (position) in advance on the basis of anatomy. For example, in a case of colorectal examination, it is possible to improve accuracy of size estimation by estimating a site (position) of a lesion area at large intestine from an insertion amount of the insertion portion and comparing the length with the circumferential length of the lumen set in advance. As described above, the lesion size analysis unit estimates the size of the lesion area Ln by comparing the size with a circular size of the lumen in the endoscope image. In a case where the estimated size of the lesion area Ln is greater than a predetermined size (for example, 5 mm) set in advance, it is determined that the oversight risk is low. On the other hand, in a case where the estimated size of the lesion area Ln is smaller than the predetermined size, it is determined that the oversight risk is high. Also, page 8, paragraph, [0097] the present item is an oversight risk determined while oversight risks based on the “shape, the position, the size”, (features), and the like, of the lesion area Ln are comprehensively taken into account. More specifically, the degree of difficulty in finding the lesion area Ln is judged using analysis results or risk determination results at the lesion shape analysis unit 34A4, the lesion position analysis unit 34A2 and the lesion size analysis unit 34A1 of the oversight risk analysis unit 34. For example, in a case where the lesion area Ln has a protruding shape, the lesion area Ln is located near the center of the observation image G1 or the size of the lesion area Ln is large, it is judged that the degree of difficulty in finding the lesion area Ln is small. Further, for example, in a case where the lesion area Ln has a flat shape, the lesion area Ln is located near the periphery portion of the observation image G1 or the size of the lesion area Ln is small, it is judged that the degree of difficulty in finding the lesion area Ln is large. In a case where the degree of difficulty in finding the lesion area Ln is small, it is determined that the oversight risk is low. On the other hand, in a case where the degree of difficulty in finding the lesion area Ln is large, it is determined that the oversight risk is high).
Regarding claim 9, Kubota discloses the medical image processing apparatus according to claim 1, wherein the processor is configured to, in the first reporting process, change a mode of the first reporting process in accordance with a sound output state in the second reporting process (see claim 1, also page 7, paragraphs, [0094-0095] the notification means selection unit 35A selects means for making a notification that the support information is displayed on the monitor 5. The notification means which can be selected includes a notification using “voice from a speaker” 5a (hereinafter, referred to as a voice notification), a notification using vibration of a portion such as the operation portion 10 grasped by the surgeon (hereinafter, referred to as a vibration notification), or the like, in addition to the above-described display of an image on the monitor 5 (hereinafter, referred to as an image notification). The notification target risk setting unit 35B sets an oversight risk for which a notification is to be controlled. The oversight risk which can be set is a risk for which the level of the risk can be determined on the basis of the analysis result of the oversight risk analysis unit 34. Setting items include, for example, (A) the number of detected lesion areas Ln, (B) a degree of difficulty in finding the lesion area Ln, (C) a size of the lesion area Ln, (D) a type of the lesion area Ln, (E) detection reliability of the lesion area Ln, and (F) an elapsed time period since the lesion area Ln has been detected. The notification target risk setting unit 35B selects an item to be set as the notification target risk among these setting items).
Regarding claim 10, Kubota discloses the medical image processing apparatus according to claim 1, wherein the processor is configured to, in the first reporting process, cause the display apparatus to superimpose and display of the information, the information being at least one of a character, a figure, or a symbol (see claim 1, also page 7, paragraphs, [0085-0086] finally, the notification control unit 35 generates support information and makes a notification on the basis of the notification method determined in step S4 (S5). More specifically, the notification control unit 35 generates a marker image G2 in accordance with the level of the oversight risk as the support information and outputs the marker image G2 to the image superimposing unit 36. The image superimposing unit 36 outputs an endoscope image in which the marker image G2 input from the notification control unit 35 is superimposed on the observation image G1 input from the image input unit 31, to the monitor 5 and causes the endoscope image to be displayed. FIG. 6 and FIG. 7 are views illustrating an example of the endoscope image displayed on the monitor 5. In other words, FIG. 6 and FIG. 7 illustrate endoscope images in which the support information is superimposed, FIG. 6 illustrates an example where the oversight risk is low, and FIG. 7 illustrates an example where the oversight risk is high. As illustrated in FIG. 6 and FIG. 7, the observation image G1 to which the marker image G2 is added is displayed in a display region D1 on a display screen 51A of the monitor 5. As illustrated in FIG. 6, in a case where the oversight risk is low, the marker image G2 having a size enclosing the circumference of a lesion area L1 is superimposed on the observation image G1. On the other hand, as illustrated in FIG. 7, in a case where the oversight risk is high, the marker image G2 having a size enclosing a periphery portion of the observation image G1 is superimposed on the observation image G1. Further, a thickness of the marker image G2 illustrated in FIG. 7 is made thicker than a thickness of the marker image G2 illustrated in FIG. 6).
Regarding claim 11, Kubota discloses an endoscope system comprising: the medical image processing apparatus according to claim 1; an endoscope to be inserted into a subject, the endoscope having an imaging unit configured to capture the medical image; the display apparatus; and the sound output apparatus (see claim 1, also page 2, paragraph, [0023-0025] the endoscope 2 of the present embodiment transmits illumination light from the light source apparatus 3 to the distal end portion 6 with a light guide cable which is disposed in the universal cable 19, the operation portion 10 and the insertion portion 9 and which is illumination means. The processor 4, which is electrically connected to the monitor 5 which “displays” an endoscope image, processes an image pickup signal which is subjected to photoelectric conversion by image pickup means such as a CCD mounted on the endoscope 2, and outputs the processed image pickup signal to the monitor 5 as an image signal. The monitor 5 is a display apparatus at which the endoscope image is to be displayed. Further, the monitor 5 includes a speaker 5a which outputs voice. Note that the monitor 5 also has a function as a notification unit).
Regarding claim 13, Kubota discloses a non-transitory computer-readable recording medium storing a program for causing, when read by a computer, the computer to execute the medical image processing method according to claim 12. (see above, also page 1, paragraph, [0008] a storage medium according to one aspect of the present invention is a non-transitory computer-readable recording medium storing a program to be executed by a computer, the computer-readable storage medium storing an endoscope image processing program for causing the computer to execute sequentially acquiring a plurality of observation images obtained by picking up images of an object with an endoscope, detecting from the observation images a lesion area which is an observation target of the endoscope).
With regard to claims 8 and 12 the arguments analogous to those presented above for claims 1, 2, 3, 4, 5, 9, 10, 11 and 13, are respectively applicable to claims 8 and 12.
Allowable Subject Matter
Claims 6 and 7, are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Seyed Azarian whose telephone number is (571) 272-7443. The examiner can normally be reached on Monday through Thursday from 6:00 a.m. to 7:30 p.m.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Matthew Bella, can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR.
Status information about the PAIR system, see http:// pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/SEYED H AZARIAN/Primary Examiner, Art Unit 2667
December 2, 2025