Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 8-10, 12, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kitamura et al. US2021/0000327 and further in view of Zou et al. US2022/0249060, Meagher US2020/0297433 and Haase US2022/0351369.
Kitamura discloses for claim 1, “A medical image processing apparatus comprising:
a processor (image processing apparatus 13; fig 1; 0028) configured to:
acquire a medical image (0034);
detect a region-of-interest in the medical image (lesion-candidate-region detecting section 131; fig 1; 0038);
set a highlight display for highlighting the detected region-of-interest and superimpose and display the highlight display on the medical image (fig 4; 0064, 0068 describes setting marker images MM1 and MS1);
wherein the processor executes a process performed by an image processing unit (lesion-candidate-region detection section 131; fig 1; 0037) and a process performed by a display control unit (display control section 133; fig 1; 0037),
wherein the process performed by the image processing unit detects the region-of-interest, sets the highlight display (0038)”,
wherein the process performed by the display control unit superimposes and displays the highlight display on the medical image (0042)”.
Kitamura does not disclose:
“determine a visibility of the highlight display from image information acquired from the medical image in which the region-of-interest is detected and the highlight display which is set;
calculate a numerical index value as the determination result of the visibility
determines the visibility and notifies”.
Zou teaches in the same field of endeavor, determining different parameters and/or attributes of indicators for marking relevant portions in an image to display on the image, the parameters including a “visibility of the indicator” (fig 19, 22; 0331 – “Therefore, in some embodiments of the present disclosure, configure the indicators for marking the intersection line or anatomical position with different parameters and/or attributes to identify different marked sections, the parameter and/or attribute include: any of color of the indicator, shape of the indicator (e.g. the various kinds of line types of the reference line, including dash lines, solid lines, and dot dash lines in the FIG. 19), symbol label, indication label (including by way of marking a guide line and a note symbol), proportion of the indicator, and visibility of the indicator, and any combination thereof.”). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Zou into the invention of Kitamura in order to configure the medical image processing apparatus e.g. as claimed because it provides relevant meta data and helps to characterize the corresponding anatomical position (0331).
Kitamura does not disclose:
“notify a user of the determination result of the visibility in a case where the index value is equal to or smaller than a preset threshold value; and not notify in a case where the index value exceeds the threshold value”.
Meagher teaches in the same field of endeavor, “A decrease in the visibility of an identified structure might additionally or alternatively result in a visual, auditory or haptic notification signal to the user” 0026. Additionally, Haase teaches in the same field of endeavor, a visibility score for a imaged subject for determining adequate and inadequate visibility based on a threshold, either above or below (0036). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Meagher in light of Haase into the invention of Kitamura in order to configure the apparatus e.g. as claimed because it allows conspicuous notification when the visibility of a marker/indicator (Meaghan: 0026) in the context of a preset reference or threshold either above or below (Haase: 0036).
Kitamura discloses for claim 2, “The medical image processing apparatus according to claim 1, wherein the processor is configured to acquire the image information from an inside of the highlight display in the medical image (0068 describes the process for generating marker images MM1 and MS1 is based on the lesion candidate information, i.e. the image information within the markers)”.
Modified Kitamura discloses for claim 9, “The medical image processing apparatus according to claim 1, wherein the processor is configured to display the determination result on a display screen (Zou: fig 19; 0331 describes the visibility of the indicator as a parameter, i.e. a numerical index value)”.
Kitamura discloses for claim 10, “The medical image processing apparatus according to claim 9, wherein the processor is configured to display the index value as the notification in a case where the index value is equal to or smaller than the preset threshold value (0086 describes the detection of a lesion based on a threshold value thereby determining the display based on the comparison. See also 0115 and 0117).
Modified Kitamura discloses for claim 12, “The medical image processing apparatus according to claim 1, wherein the processor is configured to display identification information or an identification figure according to the index value (Zou: 0331 describes the visibility of the indicator as a parameter, i.e. a numerical index value)”.
Modified Kitamura (as provided in claim 1) discloses for claim 18, “An endoscope system comprising:
a light source device (light source section 121; fig 1; 0033) that emits an illumination light beam for illuminating an observation target;
an endoscope including an imaging sensor (image pickup section 111; fig 1; 0030) which images the observation target illuminated with the illumination light beam;
a processor (image processing apparatus 13; fig 1; 0028); and
a monitor (display apparatus 14; fig 1; 0028) that displays a medical image obtained by performing signal processing on an image signal which is output by the imaging sensor,
wherein the processor is configured to:
acquire the medical image (0034);
detect a region-of-interest in the medical image (lesion-candidate-region detecting section 131; fig 1; 0038);
set a highlight display for highlighting the detected region-of-interest and superimpose the highlight display on the medical image and display the medical image on which the highlight display is superimposed on the monitor (fig 4; 0064, 0068 describes setting marker images MM1 and MS1);
determine a visibility of the highlight display from image information acquired from the medical image in which the region-of-interest is detected and the highlight display which is set (Zou: fig 19, 22; 0331 – “Therefore, in some embodiments of the present disclosure, configure the indicators for marking the intersection line or anatomical position with different parameters and/or attributes to identify different marked sections, the parameter and/or attribute include: any of color of the indicator, shape of the indicator (e.g. the various kinds of line types of the reference line, including dash lines, solid lines, and dot dash lines in the FIG. 19), symbol label, indication label (including by way of marking a guide line and a note symbol), proportion of the indicator, and visibility of the indicator, and any combination thereof.”);
calculate a numerical index value as the determination result of the visibility (Zou: fig 19, 22; 0331);
notify a user of the determination result of the visibility in a case where the index value is equal to or smaller (Meagher: 0026) than a preset threshold value; and not notify in a case where the index value exceeds the threshold value (Haase: 0036)
wherein the processor executes a process performed by an image processing unit (lesion-candidate-region detection section 131; fig 1; 0037) and a process performed by a display control unit (display control section 133; fig 1; 0037),
wherein a process performed by the image processing unit detects the region-of-interest, sets the highlight display (0038), determines the visibility, and notifies,
wherein a process performed by the display control unit superimposes and displays the highlight display on the medical image (0042)”.
Modified Kitamura (as provided in claim 1) discloses for claim 19, “A method of operating a medical image processing apparatus, the method comprising:
acquiring a medical image (0034);
detecting a region-of-interest in the acquired medical image (lesion-candidate-region detecting section 131; fig 1; 0038);
setting a highlight display for highlighting the detected region-of-interest and superimposing and displaying the highlight display on the medical image (fig 4; 0064, 0068 describes setting marker images MM1 and MS1);
determining a visibility of the highlight display from image information acquired from the medical image in which the region-of-interest is detected and the highlight display which is set (Zou: fig 19, 22; 0331 – “Therefore, in some embodiments of the present disclosure, configure the indicators for marking the intersection line or anatomical position with different parameters and/or attributes to identify different marked sections, the parameter and/or attribute include: any of color of the indicator, shape of the indicator (e.g. the various kinds of line types of the reference line, including dash lines, solid lines, and dot dash lines in the FIG. 19), symbol label, indication label (including by way of marking a guide line and a note symbol), proportion of the indicator, and visibility of the indicator, and any combination thereof.”);
calculate a numerical index value as the determination result of the visibility (Zou: fig 19, 22; 0331);
notify a user of the determination result of the visibility in a case where the index value is equal to or smaller (Meagher: 0026) than a preset threshold value; and not notify in a case where the index value exceeds the threshold value (Haase: 0036)
wherein the method is carried by a processor that executes a process performed by an image processing unit (lesion-candidate-region detection section 131; fig 1; 0037) and a process performed by a display control unit (display control section 133; fig 1; 0037),
wherein a process performed by the image processing unit detects the region-of-interest, sets the highlight display (0038), determines the visibility, and notifies,
wherein a process performed by the display control unit superimposes and displays the highlight display on the medical image (0042)”.
Modified Kitamura (as provided in claim 1) discloses for claim 20, “A non-transitory computer readable medium for storing a computer-executable program for functioning a computer as a medical image processing apparatus that acquires a medical image and performs image processing on the medical image, the computer-executable program causing the computer to execute functions of:
acquiring the medical image (0034);
detecting a region-of-interest in the medical image (lesion-candidate-region detecting section 131; fig 1; 0038);
setting a highlight display for highlighting the detected region-of-interest and superimposing and displaying the highlight display on the medical image (fig 4; 0064, 0068 describes setting marker images MM1 and MS1);
determining a visibility of the highlight display from image information acquired from the medical image in which the region-of-interest is detected and the highlight display which is set (Zou: fig 19, 22; 0331 – “Therefore, in some embodiments of the present disclosure, configure the indicators for marking the intersection line or anatomical position with different parameters and/or attributes to identify different marked sections, the parameter and/or attribute include: any of color of the indicator, shape of the indicator (e.g. the various kinds of line types of the reference line, including dash lines, solid lines, and dot dash lines in the FIG. 19), symbol label, indication label (including by way of marking a guide line and a note symbol), proportion of the indicator, and visibility of the indicator, and any combination thereof”);
calculate a numerical index value as the determination result of the visibility (Zou: fig 19, 22; 0331);
notify a user of the determination result of the visibility in a case where the index value is equal to or smaller (Meagher: 0026) than a preset threshold value; and not notify in a case where the index value exceeds the threshold value (Haase: 0036)
wherein the computer executes a process performed by an image processing unit (lesion-candidate-region detection section 131; fig 1; 0037) and a process performed by a display control unit (display control section 133; fig 1; 0037),
wherein a process performed by the image processing unit detects the region-of-interest, sets the highlight display (0038), determines the visibility, and notifies,
wherein a process performed by the display control unit superimposes and displays the highlight display on the medical image (0042, and
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kitamura, Zou, Meagher, and Haase as applied to claim 1 above, and further in view of Karino US2021/0153722.
Kitamura does not disclose for claim 3, “The medical image processing apparatus according to claim 1, wherein the processor is configured to acquire the image information from an outside of the highlight display in the medical image”. Karino teaches in the same field of endeavor, a detection result of a region of interest is based in part from color detected in a peripheral region 44 outside of the region marked by bound box 58 (fig 11; 0055). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Karino into the invention of Kitamura in order to configure the medical image processing apparatus e.g. as claimed because allows distinguishing the region of interest based on the surrounding color, helping to distinguish similar color profiles.
Claim(s) 4 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kitamura, Zou, Meagher, and Haase as applied to claim 1 above, and further in view of Ikemoto et al. US2017/0112353.
Kitamura does not disclose for claim 4, “The medical image processing apparatus according to claim 1, wherein the processor is configured to acquire a color difference between the medical image and the highlight display from color information calculated from the image information and color information calculated from the highlight display, and determine the visibility from the color difference”. Ikemoto teaches in the same field of endeavor, visibility of a mask/highlight is related to the color of the masks “with respect to the normal observation image NP” (fig 23; 0154), teaching that a comparison of an object’s color with the surroundings is relevant in visibility detection. It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Ikemoto into the invention of Kitamura in order to configure the medical image processing apparatus e.g. as claimed because it allows color differentiation as a form of visibility between the region of interest in the medical image and the mask/highlight display.
Kitamura does not disclose for claim 11, “The medical image processing apparatus according to claim 9, wherein the processor is configured to use, as the index value, a color difference calculated from the image information and the highlight display”. Ikemoto teaches in the same field of endeavor, visibility of a mask/highlight is related to the color of the masks “with respect to the normal observation image NP” (fig 23; 0154), teaching that a comparison of an object’s color with the surroundings is relevant in visibility detection. It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Ikemoto into the invention of Kitamura in order to configure the medical image processing apparatus e.g. as claimed because it allows color differentiation as a form of visibility between the region of interest in the medical image and the mask/highlight display.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kitamura, Zou, Meagher, and Haase as applied to claim 1 above, and further in view of Kawakami US2022/0182555.
Kitamura does not disclose for claim 5, “The medical image processing apparatus according to claim 1, wherein the processor is configured to calculate, as color information, an average value calculated from the image information, wherein the color information is calculated from the image information, and wherein the average value is an average of the color information and calculated based on the inside or outside of the figure as the highlight display”. Kawakami teaches in the same field of endeavor, using an average color value when extracting color information from objects in an image to compare (0106). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Kawakami into the invention of Kitamura in order to configure the medical image processing apparatus e.g. as claimed because it facilitates analysis by potentially reducing the computation required compared to a pixel-wise algorithm.
Response to Arguments
Applicant’s arguments with respect to the claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument, specifically with Meagher, and Haase.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAE K WOO whose telephone number is (571)272-0837. The examiner can normally be reached M-F 8:30-2:30p, 6p-9p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jae Woo/Examiner, Art Unit 3795
/ANHTUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795
10/5/26
