DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 03/13/2025 has been entered. Claims 1-8, 14, and 16-19 are pending in the application. Claims 1-6, 14, and 16 are amended. Claims 9-13 and 15 are canceled.
Response to Arguments
Applicant’s arguments, see pages 5-13, filed 03/13/2025, with respect to the rejection of the pending claims have been fully consider but are moot in view of the new grounds of rejection necessitated by the amendments to the claims. Claim 1 has been modified with a newly added limitation stating, “…wherein the detection result is represented by a specific geometric shape indicating the region of interest, when a first specific distance in the medical image between a specific position in the medical image and the detection position of the region of interest is within a specific range, the detection result of the region of interest is displayed according to a first display style which is associated with the first specific distance, when a second specific distance in the medical image between the specific position in the medical image and the detection position of the region of interest is greater than the specific range, the detection result of the region of interest is displayed according to a second display style which is associated with the second specific distance, the second display style which is different from the first style, and wherein the specific position is included in a center region of the medical image.” As such, the scope of the claims was substantially changed and new grounds for rejection was necessitated by such amendments, which are presented below.
Please see section 35 U.S.C §102 and 35 U.S.C §103 for further explanation.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 7, 8, 14, and 17-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Hameed et al. (US2019/0080454) hereinafter Hameed.
Regarding claim 1, Hameed discloses a medical image processing system comprising:
one or more processors (Fig. 1c processor) configured to function as:
a medical image acquisition unit (Fig. 1c front-facing imaging device and side-mounted imaging devices) that acquires a medical image (Figs. 3A-3D image ) obtained by imaging of an observation target by using an endoscope (Figs. 1a-1g endoscope 102);
a region-of-interest detection unit (Fig. 2A region of interest within the acquired image from the image device in step 202) that detects a region of interest (area near polyp within image of Figs. 3A-3D) from the medical image (Figs. 3A-3D image); and
a display control unit (Figs. 1C display) that displays the medical image (Figs. 3A-3D image) and a detection result (polyp within Figs. 3A-3D) of the region of interest (area near polyp within image of Figs. 3A-3D) on a display unit (Figs. 1C display) using a display style (Figs. 3a-d solid line circles or rectangles from Steps 208 – 210 of Fig. 2A) that differs depending on at least a detection position of the region of interest (area near polyp within image of Figs. 3A-3D) which is included in the medical image (Figs. 3A-3D image),
wherein the detection result (polyp within Figs. 3A-3D) is represented by a specific geometric shape (Fig. 3B circle) indicating the region of interest (area near polyp within image of Figs. 3A-3D),
when a first specific distance (surface peak located at a top of a fold, [0027]) in the medical image (Figs. 3A-3D image ) between a specific position in the medical image (Figs. 3A-3D image ) and the detection position of the region of interest is within a specific range ([0028-0029]), the detection result of the region of interest is displayed according to a first display style (brighter/more intense) [0027] which is associated with the first specific distance ([0006] “…identifying surface peaks in the image, identifying clusters of surface peaks based on a predetermined threshold separation distance, defining a pixel region around each of the selected surface peaks,..”[0027] whole paragraph, and [0027] “The processor may optionally convert 204 the images to grayscale (i.e., depending on whether the imaging device acquired the image(s) in color or in black and white). Method 200 may comprise identifying surface peaks 206 in the grayscale image. Surface peaks may represent surface regions or points that are located at the top surface or the bottom surface of a fold, as identified by local intensity extremums (e.g., minimums or maximums, respectively) in the image. For example, a surface peak that is located at the bottom of a fold may be further from the imaging device(s) than a surface peak that is located at the top of a fold, and the difference in distance may be determined by calculating the intensity difference between the identified surface peaks. A surface peak that is located at the top of a fold may be brighter (e.g., more intense) than a surface peak that located at the bottom of the fold, and the intensity difference may indicate the distance between the two peaks.”),
when a second specific distance (surface peak located at a bottom of a fold, [0027]) in the medical image (Figs. 3A-3D image ) between the specific position in the medical image (Figs. 3A-3D image ) and the detection position of the region of interest is greater than the specific range ([0028-0029]), the detection result of the region of interest is displayed according to a second display style (darker/less intense) [0027]) which is associated with the second specific distance (surface peak located at a bottom of a fold, [0027]) ,
the second display style (darker/less intense) [0027]) which is different from the first style (brighter/more intense) [0027], and
the specific position is included in a center region (Center region is being interpreted reasonably broadly, thus the center region excludes the darkened corners of Figs. 3a-3d, [0043] “ If the upcoming length of colon is relatively straight, the location of the darkest region of the image may remain in a central area of the image as the endoscope is advanced forward.”)) of the medical image (Figs. 3A-3D image ).
Regarding claim 2, Hameed discloses the medical image processing system according to claim 1, wherein a detection accuracy of the central region of the medical image(Figs. 3A-3d image) is greater than a detection accuracy of a peripheral area of the image (Center region is being interpreted reasonably broadly, thus the center region excludes the darkened corners of Figs. 3a-3d, [0043] “ If the upcoming length of colon is relatively straight, the location of the darkest region of the image may remain in a central area of the image as the endoscope is advanced forward.”))
Regarding claim 7, Hameed disclose the medical image processing system according to claim 1, wherein when the detection result of the region of interest is represented by a specific color region, a color of the specific color region differs depending on the detection position of the region of interest. ([0027] “The processor may optionally convert 204 the images to grayscale (i.e., depending on whether the imaging device acquired the image(s) in color or in black and white). Method 200 may comprise identifying surface peaks 206 in the grayscale image … calculating the intensity difference between the identified surface peaks. A surface peak that is located at the top of a fold may be brighter (e.g., more intense) than a surface peak that located at the bottom of the fold, and the intensity difference may indicate the distance between the two peaks.” and [0040] “In some variations, a polyp detection method may optionally comprise identifying characteristics of the polyp and its surrounding colon surface environment and storing data pertaining to those characteristics in a memory of the processor. This may allow a practitioner to determine whether a polyp has been encountered previously, or is a newly identified polyp. Examples of polyp parameters that may be stored and used to identify a polyp may include size, shape, light reflection properties, circumferential location, longitudinal location (e.g., colon segment where the polyp is located), surface texture, coloration, etc. The location of a polyp may be computed or estimated based on image analysis of travel distance relative to an origin (e.g., motion detection) and/or anatomical structures (e.g., striated muscle patterns, characteristic curves/bends/flexures, rectum folds, vascular patterns), and reference tags selected by the practitioner.”
Regarding claim 8, Hameed disclose the medical image processing system according to claim 1, wherein when the detection result of the region of interest is represented by a specific color region,([0027] and [0044]) a display time of the specific color region differs depending on the detection position of the region of interest ([0041] “The position-scanning speed plot (such as that depicted in FIG. 7A) may be generated by the processor and displayed during the scanning session (thereby providing real-time feedback to the practitioner) and/or may be generated by the processor after the conclusion of the scan.”).
Regarding claim 14, Hameed disclose the medical image processing system of claim 1, wherein when the distance in the medical image between the specific position and the detection position of the region of interest is greater than zero, the display style of the detection result of the region of interest changes ([0043] “…a processor may provide navigational guidance to a practitioner to provide advanced notice of approaching curves in the colon. This may help to reduce the likelihood of a practitioner advancing the distal end of the endoscope into the wall of the colon (which often causes discomfort or pain to the patient). In one variation, the processor may be configured to identify features in an image that indicate a change in the curvature of the colon lumen, and when a change in curvature is detected, an arrow may appear on the display that indicates the direction of the curvature change. In some variations, the processor may track the movement of the darkest region of the image, where the darkest region of the image may represent the region of the colon furthest from the endoscope. If the upcoming length of colon is relatively straight, the location of the darkest region of the image may remain in a central area of the image as the endoscope is advanced forward. If the upcoming length of colon curves, the location of the darkest region of the image may shift away from the central area of the image as the endoscope is advanced. For example, if the colon segment curves to the right, the darkest region in the image may move towards the right. If the area of the image occupied by the dark region monotonically grows, the processor may interpret such visual cue as the endoscope is moving in a trajectory that will impact or collide with the colon wall. The processor may generate a notification to prompt the practitioner to quickly steer the endoscope in the direction of the curve. The arrow may flash at a frequency that indicates the proximity of the distal end of the endoscope to the colon wall ahead of it. For example, as the distal tip nears the wall, the arrow flashing frequency may increase. Alternatively or additionally, an audible signal may be generated if the processor determines that the distal tip of the endoscope is within a predetermined distance from a colon wall. For example, the audible signal may be a tone pulsed at an initial frequency and as the distal tip nears a colon wall and is at risk of directly contacting the wall, the frequency may increase. A method for providing navigational guidance may comprise identifying the dark region of an image (e.g., lumen of the colon) from a front-facing imaging device of a colonoscope, determining whether the dark region remains in a central area of the image (or field-of-view of the front-facing imaging device) as the colonoscope is advanced, and if the dark region shifts from the central area of the image, providing an indication to the clinician to steer the colonoscope in the direction of the shift. The method may optionally comprise determining whether the area occupied by the dark region monotonically grows as the colonoscope is advanced and providing an indication to the clinician to steer the colonoscope away from the wall of the colon (e.g., steer left or right).”).
Regarding claim 17, Hameed disclose the medical image processing system according to claim 1, wherein the medical image processing system is an endoscope system (Figs. 1a-1g endoscope system 100) which comprises the endoscope (Figs. 1a-1g endoscope 102) and the one or more processors (Fig. 1c processor).
Regarding claim 18, Hameed disclose the medical image processing system of claim 1, wherein the detection result (polyp) encapsulates (the solid line around the polyp) the region of interest (area near polyp within image Figs. 3a-3d) within the medical image (Figs. 3a-3d image) being displayed.
Regarding claim 19, Hameed discloses the medical image processing system of claim 1, wherein the specific position is a predetermined location (the colon) in the center of the medical image (Figs. 3a-3d) being displayed. ([0043])
Claim 1 is rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Jia et al. (US2015/0065850) hereinafter Jia.
Regarding claim 1, Jia discloses a medical image processing system (Fig. 6 computer system 61, abstract) comprising:
one or more processors (Fig. 6 CPU 62)) configured to function as:
a medical image acquisition unit (Fig. 6 memory 67) that acquires a medical image obtained by imaging of an observation target by using an endoscope ([0036] “…images received from an endoscopy apparatus…” [0030]);
a region-of-interest detection unit ([0036] SMV classifier) that detects a region of interest (polyp) from the medical image ([0036] image); and
a display control unit (Fig. 6 display 65) that displays the medical image ([0036] image) and a detection result (polyp) of the region of interest (Fig. 2 ROI I, II, III) on a display unit (Fig. 6 display 65) using a display style ([0046] dotted line of curve 20 detection result) differs depending on at least a detection position (Fig. 2 curve 20 detection result) of the region of interest (Fig. 2 ROI I, II, III) which is included in the medical image ([0036] image),
wherein the detection result (Fig. 2 curve 20 detection result) is represented by a specific geometric shape (Fig. 2 oval) indicating the region of interest (Fig. 2 ROI I, II, III)
when a first specific distance (Fig. 2 ROI I) in the medical image ([0036] image) between a specific position (Fig. 2 ‘center’ identified by ‘plus’ sign) in the medical image ([0036] image) and the detection position (Fig. 2 curve 20 detection result) of the region of interest (Fig. 2 ROI I, II, III) is within a specific range ([0046]) the detection result (polyp) of the region of interest(Fig. 2 ROI I, II, III) is displayed according to a first display style (Fig. 2 light grey enclosed section of ROI I, [0046]) which is associated with the first specific distance (Fig. 2 ROI I),
when a second specific distance (Fig. 2 ROI III) in the medical image ([0036] image) between the specific position (Fig. 2 ‘center’ identified by ‘plus’ sign) in the medical image ([0036] image ) and the detection position (Fig. 2 curve 20 detection result) of the region of interest (Fig. 2 ROI I, II, III) is greater than the specific range ([0046]), the detection result (polyp) of the region of interest (Fig. 2 ROI I, II, III) is displayed according to a second display style (Fig. 2 dark grey enclosed section of ROI III, [0046]) which is associated with the second specific distance (Fig. 2 ROI III),
the second display style (Fig. 2 dark grey enclosed section of ROI III, [0046]) which is different from the first style (Fig. 2 light grey enclosed section of ROI I, [0046])], and
the specific position (Fig. 2 ‘center’ identified by ‘plus’ sign) is included in a center region (center of Fig. 2) of the medical image ([0036] image, abstract )
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-4, 6, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Hameed in view of Goto et al. US20090116718 hereinafter Goto.
Regarding claim 3, Hameed discloses the medical image processing system according to claim 1, but is silent as to explicitly disclose wherein a line thickness of the specific geometric shape differs depending on the detection position of the region of interest.
However Goto, in the same field of endeavor teaches a line thickness of the specific geometric shape differs depending on the detection position of the region of interest (Fig. 3 and [0051-0057]
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Hameed with the teachings of Goto to have, a line thickness of the specific geometric shape differs depending on the detection position of the region of interest for the benefit of “… set[ting] an ROI smoothly along the subject, and to accurately extract the subject” because the “shape of the marker is in accordance with the shape of the subject such as an organ or ROI…”[0078].
Regarding claim 4, Hameed discloses the medical image processing system according to claim 1, but is silent as to whether a size of the specific geometric shape differs depending on the detection position of the region of interest.
However Goto, in the same field of endeavor, teaches when the detection result of the region of interest is represented by a specific geometric shape, a size of the specific geometric shape differs depending on the detection position of the region of interest ([0073] “As shown in FIG. 7(C), the CPU 11 changes the form of the marker 172 based on the detected straddle degree, and displays the changed marker 172 c. For example, it may be set so that the size of the marker 172 is to be made small when the straddle degree is large, and the size of the marker 172 is to be made large when the straddle degree is small. The CPU 11 changes the shape of ROI 171 based on the marker 172 c, and displays the ROI 171 c.”).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify the teachings of Hameed with the teachings Goto to include when the detection result of the region of interest is represented by a specific geometric shape, a size of the specific geometric shape differs depending on the detection position of the region of interest for the benefit of “… set[ting] an ROI smoothly along the subject, and to accurately extract the subject” because the “shape of the marker is in accordance with the shape of the subject such as an organ or ROI…”[0078].
Regarding claim 6, Hameed discloses the medical image processing system according to claim 1, but is silent as to when a transparency of the specific geometric shape differs depending on the detection position of the region of interest.
However Goto, in the same field of endeavor, teaches when the detection result of the region of interest is a transparency of the specific geometric shape differs depending on the detection position of the region of interest ([0076-0078] “While the shape of the marker is illustrated in circular shape in FIG. 3˜FIG. 7, it does not have to be limited to a circular shape. The shape of the marker may be arbitrarily set in accordance with the shape of the subject such as an organ. The shape of the marker 182 shown in FIG. 8 is an ellipse. Further, the major-axis radius and the minor-axis radius of the marker 182 may be set based on the positional relationship between a central position 183 and a contact point 184 and curvature radius of a ROI 181. When the shape of the marker is circular or ellipse, contour of the organ can be smoothly extracted. The shape of the marker may be other than circular or ellipse shape, such as rectangle or squire. Also, in the case of setting ROI in a very small region such as a blood vessel, the marker may be a “dot”. In this way, by setting a shape of the marker in accordance with the shape of the subject such as an organ or ROI, it is possible to change the shape of the ROI effectively whereby improving the operationality of the display device.”).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Hameed with the teachings of Goto to have the detection result of the region of interest is represented by a specific geometric shape and a transparency of the specific geometric shape differs depending on the detection position of the region of interest for the benefit of “… set[ting] an ROI smoothly along the subject, and to accurately extract the subject” because the “shape of the marker is in accordance with the shape of the subject such as an organ or ROI…”[0078].
Regarding claim 16, Hameed discloses the medical image processing system of claim 14, but is silent as to wherein the detection result is displayed as a bounding box
However Goto, in the same field of endeavor, teaches wherein the detection result is displayed as a bounding box (fig. 3 ROI 134)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Hameed in view of Goto to include the detection result is displayed as a bounding box and the second detection result is displayed as a second bounding box for the purpose of showing the operator a positional relationship between two results [0053-0057].
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Hameed in view of Tammo Heeren (US2017/0280989) hereinafter Heeren.
Regarding claim 5, Hameed discloses the medical image processing system according to claim 1, but is silent as to explicitly disclose wherein a display time of the specific geometric shape differs depending on the detection position of the region of interest.
However Heeren, in the same field of endeavor teaches, wherein a display time of the specific geometric shape differs depending on the detection position of the region of interest ([0037] “Certain embodiments provide a user with a microscope image of eye tissue that includes a computer-generated visual indicator (e.g., a pointer, shape, icon, or other graphic element) which indicates an distance between particular eye tissue (e.g., a retina) and a surgical tool inserted in the eye (e.g., the tip of a vitrectomy probe). One or more characteristics of the visual indicator (e.g., its color, size, shape) may be modified in real-time to reflect the distance between the surgical tool and particular eye tissue. In certain embodiments, a characteristic of the visual indicator (e.g., size, color) is modified incrementally, and proportional to the change in distance, to intuitively convey the movement of the tool. The distance between the surgical tool and particular eye tissue may be determined based on data obtained by an imaging system capable of resolving depth in real time, such as an OCT imaging system, ultrasound imaging system, a multispectral imaging system, a computerized axial tomography (CAT) scan system, a magnetic resonance imaging (MRI) system, or a positron emission tomography (PET) imaging system. Certain embodiments also track movements of the surgical tool within a microscope image in real time, and may display the visual indicator as a dynamic overlay in the microscope image presented in an eyepiece or on a heads-up display. For example, a visual indicator may be displayed as a graphic overlay superimposed on a distal end of the surgical tool as it moves within in a microscope image of a retina, and the size and/or color of the overlay indicator may be updated continuously according to the distance between the distal end of the surgical tool and the retina.”.
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Hameed in view of Heeren to include whether a display time of the specific geometric shape differs depending on the detection position of the region of interest for the benefit of “convey[ing] the proximity of a surgical tool to a particular tissue” [0037].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEGAN E MONAHAN whose telephone number is (571)272-7330. The examiner can normally be reached Monday - Friday, 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Carey can be reached on (571) 270-7235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEGAN ELIZABETH MONAHAN/Examiner, Art Unit 3795
/MICHAEL J CAREY/Supervisory Patent Examiner, Art Unit 3795