Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Status of claims: claims 1-19 are pending; claims 1, 4, and 15 are amended.
The amendments to claims 1, 4 and 15 overcome the claim rejections under 35 USC § 102 set forth in the last Office Action.
Response to Arguments
Applicant’s arguments with respect to claims 1-19 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9, 11-19 are rejected under 35 U.S.C. 103 as being unpatentable over Hirokazu (WO2019/012586A1) in view of Shelton (US2019/0104919A1) and further in view of Usuda (US2021/0153720A1).
Claim 1, Hirokazu discloses a medical image processing apparatus comprising one or more processors configured to: (Fig. 2, “diagnosis support device 4” [0031]);
sequentially acquire a plurality of medical images captured through an endoscope (Fig.1; [0023], [0033]);
sequentially perform recognition of a region of interest in the medical images (detecting a suspected lesion region; [0033]);
sequentially display the ‘medial’ images on a monitor (Fig. 3; [0049]); and
start control to superimpose information indicating a position of the region of interest detected in the recognition on the medical images sequentially displayed on the monitor (Fig. 3, square frame image 33 surrounding the suspected lesion region LS; [0052]).
Hirokazu does not explicitly disclose measuring an elapsed display time of the overlay based on a user-set threshold to automatically stop the superimposition. Before going into analysis of this limitation, the following is noted from applicants’ “Remarks” (filed 3/12/2026; pp. 6-7):
“Applicant notes that the feature "monitor" recited in amended claims 1 and 15 corresponds to the second display 16B in the embodiment described in paragraphs [0168]-[0171] with reference to FIG. 14 of the present application, as described below.
In the embodiment, "[t]he reporting information display control unit 62B changes, in accordance with a reporting state and a non-reporting state on the first display 16A, a manner in which the second display 16B (an example of a second display unit) displays the reporting information. Here, the reporting information display control unit 62B displays reporting information on the second display 16B when the reporting information is in a non-reporting state" [0168]. Thereby, "in part F14E [of Fig. 14], corresponding to the timing at which the reporting information is in a non-reporting state, the frame-shaped figure Fl surrounding the region of interest RI is superimposed on the frame image 38a" [0170], and then "[i]n part[s F14D and] F14F, corresponding to the timing[s] at which the reporting information is in a reporting state, the figure F1 is not displayed" [0170], where the "non-reporting state" is switched to the "reporting state" on the first display 16A by the reporting information display control unit 62B described as "the reporting information display control unit 62B maintains a non-reporting state from when the moving image 38 does not satisfy the non-reporting condition to when a non-reporting time set by the non-reporting time setting unit 54 elapses, and after the non-reporting time has elapsed, starts a reporting state in which the reporting information is displayed" [0071], where the "non-reporting time" may be set by a user as described as "[t]he non- reporting time setting unit 54 may set a non-reporting time in accordance with a treatment state estimated by the treatment state estimating unit 50 or may set a value input by a user as a non-reporting time" [0065].” Based on the examiner’s understanding, the elapsed display time is based on the “reporting/non-reporting” state and the time duration is associated with each state and can be set in accordance with a treatment state. With this understanding, Shelton discloses measure a time elapsed from when starting the control to superimpose the information indicating the position of the region of interest; (monitoring and measuring “procedure time” and the elapsed duration of active/inactive surgical steps; [0321], [0337]);
determine whether or not the measured time exceeds a time set by a user; (comparing elapsed times against user-predefined or adjusted step-duration baseline thresholds; [0338-0340], [0380], [0383]); and
when the measured time exceeds the time set by the user, automatically stop the control to superimpose the information indicating the position of the region of interest detected in the recognition on the medical images sequentially displayed on the monitor (automatically adjusting, de-emphasizing, or stopping display overlays/views when active surgery begins to clear the field of view and reduce manual interaction; [0313], [0314], [0340], [0370], [0371]). Furthermore, to the extent that Hirokazu and Shelton do not explicitly disclose the details of performing the recognition of the region of interest using artificial intelligence, Usuda discloses performing the recognition of the region of interest in the medical images using an artificial intelligence (utilizing a machine learning algorithm such as a convolutional neural network or deep learning to recognize lesion portions [0066]).
Therefore, it would have been obvious to a PHOSITA at the time of the invention was filed to combine the display and image analysis apparatus of Hirokazu with the situational awareness, timeline-tracking, and automatic display-toggling controls of Shelton, and further incorporate the artificial intelligence recognition of Usuda, because tracking the elapsed duration of surgical steps against user-predefined thresholds allows the system to automatically stop superimposing the ROI overlay when active surgery begins, thereby preventing visual clutter, minimizing screen obstruction during critical, active surgical manipulation, and reducing clinician cognitive overload without requiring manual display adjustments.
Claim 15 rejected along the same line of reasoning as established in claim 1 for being a corresponding method claim.
Claims 2 & 17, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus and method as set forth in Claims 1 and 15. Hirokazu further discloses that the information indicating the position of the region of interest is superimposed as a frame-shaped figure surrounding the position of the region of interest (Fig. 3, square frame image 33 surrounding the suspected lesion region LS; [0052]).
Claim 3, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus of Claim 1. Hirokazu further discloses that the one or more processors are configured to, in a region different from a region where the medical images are displayed on the monitor, display information indicating that the control to superimpose the information indicating the position of the region of interest is being performed (Fig. 3, message display 34 “Analyze ON” displayed on the lower left side of the display screen 31, which is outside the central octagonal region 32 where the medical images are displayed; [0053]).
Claim 4, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus of Claim 1. Shelton further discloses that the one or more processors are configured to, in accordance with a user’s operation, stop the control to superimpose the information indicating the position of the region of interest before the measured time exceeds the time set by the user (user-actuated manual override and toggling of automatic display settings; [0339], [0340]).
Claim 5, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus of Claim 1. Hirokazu further discloses that the one or more processors are configured to, in accordance with a user’s operation, start the control to superimpose the information indicating the position of the region of interest detected in the recognition (user-initiated instruction and setting via the operation panel; [0021]).
Claim 6, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus of Claim 1. Hirokazu further discloses that the one or more processors are configured to display information indicating whether the control to superimpose the information indicating the position of the region of interest is being performed or not (displaying “Analyze ON” when active, and changing the display state to red or displaying “Not Display” when the superimposition is stopped; [0053], [0059], [0065]).
Claims 7 & 16, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus and method as set forth in Claims 1 and 15. Usuda further discloses that the region of interest includes at least one of a polyp, a cancer, an inflammation and angiodysplasia (detecting a lesion portion such as a tumor or inflammation; [0066]).
Claim 8, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus of Claim 1. Hirokazu further discloses that the one or more processors are configured not to superimpose the information indicating the position of the region of interest in a case where determining that an area of the detected region of interest is larger than an area set as a condition where the information indicating the position of the region of interest is not superimposed (determining not to display the mark when the detected size of the suspected region exceeds a preset mark non-display condition; [0037], [0038]).
Claim 9, rejected for the same reason as established in the last Office Action with respect to Hirokazu.
Claims 11, 12, & 18, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus and method as set forth in Claims 1 and 15. Usuda further discloses that the system is configured to perform the recognition of the region of interest in the medical images using an artificial intelligence (performing recognition using a deep learning machine learning algorithm; [0066]). Hirokazu further discloses that the system is configured to display, on the monitor, information relating to whether the recognition using the artificial intelligence is performed or not and display, on the monitor, information indicating that the recognition using the artificial intelligence is performed while the recognition of the region of interest in the medical images using the artificial intelligence is performed (displaying “Analyze ON” continuously while the deep learning recognition function is active; [0035], [0053]).
Claims 13 & 19, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus and method as set forth in Claims 1 and 15. Hirokazu further discloses that the system is configured to sequentially perform recognition of a region of interest in the medical images after automatically stopping the control to superimpose the information indicating the position of the region of interest detected in the recognition on the medical images sequentially displayed on the monitor (the image analysis unit 21 continues to operate and analyze images even when the display condition determination unit stops the superimposition of the mark; [0075]).
Claim 14, Hirokazu in view of Shelton and Usuda discloses the medical image processing apparatus of Claim 1. Hirokazu further discloses an endoscope system comprising: an endoscope; and the medical image processing apparatus (Fig. 1, endoscope system 1 comprising endoscope 2 and diagnosis support device 4; [0012]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Hirokazu (WO2019/012586A1) in view of Shelton (US2019/0104919A1) and Usuda (US2021/0153720A1), and further in view of Ren (TW201706890).
Claim 10, rejected for the same reason as established in the last Office Action with respect to Ren. The motivation to combine remains the same.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VU LE whose telephone number is (571)272-7332. The examiner can normally be reached M-F 8:00 - 17:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Vu Le can be reached at 2-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VU LE/ Supervisory Patent Examiner, Art Unit 2668
