DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-13 are is/are rejected under 35 U.S.C. 103 as being unpatentable over Mintz et al. (US 2017/0084027) in view of Zhao et al. (US 2009/0088634).
With respect to claims 1 and 11-12, Mintz et al. discloses an image processing device comprising: at least one processor ([0185]), wherein the processor is configured to; acquire a three-dimensional image of a subject (1410; [0172]); acquire a fluoroscopic image of the subject having a lumen structure into which an endoscope is inserted (1405; [0061]; [0171]); acquire a first real endoscopic image in the lumen structure of the subject captured at a first time point by the endoscope ([0177]; [0179]); derive a provisional virtual viewpoint in the three-dimensional image of the endoscope using the fluoroscopic image and the three-dimensional image (1445; [0179]); derive a virtual viewpoint at the first time point in the three-dimensional image of the endoscope using the provisional virtual viewpoint, the first real endoscopic image, and the three-dimensional image (1450; [0180]); and derive a virtual viewpoint at a second time point after the first time point in the three-dimensional image of the endoscope using the first real endoscopic image and a second real endoscopic image captured by the endoscope at the second time point ([0181-0182]). Mintz et al. does not teach determining an evaluation result representing a reliability degree. However, Zhao et al. teaches in the same field of endeavor determining reliability and accuracy of images (including sequential images) by comparing real images to virtual images ([0057]; [0100]; [0167]). The Examiner’s position is that comparing real images to real images or real images to virtual images is well within the skill level of one of ordinary skill in the art (MPEP 2144.04(VI)(Reversal, duplication, or rearrangement of parts)). Therefore, it would have been obvious to one of ordinary skill in the art to have provided Mintz et al. with reliability and accuracy determination as taught by Zhao et al. in order to provide accurate position and orientation information ([0004]; [0100]).
With respect to claim 2, Mintz et al. discloses wherein the processor is configured to: specify a position of the endoscope within the fluoroscopic image (1435; 1436; [00177]); derive a position of the provisional virtual viewpoint using the specified position of the endoscope ([0179]); and derive an orientation of the provisional virtual viewpoint using the position of the provisional viewpoint in the three-dimensional image (1447).
With respect to claim 3, Mintz et al. discloses wherein the processor is configured to adjust the virtual viewpoint at the first time point such that a first virtual endoscopic image In the virtual viewpoint at the first time point derived using the three-dimensional image matches the first real endoscopic image ([0177]; [0179]).
With respect to claim 4, Mintz et al. discloses wherein the processor is configured to: derive a change in viewpoint using the first real endoscopic image and the second real endoscopic image; and derive the virtual viewpoint at the second time point using the change in the viewpoint and the virtual viewpoint at the first time point (tracking; [0083]; [0127]; [0137]; [0161]).
With respect to claims 5-6, Mintz et al. discloses determining evaluation result representing a reliability degree and determination of the state of the endoscope ([0100]; [0112-0115]).
With respect to claim 7, Mintz et al. discloses wherein the processor is configured to sequentially acquire a real endoscopic image at a new time point by the endoscope and sequentially derive a virtual viewpoint of the endoscope at each time (real-time; [0177]).
With respect to claim 8, Mintz et al. discloses wherein the processor is configured to sequentially derive a virtual endoscopic image (1450) at each time point and sequentially display the real endoscopic image which is sequentially acquired and the virtual endoscopic image which is sequentially derived, using the three-dimensional image and the virtual viewpoint of the endoscope at each time point ([0180]).
With respect to claim 9, Mintz et al. discloses wherein the processor is configured to sequentially display the virtual endoscopic image at each time point and the real endoscopic image at each time point ([0177-0180]; Fig. 14B).
With respect to claim 11, Mintz et al. discloses wherein the processor is configured to sequentially display a position of the virtual viewpoint at each time point in the lumen structure in the three-dimensional image (1440; [0178]; Fig. 14B).
With respect to claim 12, Mintz et al. discloses computer-readable storage medium ([0185]).
With respect to claim 13, Mintz et al. discloses a plurality of tomographic images ([0171]; CT scan).
Response to Arguments
Applicant's arguments filed 5/28/20026 have been fully considered but they are not persuasive.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant argues the reference is directed to comparing the synthesized image with a real image to update the pose or kinematic information of the tool and does not disclose or suggest endoscopic images. However, the Examiner respectfully disagrees with the applicant. Zhao et al. is relied upon to teach determining reliability and accuracy of images (including sequential images) by comparing real images to virtual images. The Examiner’s position is that determining reliability by comparing one image to another can be applied to any technological field and is not limited to comparison of images with tools. The Examiner’s position is that comparing real images to real images or real images to virtual images is well within the skill level of one of ordinary skill in the art (MPEP 2144.04(VI)(Reversal, duplication, or rearrangement of parts)). Furthermore, both Mintz et al. and Zhao et al. disclose endoscopic images.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER LUONG whose telephone number is (571)270-1609. The examiner can normally be reached M-F 9-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan T Nguyen can be reached at (571)272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER LUONG/Primary Examiner, Art Unit 3797
Prosecution Insights