DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Group I, claims 1-16 and Species A from Group 1, Species A from Group 2, Species A from Group 3 and Species A from Group 4, readable on claims 1-16 in the reply filed on 03/18/2026 is acknowledged.
Claims 17-18 are directed towards non-elected Group II and therefore are withdrawn from consideration as being directed toward a non-elected invention.
Examiner’s Comments
The present rejection(s) reference specific passages from cited prior art. However, Applicant is advised that the rejections are based on the entirety of each cited prior art. That is, each cited prior art reference “must be considered in its entirety”. Therefore, Applicant is advised to review all portions of the cited prior art if traversing a rejection based on the cited prior art.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 5-11, 13-16 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Pandya et al. (US 20130331644 A1, hereinafter Pandya).
Regarding Claim 1, Pandya discloses
An endoscope system (surgical system 400, FIG. 4) comprising:
an endoscope (endoscopic camera 428/ camera 2210, FIG. 22) to be inserted into a subject (patient site 426, FIG. 4) to acquire an image of an inside of the subject (par. 71 discloses camera images the patient site);
a robot arm (AESOP robot 434/ 610, FIG. 4/6) configured to change a position and a posture of the endoscope (par. 192 discloses AESOP robot holds laparoscopic camera and adjusts position and posture of camera); and
a controller (motion controller 422 + 432, FIG. 4) comprising at least one processor (control software 416 + control software 430, FIG. 4),
wherein the controller is configured to:
acquire treatment instrument information concerning a position or a movement of a treatment instrument (tool 2220, FIG. 22) to be inserted into the subject (par. 53 discloses tools inserted into patient’s body; par. 169, 182 disclose position of treatment instrument acquired by program software),
determine whether the treatment instrument has been removed based on the treatment instrument information (par. 186 discloses program software determines tool is out of view, i.e. removed, based on acquired calculations),
in response to determining that the treatment instrument has been removed, execute an overlooking mode (par. 188, 192 disclose zoom out mode activated by program software when tool is out of view and/or at edge of view), and
in the overlooking mode, control at least one of the endoscope or the robot arm to thereby automatically zoom out the image while maintaining, in the image, a specific point in the subject (par. 192 discloses autonomous zoom control allowing Aesop robot to adjust the zoom level of the camera, FIGS. 24a-c disclose a specific point in the image is maintained).
Regarding Claim 2, Pandya discloses
The endoscope system according to claim 1, wherein the controller is configured to start and end the overlooking mode based on the treatment instrument information (par. 192 discloses program software allows zoom out or zoom in depending on position of the tool, i.e. whether or not tool is in view).
Regarding Claim 3, Pandya discloses
The endoscope system according to claim 1, wherein the controller is configured to control the robot arm and move the endoscope in a direction away from the specific point to thereby zoom out the image (par. 192 discloses autonomous zoom control allowing Aesop robot to adjust the zoom level of the camera by moving the camera up or down, i.e. away or closer).
Regarding Claim 5, Pandya discloses
The endoscope system according to claim 1, wherein the controller is configured to detect the treatment instrument information from the image acquired by the endoscope (par. 75 discloses position of the tools may be detected using the images from the laparoscopic camera).
Regarding Claim 6, Pandya discloses
The endoscope system according to claim 1, further comprising a sensor configured to detect the position of the treatment instrument (par. 75 discloses position sensors to detect the position of the tool).
Regarding Claim 7, Pandya discloses
The endoscope system according to claim 1,
wherein the controller is configured to switch a manual mode to the overlooking mode and/or switch the overlooking mode to the manual mode (par. 192-193 disclose autonomous zoom mode/ control, par. 173, 183 disclose manual zoom mode which can be switched into, i.e. from autonomous zoom mode), and
in the manual mode, the controller is configured to permit operation of the endoscope by a user (par. 74 discloses camera operated by surgeon to change view, par. 148 discloses surgeon operates tools on the robots, i.e. endoscope, using master controller).
Regarding Claim 8, Pandya discloses
The endoscope system according to claim 1,
wherein the controller is configured to switch a following mode to the overlooking mode and/or switch the overlooking mode to the following mode (par. 194 discloses autonomous follow mode, par. 198 discloses system can combine both zoom mode and follow mode for joint operation where functions of each are utilized, i.e. switch between the two for maximum efficiency), and
in the following mode, the controller is configured to control the robot arm based on the position of the treatment instrument to thereby cause the endoscope to follow the treatment instrument (par. 194 discloses camera follows the tool in autonomous follow mode, par. 190 discloses Aesop robot causes camera to follow tool).
Regarding Claim 9, Pandya discloses
The endoscope system according to claim 1, wherein, in the overlooking mode,
the controller is configured to control the at least one of the endoscope or the robot arm to thereby move a visual field of the endoscope toward a distal end of a trocar through which the treatment instrument pierces (par. 118 discloses tool is placed through trocar point; par. 40 discloses controller can position camera, i.e. visual field, towards trocar point).
Regarding Claim 10, Pandya discloses
The endoscope system according to claim 9, wherein the controller is configured to end the movement of the visual field in response to the specific point having reached a peripheral edge region in the image (par. 188 discloses camera is stopped from moving once distance threshold between camera and trocar is reached, i.e. camera stops moving once tool/ trocar is at a certain point in image) or
in response to the distal end of the trocar through which the treatment instrument pierces having reached a center region in the image.
Regarding Claim 11, Pandya discloses
The endoscope system according to claim 1,
wherein the treatment instrument information includes any one of presence or absence of the treatment instrument in the image (par. 188 discloses camera zooms out to get tool back in view, i.e. absence of treatment instrument in image),
length of a disappearance time in which the treatment instrument is continuously absent in the image,
a position,
speed, and
a path of the treatment instrument, and
a combination thereof.
Regarding Claim 13, Pandya discloses
The endoscope system according to claim 1,
wherein the treatment instrument information includes presence or absence of the treatment instrument in the image (par. 188 discloses camera zooms out to get tool back in view, i.e. absence of treatment instrument in image), and
in response to the presence of the treatment instrument being detected in the image, the controller is configured to end the overlooking mode (par. 188, 192 disclose camera zooms in once tool is within view, i.e. presence of treatment instrument being detected activates zoom in mode and deactivate zoom out mode, i.e. overlooking mode).
Regarding Claim 14, Pandya discloses
The endoscope system according to claim 1, wherein the controller ends the zooming out of the image when an area of a trocar in the image has reached a predetermined area threshold (par. 188 discloses camera is programmed to stop zooming out if distance between camera and trocar is less than a predetermined threshold).
Regarding Claim 15, Pandya discloses
The endoscope system according to claim 1, wherein the controller is configured to end the zooming out of the image in response to a distance from the specific point to a distal end of the endoscope having reached a predetermined distance threshold (par. 192 discloses distance between camera and tool is checked, i.e. at specific point, and the camera is not zoomed out further once a threshold is reached).
Regarding Claim 16, Pandya discloses
The endoscope system according to claim 15,
wherein the predetermined distance threshold includes at least one of a first threshold determined based on a far point of a depth of field of the endoscope (par. 33-35 disclose threshold determined based on a viewing axis of the camera and controller may prevent the camera from being adjusted away from the tool, i.e. zoomed out, based on the threshold distance),
a second threshold determined based on resolution of the image, and
a third threshold determined based on accuracy of stereoscopic measurement of the endoscope.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pandya et al. (US 20130331644 A1, hereinafter Pandya) as applied to claim 1 above, and further in view of Gane et al. (US 20210113071 A1, hereinafter Gane).
Regarding Claim 4, Pandya discloses all of the elements of the current invention disclosed in claim 1, however, Pandya does not disclose wherein the controller is configured to control the endoscope to optically zoom out the image.
Gain teaches an analogous system (optical imaging system 500) which comprises a primary camera (535, FIG. 5, i.e. endoscope) and is capable of being attached to a positioning system (208, FIG. 2, i.e. robotic arm) which controls the position and orientation of the imaging system [0057-0058]. The imaging system (500) features a controller (530, FIG. 5) capable of controlling the operation of zoom optics (510, FIG. 5) found in an optical assembly (505, FIG. 5) which can be provided within housing of endoscope camera [0058, 0066].
It would have been obvious to one of ordinary skill in the art at the effective filing date of the invention to provide the camera of Pandya with the zoom lens of Gane in order to provide an improved lens to the endoscope camera capable of being optically adjusted and movable by the controller/ processor system [Gane - 0058-0060].
Additionally, the camera (2210) of Pandya features a lens [Pandya - 0188] which can be modified to be a zoom lens for the ability to independently zoom without the required movement of the robotic arms [Gane - 0058].
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pandya et al. (US 20130331644 A1, hereinafter Pandya) as applied to claim 11 above, and further in view of Larkin et al. (US 20080004603 A1, hereinafter Larkin).
Regarding Claim 12, Pandya discloses all of the elements of the current invention disclosed in claim 11, and Pandya further discloses
the controller is configured to start the overlooking mode in response to the treatment instrument information having exceeded a predetermined threshold (par. 192 discloses program software enables zoom out mode based on threshold in relation to position of treatment instrument).
However, Pandya does not disclose wherein the treatment instrument information is the length of the disappearance time, the speed of the treatment instrument and the path of the treatment instrument, and a combination thereof.
Larkin teaches an analogous endoscope system (100, FIG. 1) having an endoscope (140, FIG. 1) with a robotic arm (122, FIG. 1) controlling its position and posture [0030]. The system (100) has a processor (102, FIG. 1, i.e. controller) which controls the robotic arm and endoscope [0028], and the processor performs the function of indicating positions of a tool when the tool is outside a camera captured view [0033]. The processor is capable of providing symbols related to the position of the treatment tool [0039], and the symbol can be determined using an amount of time that a tool is out of view from the endoscope, i.e. length of disappearance time [0047].
It would have been obvious to one of ordinary skill in the art at the effective filing date of the invention to provide the endoscope system of Pandya with the controller of Larkin in order to provide a system capable of defining the trajectory of an out-of-view tool, while it remains out-of-view, based on the length of disappearance time of the tool [Larkin - 0047].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDUL HADI ABBASI whose telephone number is (571)272-4076. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDUL HADI ABBASI/Examiner, Art Unit 3795
/RYAN N HENDERSON/Primary Examiner, Art Unit 3795
Prosecution Insights