DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7, 9-13, 24, and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barral et al. (Pub. No.: US 2019/0110855 A1); hereinafter referred to as “Barral”; in view of Savall et al. (Pub. No.: US 2022/0087748 A1); hereinafter referred to as “Savall”.
Regarding claims 1 and 24, Barral discloses a system (e.g. see figure 1 element 100) for performing robotic surgery (e.g. see [0021]) at a surgical site on a patient, the system comprising: one or more displays (e.g. see figure 2 elements 215, 217); and a processor (e.g. see [0022]), configured to: cause a robotic arm (e.g. see figure 1 element 141, [0021]), which holds a surgical tool (e.g. see [0021], “each arm 141 holds an instrument at the distal ends of the arm”. Thus the “instrument” is the “surgical tool”), to manipulate the surgical tool (e.g. see [0021]); cause the displays to display a real-time video of the surgical site (e.g. see [0032]), in which the surgical site is magnified (e.g. see [0032]), to the operator while the operator controls the surgical tool; and cause the displays to display at least one three-dimensional (3D) image (e.g. see [0029], [0034]) of the surgical site, in which the surgical site is shown with less magnification relative to the video (e.g. see [0032], figure 2 elements 253, 263, 265. Note: In figure 2 the surgical site overview 263 shows the entire chest and abdomen while the video 265 shows the lung area in box 253. [0032] states “Preoperative image 263 includes an accentuated region 253 including a bounding box containing the lung shown in video 265. As the location of video feed 265 moves in the body of the patient, the accentuated region 265 will change location and size on preoperative image 263”).
Barral discloses the invention but is silent as to a control tool configured for manipulation by an operator to control the surgical tool. Savall teaches it is known to use such a modification as set forth in figure 6 elements 602-608, “user interface device” to provide accurate control of robotically-assisted tools to reach a favorable clinical outcome with robotic surgery (e.g. see [0004]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a control tool as taught by Savall in the system/method of Barral, since said modification would provide the predictable results of accurate control of robotically-assisted tools to reach a favorable clinical outcome with robotic surgery.
Regarding claim 2, Barral discloses a spatial relationship between the control tool and the surgical site as shown in the 3D image corresponds to a spatial relationship between the surgical tool and the surgical site (e.g. see figure 1C elements 141, 153, 163, 165).
Regarding claim 3, Barral discloses the surgical site is shown in the 3D image without any magnification (e.g. see [0029], [0032], [0034]. Note: [0032] states “As the location of video feed 265 moves in the body of the patient, the accentuated region 265 will change location and size on preoperative image 263 (e.g., the bounding box may move, or grow larger or smaller depending on how “zoomed in” video 265 is)”. Thus the bounding box 253 may be zoomed in, but is not necessarily zoomed in and thus is configured to meet the functional use limitation of the system claim “without any magnification”).
Regarding claim 4, Barral discloses a size of the surgical site as shown in the 3D image corresponds to an actual size of the surgical site (e.g. see [0029], [0032], [0034]).
Regarding claim 5, Barral discloses a size of the surgical site as shown in the 3D image is scaled, relative to an actual size of the surgical site, by an inverse of the scale factor (e.g. see [0029], [0032], [0034]. Note: [0032] states “As the location of video feed 265 moves in the body of the patient, the accentuated region 265 will change location and size on preoperative image 263 (e.g., the bounding box may move, or grow larger or smaller depending on how “zoomed in” video 265 is)”. Thus, the bounding box 253 is capable of being zoomed in/out and thus is configured to meet the functional use limitation of the system claim “scaled, relative to an actual size of the surgical site, by an inverse of the scale factor ”) but is silent as to the processor is configured to scale movement of the surgical tool, relative to movement of the control tool, by a scale factor. Savall teaches it is known to use such a modification as set forth in figure 6 elements 602-608, “user interface device” to provide accurate control of robotically-assisted tools to reach a favorable clinical outcome with robotic surgery (e.g. see [0004]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a processor being configured to scale movement of the surgical tool, relative to movement of the control tool, by a scale factor as taught by Savall in the system/method of Barral, since said modification would provide the predictable results of accurate control of robotically-assisted tools to reach a favorable clinical outcome with robotic surgery.
Regarding claim 7, Barral discloses the processor is further configured to mark a control zone in the 3D image (e.g. see figures 1C and 2, elements 153, 253) the control zone being a predefined volume in which the surgical tool is controllable by the operator (e.g. see figure 1C element 141).
Regarding claim 9, Barral discloses the 3D image shows the surgical tool (e.g. see figure 1C elements 141, 153, 163).
Regarding claim 10, Barral discloses a field of view (FOV) shown in the 3D image is larger than an FOV shown in the video (e.g. see figure 2 elements 263/265, 215/217, [0032]).
Regarding claim 11, Barral discloses the surgical site includes part of an eye of the patient, and wherein the 3D image shows a face of the patient (e.g. see figure 2 elements 263/265, 215/217. Note: The prior art is capable of and/or configured to perform this functional use recitation. Specifically, the larger 3D image 263 is configured to be a face and the smaller video 265 of the surgical site is configured to be an eye. These are system claims and so the prior art needs only to be “configured to” meet the claim limitations).
Regarding claim 12, Barral discloses the processor is further configured to mark, in the 3D image, the FOV shown in the video (e.g. see figures 1C and 2, elements 153, 253).
Regarding claim 13, Barral discloses the displays comprise a 3D display (e.g. see [0029], [0034]), and wherein the processor (e.g. see [0022]) is configured to cause the 3D display to display the 3D image (e.g. see [0029], [0034]).
Regarding claim 25, Barral discloses the device (e.g. see the rejection for claim 1 above) and further discloses the non-transitory computer readable medium (e.g. see [0021]-[0023], “memory”).
Claim(s) 6 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barral and Savall as applied to claim 1 above, and further in view of Komp et al. (Pub. No.: US 2021/0055537 A1); hereinafter referred to as “Komp”.
Regarding claim 6, Barral and Savall disclose the claimed invention except for the real-time video is 3D. Komp teaches it is known to use such a modification as set forth in [0047], [0049], figures 4 and 5 to provide videos with an additional dimension which provides more detail and data. It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a 3D video as taught by Komp in the system/method of Barral and Savall, since said modification would provide the predictable results of videos with an additional dimension which provides more detail and data.
Regarding claim 8, Barral and Savall disclose the claimed invention except for the at least one 3D image includes a real-time 3D video. Komp teaches it is known to use such a modification as set forth in [0047], [0049], figures 4 and 5 to provide videos with an additional dimension which provides more detail and data. It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a 3D video as taught by Komp in the system/method of Barral and Savall, since said modification would provide the predictable results of videos with an additional dimension which provides more detail and data.
Claim(s) 14-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barral and Savall as applied to claims 1 and 13 above, and further in view of Garcia Kilroy et al. (Pub. No.: US 2020/0138534 A1); hereinafter referred to as “Garcia Kilroy”.
Regarding claims 14-16, Barral and Savall disclose the claimed invention except for a sensor configured to communicate, to the processor, a signal indicating a gaze direction of the operator, wherein the processor is further configured to lock the control tool responsively to the signal indicating that the operator is looking toward the 3D display, the sensor comprises an imaging sensor, and the 3D display is configured to use the imaging sensor to track eyes of the operator as the operator looks toward the 3D display, and to display the 3D image responsively to tracking the eyes of the operator. Garcia Kilroy teaches it is known to use such a modification as set forth in [0050]-[0051] to increase patient safety by pausing or locking the surgical robot if the user is not observing the surgical site for some time (e.g. see [0051]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a locking mechanism dependent on the surgeon gaze as taught by Garcia Kilroy in the system/method of Barral and Savall, since said modification would provide the predictable results of increasing patient safety by pausing or locking the surgical robot if the user is not observing the surgical site for some time.
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barral, Savall, and Garcia Kilroy as applied to claims 1, 13, and 14 above, and further in view of Shelton, IV et al. (Pub. No.: US 2022/0104910 A1); hereinafter referred to as “Shelton”.
Regarding claim 17, Barral, Savall, and Garcia Kilroy disclose the claimed invention (see the rejection for claims 1, 13, and 14 above) but are silent as to the sensor comprises an orientation sensor, and wherein the system further comprises a device wearable over eyes of the operator and comprising: lenses configured for viewing the 3D image via the lenses; and the orientation sensor. Shelton teaches it is known to use such a modification as set forth in figure 72, element 36500, [0804]-[0806] to provide one or more sensors that may be used to generate geometric 3D data, which may be used to determine a line of sight for surgeon and/or the visual focus of surgeon (e.g. see [0804]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use the safety glasses as taught by Shelton in the system/method of Barral, Savall, and Garcia Kilroy, since said modification would provide the predictable results of one or more sensors that may be used to generate geometric 3D data, which may be used to determine a line of sight for surgeon and/or the visual focus of surgeon.
Claim(s) 18, 20, 21, and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barral and Savall as applied to claim 1 above, and further in view of Lee et al. (Pub. No.: US 2014/0275760 A1); hereinafter referred to as “Lee”.
Regarding claim 18, Barral and Savall disclose the 3D display (e.g. see [0029]-[0034] of Barral) but are silent as to the display is positioned underneath the control tool. Lee teaches it is known to use such a modification as set forth in figure 1 element 111, 113, 120 to provide an augmented reality image display system capable of instinctively observing the inside of a patient's body and a surgical robot system including the same (e.g. see [0010]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a display positioned underneath the control tool as taught by Lee in the system/method of Barral and Savall, since said modification would provide the predictable results of an augmented reality image display system capable of instinctively observing the inside of a patient's body and a surgical robot system including the same.
Regarding claims 20, 21, and 23, Barral and Savall disclose the claimed invention except for the displays comprise an extended-reality display, and wherein the processor is configured to cause the extended-reality display to display the 3D image and the extended-reality display is an augmented-reality display, a device wearable over eyes of the operator and comprising: the extended-reality display; and an orientation sensor configured to communicate, to the processor, a signal indicating an orientation of the device, wherein the processor is configured to cause the extended-reality display to display the 3D image in response to the signal indicating that the operator is looking toward a vicinity of the control tool, and wherein the processor is further configured to augment the control tool with one or more portions of the surgical tool in the 3D image. Lee teaches it is known to use such a modification as set forth in figure 3, figure 11, [0075]-[0077] to provide an augmented reality image display system capable of instinctively observing the inside of a patient's body and a surgical robot system including the same (e.g. see [0010]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a locking mechanism dependent on the surgeon gaze as taught by Lee in the system/method of Barral and Savall, since said modification would provide the predictable results of an augmented reality image display system capable of instinctively observing the inside of a patient's body and a surgical robot system including the same.
Claim(s) 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barral, Savall, and Lee as applied to claim 1 above, and further in view of Garcia Kilroy.
Regarding claim 22, Barral, Savall, and Lee disclose the claimed invention except for the processor is further configured to lock the control tool responsively to the signal indicating that the operator is looking toward the vicinity of the control tool. Garcia Kilroy teaches it is known to use such a modification as set forth in [0050]-[0051] to increase patient safety by pausing or locking the surgical robot if the user is not observing the surgical site for some time (e.g. see [0051]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a locking mechanism dependent on the surgeon gaze as taught by Garcia Kilroy in the system/method of Barral, Savall, and Lee, since said modification would provide the predictable results of increasing patient safety by pausing or locking the surgical robot if the user is not observing the surgical site for some time.
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barral and Savall as applied to claims 1 and 13 above, and further in view of Hondori et al. (Pub. No.: US 2020/0015598 A1); hereinafter referred to as “Hondori”.
Regarding claims 19, Barral and Savall disclose the 3D display (e.g. see [0029]-[0034] of Barral) but are silent as to the display is tilted such that a top of the display, which is positioned to be farther from the operator, is higher than a bottom of the display, which is positioned to be closer to the operator. Hondori teaches it is known to use such a modification as set forth in [0040], figure 3 element 322 to provide greater comfort which can lead to better surgical performance and surgical outcomes (e.g. see [0023]). It would have been obvious to one having ordinary skill in the art at the time the invention was filed to use a tilting display as taught by Hondori in the system/method of Barral and Savall, since said modification would provide the predictable results of greater comfort which can lead to better surgical performance and surgical outcomes.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Noonan et al. (Pub. No.: 2022/0361968 A1) is pertinent to claims 14-17 and 22 (e.g. see [0154]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHILIP C EDWARDS whose telephone number is (571)270-1804. The examiner can normally be reached Mon-Fri, 9:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Unsu Jung can be reached at 571-272-8506. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/P.C.E/Examiner, Art Unit 3792
/AMANDA L STEINBERG/Examiner, Art Unit 3792
Prosecution Insights