Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Qian et al. (NPL: “ARssist: augmented reality on a head-mounted display for the first assistant in robotic surgery”).
Regarding claim 1, Qian et al. discloses a method comprising:
receiving image data of a component of a surgical robotic system (Implementation of ARssist: We chose the da Vinci Research kit (dVRK) [28] as the robotic platform and Microsoft HoloLens as the HMD, section 4.1), wherein the image data is captured by an image sensor (visualisation results captured by the eye simulating cameras, figure 4) of a head-mounted display (HMD) (Microsoft HoloLens is a binocular OST-HMD featuring a holographic waveguide-based optical system, stable self-localisation capability, sufficient computational power for tracking, and good support from development tools, section 4.1.2);
determining spatial information (display pre operatively planned trajectories on a virtual model, and (iv) it increases the surgeon’s spatial awareness, section 2) or real-time system state information associated with the component of the surgical robotic system (uses the fiducial tracking, the kinematics data, the model of the robot, and the pivot calibration that determines the pose of the marker relative to a certain joint of the robot, section 3.1);
creating a three-dimensional (3D) virtual model of the component of the surgical robotic system based on 1) the image data (ARssist, an application based on the integration of a surgical robot and an OST-HMD. ARssist provides various AR information to the first assistant, including (i) three-dimensional (3D) real-time rendering of the endoscope, robotic instruments and hand-held instruments within the patient body, section 1) and 2) the spatial information or the real-time system state information associated with the component of the surgical robotic system (real-time stereo endoscopy that is configurable for the FA’s preferred hand–eye coordination, section 1); and
displaying the 3D virtual model of the component of the surgical robotic system on the HMD (A video see-through head mounted display has access to every pixel that the user sees and therefore is able to blend the overlaid graphics perfectly with the background, section 3.2.1).
Regarding claim 2, Qian et al. discloses the method of claim 1, wherein the component comprises an operating table (while the patient-side assistant stands or sits at the bedside assisting the operation (see Fig. 1), Introduction) or a robotic arm (an instrument for the main surgeon, he/she has to manually and blindly adjust the robotic arm in order for the instrument to appear in the operative field or have the console surgeon reposition the endoscope to visualise the instrument until it arrives at the desired location, Introduction) coupled to the operating table.
Regarding claim 3, Qian et al. discloses the method of claim 2, wherein the 3D virtual model of the component of the surgical robotic system provides real-time guidance on maneuvering the robotic arm manually or robotically (the FA is responsible for or takes an important role in trocar placement, docking of the robot, and preparing the operative field. During the surgery, the FA exchanges the instrument for the main surgeon manipulates certain laparoscopic instruments, e.g. gripper and vessel sealer, and extracts specimen, see Introduction; provides guidance to critical targets and structures, section 2).
Regarding claim 4, Qian et al. discloses the method of claim 2, wherein the 3D virtual model of the component of the surgical robotic system provides real-time rendering of a workspace of the robotic arm (system restores the hand–eye coordination for the main surgeon by providing an immersive endoscopic operative field and letting him/her intuitively control the robotic instruments, which appear registered with his/her hand motion [5]. However, this configuration and resulting benefits are not provided to the FA. For example, when the FA needs to install or exchange an instrument for the main surgeon, he/she has to manually and blindly adjust the robotic arm in order for the instrument to appear in the operative field or have the console surgeon reposition the endoscope to visualise the instrument until it arrives at the desired location, Introduction).
Regarding claim 5, Qian et al. discloses the method of claim 1, wherein the 3D virtual model of the component is displayed within an environment in which a user who is wearing the HMD is located, wherein a position and orientation of the 3D virtual model of the component in the environment is spatially-fixed at the position and orientation as the user moves about the environment (FA can ‘‘move’ the virtual monitor to an arbitrary place that he/she feels the most comfortable with. For example, the virtual monitor can be placed on top of the trocar, so that the FA can see both the endoscopy and the external condition of the patient without turning his/her head, section 3.3.2).
Regarding claim 6, Qian et al. discloses the method of claim 1 further comprising establishing a global coordinate frame between the HMD and the surgical robotic system based on the image data, wherein the 3D virtual model of the component is in a position and orientation in the global coordinate frame (In the setup of a da Vinci robot-assisted laparoscopic surgery, AR can be implemented on the video display of the surgeon console by overlaying virtual information on the real-time endoscopy, section 2).
Regarding claim 7, Qian et al. discloses the method of claim 6, wherein creating the 3D virtual model of the component comprises rendering the component of the surgical robotic system at the position and orientation in the global coordinate frame that matches an actual position and orientation of the component or a target position and orientation of the component (when the FA needs to install or exchange an instrument for the main surgeon, he/she has to manually and blindly adjust the robotic arm in order for the instrument to appear in the operative field or have the console surgeon reposition the endoscope to visualise the instrument until it arrives at the desired location, Introduction).
Claim 8, a head-mounted display (HMD) claim, is rejected for the same reason as claim 1.
Claim 9, a head-mounted display (HMD) claim, is rejected for the same reason as claim 2.
Claim 10, a head-mounted display (HMD) claim, is rejected for the same reason as claim 3.
Claim 11, a head-mounted display (HMD) claim, is rejected for the same reason as claim 4.
Claim 12, a head-mounted display (HMD) claim, is rejected for the same reason as claim 5.
Claim 13, a head-mounted display (HMD) claim, is rejected for the same reason as claim 6.
Claim 14, a head-mounted display (HMD) claim, is rejected for the same reason as claim 7.
Claim 15, a surgical robotic system claim, is rejected for the same reason as claim 1 and see section 4.1.1 for processing CPU.
Claim 16, a surgical robotic system claim, is rejected for the same reason as claim 2.
Regarding claim 17, Qian et al. discloses the surgical robotic system of claim 16, wherein the 3D virtual model of the operating table or the robotic arm comprises context-sensitive real-time information of the operating table or the robotic arm to aid a user of the surgical robotic system to maneuver or troubleshoot the operating table or the robotic arm (display relative calibration (DRC), section 3.2.1).
Claim 18, a surgical robotic system claim, is rejected for the same reason as claim 5.
Claim 19, a surgical robotic system claim, is rejected for the same reason as claim 6.
Claim 20, a surgical robotic system claim, is rejected for the same reason as claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS J LETT whose telephone number is (571)272-7464. The examiner can normally be reached Mon-Fri 9-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS J LETT/Primary Examiner, Art Unit 2611
Prosecution Insights