DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/26/2024 is being considered by the examiner.
Specification
35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, requires the specification to be written in “full, clear, concise, and exact terms.” The specification is replete with terms which are not clear, concise and exact. The specification should be revised carefully in order to comply with 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112. Examples of some unclear, inexact or verbose terms used in the specification are: Terms that are cited in Drawings and in the Detailed Description a couple of times, are not cited throughout the rest of paragraphs in the Detailed Description. In paragraph 0013, it discloses and labels “Information processing device” with reference number 101. Then, further down in paragraph 0022 and on, it mentions “Information processing device” multiple times, with either no proper label or a different label, 1 or 501. Then, in paragraph 0084, it discloses Fig. 12 which mentions an “Information processing device” with no clear label. As well as many other paragraphs without a proper label for “Information processing device” when mentioned. This goes for many different terms throughout the detailed disclosure that do not have a proper label reference, but is referenced from the drawings. Furthermore, inconsistent label numberings for terms that were previously labeled. For example, “Display Surface” is referenced by different numbers, 12, 92. This goes for many different terms throughout the detailed disclosure that have inconsistent reference numbering. Further revision is required, additional typos should be fixed accordingly.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by LEE (No. US-20110306986-A1 “Lee”).
Regarding claim 1, Lee teaches, “A surgical robot system comprising:” (surgical robot system; 0013);
“a spatial reproduction display capable of displaying a surgical video visually recognized as a stereoscopic image in a stereoscopic space formed by a plane through which a lower side of a display surface of a monitor and a normal surface of an installation surface pass and a plane through which an upper side of the display surface and a parallel surface of the installation surface pass;” (the simulation mode, the master robot 1 may serve as a surgery simulator that uses the characteristics (e.g. shape, texture, tactile feel during incision, etc.) of an organ shaped in 3 dimensions by 3-dimensional modeling; 0191, Figs. 2 & 5);
(which the characteristics of an organ can be coupled with a 3-dimensional shape obtained by using a stereo endoscope, etc; 0191);
(If the liver is outputted through the screen display unit 320, a stereo endoscope can be used to identify the shape of the liver; 0192);
“an imaging device that captures the surgical video;” (a magnetic resonance imaging (MRI) picture; 0047);
“an operating device connected to the spatial reproduction display; and” (One or more slave robots 2 can be used to perform surgery on a patient, and the laparoscope 5 for displaying the surgical site on the monitor unit 6 as an on-screen image can be implemented on an independent slave robot; 0173);
“a surgical robot that operates by an operation of an operator via the operating device.” (a master robot 1, by which the operator remotely controls the slave robot; 0164);
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-5 are rejected under 35 U.S.C. 103 as being unpatentable over LEE in view of SHEKHAR (No. US-20140303491-A1 “Shekhar”).
Regarding claim 2, while Lee fails to teach all of claim 2, Shekhar teaches “The surgical robot system according to claim 1, wherein
the spatial reproduction display displays a composite image obtained by combining the surgical video, reference information indicating preoperative information or intraoperative information, and a base unit that increases stereoscopic effect of the surgical video.” (creating composite images by augmenting the endoscopic video with 2D or 3D tomographic images of the surgical anatomy; 0017);
(An existing feature-based registration method is then applied to register pre-operative tomographic images with intra-operative tomographic image volume; 0093);
(Fig. 9 Shows a Registration Module for intra-operative and pre-operative images.)
(An improved multimodality surgical visualization, wherein 3D ultrasound images are combined with laparoscopic video);
Shekhar teaches how to get composite images from a video of a surgical anatomy. Thus, a registration module is applied to the tomographic images for identifying intra-operative and pre-operative images. The images are also combined with laparoscopic video that relates to the increase of stereoscopic effects, to get and improved display of composite images.
Lee and Shekhar are analogous art as both of them are related to image processing for surgical robotic systems.
The motivation for the above is have a more efficient and accurate display of images in relation to surgery.
Therefore, it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Lee by combining the surgical video, reference information indicating preoperative information or intraoperative information, and a base unit that increases stereoscopic effect of the surgical video as taught by Shekhar.
Regarding claim 3, while Lee fails to teach “the spatial reproduction display displays a composite image obtained by combining the surgical video, reference information indicating preoperative information or intraoperative information, a base unit that increases stereoscopic effect of the surgical video,”
Shekhar teaches “the spatial reproduction display displays a composite image obtained by combining the surgical video, reference information indicating preoperative information or intraoperative information, and a base unit that increases stereoscopic effect of the surgical video.” (creating composite images by augmenting the endoscopic video with 2D or 3D tomographic images of the surgical anatomy; 0017);
(An existing feature-based registration method is then applied to register pre-operative tomographic images with intra-operative tomographic image volume; 0093);
(Fig. 9 Shows a Registration Module for intra-operative and pre-operative images.)
(An improved multimodality surgical visualization, wherein 3D ultrasound images are combined with laparoscopic video);
Claim 2 limitation is similar in scope and functions as this limitation, therefore, this claim limitation is also rejected under the same rationale as claim 2.
However, Lee does teach “and an interactive part that is interactively placeable, movable or rotatable on a basis of an operation of a user.”
(The operator's manipulation of the handles 10 can cause the robot arm 3 to perform a position movement, rotation, cutting operation, etc; 0165);
Lee discloses how the operator can manipulate and control the surgical robot with handles, to preform movements, etc. This relates to the interactive part that is placeable, movable, or rotatable by the operation of the user.
The motivation for the above is have a more efficient and accurate display of images in relation to surgery.
Regarding claim 4, while Lee fails to teach all of claim 2, Shekhar teaches “The surgical robot system according to claim 1, wherein
the spatial reproduction display displays a composite image arranged on a virtual space such that reference information indicating preoperative information or intraoperative information is displayed in front of the surgical video.”
(creating composite images by augmenting the endoscopic video with 2D or 3D tomographic images of the surgical anatomy; 0017);
(The availability of volumetric data may also ease the interpretation problem of conventional ultrasound images and allow presenting to operating surgeons 3D rendered views and reformatted cross-sectional views different from the acquisition planes; 0022);
(The pre-operative tomographic image volume is registered with intra-operative tomographic image volume using an intensity-based deformable image registration algorithm; 0092);
Shekhar discloses composite images that allow the surgeons to see rendered viewpoints of the surgical site and have available data that help understand the images. This relates to seeing reference information displayed on the front of the surgical video due to the different viewpoints.
The motivation for the above is an accurate display of information on the surgical video.
Therefore, it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Lee by arranging on a virtual space such that reference information indicating preoperative information or intraoperative information is displayed in front of the surgical video as taught by Shekhar.
Regarding claim 5, while Lee fails to teach all of claim 2, Shekhar teaches “The surgical robot system according to claim 1, wherein
the spatial reproduction display displays a composite image arranged on a virtual space such that reference information indicating preoperative information or intraoperative information is displayed on a back side of the surgical video.”
(creating composite images by augmenting the endoscopic video with 2D or 3D tomographic images of the surgical anatomy; 0017);
(The availability of volumetric data may also ease the interpretation problem of conventional ultrasound images and allow presenting to operating surgeons 3D rendered views and reformatted cross-sectional views different from the acquisition planes; 0022); ‘
(The pre-operative tomographic image volume is registered with intra-operative tomographic image volume using an intensity-based deformable image registration algorithm; 0092);
Shekhar discloses composite images that allow the surgeons to see rendered viewpoints of the surgical site and have available data that help understand the images. This relates to seeing reference information displayed on the back side of the surgical video due to the different viewpoints.
The motivation for the above is an accurate display of information on the surgical video.
Therefore, it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Lee by arranging on a virtual space such that reference information indicating preoperative information or intraoperative information is displayed on a back side of the surgical video as taught by Shekhar.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US-20200397531-A1 (Schrader) – Discloses an instrument holder for holding an observation instrument that is equipped with an image capturing unit for capturing an image section of an object plane. And a robotic handling unit that supports the instrument holder and a control device that comprises a handling control unit for controlling the robotic handling unit and an instrument control unit for controlling the observation instrument.
WO-2012060586-A2 (Choi) – Discloses a robot system and a laparoscope manipulation where the surgeon can control the position and the image-input angle of the laparoscope with just a gesture of intent to view a desired surgical site.
WO-2019210322-A1 (Ramirez) – Discloses a robotic arm, a stereoscopic camera, and a sensor positioned between the robotic arm and the stereoscopic camera. The sensor transmits output data that is indicative of translational and rotational force imparted on the stereoscopic camera by an operator.
WO-2020203139-A1 (Itotani) – Discloses a master-slave system that enables an observer to more intuitively observe a target object and controls an imaging unit that captures an image of the target object such that a second positional relationship, between the imaging unit and a second point corresponding to the first point in a slave-side three-dimensional coordinate system.
WO-2011125007-A1 (Ruiz) – Discloses a robotized surgery system comprises at least one robot arm which acts under the control of a control console intended for the surgeon. The console comprises an eye tracking system for detecting the direction of the surgeon's gaze and for entering commands depending on the directions of the gaze detected.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIGITER D PROTAZI whose telephone number is (571)272-7995. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A Broome can be reached at 5712722931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.D.P./Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612