DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/25/2026 has been entered.
Status of Claims
Claims 11-26 are pending in this application. Claims 14 and 20-26 remain withdrawn. Claims 11-13, 15-19, and 27-29 have been examined on the merits.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11-13, 15-19, and 27-29 are rejected under 35 U.S.C. 103 as being unpatentable over Fan (WO2013116694A1) in view of Gullotti (US20190209080A1).
Regarding Claim 11,
Fan teaches A surgical assistance system for use in a surgical intervention (corresponding disclosure in at least [0004], where the system is used for surgery and determining intraoperative displacement (surgical intervention) “using displacements of the surface features, determined surgical cavity volume and location, and presence and location of surgical tools to determine intraoperative displacements of the tumors or other inclusions and structures”), the surgical assistance system comprising:
at least one imaging 3D capturing device adapted to create a three-dimensional intracorporeal image for a patient and provide the three-dimensional intracorporeal image in a computer-readable manner (corresponding disclosure in at least [0004], where a 3D image is generated based on the intraoperative location “determining intraoperative locations of tumors or other inclusions and structures in mammalian tissues while performing surgery. In particular, this document relates to using a stereo 3-dimensional surface-mapping device”);
a tracking system adapted to detect and track at least one surgical intervention region of the patient (corresponding disclosure in at least [0047], where there are tracking sensors or the microscope (surgical intervention) position “The microscope location and orientation is tracked relative to the patient using tracking sensors 142, microscope location sensors 144 and patient tracking sensors 146”);
a data provision unit adapted to provide digital 3D image data of the patient (corresponding disclosure in at least [0037], where there is an image processing system for the 3D images (3D images were mentioned prior, alongside the three cameras 120 122 and 132, which would provide a 3-dimensional image) “Image processing system 126 also has a memory 178 into which image capture interface 124 saves images received from cameras 120, 122, 132”),;
and a control unit adapted to process the three-dimensional intracorporeal image, data of the tracking system and the digital 3D image data of the patient (corresponding disclosure in at least [0150], where there is a processor with a memory with instructions for the 3D images “a processor configured with machine readable instructions in the memory, the memory further configured with machine readable instructions for determining three-dimensional (3D) warping parameters for warping a first image of the warped stereo image into a second image of the stereo image”, and further in [0035], where with the processing system, there is a tracking interface “ a tracker interface 140 of the image processing system 126 is coupled to use tracking sensors”),
the surgical assistance system being adapted to register the digital 3D image data of the patient on an outer surface of the patient as a first registration (corresponding disclosure in at least [0045], where registration is complete with the skull (outer surface) and the pMR model (3D image data) “the patient is prepared for surgery, and patient tracking sensors 146 are attached to the patient's skull. The patient tracking sensors are registered to the patient-centered coordinate system of the pMR model”),
and based on the first registration and based on the intracorporeal reference, to register the digital 3D image data to the intracorporeal structure as a second registration to increase accuracy of registration (corresponding disclosure in at least [0047], where the pMR model (3D image data) is registered to the dural surface map (intracorporeal structure) “5) Transforming the extracted dural surface map to the patient-centered coordinate system of the pMR model by applying any necessary rotations and translations.
Once consent is obtained, the patient is prepared for surgery, and patient tracking sensors 146 are attached to the patient's skull. The patient tracking sensors are registered to the patient-centered coordinate system of the pMR model”),
and the control unit being adapted to perform the second registration, based on a rigid transformation (corresponding disclosure in at least [0047], where the second registration is based on transformation “5) Transforming the extracted dural surface map to the patient-centered coordinate system of the pMR model by applying any necessary rotations and translations”).
Fan does not teach indicating a position of the surgical instrument based on a rigid transformation, wherein the second registration, and indicating the position of the surgical instrument are performed without taking into account any elastic deformation of the digital 3D image data of the patient.
Gullotti, in a similar field of endeavor, teaches a similar concept (image registration) of indicating a position of the surgical instrument based on a rigid transformation, wherein the second registration, and indicating the position of the surgical instrument are performed without taking into account any elastic deformation of the digital 3D image data of the patient (corresponding disclosure in at least [0752], where the position of the surgical tool is determined, “the system utilizes the known distance of the 3D-tracked surgical tool away from the X-ray imaging system, the size and dimensions of the surgical tool, the location and orientation of the surgical tool, and the location and orientation of the imaging system”, further in [0652], where the position of the surgical instrument is indicated with the tool location being tracked “With a 3D-tracking camera having recorded the location of the emitter, and thereby the conical imaging volume, when an X-ray is taken, the acquisition system can determine when any component of the tracked surgical tool enters within the volume. When the surgical tool 4689 is positioned within the volume, its virtual projection can be overlaid on the previously-acquired x-ray image, as shown in FIG. 46D” and further in [0465], where the rigid transformation, which is used for indicating the position of the instrument is performed without taking into account deformation “The 3D rigid transform is utilized when the system needs to transform the relations of an object from one coordinate axes to another, without deformation of the object. For example, instead of having a 3D-tracked tool's location coordinates and orientation values to be in reference to a 3D-tracking, acquisition system, the 3D-tracked tool can be substantially rigidly transformed to be in reference to the coordinates and orientation of another 3D-tracked tool or DRF within the scene”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date to have incorporated indicating the position of the surgical instrument, where elastic deformation is not taken into account as taught by Gullotti. One of the ordinary skill in the art would have been motivated to incorporate this because it preserves the shape of the imaged objects, especially beneficial in imaging where there are rigid constraints.
Regarding Claim 12, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches the system according to claim 11 wherein the surgical instrument comprises: a surgical 3D microscope; a surgical 3D endoscope; or a medical instrument with an optical 3D camera as a 3D capturing device for creating the three-dimensional intracorporeal image (corresponding disclosure in at least [0037], where there are cameras for creating the 3D image “Image processing system 126 also has a memory 178 into which image capture interface 124 saves images received from cameras 120, 122, 132”, and further in [0091] where this relationship is further described “A 3D point in world space (X, Y, Z) is transformed into the camera image coordinates (x, y) using a perspective projection matrix… incorporate the perspective projection from camera to sensor coordinates and the transformation from sensor to image coordinates, (Cx, Cy) is the image center, and T is a rigid body transformation describing the geometrical relationship of the effective optical centers between the views of the two cameras, 120, 122”).
Regarding Claim 13, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches the system according to claim 11 and wherein the control unit is adapted to determine at least one point-like landmark and/or at least one surface for the second registration in the three-dimensional intracorporeal image (corresponding disclosure in at least [0047], where steps are outlined describing how the points are determined for the surface for registering the 3D intracorporeal image “The microscope location and orientation is tracked relative to the patient using tracking sensors 142, microscope location sensors 144 and patient tracking sensors 146… Stereo visual surface extraction (FIG. 4) is performed of the dural surface in the images to create a dural surface map… 4) Constructing 308 an extracted dural surface map from the point cloud of three-dimensional locations…. 5) Transforming the extracted dural surface map to the patient-centered coordinate system of the pMR model by applying any necessary rotations and translations”).
Regarding Claim 15, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches wherein the tracking system comprises at least one of: an infrared-based navigation system; an electromagnetic navigation system; or an image-processing navigation system (corresponding disclosure in at least [0035], where the tracking system has an image processing navigation system (processing system with the tracking system for location) “a tracker interface 140 of the image processing system 126 is coupled to use tracking sensors 142 attached to a reference location within an operating room to track relative locations of microscope location sensors 144 and patient location sensors 146”).
Regarding Claim 16, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches wherein the control unit is adapted to perform the first registration via a point-to-point matching and/or a surface matching using an outer surface of the patient (corresponding disclosure in at least [0058], where surface matching is use for the registration of the registration “While these features have similar shape and similar relative positions in both the pMR textured surface map and in the post-dural-opening surface map, their exact positions will differ because of post-dural-opening swelling and/or sagging of the brain. The detected global shift is applied to account for lateral movement of the brain, then a 2D nonrigid deformation field, representing local movement of the brain, is determined by using a block matching registration algorithm based on mutual-information to determine local shifts of each block”).
Regarding Claim 17, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches wherein the control unit is adapted to perform the first registration via the tracking system on an external surface of the patient (corresponding disclosure in at least [0047], where registration is completed via the tracking system “The microscope location and orientation is tracked relative to the patient using tracking sensors 142, microscope location sensors 144 and patient tracking sensors 146. A first pair of stereo images is then taken 208. Once taken, this first pair of stereo images is then processed using any features visible on the dural surface”).
Regarding Claim 18, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches wherein the control unit is adapted to perform the first registration via an external 3D stereo camera of the tracking system on a face of the patient (corresponding disclosure in at least [0090] and [0092], where registration is based on the camera and its tracking of the outer face as it determines the multiple parameters for registration “Note that we now have a camera model that projects a point in the world to its image coordinates, the next step is to determine (i.e., calibrate) several unknown parameters among the equations presented above. In particular, the extrinsic camera parameters to be calibrated are the rotation and translation matrices (R; T) and the intrinsic parameters are the focal length (f), lens distortion coefficient ·, scale factor (Sx), and image center (Cx; Cy)”).
Regarding Claim 19, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches wherein the control unit is adapted to perform the second registration a plurality of times intraoperatively (corresponding disclosure in at least [0055], where automatic image registration is completed for second registration, which would update the registration multiple times (plurality of times) “corresponding surface features, such as blood vessels and sulci, visible in the post-dural-opening surface map and post-dural-opening stereo images, and also visible in the pMR textured surface map are identified... this is performed by smoothing the images to retain larger shapes, then using a rigid mutual - information-based automatic image registration to detect global shift and gain error”).
Regarding Claim 27, the combined references of Fan and Gullotti recite the limitations of Claim 11, and further teaches wherein indicating the position of the surgical instrument based on the rigid transformation comprises displaying the position of the surgical instrument in the digital 3D image data (corresponding disclosure in at least [0037] of Fan) relative to a targeted anatomical structure (corresponding disclosure in at least [0750] of Gullotti, where the position of the surgical instrument is displayed relative to an anatomical structure “overlaying a surgical instrument using 3D-tracking dynamic reference markers to approximate the 2D, projected shape of the instrument on the 2D radiograph of an anatomical region of interest”).
Regarding Claim 28, the combined references of Fan and Gullotti recite the limitations of Claim 19, and Fan further teaches wherein the control unit is adapted to perform the second registration continuously intraoperatively (corresponding disclosure in at least [0055], where automatic image registration is completed for second registration, which would update the registration continuously (automatic) “corresponding surface features, such as blood vessels and sulci, visible in the post-dural-opening surface map and post-dural-opening stereo images, and also visible in the pMR textured surface map are identified... this is performed by smoothing the images to retain larger shapes, then using a rigid mutual - information-based automatic image registration to detect global shift and gain error”).
Regarding Claim 29, the combined references of Fan and Gullotti recite the limitations of Claim 11, and Fan further teaches wherein the control unit is adapted to indicate the position of the surgical instrument, based on the rigid transformation, relative to the digital 3D image data, and wherein the digital 3D image data comprises pre-operative image data (corresponding disclosure in at least [0050], where using rigid transformation, the instrument position is tracked, relative to the 3D image data (the transformation uses the pre-operative image data, keeping the position relative to it) “The registration result may be represented by means of a rigid transformation matrix, which may allow tracked instruments in the tracking device's coordinate system to be transformed to the patient's coordinate system. Using the result from the pre-operative calibration and co-registration, projector 14 may be tracked in the patient's coordinate system and co-registered in the image space”).
Response to Arguments
Applicant's arguments filed 04/25/2026 regarding the 35 U.S.C. 112b rejection have been fully considered and the rejection is withdrawn in light of the amendments.
Applicant’s arguments, see pg. 7, filed 04/25/2026, with respect to the rejection(s) of claim 11 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Gullotti.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Such prior arts include McLeod (US20220296303A1), in regards to image registration and Saget (US20190122330A1) in regards to interoperative surgical guidance and image registration.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN KIM whose telephone number is (571)272-1821. The examiner can normally be reached Monday-Friday 6-2 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne Kozak can be reached at (571) 270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.E.K./Examiner, Art Unit 3797
/ANNE M KOZAK/Supervisory Patent Examiner, Art Unit 3797
Prosecution Insights