Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Allowable Subject Matter
Claims 11-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The claims recite “identify, based on the registration, surface vertices on a surface of the 3D virtual model that correspond to the surface points on the one or more anatomical objects, the material properties of the 3D virtual model affecting how the surface vertices are displaced in response to manipulation of the 3D virtual model apply, to the 3D virtual model, a virtual force that simulates the physical force and determine an additional set of displacement values representative of a displacement of one or more of the surface vertices of the 3D virtual model caused by the virtual force” The prior art does not teach these limitations in combination with the other limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 7-9, 16-17, 19, 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Toporek U.S. Patent/PG Publication 20210275275.
Regarding claim 1 (independent):
A system comprising: a memory storing instructions and a processor communicatively coupled to the memory and configured to execute the instructions to: (Toporek [0099] In one embodiment, robot controller 131, surface scanning controller 132 and display controller 137 each may include a processor, a memory, a user interface, a network interface, and a storage interconnected via one or more system buses. ).
identify surface points on one or more anatomical objects (Toporek [0046] In practice, surface scanning end-effector 43 is mounted onto scanning robot 41 whereby robot controller 42 controls scanning robot 41 in accordance with robot position commands 55 from surface scanning controller 50 to implement a robotic surface scanning 12 of force sensed surface scanning method 10 of FIG. 1A as will be further explained herein.)(Toporek [0053] Path planning phase 11 of method 10 further encompasses surface scanning controller 50 executing a scan path planning 51 involving a definition of a path along one or more segments or an entirety of a surface of preoperative image segmented volume model 15 of the anatomical organ as known in the art of the present disclosure.)
determine a set of displacement values representative of a displacement of one or more of the surface points of the one or more anatomical objects caused by a physical force being applied by a physical tool to the one or more anatomical objects (Toporek [0065] Surface sensing data 16 further includes force sensing data 46 informative of a contact force applied by the surface scanning end-effector 43 to the anatomical organ, and for imaging embodiments of surface scanning end-effector 43, surface sensing data 16 further includes scan image data 47 representative of a current image slice of the anatomical image.)(Toporek [0079] Referring to FIG. 3A, surface scanning end-effector 43 is shown deforming an anatomical organ prior to a scanning of the surface of the anatomical organ.).
and define, based on the set of displacement values and data representative of the physical force, (Toporek [0079] More particularly, surface scanning controller 50 controls a positioning of scanning end-effector 43 relative to the anatomical organ to initially apply a contact force unto the tissue of the anatomical organ resulting in an OFFSET.sub.I between undeformed anatomical tissue UAT and deformed anatomical tissue DAT.sub.I. The positioning of scanning end-effector 43 is adjusted until a sensed contact force SCF.sub.I per force sensing data FSD equals a desired contact force DCF whereby OFFSET.sub.I between undeformed anatomical tissue UAT and deformed anatomical tissue DAT.sub.I is deemed to equate the defined surface deformation offset u.sub.SDO of the anatomical organ as previously described herein.)
one or more material property values for a three-dimensional (3D) virtual model of the one or more anatomical objects, (Toporek [0016] The model constructor (134) is employed for constructing the intraoperative volume model of the anatomical organ responsive to the force sensing data generated by the surface scanning end-effector indicating a defined surface deformation offset of the anatomical organ.)
the one or more material property values representative of a material property of the one or more anatomical objects (Toporek [0069] In one embodiment of model construction 52, an operator of surface scanning controller 50 via input devices and/or graphical interfaces provides or selects a visocleastic property parameter k as a constant value representative viscoelastic properties of the subject anatomical organ, and further provides or selects a scanning force parameter f.sub.DC at which the surface of the anatomical organ will be scanned (e.g., a contact force in meganewtons). A surface deformation offset u.sub.SDO is calculated from the provided/selected visocleastic property parameter k and scanning force parameter f.sub.DC to support the construction of the inoperative volume model 17 of the anatomical organ.).
Toporek discloses the above elements in several embodiments. With the embodiments being disclosed in a single reference, one of ordinary skill in the art at the time of the filing of the invention being aware of one embodiment would also have been aware of the others, and it would have been obvious to one of ordinary skill in the art at the time of the filing of the invention to have combined these elements from two or more embodiments into a single arrangement for the benefit of enjoying the advantages of all the embodiments disclosed combined into a single arrangement.
Regarding claim 2:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the one or more material property values representative of the material property of the one or more anatomical objects define how the 3D virtual model deforms in response to manipulation of the 3D virtual model (Toporek [0068] From the assumption, model construction 52 involves a designation of a defined scanning force parameter f.sub.DC and of a defined visocleastic property parameter k whereby a surface deformation offset u.sub.SDO may be calculated to support the construction of the inoperative volume model 17 of the anatomical organ as will be further explained herein.).
Regarding claim 3:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the determining the set of displacement values comprises:
generating a point cloud having a plurality of nodes representative of the surface points on the one or more anatomical objects (Toporek [0072] Each recorded positon of the calibrated scanned reference of scanning surface end-effector 43 is deemed a digitized model point suitable for a generation of a sparse point cloud representation of the anatomical organ on the assumption of a uniform deformation offset of each recorded position of a digitized model point.).
and tracking movement of the plurality of nodes over time while the physical force is applied to the one or more anatomical objects (Toporek [0073] In practice, as will be further explained herein, a line sampling scan path generates a sparse point cloud representation of the anatomical organ in view of a subset of positons of the calibrated scanned reference of scanning surface end-effector 43 corresponding to a contact force applied by surface scanning end-effector 43 to the anatomical organ equaling scanning force parameter f.sub.DC and further in view a subset of positons of the calibrated scanned reference of scanning surface end-effector 43 failing to correspond to a contact force applied by surface scanning end-effector 43 to the anatomical organ equaling scanning force parameter f.sub.DC.).
Regarding claim 4:
The system of claim 3, has all of its limitations taught by Toporek. Toporek further teaches wherein the tracking movement of the plurality of nodes comprises determining a depth of the plurality of nodes relative to an initial position of the plurality of nodes prior to application of the physical force (Toporek [0085] In mesh embodiments of intraoperative volume model 17, surface scanning controller 50 may execute a point-by-point registration technique for registering preoperative segmented volume model 15 and intraoperative volume model 17. Examples of such a point-by-point registration technique include, but are not limited to, a rigid or non-rigid Iterative Closer Point (ICP) registration, a rigid or non-rigid Robust Point Matching (RPM) registration and a particle filter based registrations.) since it is registering two points, it is determining a depth.
Regarding claim 7:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the data representative of the physical force comprises a measure of the physical force based on kinematic data representative of movement of the physical tool over time, the kinematic data generated by a computer-assisted medical system communicatively coupled with the physical tool (Toporek [0042] In one exemplary embodiment, a scanning robot 41 is a snake scanning robot equipped with a rotary encoder embedded in each joint of the snake scanning robot for tracking a pose of the snake scanning robot as known in the art of the present disclosure, and further equipped with a force sensor, a pressure sensor, or an optical fiber for sensing a contact force between an end-effector of the snake scanning robot and an anatomical organ as known in the art of the present disclosure.)(Toporek [0069] In one embodiment of model construction 52, an operator of surface scanning controller 50 via input devices and/or graphical interfaces provides or selects a visocleastic property parameter k as a constant value representative viscoelastic properties of the subject anatomical organ, and further provides or selects a scanning force parameter f.sub.DC at which the surface of the anatomical organ will be scanned (e.g., a contact force in meganewtons). A surface deformation offset u.sub.SDO is calculated from the provided/selected visocleastic property parameter k and scanning force parameter f.sub.DC to support the construction of the inoperative volume model 17 of the anatomical organ.).
Regarding claim 8:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the data representative of the physical force comprises a measure of the physical force based on force data received by a force sensor communicatively coupled with the physical tool (Toporek [0042] In one exemplary embodiment, a scanning robot 41 is a snake scanning robot equipped with a rotary encoder embedded in each joint of the snake scanning robot for tracking a pose of the snake scanning robot as known in the art of the present disclosure, and further equipped with a force sensor, a pressure sensor, or an optical fiber for sensing a contact force between an end-effector of the snake scanning robot and an anatomical organ as known in the art of the present disclosure.)(Toporek [0069] In one embodiment of model construction 52, an operator of surface scanning controller 50 via input devices and/or graphical interfaces provides or selects a visocleastic property parameter k as a constant value representative viscoelastic properties of the subject anatomical organ, and further provides or selects a scanning force parameter f.sub.DC at which the surface of the anatomical organ will be scanned (e.g., a contact force in meganewtons). A surface deformation offset u.sub.SDO is calculated from the provided/selected visocleastic property parameter k and scanning force parameter f.sub.DC to support the construction of the inoperative volume model 17 of the anatomical organ.).
Regarding claim 9:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the data representative of the physical force comprises one or more physical properties of the physical tool (Toporek [0042] In one exemplary embodiment, a scanning robot 41 is a snake scanning robot equipped with a rotary encoder embedded in each joint of the snake scanning robot for tracking a pose of the snake scanning robot as known in the art of the present disclosure, and further equipped with a force sensor, a pressure sensor, or an optical fiber for sensing a contact force between an end-effector of the snake scanning robot and an anatomical organ as known in the art of the present disclosure.)(Toporek [0069] In one embodiment of model construction 52, an operator of surface scanning controller 50 via input devices and/or graphical interfaces provides or selects a visocleastic property parameter k as a constant value representative viscoelastic properties of the subject anatomical organ, and further provides or selects a scanning force parameter f.sub.DC at which the surface of the anatomical organ will be scanned (e.g., a contact force in meganewtons). A surface deformation offset u.sub.SDO is calculated from the provided/selected visocleastic property parameter k and scanning force parameter f.sub.DC to support the construction of the inoperative volume model 17 of the anatomical organ.).
Regarding claim 16:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the processor is further configured to execute the instructions to generate the 3D virtual model based on one or more preoperative images of the one or more anatomical objects (Toporek [0039] Volume imaging modality 31 is an imaging modality for generating a preoperative volume image of an anatomical region as known in the art of the present disclosure (e.g., a computed tomography imaging, a magnetic resonance imaging, an ultrasound imaging modality, a positron emission tomography imaging, and a single photo emission computed tomography imaging of a thoracic region, a cranial region, an abdominal region or a pelvic region).).
Regarding claim 17:
The system of claim 16, has all of its limitations taught by Toporek. Toporek further teaches wherein the generating the 3D virtual model comprises deriving a volumetric mesh model (Toporek [0077] For imaging embodiments of scanning surface end-effector 43, robotic surface scanning phase 12 of method 10 further encompasses surface scanning controller 50 stitching images associated with each point of the mesh, unsized or sized to thereby render intraoperative volume model 17 as an image of the anatomical organ. In practice, while stitching images associated with each point of the mesh, surface scanning controller 50 may interpolate images missing from the mesh due to unrecorded positions of the calibrated scanned reference of scanning surface end-effector 43.) based on the one or more preoperative images of the one or more anatomical objects (Toporek [0039] Volume imaging modality 31 is an imaging modality for generating a preoperative volume image of an anatomical region as known in the art of the present disclosure (e.g., a computed tomography imaging, a magnetic resonance imaging, an ultrasound imaging modality, a positron emission tomography imaging, and a single photo emission computed tomography imaging of a thoracic region, a cranial region, an abdominal region or a pelvic region).).
Regarding claim 19 (independent):
The claim is a parallel version of claim 1. As such it is rejected under the same teachings.
Regarding claim 30 (independent):
The claim is a parallel version of claim 1. As such it is rejected under the same teachings.
Claim(s) 5-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Toporek U.S. Patent/PG Publication 20210275275 in view of Fan U.S. Patent/PG Publication 20140369584.
Regarding claim 5:
The system of claim 4, has all of its limitations taught by Toporek. Toporek further teaches wherein the identifying surface points on the one or more anatomical objects is based on (Toporek [0072] Each recorded positon of the calibrated scanned reference of scanning surface end-effector 43 is deemed a digitized model point suitable for a generation of a sparse point cloud representation of the anatomical organ on the assumption of a uniform deformation offset of each recorded position of a digitized model point.).
Toporek does not teach use stereoscopic images for depth. In a related field of endeavor, Fan teaches:
wherein the identifying surface points on the one or more anatomical objects is based on stereoscopic images of the one or more anatomical objects that are captured by an imaging device, and wherein determining the depth of the plurality of nodes comprises processing the stereoscopic images (Fan A first pair of stereo images is then taken 208. Once taken, this first pair of stereo images is then processed using any features visible on the dural surface as follows: [0048] a) Stereo visual surface extraction (FIG. 4) is performed of the dural surface in the images to create a dural surface map by [0049] 1) Warping 302 the images to equivalent images as if taken at the reference settings; [0050] 2) Identifying 304 corresponding features in both warped images; [0051] 3) Tracing rays from the corresponding features to determine 306 three-dimensional locations of those features, the 3-dimensional locations forming a point cloud; [0052] 4) Constructing 308 an extracted dural surface map from the point cloud of three-dimensional locations. [0053] 5) Transforming the extracted dural surface map to the patient-centered coordinate system of the pMR model by applying any necessary rotations and translations. [0054] b) Correlating the pre-opening surface map with dural surface as observed in the pMR model and scans. [0055] c) A pMR-textured surface map is constructed 210 from the extracted dural surface map, annotated with positions of brain surface features, such as blood vessels and sulci that are visible in the pMR model and images. )(Fan [0082] A 2D deformation field that maps the pMR textured surface map into the post-dural-opening surface map is then determined 216 by the processor from the global shifts and local shifts of the identified corresponding surface features. Since three-dimensional locations of each feature are known both in the pMR textured surface map and in the post-dural-opening surface map, a three dimensional deformation field of the brain surface can be determined by finding the 3D locations of the starting and ending points of each vector in the 2D displacement field.).
Therefore, it would have been obvious before the effective filing date of the claimed invention to use stereoscopic images as taught by Fan. The rationale for doing so would have been that it is a simple substitution of one known element for another, where Toporek is determining points and depth, and Fan is determining points and depths using stereoscopic images, where the end result for both is the same data for accurately mapping a deforming organ to create a model. Therefore it would have been obvious to combine Fan with Toporek to obtain the invention.
Regarding claim 6:
The system of claim 4, has all of its limitations taught by Toporek. Fan further teaches wherein the determining the depth of the plurality of nodes comprises generating, based on depth measurements obtained by a depth sensor, a depth map of the plurality of nodes (Fan [0096] The spatial mapping of the ISM to the plane, and the PIR model to the plane is then inverted and subtracted to determine 726 a three-dimensional displacement vector of difference between the ISM and PIR models. The 3-D displacement vector in turn is used to derive 728 a 3D difference model (delta-M), a volume of the difference model being an initial estimate of the volume of resected tissue. In an alternative embodiment, the spatial mappings of the ISM to the plane and PIR to the plane are determined at planar grid points, inverted, and subtracted to determine 728 the three-dimensional difference model. ) where the displacement vectors create a map of depth.
Therefore, it would have been obvious before the effective filing date of the claimed invention to use depth map information as taught by Fan. The rationale for doing so would have been that it is a simple substitution of one known element for another, where Toporek is determining points and depth, and Fan is determining points and depths, where the end result for both is the same data for accurately mapping a deforming organ to create a model simply using alternative forms of obtaining depth information. Therefore it would have been obvious to combine Fan with Toporek to obtain the invention.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Toporek U.S. Patent/PG Publication 20210275275 in view of Pheiffer U.S. Patent/PG Publication 20170084036.
Regarding claim 10:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the processor is further configured to execute the instructions to perform a registration of the 3D virtual model with a 3D model derived from one or more (Toporek [0085] In mesh embodiments of intraoperative volume model 17, surface scanning controller 50 may execute a point-by-point registration technique for registering preoperative segmented volume model 15 and intraoperative volume model 17.)(Toporek [0066] Surface scanning controller 50 processing robot position data 45, force sensing data 46 and scan image data 47 (if applicable) to construct an inoperative volume model 17 of the anatomical organ based on a physical behavior of a soft tissue of an anatomical organ under a minor deformation by scanning surface end-effector 42 (e.g., a tissue deformation in nanometers).).
Toporek does not teach creating 3d models from video. In a related field of endeavor, Pheiffer teaches:
wherein the processor is further configured to execute the instructions to perform a registration of the 3D virtual model with a 3D model derived from one or more video images depicting the one or more anatomical objects (Pheiffer [0048] In act 26, the processor registers coordinates systems of the medical scan data from the medical scanner with the intraoperative camera using the identified features. The salient features are used to register. Rather than using a tracking sensor external to the patient, the features are used to align the coordinate systems or transform one coordinate system to the other. In alternative embodiments, an external tracking sensor is also used. [0049] Correspondence between salient anatomical features in each image modality guides the registration process. For example, the three-dimensional distribution from the camera is registered with the preoperative volume using the salient features of the three-dimensional distribution and the salient features of the preoperative volume. The 3D point cloud reconstructed from the intraoperative video data is registered to the preoperative image volume using the salient features. The feature correspondences in the two sets of data are used to calculate registration between video and medical imaging.).
Therefore, it would have been obvious before the effective filing date of the claimed invention to use video as taught by Pheiffer. The rationale for doing so would have been that it is a simple substitution of one known element for another to obtain predictable results, where Toporek uses an image and Lee uses an image or video (Pheiffer [0004] Other strategies rely only on the camera information in order to perform the registration. A patient-specific 3D model of the organ of interest is created by stitching together sequences of 2D or 2.5D images or video from the camera.) where there are predictable results since 3D models are still being generated. Therefore it would have been obvious to combine Pheiffer with Toporek to obtain the invention.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Toporek U.S. Patent/PG Publication 20210275275 in view of Beudeker U.S. Patent/PG Publication 20180301063.
Regarding claim 18:
The system of claim 1, has all of its limitations taught by Toporek. Toporek further teaches wherein the processor is further configured to execute the instructions to use the 3D virtual model during a simulation session (Toporek [0139] Referring to FIGS. 1-7, those having ordinary skill in the art will appreciate numerous benefits of the present disclosure including, but not limited to, an improvement over surface scanning systems, devices, controllers and methods by the inventions of the present disclosure providing a construction of an intraoperative scanned volume model of an anatomical organ based upon a sensing of a contact force applied by an surface scanning end-effector of a scanning robot to the anatomical organ whereby the contact force is indicative of a defined surface deformation offset of the anatomical organ, thereby enhancing a registration of the intraoperative surface scanned volume model of the anatomical organ with a preoperative image segmented volume model of the anatomical organ.).
Toporek does not teach a simulation session. In a related field of endeavor, Beudeker teaches:
further configured to execute the instructions to use the 3D virtual model during a simulation session in which a user interacts with the 3D virtual model to simulate physical interaction with the one or more anatomical objects
Therefore, it would have been obvious before the effective filing date of the claimed invention to have a simulation session as taught by Beudeker. The motivation for doing so would have been that deformable simulations provide realism for training, where Toporek already has a deformable model. Further, the rationale for doing so would have been that it combines prior art elements according to known methods to yield predictable results since Toporek has a deformable 3D model of an organ, and Beudeker is creating a simulation using a deformable 3D model of an organ, where there are predictable results since they are using the same generated information, merely for a different purpose once created. Therefore it would have been obvious to combine Beudeker with Toporek to obtain the invention.
Conclusion
For the prior art referenced and the prior art considered pertinent to Applicant’s disclosure but not relied upon, see PTO-892 “Notice of References Cited”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON PRINGLE-PARKER whose telephone number is (571) 272-5690 and e-mail is jason.pringle-parker@uspto.gov. The examiner can normally be reached on 8:30am-5:00pm est Monday-Friday. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, King Poon can be reached on (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, seehttp://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASON A PRINGLE-PARKER/
Primary Examiner, Art Unit 2617