DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Independent claim 1 recites, for example, the following abstract idea: “…move an imaging device…”, “…determine a first distance…”, “…determine…first coordinates…”, “…move the imaging device to a second pose…”, “…determine a second distance…”,“…determine…second coordinates…” and “…determine…third distance…”, fall within mental process. There is/are no specific machine(s) or device that is not known or generic recited in the claim limitations, see MPEP § 2106.05(b). The limitations have no specifics to the algorithmic foundation or dimensionality associated with the imaging data utilized in the various steps and as such can be considered computations that can be performed in the mind using visual inspection or simple pen and paper. Further, limitations of “move the imaging device” can be considered virtual simulation based “movement” not a “physical” movement of an actual device and thus, can be considered a mental process performed in the mind as clinicians are known to prepare for a surgical procedure by mentally performing the surgical steps to be performed and where device(s)/tool(s) will be located and moved as the procedure progresses.
The judicial exceptions are not integrated into a “practical application” as defined by the Subject Matter Eligibility Analysis documented in Federal Register 84(4), issued on 07 January 2019, and MPEP § 2106. The limitation of “…controller communicatively coupled to the memory…” in claim 1, simply represents implementing the abstract ideas with a computer. MPEP § 2106.05(f) notes that “using a computer as a tool to perform the abstract idea” is not sufficient to integrate a judicial exception into a practical application as interpreted by the court(s). Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972) “held that simply implementing a mathematical principle on a physical machine, namely a computer, was not a patentable application of that principle and Intellectual Ventures LLC v. Symantec Corp., 838 F.3d 1307, 1318 (Fed. Cir. 2016) established that mental processes encompass acts which, absent anything beyond generic computer components, may be “performed by a human, mentally or with pen and paper.” Intellectual Ventures additionally established that if a claim, under its broadest reasonable interpretation, covers performance in the mind but for the recitation of generic computer components, then it is still in the mental processes category of abstract ideas unless the step(s) cannot be practically performed in the mind. Therefore, a positive recitation of the associated computer would not necessarily result in patent eligible subject matter.
Independent claims 19-20 recite similar limitations of claim 1 and are rejected under the same rationale.
The dependent claims 2-18 do not sufficiently link the subject matter to a practical application or recite element(s) which constitute significantly more than the abstract ideas identified. The depending claims are directed to additional limitations which encompass abstract ideas consistent with those identified above that are well-understood, routine and/or conventional activity. Further, dependent claims 2-18 merely include limitations that either further define the abstract idea (and thus don’t make the abstract idea any less abstract) or amount to no more than generally linking the use of the abstract idea to a particular technological environment or field of use because they’re merely incidental or token additions to the claims that do not alter or affect how the process steps are performed.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 9-17 and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticiapted by Panescu et. al. (U.S. 20170172662, June 22, 2017)(hereinafter, “Panescu”).
Regarding Claim 1, Panescu teaches: A system for measuring a distance between two points in a surgical space (see e.g. Figs. 8, 20 and 25, [0130]), the system comprising:
a memory (“…the controller 106, which includes a processor and a storage device…computes the physical quantitative 3D coordinates of the points in a scene adjacent the tip 208 of the elongate portion 202 and drives both the video processor 104 and a 3D display driver 109 to compose 3D scenes, which then can be displayed on a display 110…” [0073]); and
a controller communicatively coupled to the memory (“Images captured by the imager sensor array 210 are sent over a data bus 212 to a video processor 104, which communicates via bus 105 with a controller 106.” [0071]; “…the controller 106, which includes a processor and a storage device…computes the physical quantitative 3D coordinates of the points in a scene adjacent the tip 208 of the elongate portion 202 and drives both the video processor 104 and a 3D display driver 109 to compose 3D scenes, which then can be displayed on a display 110…” [0073]), the controller configured to:
move an imaging device to a first pose to capture first imaging data showing a first region of a surface of an anatomical structure (“…a series of geometric transformations are performed based upon information in the Q3D model of the scene 2508 to virtually “move” the Q3D endoscope 2502 so that it takes the 3D pose of instrument 2506.” [0136]);
determine a first distance between the imaging device in the first pose and a first point on the surface (“…FIG. 25, d.sub.1 is distance from Q3D endoscope tip 2512 to the closest point on target 2508-1.” [0135]);
determine, based on the first pose and the first distance, first coordinates of the first point on the surface (“Module 2602 configures the computer 151 (or controller 106; references to computer 151 apply to controller 106 as well in the following descriptions) from FIG. 8, to create a Q3D model 2650 of the scene 2508. As explained above with reference to FIGS. 9-19, the computer 151 is configured to determine the (x, y, z) coordinates of points determined in the endoscope FOV.sub.e. In doing so, the system creates a Q3D model 2650 of the scene 2508. The Q3D model 2650 is stored in a non-transitory computer-readable storage device 2652.” [0134]);
move the imaging device to a second pose to capture second imaging data showing a second region of the surface of the anatomical structure (“Module 2610 configures the computer system 151 to compute transformations that virtually “move” endoscope 2502 to the pose of the instrument 2506. This “move” includes a series of virtual rotations and virtual translations that virtually align the z-axis 2518 of the endoscope 2502 with the axis 2514 of the target instrument 2506.” [0140]);
determine a second distance between the imaging device in the second pose and a second point on the surface (“Similarly, module 2604 configures the computer 151 to determine a distance d.sub.2 between a tip 2516 of the target surgical instrument 2506 and the surgical scene 2508. Distance d.sub.2 represents the length from tip 2516 to the closest point of scene 2508. In the example provided in FIG. 25, d.sub.2 is distance from instrument tip 2516 to the closest point on target 2508-1.” [0135]);
determine, based on the second pose and the second distance, second coordinates for the second point on the surface (“Module 2606 configures the computer 151, or controller 106, to determine the (x, y, z) coordinates of at least two points located on a visible portion of the instrument 2506 from the Q3D model 2650. Preferably, for increased accuracy, more points can be determined. Assume, for example, that these points are P.sub.1 and P.sub.2 shown in FIG. 25. As discussed above, it is assumed that endoscope 2502 lies on the z-axis 2518 of the 3D system of coordinates, with its tip at the origin.” [0137]); and
based on each of the first and second distances to the first and the second points on the surface, respectively, being within a threshold, determine, based on the first coordinates and the second coordinates, a third distance between the first point and the second point (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135]).
Regarding Claim 2, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the third distance is a linear distance (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135]; “Module 2608 configures the computer system 151 to determine the axis 2514 of instrument 2506, which includes a 3D line defined by the following equation, in relation to points P.sub.1(x.sub.1, y.sub.1, z.sub.1) and P.sub.2(x.sub.2, y.sub.2, z.sub.2)…x-x1x2-x1=y-y1y2-y1=z-z1z2-z1…where Eq. (9) defines any point (x, y, z) on the axis 2514 of the instrument 2506.” [0138-0139]).
Regarding Claim 3, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the third distance is a distance on the surface (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135]; “Module 2608 configures the computer system 151 to determine the axis 2514 of instrument 2506, which includes a 3D line defined by the following equation, in relation to points P.sub.1(x.sub.1, y.sub.1, z.sub.1) and P.sub.2(x.sub.2, y.sub.2, z.sub.2)…x-x1x2-x1=y-y1y2-y1=z-z1z2-z1…where Eq. (9) defines any point (x, y, z) on the axis 2514 of the instrument 2506.” [0138-0139]).
Regarding Claim 4, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the controller is further configured to update the third distance based on movement of the surface of the anatomical structure (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135]; “Module 2608 configures the computer system 151 to determine the axis 2514 of instrument 2506, which includes a 3D line defined by the following equation, in relation to points P.sub.1(x.sub.1, y.sub.1, z.sub.1) and P.sub.2(x.sub.2, y.sub.2, z.sub.2)…x-x1x2-x1=y-y1y2-y1=z-z1z2-z1…where Eq. (9) defines any point (x, y, z) on the axis 2514 of the instrument 2506.” [0138-0139]).
Regarding Claim 5, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein determining the first distance comprises: determining a distance between a left camera of the imaging device and the first point; determining a distance between a right camera of the imaging device and the first point; and determining the first distance based on the distance between the left camera and the first point and the distance between the right camera and the first point (“…a left image 400L and a right image 400R of the surgical site include any instruments 400 and a target 410 respectively in a left viewing element 401L and a right viewing element 401R. The images 400L and 400R in the viewing elements may be provided by a left display device 402L and a right display device 402R, respectively.” [0076]; “A viewing system having two viewing elements 401R, 401L can provide a good 3D viewing perspective. The Q3D imaging system supplements this viewing perspective with physical dimension information for physical structures in the surgical scene. The stereo viewer 312 used in conjunction with a Q3D endoscopy system, can display Q3D information overlayed onto the stereo image of the surgical scene... a numerical distance value “d_Instr_Trgt” is shown displayed within the scene between instrument 400 and target 410.” [0077]; “Module 402.5 configures the controller to compute two or more pixel coordinates (N.sub.x, N.sub.y) coordinates for the selected target to determine whether the candidate pixels are illuminated by a projection from the same target. Decision module 402.6 determines whether the computed 2D pixel coordinate values indicate that the candidate pixels are illuminated by a projection from the same target. The image diversity caused by viewing the same scene with multiple sensors S.sub.ij plays a role in correctly identifying (N.sub.x, N.sub.y) associated with a specific target in the various individual images S.sub.ij.” [0081]. See Fig. 4.)
Regarding Claim 9, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the controller is further configured to control a repositionable structure to move the imaging device to the first pose and the second pose (“Module 2610 configures the computer system 151 to compute transformations that virtually “move” endoscope 2502 to the pose of the instrument 2506. This “move” includes a series of virtual rotations and virtual translations that virtually align the z-axis 2518 of the endoscope 2502 with the axis 2514 of the target instrument 2506.” [0140]).
Regarding Claim 10, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the controller is further configured to: present, on a display, a video from the imaging device showing the surface of the anatomical structure (“The operator can view video frames of images of a surgical site inside a patient's body through a stereo display device 164, which includes the viewer 312 described above with reference to FIG. 4.” [0063]; “…the controller 106, which includes a processor and a storage device (not shown), computes the physical quantitative 3D coordinates of the points in a scene adjacent the tip 208 of the elongate portion 202 and drives both the video processor 104 and a 3D display driver 109 to compose 3D scenes, which then can be displayed on a display 110, which can be a stereoscopic display or a volumetric (e.g., holographic) 3D display.” [0073]); and
add, to the video, a first virtual marker overlaid on the first point; and add, to the video, a second virtual marker overlaid on the second point (“Numerical values representing physical dimension and/or location information for physical structures in the surgical scene are shown overlaid onto the surgical scene image.” [0060]; “ A viewing system having two viewing elements 401R, 401L can provide a good 3D viewing perspective. The Q3D imaging system supplements this viewing perspective with physical dimension information for physical structures in the surgical scene. The stereo viewer 312 used in conjunction with a Q3D endoscopy system, can display Q3D information overlayed onto the stereo image of the surgical scene.” [0077]).
Regarding Claim 11, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the first distance is determined based on an axis of the imaging device (“Module 2602 configures the computer 151 (or controller 106; references to computer 151 apply to controller 106 as well in the following descriptions) from FIG. 8, to create a Q3D model 2650 of the scene 2508. As explained above with reference to FIGS. 9-19, the computer 151 is configured to determine the (x, y, z) coordinates of points determined in the endoscope FOV.sub.e. In doing so, the system creates a Q3D model 2650 of the scene 2508. The Q3D model 2650 is stored in a non-transitory computer-readable storage device 2652.” [0134]; “…FIG. 25, d.sub.1 is distance from Q3D endoscope tip 2512 to the closest point on target 2508-1.” [0135]. See Fig. 25).
Regarding Claim 12, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein, when viewing the second region, the first region is out of view of the imaging device (“Module 2610 configures the computer system 151 to compute transformations that virtually “move” endoscope 2502 to the pose of the instrument 2506. This “move” includes a series of virtual rotations and virtual translations that virtually align the z-axis 2518 of the endoscope 2502 with the axis 2514 of the target instrument 2506.” [0140]. See Fig. 25).
Regarding Claim 13, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the controller is further configured to present the third distance on a display (“The operator can view video frames of images of a surgical site inside a patient's body through a stereo display device 164, which includes the viewer 312 described above with reference to FIG. 4.” [0063]; “Images captured by the imager sensor array 210 are sent over a data bus 212 to a video processor 104, which communicates via bus 105 with a controller 106.” [0071]; “…the controller 106, which includes a processor and a storage device (not shown), computes the physical quantitative 3D coordinates of the points in a scene adjacent the tip 208 of the elongate portion 202 and drives both the video processor 104 and a 3D display driver 109 to compose 3D scenes, which then can be displayed on a display 110, which can be a stereoscopic display or a volumetric (e.g., holographic) 3D display.” [0073]).
Regarding Claim 14, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the controller is further configured to update the third distance based on movement of the imaging device or movement of the anatomical structure (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135]; “Module 2608 configures the computer system 151 to determine the axis 2514 of instrument 2506, which includes a 3D line defined by the following equation, in relation to points P.sub.1(x.sub.1, y.sub.1, z.sub.1) and P.sub.2(x.sub.2, y.sub.2, z.sub.2)…x-x1x2-x1=y-y1y2-y1=z-z1z2-z1…where Eq. (9) defines any point (x, y, z) on the axis 2514 of the instrument 2506.” [0138-0139]).
Regarding Claim 15, Panescu teaches the claim limitations as noted above.
Panescu further teaches: wherein the controller is further configured to communicate an alert in response to determining that the third distance reached or fell below a target distance (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm. The alarm may include a sound, a visual queue such as a blinking light, locking of instrument movement to avoid collision, or other haptic feedback. In response to a determination that the proximity threshold has not been crossed, control flows back to monitoring module 2014. Decision module 2020 configures the computer to wait for the prescribed time interval for receipt of user input to enter the proximity threshold and to end operation of decision module 2012 in response to no receipt of user input within the “time out” interval.” [0124]).
Regarding Claim 16, Panescu teaches the claim limitations as noted above.
wherein indication of the first point and the second point on the surface of the anatomical structure is independent of entering a virtual mode and moving a virtual cursor (“…a virtual Q3D perspective of the surgical scene 2508 is produced from a viewing perspective along longitudinal axis 2514 of the target surgical instrument 2506, which is disposed within the endoscope FOV.sub.e of the sensor array associated with the endoscope 2502. A determination is made as to transformations of orientation and position of the endoscope 2502 to align it with the axis 2514 of the target surgical instrument 2506. These orientation and position transformations are used as a basis for transformation of a Q3D rendering of the scene 2508 from a Q3D view within the FOV.sub.e of the endoscope 2502 to a Q3D view within an instrument field of view (FOV.sub.i) along the axis 2514 of the surgical instrument 2506. The view of the surgical scene 2508 from the FOV.sub.i is may be visualized using the 3D viewer 312 of FIG. 4.” [0133]).
Regarding Claim 17, Panescu teaches the claim limitations as noted above.
wherein indication of the first point and the second point on the surface of the anatomical structure is independent of moving a surgical instrument (“…a virtual Q3D perspective of the surgical scene 2508 is produced from a viewing perspective along longitudinal axis 2514 of the target surgical instrument 2506, which is disposed within the endoscope FOV.sub.e of the sensor array associated with the endoscope 2502. A determination is made as to transformations of orientation and position of the endoscope 2502 to align it with the axis 2514 of the target surgical instrument 2506. These orientation and position transformations are used as a basis for transformation of a Q3D rendering of the scene 2508 from a Q3D view within the FOV.sub.e of the endoscope 2502 to a Q3D view within an instrument field of view (FOV.sub.i) along the axis 2514 of the surgical instrument 2506. The view of the surgical scene 2508 from the FOV.sub.i is may be visualized using the 3D viewer 312 of FIG. 4.” [0133]).
Regarding Claim 19, Panescu teaches: A method for measuring a distance between two points in a surgical space (see e.g. Figs. 8, 20 and 25, [0130]), the method comprising:
moving an imaging device to a first pose to capture first imaging data showing a first region of a surface of an anatomical structure (“…a series of geometric transformations are performed based upon information in the Q3D model of the scene 2508 to virtually “move” the Q3D endoscope 2502 so that it takes the 3D pose of instrument 2506.” [0136]);
determining a first distance between the imaging device in the first pose and a first point on the surface (“…FIG. 25, d.sub.1 is distance from Q3D endoscope tip 2512 to the closest point on target 2508-1.”);
determining, based on the first pose and the first distance, first coordinates of the first point on the surface (“Module 2602 configures the computer 151 (or controller 106; references to computer 151 apply to controller 106 as well in the following descriptions) from FIG. 8, to create a Q3D model 2650 of the scene 2508. As explained above with reference to FIGS. 9-19, the computer 151 is configured to determine the (x, y, z) coordinates of points determined in the endoscope FOV.sub.e. In doing so, the system creates a Q3D model 2650 of the scene 2508. The Q3D model 2650 is stored in a non-transitory computer-readable storage device 2652.” [0134]);
moving the imaging device to a second pose to capture second imaging data showing a second region of the surface of the anatomical structure (“Module 2610 configures the computer system 151 to compute transformations that virtually “move” endoscope 2502 to the pose of the instrument 2506. This “move” includes a series of virtual rotations and virtual translations that virtually align the z-axis 2518 of the endoscope 2502 with the axis 2514 of the target instrument 2506.” [0140]);
determining a second distance between the imaging device in the second pose and a second point on the surface (“Similarly, module 2604 configures the computer 151 to determine a distance d.sub.2 between a tip 2516 of the target surgical instrument 2506 and the surgical scene 2508. Distance d.sub.2 represents the length from tip 2516 to the closest point of scene 2508. In the example provided in FIG. 25, d.sub.2 is distance from instrument tip 2516 to the closest point on target 2508-1.” [0135]);
determining, based on the second pose and the second distance, second coordinates for the second point on the surface (“Module 2606 configures the computer 151, or controller 106, to determine the (x, y, z) coordinates of at least two points located on a visible portion of the instrument 2506 from the Q3D model 2650. Preferably, for increased accuracy, more points can be determined. Assume, for example, that these points are P.sub.1 and P.sub.2 shown in FIG. 25. As discussed above, it is assumed that endoscope 2502 lies on the z-axis 2518 of the 3D system of coordinates, with its tip at the origin.” [0137]); and
based on each of the first and second distances to the first and the second points on the surface, respectively, being within a threshold, determining, based on the first coordinates and the second coordinates, a third distance between the first point and the second point (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135]).
Regarding Claim 20, Panescu teaches: A non-transitory machine-readable medium storing instructions for measuring a distance between two points in a surgical space (“…the controller 106, which includes a processor and a storage device…computes the physical quantitative 3D coordinates of the points in a scene adjacent the tip 208 of the elongate portion 202 and drives both the video processor 104 and a 3D display driver 109 to compose 3D scenes, which then can be displayed on a display 110…” [0073]; “The Q3D model 2650 is stored in a non-transitory computer-readable storage device 2652.” [0134]), when executed by a processor, cause the processor to:
move an imaging device to a first pose to capture first imaging data showing a first region of a surface of an anatomical structure (“…a series of geometric transformations are performed based upon information in the Q3D model of the scene 2508 to virtually “move” the Q3D endoscope 2502 so that it takes the 3D pose of instrument 2506.” [0136]);
determine a first distance between the imaging device in the first pose and a first point on the surface (“…FIG. 25, d.sub.1 is distance from Q3D endoscope tip 2512 to the closest point on target 2508-1.”);
determine, based on the first pose and the first distance, first coordinates of the first point on the surface (“…FIG. 25, d.sub.1 is distance from Q3D endoscope tip 2512 to the closest point on target 2508-1.”);
move the imaging device to a second pose to capture second imaging data showing a second region of the surface of the anatomical structure (“Module 2610 configures the computer system 151 to compute transformations that virtually “move” endoscope 2502 to the pose of the instrument 2506. This “move” includes a series of virtual rotations and virtual translations that virtually align the z-axis 2518 of the endoscope 2502 with the axis 2514 of the target instrument 2506.” [0140]);
determine a second distance between the imaging device in the second pose and a second point on the surface (“Similarly, module 2604 configures the computer 151 to determine a distance d.sub.2 between a tip 2516 of the target surgical instrument 2506 and the surgical scene 2508. Distance d.sub.2 represents the length from tip 2516 to the closest point of scene 2508. In the example provided in FIG. 25, d.sub.2 is distance from instrument tip 2516 to the closest point on target 2508-1.” [0135]);
determine, based on the second pose and the second distance, second coordinates for the second point on the surface (“Module 2606 configures the computer 151, or controller 106, to determine the (x, y, z) coordinates of at least two points located on a visible portion of the instrument 2506 from the Q3D model 2650. Preferably, for increased accuracy, more points can be determined. Assume, for example, that these points are P.sub.1 and P.sub.2 shown in FIG. 25. As discussed above, it is assumed that endoscope 2502 lies on the z-axis 2518 of the 3D system of coordinates, with its tip at the origin.” [0137]); and
based on each of the first and second distances to the first and the second points on the surface, respectively, being within a threshold, determine, based on the first coordinates and the second coordinates, a third distance between the first point and the second point (“…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Panescu in view of Itkowitz et. al. (U.S. 20170000574, January 5, 2017)(hereinafter, “Itkowitz”).
Regarding Claim 6, Panescu teaches the claim limitations as noted above.
With regards to limitation: wherein the threshold is determined based on at least one of a depth measurement at the first point, a depth measurement at the second point, or kinematics errors, Panescu further teaches, “Stereoscopic images have been captured, which allow the perception of depth during a surgical procedure.” [0061]; “…the numerical Q3D depth information can be used to annotate a stereoscopic image of a surgical scene with distance information or surface contour information.” [0073]; “…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124].
Panescu does not explicitly teach the specifics of the threshold determination being based on at least one of a depth measurement at the first point, a depth measurement at the second point, or kinematics errors.
Itkowitz in the field of image guided surgery systems teaches: “Using the geometries of the poses and knowledge of a preferred working distance for the imaging device, a desired pose for the imaging system is determined which places the end effectors within the view space of the imaging device. The pose and one or more kinematic models of the imaging device may then be used to determine the desired view recentering move for the imaging device.” [0037]; “At a process 250, it is determined whether the view recentering move and/or the input control recentering moves are valid. Using the kinematic models of the imaging device and the desired recentering move for the imaging device determined during process 230, it is determined whether the desired recentering move for the image device is valid.” [0039]; “…force and/or torque on one or more of the joints used to manipulate the imaging device may be monitored using suitable sensors to determine whether unexpected forces and/or torques may indicate that the imaging device is in unacceptable contact with the anatomy of the patient and/or other obstacles. In some examples, errors between the commanded positions and/or velocities and actual positions and/or velocities of the imaging device and/or the joints used to manipulate the imaging device may be monitored to determine whether the errors exceed a configurable threshold. In some examples, the configurable threshold may be different for each of the joints.” [0067].
Therefore, it would be obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the threshold determination of Panescu to be based on kinematics as taught in Itkowitz “…to determine the desired view recentering move for the imaging device.” (Itkowitz, [0037]).
Regarding Claim 8, Panescu teaches the claim limitations as noted above.
With regards to limitation: wherein the controller is further configured to determine the first pose and the second pose based on kinematics data associated with the imaging device, Panescu further teaches: “…module 2014 configures the computer to use Q3D information to monitor proximity between two or more objects within the surgeon's field of view. Decision module 2016 determines whether the proximity threshold has been crossed. In response to a determination that the proximity threshold has been crossed, module 2018 configures the computer to activate an alarm.” [0124]; “Module 2602 configures the computer 151 (or controller 106; references to computer 151 apply to controller 106 as well in the following descriptions) from FIG. 8, to create a Q3D model 2650 of the scene 2508. As explained above with reference to FIGS. 9-19, the computer 151 is configured to determine the (x, y, z) coordinates of points determined in the endoscope FOV.sub.e. In doing so, the system creates a Q3D model 2650 of the scene 2508. The Q3D model 2650 is stored in a non-transitory computer-readable storage device 2652.” [0134]; “Distances d.sub.1 and d.sub.2 can be determined by the system based upon the details of the Q3D model. Distances d.sub.1 and d.sub.2 can be computed using algorithms described with reference to FIGS. 9 and 20. The algorithm described for FIG. 9 determines the coordinates of all points in FOV.sub.e. The algorithm described for FIG. 20 can then be used to determine the distance between any two objects, such as between 2512 and 2508-1, or such as between 2516 and 2508-1. As explained more fully below, these distance parameters are used when the rendering of the 3D perspective is described.” [0135].
Panescu does not explicitly teach kinematics.
Itkowitz in the field of image guided surgery systems teaches: “Using the geometries of the poses and knowledge of a preferred working distance for the imaging device, a desired pose for the imaging system is determined which places the end effectors within the view space of the imaging device. The pose and one or more kinematic models of the imaging device may then be used to determine the desired view recentering move for the imaging device.” [0037]; “At a process 250, it is determined whether the view recentering move and/or the input control recentering moves are valid. Using the kinematic models of the imaging device and the desired recentering move for the imaging device determined during process 230, it is determined whether the desired recentering move for the image device is valid.” [0039]; “…force and/or torque on one or more of the joints used to manipulate the imaging device may be monitored using suitable sensors to determine whether unexpected forces and/or torques may indicate that the imaging device is in unacceptable contact with the anatomy of the patient and/or other obstacles. In some examples, errors between the commanded positions and/or velocities and actual positions and/or velocities of the imaging device and/or the joints used to manipulate the imaging device may be monitored to determine whether the errors exceed a configurable threshold. In some examples, the configurable threshold may be different for each of the joints.” [0067].
Therefore, it would be obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Panescu to be based on kinematics as taught in Itkowitz “…to determine the desired view recentering move for the imaging device.” (Itkowitz, [0037]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Panescu as applied to claim 1 above, and further in view of Zhang et. al. (U.S. 12551108, September 29, 2022, excerpts from PG-Pub U.S. 20240108224 used below)(hereinafter, “Zhang”).
Regarding Claim 7, Panescu teaches the claim limitations as noted above.
Panescu does not teach: wherein the threshold is determined based on a received error tolerance and a lookup table.
Zhang in the field of surgical image-based systems teaches: “…the result may have, and one or more method embodiments may further include, error metrics calculation, such as, but not limited to, RMSE (Root Mean Square Error). [0109] e) Estimating the angio delay time (see e.g., step S605 in FIG. 6). For example, in one or more embodiments, the estimating step may include estimating the angio delay time (or the delay time) and a related tolerance error, and, in a case where the estimated angio delay time and/or the related tolerance error are accepted or within acceptable threshold limits, then the angio delay time (or the delay time) and/or the related tolerance error is/are entered into a result table…” [0108].
Therefore, it would be obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the threshold determination of Panescu to be based on a received error tolerance and a lookup table as taught on Zhang to be able to achieve a predetermined/desired accuracy system values when considering settings (Zhang, [0084]).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Panescu as applied to claim 1 above, and further in view of Ummalaneni (U.S. 20180368920, December 27, 2018)(hereinafter, “Ummalaneni”).
Regarding Claim 18, Panescu teaches the claim limitations as noted above.
Panescu does not teach: wherein determining the first distance comprises generating a first depth map of the surface of the anatomical structure based on the first imaging data, wherein the first distance is determined based on the first depth map.
Ummalaneni in the field of robot surgery systems teaches: “The feature extractor 450 can also receive data from the endoscope imaging data repository 480, generate a depth map representing the distance between the endoscope imaging device and the imaged tissue represented by pixels of the image, and derive features from the generated depth map... the feature extractor 450 can use photoclinometry (e.g., shape by shading) processing to generate a depth map based on a single image…the feature extractor 450 can use a stereoscopic image set depicting the imaged region to generate a depth map.” [0153]; “The features can include, for example, positions of local maxima within the depth map…positions along a curve peak surrounding a local maxima, a value representing the distance…separating two local maxima, and/or the size, shape, and orientation of a line or polygon connecting a number of local maxima. A curve peak represents a region in the depth map at which depth values of pixels on one side of the curve peak are increasing while depth values on the other side of the curve peak are decreasing. The curve peak can include a local maximum where the depth associated with a pixel is greater than depths associated with pixels on either side of the pixel.” [0154].
Therefore, it would be obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the first distance determination of Panescu to comprise generating a first depth map as taught in Ummalaneni to incorporate image pixel information and feature information to assist in evaluation and identification of best or closest feature matches can be ascertained (Ummalaneni, [0154]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gregerson et. al. U.S. 20180185113 teaches a computer-assisted robotic surgery system.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMAL FARAG whose telephone number is (571)270-3432. The examiner can normally be reached 8:30 - 5:30 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Keith Raymond can be reached at (571) 270-1790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMAL ALY FARAG/Primary Examiner, Art Unit 3798
Prosecution Insights