DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Preliminary Amendment
Applicant’s preliminary amendment filed 2/10/2025 has been entered. The claims 1-11 have been amended. The claims 1-11 are pending in the current application.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claim 10 recites “that is to say” at line 30 of the claim 10. The language is indefinite as to which term is used as a claim limitation. It is indefinite as to whether “an end digital point” or “the pointing end” is alternatively used or refer to the same meaning. It is not clear whether two different terms are used to mean the same thing, if it is intended to do so. The claim 11 is subject to the same rationale of rejection as the claim 10.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 are rejected under 35 U.S.C. 103 as being unpatentable over Nicolosi US- Theodore et al. US-PGPUB No. 2024/0016550 (hereinafter Theodore) /PGPUB No. 2022/0093008 (hereinafter Nicolosi) in view of Sela US-PGPUB No. 2021/0289189 (hereinafter Sela); Kockro US-PGPUB No. 2005/0015005 (hereinafter Kockro).
Re Claim 1:
Nicolosi/Theodore teaches a surgical navigation system comprising:
a mobile electronic device (5) transportable in an operator's hand, for example optionally the mobile electronic device being a tablet or a smartphone, said the mobile electronic device (5) comprising at least one electronic processing unit, a display (52) and a camera (51) (see Nicolosi FIG. 7 and Paragraph 0145 and Theodore FIG. 2 and Paragraph 0061 the mobile communication device 61 including camera 13, display and processor and Paragraph 0082 a second camera 42 is mounted to the pointer device 11);
a marker, optionally the marker being a depiction of a QR-code, detectable by the camera of the mobile electronic device and suitable for being positioned near a portion of a human body or near a three-dimensional physical reproduction which at least partially simulates an anatomical part of the human body (see Nicolosi FIG. 7 and Paragraph 0059 and 0145 and Theodore FIG. 2 and Paragraph 0057 and Paragraph 0121 the markers 19 and 21 detectable by the second camera 43 at a mobile communication device); and
[a pointer device, having a pointing end; wherein said the pointer device is fixed to the mobile electronic device or is releasably fixed to the mobile electronic device, so that, when the pointer device is fixed to the mobile electronic device, said the pointer device is integral in rototranslation to the mobile electronic device and said the pointing end is visible in a field of view of the camera during a surgical navigation and/or during a calibration procedure],
and wherein the electronic processing unit is configured to perform geometric operations for defining a 3D scenario reference system associated with the marker in a 3D scenario space and for calculating a three-dimensional position of the pointing end in said the 3D scenario reference system (Nicolosi teaches at FIG. 7, Paragraph 0118-0119 and Paragraph 0161; Theodore at Paragraph 0103 and FIG. 6a and Paragraph 0106-0107 the system may require a sequence of transformation to bring the various data into a single desired coordinate system and this includes the camera-base tracking system 3 determining 205 locations in space of the respective pointer markers 19 and tissue markers 21 so that appropriate transformations can be applied to the data so that the surface point cloud 16 can be generated relative to a frame of reference of the anatomical feature 9.).
Nicolosi is silent to the claim limitation of
a pointer device, having a pointing end; wherein said the pointer device is fixed to the mobile electronic device or is releasably fixed to the mobile electronic device, so that, when the pointer device is fixed to the mobile electronic device, said the pointer device is integral in rototranslation to the mobile electronic device and said the pointing end is visible in a field of view of the camera during a surgical navigation and/or during a calibration procedure.
Theodore at least suggests the claim limitation of
a pointer device, having a pointing end; wherein said the pointer device is fixed to the mobile electronic device or is releasably fixed to the mobile electronic device, so that, when the pointer device is fixed to the mobile electronic device, said the pointer device is integral in rototranslation to the mobile electronic device and said the pointing end is visible in a field of view of the camera during a surgical navigation and/or during a calibration procedure (Theodore FIG. 2).
However, Sela/Kockro teaches the claim limitation of a pointer device, having a pointing end; wherein said the pointer device is fixed to the mobile electronic device or is releasably fixed to the mobile electronic device, so that, when the pointer device is fixed to the mobile electronic device, said the pointer device is integral in rototranslation to the mobile electronic device and said the pointing end is visible in a field of view of the camera during a surgical navigation and/or during a calibration procedure (Sela teaches at Paragraph 0054 that the tracked pointer 22 may include markers 212 and at FIG. 2 and Paragraph 0056-0057 that the fiducial markers 212 are coupled with or placed on or otherwise fixed to in rigid connection any of the positioning system 208, the access point 12, the transducer 210, the tracked pointer 22 and/or other tracked instruments. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target.
Kockro teaches at FIG. 2 and FIGS. 20-24 and Paragraph 0037- 0040 that the pointer 130 is attached to the front end of the probe 110 containing a video camera 120 such that a user can see computer generated 3D representations of preoperative imaging data and Paragraph 0064-0066 that three-dimensional positions of the landmarks/fiducials from the patient image data set can be extracted and actual 3D positions of the landmarks/fiducials on the physical patient can be determined within the reference frame of the tracking device and the camera within the probe 1802 sends real time video of the patient 1801 to a computer 1861, where it can be stored, for example, in memory buffer 1807. The probe's 3D position and orientation can be detected by a tracking device 1803).
Additionally, Sela/Kockro teaches the claim limitation wherein the electronic processing unit is configured to perform geometric operations for defining a 3D scenario reference system associated with the marker in a 3D scenario space and for calculating a three-dimensional position of the pointing end in said the 3D scenario reference system (
Sela teaches at Paragraph 0089 that the calibration module may also determine transforms, e.g. homographic transforms, to apply to images of the cameras (camera reference frame) for providing 3D visualization of the viewing target (in the 3D augmented reality image space or 3D scenario reference/virtual space).
Sela teaches at Paragraph 0057 and 0066 converting the 3D position of the tool position and/or marker position from the world reference frame to the 3D position in the virtual space for stereoscopic view of the target in the virtual space or the intraoperative reference frame---the 3D scenario reference system.
Kockro teaches at FIG. 2 and FIGS. 20-24 and Paragraph 0037- 0040 that the pointer 130 is attached to the front end of the probe 110 containing a video camera 120 such that a user can see computer generated 3D representations of preoperative imaging data and Paragraph 0064-0066 that three-dimensional positions of the landmarks/fiducials from the patient image data set can be extracted and actual 3D positions of the landmarks/fiducials on the physical patient can be determined within the reference frame of the tracking device and the camera within the probe 1802 sends real time video of the patient 1801 to a computer 1861, where it can be stored, for example, in memory buffer 1807. The probe's 3D position and orientation can be detected by a tracking device 1803).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Sela/Kockro’s pointer device into Thedore/Nicolosi’s surgical navigation system as one of the ordinary skill in the art find that Sela/Kockro’s surgical navigation device belongs to the same technical field, addresses the problem of scanning an anatomical part of the human body, where the pointer is fixed in rigid connection to the access point 12 or the positioning system 208 or the transducer and the pointer device is transportable in a surgeon’s hand and the pointer device is in the field of view of the camera. The skilled person would apply the solution disclosed in Sela/Nickro to Nicolosi because the references relate to surgical navigation system used in surgery, and disclose hand-held mobile surgical navigation systems with augmented reality. By applying the solution disclosed in Sela/Nickro, the skilled person would arrive at the claimed solution, i.e. attaching a pointer device to the mobile electronic device. As the geometric relationship between the origin of the reference system of the camera and the pointer device is known, the processor is able to determine the position of the anatomical part to perform geometric calculations for calculating the 3D position of the pointing end of the pointer device.
Re Claim 2:
The claim 2 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that a three-dimensional physical reproduction suitable for at least partially simulating an anatomical part of the human body or an anatomical portion of the human body, optionally the anatomical portion of the human body being a skull, wherein the marker is removably fixed near the three-dimensional physical reproduction or near the anatomical portion of the human body.
However, Sela/Kockro further teaches the claim limitation that a three-dimensional physical reproduction suitable for at least partially simulating an anatomical part of the human body or an anatomical portion of the human body, optionally the anatomical portion of the human body being a skull, wherein the marker is removably fixed near the three-dimensional physical reproduction or near the anatomical portion of the human body (Sela teaches at Paragraph 0046-0047 digital surgical microscopes provide magnified views of anatomical structure during surgery and at Paragraph 0063 that the navigation system 205 provides tools to the neurosurgeon for removal of brain tumors…in skull-based procedures. Sela teaches at Paragraph 0054 that the tracked pointer 22 may include markers 212 and at FIG. 2 and Paragraph 0056-0057 that the fiducial markers 212 are coupled with or placed on or otherwise fixed to in rigid connection any of the positioning system 208, the access point 12, the transducer 210, the tracked pointer 22 and/or other tracked instruments. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target.
Kockro teaches at FIG. 2 and FIGS. 20-24 and Paragraph 0037- 0040 that the pointer 130 is attached to the front end of the probe 110 containing a video camera 120 such that a user can see computer generated 3D representations of preoperative imaging data and Paragraph 0064-0066 that three-dimensional positions of the landmarks/fiducials from the patient image data set can be extracted and actual 3D positions of the landmarks/fiducials on the physical patient can be determined within the reference frame of the tracking device and the camera within the probe 1802 sends real time video of the patient 1801 to a computer 1861, where it can be stored, for example, in memory buffer 1807. The probe's 3D position and orientation can be detected by a tracking device 1803).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Sela/Kockro’s pointer device into Nicolosi’s surgical navigation system as one of the ordinary skill in the art find that Sela/Kockro’s surgical navigation device belongs to the same technical field, addresses the problem of scanning an anatomical part of the human body, where the pointer is fixed in rigid connection to the access point 12 or the positioning system 208 or the transducer and the pointer device is transportable in a surgeon’s hand and the pointer device is in the field of view of the camera. The skilled person would apply the solution disclosed in Sela/Nickro to Nicolosi because the references relate to surgical navigation system used in surgery, and disclose hand-held mobile surgical navigation systems with augmented reality. By applying the solution disclosed in Sela/Nickro, the skilled person would arrive at the claimed solution, i.e. attaching a pointer device to the mobile electronic device. As the geometric relationship between the origin of the reference system of the camera and the pointer device is known, the processor is able to determine the position of the anatomical part to perform geometric calculations for calculating the 3D position of the pointing end of the pointer device.
Re Claim 3:
The claim 3 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that a pointer coupling device suitable for coupling to the mobile electronic device and comprising a coupling seat shaped to receive a rear end of the pointer device, opposite the pointing end.
However, Sela/Kockro further teaches the claim limitation that a pointer coupling device suitable for coupling to the mobile electronic device and comprising a coupling seat shaped to receive a rear end of the pointer device, opposite the pointing end (
Sela teaches at Paragraph 0046-0047 digital surgical microscopes provide magnified views of anatomical structure during surgery and at Paragraph 0063 that the navigation system 205 provides tools to the neurosurgeon for removal of brain tumors…in skull-based procedures. Sela teaches at Paragraph 0054 that the tracked pointer 22 may include markers 212 and at FIG. 2 and Paragraph 0056-0057 that the fiducial markers 212 are coupled with or placed on or otherwise fixed to in rigid connection any of the positioning system 208, the access point 12, the transducer 210, the tracked pointer 22 and/or other tracked instruments. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target.
Kockro teaches at FIG. 2 and FIGS. 20-24 and Paragraph 0037- 0040 that the pointer 130 is attached to the front end of the probe 110 containing a video camera 120 such that a user can see computer generated 3D representations of preoperative imaging data and Paragraph 0064-0066 that three-dimensional positions of the landmarks/fiducials from the patient image data set can be extracted and actual 3D positions of the landmarks/fiducials on the physical patient can be determined within the reference frame of the tracking device and the camera within the probe 1802 sends real time video of the patient 1801 to a computer 1861, where it can be stored, for example, in memory buffer 1807. The probe's 3D position and orientation can be detected by a tracking device 1803).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Sela/Kockro’s pointer device into Nicolosi’s surgical navigation system as one of the ordinary skill in the art find that Sela/Kockro’s surgical navigation device belongs to the same technical field, addresses the problem of scanning an anatomical part of the human body, where the pointer is fixed in rigid connection to the access point 12 or the positioning system 208 or the transducer and the pointer device is transportable in a surgeon’s hand and the pointer device is in the field of view of the camera. The skilled person would apply the solution disclosed in Sela/Nickro to Nicolosi because the references relate to surgical navigation system used in surgery, and disclose hand-held mobile surgical navigation systems with augmented reality. By applying the solution disclosed in Sela/Nickro, the skilled person would arrive at the claimed solution, i.e. attaching a pointer device to the mobile electronic device. As the geometric relationship between the origin of the reference system of the camera and the pointer device is known, the processor is able to determine the position of the anatomical part to perform geometric calculations for calculating the 3D position of the pointing end of the pointer device.
Claims 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Theodore et al. US-PGPUB No. 2024/0016550 (hereinafter Theodore)/Nicolosi US-PGPUB No. 2022/0093008 (hereinafter Nicolosi) in view of Sela US-PGPUB No. 2021/0289189 (hereinafter Sela); Kockro US-PGPUB No. 2005/0015005 (hereinafter Kockro) and Sela et al. US-PGPUB No. 2018/0263707 (hereinafter Sela ‘707).
Re Claim 7:
The claim 7 is in parallel with the claim 1 in a method form. The claim 1 is subject to the same rationale of rejection as the claim 1. Moreover, the claim 7 recites additional claim limitation that the surgical navigation system further comprises
a three-dimensional physical reproduction suitable for at least partially simulating an anatomical part of the human body,
or an anatomical portion of the human body, optionally the anatomical portion of the human body being a skull, and
wherein the marker is removably fixed near the three-dimensional physical reproduction or near the anatomical portion of the human body;
ii) providing digital images related to a virtual digital representation of the three-dimensional physical reproduction or of the anatomical portion of the human body or part thereof, optionally the digital images being magnetic resonance (MRI) images or computed tomography (CT) images, the digital images being positioned in a virtual image space, with respect to a virtual reference system;
iii) framing a region of the three-dimensional physical reproduction or of the anatomical portion of the human body, with the camera;
iv) simultaneously with step iii), framing the pointing end with the camera;
v) a movement of the mobile electronic device, causing a movement of the pointing end and bringing the pointing end closer to a physical point of the three-dimensional physical reproduction or of the anatomical portion of the human body; and
vi) on the electronic processing unit:
aa) calculating a virtual three-dimensional position of the pointing end when located in the physical point with respect to the virtual reference system, that is to say, in the virtual image space; and
bb) selecting one or more digital images of said the digital images, positioned at spatial coordinates of the virtual three-dimensional position of the pointing end, for a display thereof.
Theodore/Nicolosi is silent to the claim limitation of
claim limitation that the surgical navigation system further comprises
a three-dimensional physical reproduction suitable for at least partially simulating an anatomical part of the human body,
or an anatomical portion of the human body, optionally the anatomical portion of the human body being a skull, and
wherein the marker is removably fixed near the three-dimensional physical reproduction or near the anatomical portion of the human body;
ii) providing digital images related to a virtual digital representation of the three-dimensional physical reproduction or of the anatomical portion of the human body or part thereof, optionally the digital images being magnetic resonance (MRI) images or computed tomography (CT) images, the digital images being positioned in a virtual image space, with respect to a virtual reference system;
iii) framing a region of the three-dimensional physical reproduction or of the anatomical portion of the human body, with the camera;
iv) simultaneously with step iii), framing the pointing end with the camera;
v) a movement of the mobile electronic device, causing a movement of the pointing end and bringing the pointing end closer to a physical point of the three-dimensional physical reproduction or of the anatomical portion of the human body; and
vi) on the electronic processing unit:
aa) calculating a virtual three-dimensional position of the pointing end when located in the physical point with respect to the virtual reference system, that is to say, in the virtual image space; and
bb) selecting one or more digital images of said the digital images, positioned at spatial coordinates of the virtual three-dimensional position of the pointing end, for a display thereof.
However, Nicolosi/Sela/Kockro teaches the claim limitation of claim limitation that the surgical navigation system further comprises
a three-dimensional physical reproduction suitable for at least partially simulating an anatomical part of the human body,
or an anatomical portion of the human body, optionally the anatomical portion of the human body being a skull (Sela teaches at Paragraph 0063 that the navigation system 205 is applied in the context of port-based neurosurgery for removal of brain tumors in a skull-based procedure. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target), and
wherein the marker is removably fixed near the three-dimensional physical reproduction or near the anatomical portion of the human body (Sela teaches at Paragraph 0054 that markers are placed on various objects of interest);
ii) providing digital images related to a virtual digital representation of the three-dimensional physical reproduction or of the anatomical portion of the human body or part thereof, optionally the digital images being magnetic resonance (MRI) images or computed tomography (CT) images, the digital images being positioned in a virtual image space, with respect to a virtual reference system (Sela teaches at Paragraph 0063 that the navigation system 205 is applied in the context of port-based neurosurgery for removal of brain tumors in a skull-based procedure. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target);
iii) framing a region of the three-dimensional physical reproduction or of the anatomical portion of the human body, with the camera (Sela teaches at Paragraph 0089 that the processing unit is configured to generate 3D stereoscopic images of a viewing target, based on images acquired by the primary and secondary cameras and at Paragraph 0066 that The tracking system 321 is employed to track one or more medical instruments 360 and spatially register the one or more tracked medical instruments to an intraoperative reference frame);
iv) simultaneously with step iii), framing the pointing end with the camera (Sela teaches at Paragraph 0054 that the navigation system 205 comprises a tracked pointer 222 which include markers 212 to enable tracking by a tracking camera 213 to identify points on a patient and at Paragraph 0057 that a tracked tool is defined by a grouping of markers 212, thereby defining a rigid body to the tracking system. Defining the rigid body to the tracking system, in turn, is used to determine the position and/or orientation in 3D of a tracked tool in a virtual space and at Paragraph 0066 that the tracking system 321 is employed to track one or more medical instruments 360 and spatially register the one or more tracked medical instruments to an intraoperative reference frame);
v) a movement of the mobile electronic device, causing a movement of the pointing end and bringing the pointing end closer to a physical point of the three-dimensional physical reproduction or of the anatomical portion of the human body (Sela teaches at Paragraph 0054 that the navigation system 205 comprises a tracked pointer 222 which include markers 212 to enable tracking by a tracking camera 213 to identify points on a patient and at Paragraph 0057 that a tracked tool is defined by a grouping of markers 212, thereby defining a rigid body to the tracking system. Defining the rigid body to the tracking system, in turn, is used to determine the position and/or orientation in 3D of a tracked tool in a virtual space and at Paragraph 0075 that the 3D optics module 630 enables the augmented optical imaging system 600 to obtain 3D information of a viewing target.); and
vi) on the electronic processing unit:
aa) calculating a virtual three-dimensional position of the pointing end when located in the physical point with respect to the virtual reference system, that is to say, in the virtual image space (Sela teaches at Paragraph 0054 that the navigation system 205 comprises a tracked pointer 222 which include markers 212 to enable tracking by a tracking camera 213 to identify points on a patient and at Paragraph 0057 that a tracked tool is defined by a grouping of markers 212, thereby defining a rigid body to the tracking system. Defining the rigid body to the tracking system, in turn, is used to determine the position and/or orientation in 3D of a tracked tool in a virtual space and at Paragraph 0066 that the tracking system 321 is employed to track one or more medical instruments 360 and spatially register the one or more tracked medical instruments to an intraoperative reference frame); and
bb) selecting one or more digital images of said the digital images, positioned at spatial coordinates of the virtual three-dimensional position of the pointing end, for a display thereof (Sela teaches at Paragraph 0054 that the navigation system 205 comprises a tracked pointer 222 which include markers 212 to enable tracking by a tracking camera 213 to identify points on a patient and at Paragraph 0057 that a tracked tool is defined by a grouping of markers 212, thereby defining a rigid body to the tracking system. Defining the rigid body to the tracking system, in turn, is used to determine the position and/or orientation in 3D of a tracked tool in a virtual space and at Paragraph 0075 that the 3D optics module 630 enables the augmented optical imaging system 600 to obtain 3D information of a viewing target.
Sela teaches at Paragraph 0054 that the tracked pointer 22 may include markers 212 and at FIG. 2 and Paragraph 0056-0057 that the fiducial markers 212 are coupled with or placed on or otherwise fixed to in rigid connection any of the positioning system 208, the access point 12, the transducer 210, the tracked pointer 22 and/or other tracked instruments. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Sela’s pointer device into Theodore/Nicolosi’s surgical navigation system as one of the ordinary skill in the art find that Sela’s surgical navigation device belongs to the same technical field, addresses the problem of scanning an anatomical part of the human body, where the pointer is fixed in rigid connection to the access point 12 or the positioning system 208 or the transducer and the pointer device is transportable in a surgeon’s hand and the pointer device is in the field of view of the camera. The skilled person would apply the solution disclosed in Sela to Nicolosi because the references relate to surgical navigation system used in surgery, and disclose hand-held mobile surgical navigation systems with augmented reality. By applying the solution disclosed in Sela, the skilled person would arrive at the claimed solution, i.e. attaching a pointer device to the mobile electronic device. As the geometric relationship between the origin of the reference system of the camera and the pointer device is known, the processor is able to determine the position of the anatomical part to perform geometric calculations for calculating the 3D position of the pointing end of the pointer device.
Additionally, Sela et al. US-PGPUB No. 2018/0263707 teaches the claim limitation that ii) providing digital images related to a virtual digital representation of the three-dimensional physical reproduction or of the anatomical portion of the human body or part thereof, optionally the digital images being magnetic resonance (MRI) images or computed tomography (CT) images, the digital images being positioned in a virtual image space, with respect to a virtual reference system (Sela ‘707 teaches [0080] that the method 600 performs a transformation mapping to create a single unified virtual coordinate space based on the 3D scan data, the position data, and the medical image data, and updates registration data of the medical navigation system based on the transformation mapping. In one example, the transformation mapping first includes a surface matching calculation using a 3D scanner point cloud based on the 3D scan data and at least one of the MR and CT coordinates, indicated by reference 622 in FIG. 7. The transformation mapping may further include registering the tracking system to create a single unified virtual coordinate space for the 3D scanner point cloud, at least one of the MR and CT coordinates, and the position data from the tracking system based on the locations of the markers, for example when the tip of the pointer tool is placed on the targets. In one example, registering the tracking system to the aligned surfaces from the 3D scanner point cloud based on the 3D scan data and at least one of the MR and CT coordinates may be performed using a point wise correspondence approach.).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Sela ‘707 into Sela as both references are related to the same electronic device during a surgical procedure in creation of one or more anatomical landmarks on the patient body. One of the ordinary skill in the art would have mapped/transformed/registered the images captured by different modalities into a single reference frame for viewing the anatomical part in the virtual image space.
Re Claim 8:
The claim 8 encompasses the same scope of invention as that of the claim 7 except additional claim limitation of the following operating step:
vii) after step vi), displaying the one or more digital images selected in step vi) on the display of the mobile electronic device.
Sela further teaches the claim limitation of the following operating step:
vii) after step vi), displaying the one or more digital images selected in step vi) on the display of the mobile electronic device (Sela teaches at Paragraph 0054 that the navigation system 205 comprises a tracked pointer 222 which include markers 212 to enable tracking by a tracking camera 213 to identify points on a patient and at Paragraph 0057 that a tracked tool is defined by a grouping of markers 212, thereby defining a rigid body to the tracking system. Defining the rigid body to the tracking system, in turn, is used to determine the position and/or orientation in 3D of a tracked tool in a virtual space and at Paragraph 0075 that the 3D optics module 630 enables the augmented optical imaging system 600 to obtain 3D information of a viewing target.
Sela teaches at Paragraph 0054 that the tracked pointer 22 may include markers 212 and at FIG. 2 and Paragraph 0056-0057 that the fiducial markers 212 are coupled with or placed on or otherwise fixed to in rigid connection any of the positioning system 208, the access point 12, the transducer 210, the tracked pointer 22 and/or other tracked instruments. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target).
Re Claim 9:
The claim 9 encompasses the same scope of invention as that of the claim 8 except additional claim limitation that simultaneously with step vii), a current image of the three-dimensional physical reproduction or of the anatomical portion of the human body, or a part thereof, captured by the camera is also displayed on the display.
Sela further teaches the claim limitation that simultaneously with step vii), a current image of the three-dimensional physical reproduction or of the anatomical portion of the human body, or a part thereof, captured by the camera is also displayed on the display (Sela teaches at Paragraph 0054 that the navigation system 205 comprises a tracked pointer 222 which include markers 212 to enable tracking by a tracking camera 213 to identify points on a patient and at Paragraph 0057 that a tracked tool is defined by a grouping of markers 212, thereby defining a rigid body to the tracking system. Defining the rigid body to the tracking system, in turn, is used to determine the position and/or orientation in 3D of a tracked tool in a virtual space and at Paragraph 0075 that the 3D optics module 630 enables the augmented optical imaging system 600 to obtain 3D information of a viewing target.
Sela teaches at Paragraph 0054 that the tracked pointer 22 may include markers 212 and at FIG. 2 and Paragraph 0056-0057 that the fiducial markers 212 are coupled with or placed on or otherwise fixed to in rigid connection any of the positioning system 208, the access point 12, the transducer 210, the tracked pointer 22 and/or other tracked instruments. Sela teaches at Paragraph 0049 and Paragraph 0084 that the system provides a 3D visualization of the target by combining video frames from the cameras to produce stereoscopic images of the target and at Paragraph 0088-0089 that the navigation system 200 may incorporate a 3D visualization setup for use during a medical procedure for providing 3D visualization of the viewing target).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Theodore et al. US-PGPUB No. 2024/0016550 (hereinafter Theodore)/Nicolosi US-PGPUB No. 2022/0093008 (hereinafter Nicolosi) in view of Sela US-PGPUB No. 2021/0289189 (hereinafter Sela); Kockro US-PGPUB No. 2005/0015005 (hereinafter Kockro) and Barreto US-PGPUB No. 2018/0071032 (hereinafter Barreto) and Sela et al. US-PGPUB No. 2016/0000515 (hereinafter Sela ‘515).
Re Claim 10:
The claim 10 is in parallel with the claim 1 in a method form. The claim 10 is subject to the same rationale of rejection as the claim 1.
Moreover, the claim 10 recites additional claim limitation of the calibration method comprising the following operating steps:
a) providing the pointer device fixed to the mobile electronic device so that the pointing end is visible in the field of view of the camera and integral in motion therewith;
b) the camera, acquiring one or more images of the marker and, on the electronic processing unit, constructing the 3D scenario space in the 3D scenario reference system and calculating a position of a 3D camera reference system- integral with the camera, in the 3D scenario reference system;
c) the camera, acquiring a first 2D pointer image containing an end digital image of the pointing end, that is to say, an end digital point;
d) identifying two end coordinates preferably in pixels, of the end digital image and storing the two end coordinates with respect to the 3D camera reference system, on a storage device of the mobile electronic device;
e) positioning the pointing end in contact with a physical calibration point, said the physical calibration point being a physical point with known three- dimensional coordinates in the 3D scenario reference system;
f) acquiring the position of the 3D camera reference system in the 3D scenario reference system; and
g) on the electronic processing unit, calculating a geometric distance between the 3D camera reference system and the physical calibration point in the 3D camera reference system and, as a function of the geometric distance, calculating a third end coordinate, which together with the two end coordinates defines the position of the pointing end with respect to the 3D camera reference system.
Barreto US-PGPUB No. 2018/0071032 (hereinafter Barreto) teaches the calibration method comprising the following operating steps (Barreto teaches at Paragraph 0138 that the CAMT can be calibrated in the OR from a single image of a known grid of checkerboard pattern):
a) providing the pointer device fixed to the mobile electronic device so that the pointing end is visible in the field of view of the camera and integral in motion therewith (Barreto teaches at FIGS. 7A-7C and Paragraph 0043-0046 a Camera Tool including a pointer device and at Paragraph 0140 that the pointer has a visual marker attached);
b) the camera, acquiring one or more images of the marker and, on the electronic processing unit, constructing the 3D scenario space in the 3D scenario reference system and calculating a position of a 3D camera reference system- integral with the camera, in the 3D scenario reference system (Barreto teaches at Paragraph 0138 that the surgeon acquires the calibration frame by positioning the camera such that the pattern is visible in image and the tool tip touches a particular point whose coordinates are known in the coordinate system of the grid and at Paragraph 0139 that Section 5 discloses a method for the 3D reconstruction where the surgeon uses a calibrated touch -probe to pinpoint points of interest while the camera observes both the WM and the TM of the tool and at Paragraph 0140 that the pointer has a visual marker attached);
c) the camera, acquiring a first 2D pointer image containing an end digital image of the pointing end, that is to say, an end digital point (Barreto teaches at FIG. 7C and Paragraph 0138 that the surgeon acquires the calibration frame by positioning the camera such that the pattern is visible in image and the tool tip touches a particular point);
d) identifying two end coordinates preferably in pixels, of the end digital image and storing the two end coordinates with respect to the 3D camera reference system, on a storage device of the mobile electronic device (Barreto teaches at Paragraph 0139 that Section 5 discloses a method for the 3D reconstruction where the surgeon uses a calibrated touch -probe to pinpoint points of interest while the camera observes both the WM and the TM of the tool and at Paragraph 0140 that the pointer has a visual marker attached and at Paragraph 0058 that all measurements are stored in the coordinate system of the WM );
e) positioning the pointing end in contact with a physical calibration point, said the physical calibration point being a physical point with known three- dimensional coordinates in the 3D scenario reference system (Barreto teaches at Paragraph 0138 that the surgeon acquires the calibration frame by positioning the camera such that the pattern is visible in image and the tool tip touches a particular point whose coordinates are known in the coordinate system of the grid and at Paragraph 0139 that Section 5 discloses a method for the 3D reconstruction where the surgeon uses a calibrated touch -probe to pinpoint points of interest while the camera observes both the WM and the TM of the tool and at Paragraph 0140 that the pointer has a visual marker attached);
f) acquiring the position of the 3D camera reference system in the 3D scenario reference system (Barreto teaches at Paragraph 0138 that the coordinates of the grid reference frame are mapped into coordinates in the camera reference frame and at Paragraph 0044 that the 3D coordinates of the tip of the probe are known in the camera reference frame); and
g) on the electronic processing unit, calculating a geometric distance between the 3D camera reference system and the physical calibration point in the 3D camera reference system and, as a function of the geometric distance, calculating a third end coordinate, which together with the two end coordinates defines the position of the pointing end with respect to the 3D camera reference system (Barreto teaches at Paragraph 0138 that the surgeon acquires the calibration frame by positioning the camera such that the pattern is visible in image and the tool tip touches a particular point whose coordinates are known in the coordinate system of the grid and the rigid transformation maps coordinates in the grid reference frame into coordinates in the camera reference frame and at Paragraph 0139 that Section 5 discloses a method for the 3D reconstruction where the surgeon uses a calibrated touch -probe to pinpoint points of interest while the camera observes both the WM and the TM of the tool and at Paragraph 0145 that contactless 3D reconstruction can also be accomplished using an Active Contactless Probe…that is able to measure distances along the direction of the beam. The LRF has a visual marker attached and it is assumed to be calibrated such that the origin and unit direction of measurement are known in the local reference frame of TM and at Paragraph 0146 the method of section 4 is applied to determine the 3D pose of two markers and rigid transformation maps coordinates in the auxiliary marker into world coordinates).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated the features of Barreto’s pointer tool (camera tool) into Nicolosi and Sela/Kockro’s pointer tool to have provided a calibrated method to establish a camera reference frame. One of the ordinary skill in the art would have been motivated to have provided a calibrated camera reference system.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Theodore et al. US-PGPUB No. 2024/0016550 (hereinafter Theodore)/Nicolosi US-PGPUB No. 2022/0093008 (hereinafter Nicolosi) in view of Sela US-PGPUB No. 2021/0289189 (hereinafter Sela); Kockro US-PGPUB No. 2005/0015005 (hereinafter Kockro) and Barreto US-PGPUB No. 2018/0071032 (hereinafter Barreto) and Sela et al. US-PGPUB No. 2018/0263707 (hereinafter Sela ‘707).
Re Claim 11:
The claim 11 is in parallel with the claim 10 in a method form. The claim 11 is subject to the same rationale of rejection as the claim 10.
The claim 11 recites additional claim limitation of the surgical navigation method comprises the following operating steps
for calculating the virtual three-dimensional position of the pointing end in step aa):
converting the position of the pointing end from the 3D camera reference system to the 3D scenario reference system, the position of the 3D camera reference system in the 3D scenario reference system being known, obtaining the position of the pointing end in the 3D scenario reference system; and
converting the position of the pointing end from the 3D scenario reference system (W) to the virtual reference system, that is to say, in the virtual image space, the virtual reference system having been pre-registered with the 3D scenario reference system by a space registration technique, optionally by corresponding point registration technique or by image morphing registration technique.
Barreto US-PGPUB No. 2018/0071032 (hereinafter Barreto) teaches the claim limitation of the surgical navigation method comprises the following operating steps
for calculating the virtual three-dimensional position of the pointing end in step aa):
converting the position of the pointing end from the 3D camera reference system to the 3D scenario reference system, the position of the 3D camera reference system in the 3D scenario reference system being known, obtaining the position of the pointing end in the 3D scenario reference system (Barreto teaches at Paragraph 0044-0048 that the 3D coordinates of the tip or the probe are known in the camera reference frame by converting from the 3D camera reference system to the 3D scenario reference system where the laser pointer produces a colored dot on the bone/tissue surface and it is seen in the image and the point is reconstructed in 3D. This enables registration of the auxiliary marker in the WM reference frame and it is sufficient for the camera to see one of the markers in order to be registered in the global reference frame); and is silent to converting the position of the pointing end from the 3D scenario reference system (W) to the virtual reference system, that is to say, in the virtual image space, the virtual reference system having been pre-registered with the 3D scenario reference system by a space registration technique, optionally by corresponding point registration technique or by image morphing registration technique.
However, Sela ‘707 teaches converting the position of the pointing end from the 3D scenario reference system (W) to the virtual reference system, that is to say, in the virtual image space, the virtual reference system having been pre-registered with the 3D scenario reference system by a space registration technique, optionally by corresponding point registration technique or by image morphing registration technique (Sela ‘707 teaches at Paragraph 0069 that A transformation between the tracking system's camera space and the 3D scanner space may be identified so that the point cloud provided by the 3D scanner 309 and the tracking system 321 can be registered to the patient space. A transformation similar to or based on the transformation described in connection with FIG. 5 may be used.
Sela ‘707 teaches at Paragraph [0080] that the method 600 performs a transformation mapping to create a single unified virtual coordinate space based on the 3D scan data, the position data, and the medical image data, and updates registration data of the medical navigation system based on the transformation mapping. In one example, the transformation mapping first includes a surface matching calculation using a 3D scanner point cloud based on the 3D scan data and at least one of the MR and CT coordinates, indicated by reference 622 in FIG. 7. The transformation mapping may further include registering the tracking system to create a single unified virtual coordinate space for the 3D scanner point cloud, at least one of the MR and CT coordinates, and the position data from the tracking system based on the locations of the markers, for example when the tip of the pointer tool is placed on the targets. In one example, registering the tracking system to the aligned surfaces from the 3D scanner point cloud based on the 3D scan data and at least one of the MR and CT coordinates may be performed using a point wise correspondence approach.).
Moreover, Sela ‘707 teaches converting the position of the pointing end from the 3D camera reference system to the 3D scenario reference system, the position of the 3D camera reference system in the 3D scenario reference system being known, obtaining the position of the pointing end in the 3D scenario reference system (Sela ‘707 teaches at Paragraph 0069 that A transformation between the tracking system's camera space and the 3D scanner space may be identified so that the point cloud provided by the 3D scanner 309 and the tracking system 321 can be registered to the patient space. A transformation similar to or based on the transformation described in connection with FIG. 5 may be used).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated the features of Sela ‘707’s pointer tool (camera tool) into Nicolosi and Sela/Kockro’s pointer tool to have provided a calibrated method to establish a camera reference frame. One of the ordinary skill in the art would have been motivated to have provided a calibrated camera reference system.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Theodore et al. US-PGPUB No. 2024/0016550 (hereinafter Theodore)/Nicolosi US-PGPUB No. 2022/0093008 (hereinafter Nicolosi) in view of Sela US-PGPUB No. 2021/0289189 (hereinafter Sela); Kockro US-PGPUB No. 2005/0015005 (hereinafter Kockro) and Lavalle et al. US-PGPUB No. 2011/0295109 (hereinafter Lavalle).
Re Claim 4:
The claim 4 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that a magnetic pointer coupling device comprising a magnetic or ferromagnetic material and suitable for being joined to the mobile electronic device, wherein the pointer device comprises a rear end of the pointer device (7), opposite the pointing end and provided with a magnetic or ferromagnetic material for magnetic coupling with the magnetic pointer coupling device.
However, Lavalle et al. US-PGPUB No. 2011/0295109 (hereinafter Lavalle) further teaches the claim limitation that a magnetic pointer coupling device comprising a magnetic or ferromagnetic material (Lavalle teaches at Paragraph 0107 that A magnetic tracker 6 is then fixed on an instrument 5 that allows drawing of an entry point on the patient skin, such instrument being also called a pointer. The pointer is navigated with regards to R1 using the magnetic trackers. Combining magnetic localisation and registration process, the pointer can be displayed in the images of the patient, on a so called navigation image which is displayed by the display unit 9).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have further incorporated Lavalle’s magnetic pointer to have modified Sela/Nickro’s pointer device to have used magnetic tracker to have tracked positions of the pointer tip. One of the ordinary skill in the art would have been motivated to have provided a magnetic pointer.
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Theodore et al. US-PGPUB No. 2024/0016550 (hereinafter Theodore)/Nicolosi US-PGPUB No. 2022/0093008 (hereinafter Nicolosi) in view of Sela US-PGPUB No. 2021/0289189 (hereinafter Sela); Kockro US-PGPUB No. 2005/0015005 (hereinafter Kockro) and Litsch et al. US-PGPUB No. 2021/0165197 (hereinafter Litsch).
Re Claim 5:
The claim 5 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the pointer device is a pointing stick, mainly extending between the pointing end and a rear end arranged on an opposite side with respect to the pointing end, and
wherein said the pointing stick comprises:
a proximal portion mainly extending along a first longitudinal direction and being arranged near the rear end;
a distal portion, comprising the pointing end and mainly extending along a second longitudinal direction, spaced, and preferably parallel, with respect to the first longitudinal direction; and
a connecting portion connecting the proximal portion with the distal portion.
However, Litsch et al. US-PGPUB No. 2021/0165197 (hereinafter Litsch) further teaches the claim limitation that the pointer device is a pointing stick, mainly extending between the pointing end and a rear end arranged on an opposite side with respect to the pointing end, and
wherein said the pointing stick comprises:
a proximal portion mainly extending along a first longitudinal direction and being arranged near the rear end;
a distal portion, comprising the pointing end and mainly extending along a second longitudinal direction, spaced, and preferably parallel, with respect to the first longitudinal direction; and
a connecting portion connecting the proximal portion with the distal portion (Litsch FIG. 7).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have further incorporated Litsch’s stick pointer to have modified Sela/Nickro’s pointer device to have used stick-shaped pointer as a design choice. One of the ordinary skill in the art would have been motivated to have provided a stick pointer.
Re Claim 6:
The claim 6 encompasses the same scope of invention as that of the claim 5 except additional claim limitation that the pointing stick is shaped according to has a sigmoidal or "S" or "Z" shape.
However, Litsch et al. US-PGPUB No. 2021/0165197 (hereinafter Litsch) further teaches the claim limitation that the pointing stick is shaped according to has a sigmoidal or "S" or "Z" shape (Litsch FIG. 7).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have further incorporated Litsch’s stick pointer to have modified Sela/Nickro’s pointer device to have used stick-shaped pointer as a design choice. One of the ordinary skill in the art would have been motivated to have provided a stick pointer.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIN CHENG WANG whose telephone number is (571)272-7665. The examiner can normally be reached Mon-Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIN CHENG WANG/Primary Examiner, Art Unit 2617
Prosecution Insights