DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to the U.S.C. 102(a)(2) rejection of claim(s) 1-22 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Regarding the U.S.C. 112(b) rejection of Claims 1- 22 the applicant argues the following:
Claims 1-20 stand rejected under 35 USC 112(b) as being indefinite. Specifically, the Examiner asserts that the phrase "the anatomy of the patient" in line 4 of independent claim 1 lacks antecedent basis. (Action, p. 2.) However, the preamble of claim 1, which recites "A method for tracking a location of interest of anatomy of a patient," provides antecedent basis for this term. Thus, the rejection of claim 1 should be withdrawn. Independent claims 13 and 20 recite the same language and, thus, the rejection of claims 13 and 20 should be withdrawn. The rejections of claims 2-12 and 14-19 should be withdrawn at least for the respective dependencies of the claims.
The 112(b) rejection of Claim 1 has been dropped; however, it is noted that both Independent claims 13 and 20 do not disclose “an anatomy of the patient” in the preamble therefore the rejections still remain.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 13-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 13 recites the limitation "the anatomy of the patient" in Line 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 20 recites the limitation "the anatomy of the patient" in Line 4. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 7, 13, & 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Grasa et al (O. G. Grasa, E. Bernal, S. Casado, I. Gil, and J. M. Montiel, “Visual slam for handheld monocular endoscope,” IEEE Transactions on Medical Imaging, vol. 33, no. 1, pp. 135–146, Jan. 2014; hereinafter referred to as Grasa).
Regarding Claim 1, Grasa discloses a method for tracking a location of interest of anatomy of a patient in medical imaging comprising (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate.” [Abstract]), at a computing system:
receiving a series of medical imaging video frames captured by a medical imager imaging the anatomy of the patient (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate.” [Abstract]);
determining a scale of features in at least one frame of the series of medical imaging video frames based on detecting an object of known size or geometry in the at least one frame (“Exploration of a body cavity with an endoscope can be posed as a monocular SLAM problem where an up-to-scale 3-D map of the observed cavity is estimated from the sole input of an image sequence, gathered from a standard handheld monocular endoscope… the absolute scale of the map is recovered from the observation of a known size surgical tool.” [Introduction], “The goal of the experimental validation1 is to prove the feasibility of using monocular visual SLAM in real surgical procedures. We have selected ventral hernia repair as a paradigmatic example because of the following. 1) The scene is almost rigid and textured. 2) The standard procedure already includes accurate distance measurements that can be used as ground-truth to assess the visual SLAM geometrical accuracy. 3) The flexibility and robustness of visual SLAM methods are clearly tested because the surgical procedure has not been modified at all, except for the addition of an exploratory endoscope maneuver with a trajectory similar to other endoscope routine motions. 4) The SLAM version, just by making better use of the images, would simplify the surgical procedure without a dis ruptive modification of the workflow. 5) The image sequences exhibit significant inter-patient vari ability in texture, illumination, input port placement, and exploratory trajectory.” [Pg. 6]);
analyzing the at least one frame to determine a position of the location of interest relative to the medical imager, wherein the position of the location of interest relative to the medical imager is determined based on the scale of the features in the at least one frame (“The first two equations[(5),(6)] encode the prediction step. The EKF prediction provides an estimate for the relative pose of every map point with respect to the camera. It is accurate enough to synthesize in a patch the point image appearance, compensating for rotation and scale variations along the sequence. Then, the synthesized patch is exhaustively searched inside the elliptical region defined by innovation and its covariance[(7), (8)] by means of normalized image correlation[see Fig.1(a)].” [Pg. 3]);
determining at least one estimate of relative motion between the medical imager and the anatomy of the patient (“The probabilistic representation of the world map and the camera location at step is coded in a unique state vector modelled as a multivariate Gaussian, (1) It is composed of camera state, ,and the map defined by location of every point,. See Section III-E for map point management details. The camera state, is formed from position, orientation encoded in a quaternion, , and linear and angular velocities, and. The map is composed of point features whose locations are encoded either in Euclidean coordinates, , or in inverse depth (ID), . Details for the ID parametrization can be found in[32]. Regarding the state transition equation for the camera, we propose a dynamic constant velocity model to encode its smooth motion (2) where is the quaternion defined by the rotation vector.” [Pg. 3]);
and tracking the position of the location of interest relative to the medical imager based on the position of the location of interest determined from the at least one frame and the at least one estimate of relative motion between the medical imager and the anatomy of the patient (“Fig. 3. Measurement processes, 3-D map, camera trajectory, and the estimated ellipses for the operation in Fig. 5(b). (a) Two points over a clinch define the scale (magenta). Five or more points over the hernia defect boundary (yellow). (b) Tape measurement considered as ground-truth. (c) Handheld exploratory laparoscope motion. (d) SLAM measurement, Map and ellipses projected as augmented reality over a sequence frame. (e) Camera trajectory, 3-D map and ellipses in 3-D. Top view.” [Pg. 6], see Fig. 3 below).
PNG
media_image1.png
507
347
media_image1.png
Greyscale
1 Figure 3
Regarding Claim 7, Grasa discloses that the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined by analyzing a plurality of frames of the series of medical imaging video frames (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate” [Abstract], “The first two equations [(5), (6)] encode the prediction step. The EKF prediction x^k|k−1 provides an estimate for the relative pose of every map point with respect to the camera. It is accurate enough to synthesize in a patch the point image appearance, compensating for rotation and scale variations along the sequence. Then, the synthesized patch is exhaustively searched inside the elliptical region defined by innovation and its covariance [(7), (8)] by means of normalized image correlation [see Fig. 1(a)]. The pixel scoring highest, zi, if over a threshold, is selected as the match in the new image. ” [Pg. 6]).
Regarding Claim 13, Grasa discloses A system comprising one or more processors, memory, and one or more programs stored in the memory for execution by the one or more processors (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate.” [Abstract]), the one or more programs including instructions for:
receiving a series of medical imaging video frames captured by a medical imager imaging the anatomy of the patient (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate.” [Abstract]);
determining a scale of features in at least one frame of the series of medical imaging video frames based on detecting an object of known size or geometry in the at least one frame (“Exploration of a body cavity with an endoscope can be posed as a monocular SLAM problem where an up-to-scale 3-D map of the observed cavity is estimated from the sole input of an image sequence, gathered from a standard handheld monocular endoscope… the absolute scale of the map is recovered from the observation of a known size surgical tool.” [Introduction], “The goal of the experimental validation1 is to prove the feasibility of using monocular visual SLAM in real surgical procedures. We have selected ventral hernia repair as a paradigmatic example because of the following. 1) The scene is almost rigid and textured. 2) The standard procedure already includes accurate distance measurements that can be used as ground-truth to assess the visual SLAM geometrical accuracy. 3) The flexibility and robustness of visual SLAM methods are clearly tested because the surgical procedure has not been modified at all, except for the addition of an exploratory endoscope maneuver with a trajectory similar to other endoscope routine motions. 4) The SLAM version, just by making better use of the images, would simplify the surgical procedure without a dis ruptive modification of the workflow. 5) The image sequences exhibit significant inter-patient vari ability in texture, illumination, input port placement, and exploratory trajectory.” [Pg. 6]);
analyzing the at least one frame to determine a position of the location of interest relative to the medical imager, wherein the position of the location of interest relative to the medical imager is determined based on the scale of the features in the at least one frame (“The first two equations[(5),(6)] encode the prediction step. The EKF prediction provides an estimate for the relative pose of every map point with respect to the camera. It is accurate enough to synthesize in a patch the point image appearance, compensating for rotation and scale variations along the sequence. Then, the synthesized patch is exhaustively searched inside the elliptical region defined by innovation and its covariance[(7), (8)] by means of normalized image correlation[see Fig.1(a)].” [Pg. 3]);
determining at least one estimate of relative motion between the medical imager and the anatomy of the patient (“The probabilistic representation of the world map and the camera location at step is coded in a unique state vector modelled as a multivariate Gaussian, (1) It is composed of camera state, ,and the map defined by location of every point,. See Section III-E for map point management details. The camera state, is formed from position, orientation encoded in a quaternion, , and linear and angular velocities, and. The map is composed of point features whose locations are encoded either in Euclidean coordinates, , or in inverse depth (ID), . Details for the ID parametrization can be found in[32]. Regarding the state transition equation for the camera, we propose a dynamic constant velocity model to encode its smooth motion (2) where is the quaternion defined by the rotation vector.” [Pg. 3]);
and tracking the position of the location of interest relative to the medical imager based on the position of the location of interest determined from the at least one frame and the at least one estimate of relative motion between the medical imager and the anatomy of the patient (“Fig. 3. Measurement processes, 3-D map, camera trajectory, and the estimated ellipses for the operation in Fig. 5(b). (a) Two points over a clinch define the scale (magenta). Five or more points over the hernia defect boundary (yellow). (b) Tape measurement considered as ground-truth. (c) Handheld exploratory laparoscope motion. (d) SLAM measurement, Map and ellipses projected as augmented reality over a sequence frame. (e) Camera trajectory, 3-D map and ellipses in 3-D. Top view.” [Pg. 6], see Fig. 3 below).
PNG
media_image1.png
507
347
media_image1.png
Greyscale
2 Figure 3
Regarding Claim 19, Grasa discloses that the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined by analyzing a plurality of frames of the series of medical imaging video frames (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate” [Abstract], “The first two equations [(5), (6)] encode the prediction step. The EKF prediction x^k|k−1 provides an estimate for the relative pose of every map point with respect to the camera. It is accurate enough to synthesize in a patch the point image appearance, compensating for rotation and scale variations along the sequence. Then, the synthesized patch is exhaustively searched inside the elliptical region defined by innovation and its covariance [(7), (8)] by means of normalized image correlation [see Fig. 1(a)]. The pixel scoring highest, zi, if over a threshold, is selected as the match in the new image. ” [Pg. 6]).
Regarding Claim 20, Grasa discloses a non-transitory computer readable medium storing one or more programs for execution by the one or more processors of a computing system (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate.” [Abstract]), the one or more programs including instructions for:
receiving a series of medical imaging video frames captured by a medical imager imaging the anatomy of the patient (“we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate.” [Abstract]);
determining a scale of features in at least one frame of the series of medical imaging video frames based on detecting an object of known size or geometry in the at least one frame (“The goal of the experimental validation1 is to prove the fea sibility of using monocular visual SLAM in real surgical proce dures. We have selected ventral hernia repair as a paradigmatic example because of the following. 1) The scene is almost rigid and textured. 2) The standard procedure already includes accurate distance measurements that can be used as ground-truth to assess the visual SLAM geometrical accuracy. 3) The flexibility and robustness of visual SLAM methods are clearly tested because the surgical procedure has not been modified at all, except for the addition of an exploratory endoscope maneuver with a trajectory similar to other endoscope routine motions. 4) The SLAM version, just by making better use of the images, would simplify the surgical procedure without a dis ruptive modification of the workflow. 5) The image sequences exhibit significant inter-patient vari ability in texture, illumination, input port placement, and exploratory trajectory.” [Pg. 6]);
analyzing the at least one frame to determine a position of the location of interest relative to the medical imager, wherein the position of the location of interest relative to the medical imager is determined based on the scale of the features in the at least one frame (“The first two equations[(5),(6)] encode the prediction step. The EKF prediction provides an estimate for the relative pose of every map point with respect to the camera. It is accurate enough to synthesize in a patch the point image appearance, compensating for rotation and scale variations along the sequence. Then, the synthesized patch is exhaustively searched inside the elliptical region defined by innovation and its covariance[(7), (8)] by means of normalized image correlation[see Fig.1(a)].” [Pg. 3]);
determining at least one estimate of relative motion between the medical imager and the anatomy of the patient (“The probabilistic representation of the world map and the camera location at step is coded in a unique state vector modelled as a multivariate Gaussian, (1) It is composed of camera state, ,and the map defined by location of every point,. See Section III-E for map point management details. The camera state, is formed from position, orientation encoded in a quaternion, , and linear and angular velocities, and. The map is composed of point features whose locations are encoded either in Euclidean coordinates, , or in inverse depth (ID), . Details for the ID parametrization can be found in[32]. Regarding the state transition equation for the camera, we propose a dynamic constant velocity model to encode its smooth motion (2) where is the quaternion defined by the rotation vector.” [Pg. 3]);
and tracking the position of the location of interest relative to the medical imager based on the position of the location of interest determined from the at least one frame and the at least one estimate of relative motion between the medical imager and the anatomy of the patient (“Fig. 3. Measurement processes, 3-D map, camera trajectory, and the estimated ellipses for the operation in Fig. 5(b). (a) Two points over a clinch define the scale (magenta). Five or more points over the hernia defect boundary (yellow). (b) Tape measurement considered as ground-truth. (c) Handheld exploratory laparoscope motion. (d) SLAM measurement, Map and ellipses projected as augmented reality over a sequence frame. (e) Camera trajectory, 3-D map and ellipses in 3-D. Top view.” [Pg. 6], see Fig. 3 below).
PNG
media_image1.png
507
347
media_image1.png
Greyscale
3 Figure 3
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-3, & 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Grasa in view of Patel et al (US20230218146A1; hereinafter referred to as Patel).
Regarding Claim 2, Grasa discloses all limitations noted above except that comprising receiving medical imager motion data associated with motion of the medical imager and determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient based on the medical imager motion data.
However, in a similar field of endeavor, Patel teaches a portable endoscopic system comprising an imaging unit for an endoscopic procedure [Abstract].
Patel also teaches comprising receiving medical imager motion data associated with motion of the medical imager and determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient based on the medical imager motion data (“ an imaging unit for an endoscopic procedure is presented. The imaging unit comprises a housing and a display integrated into the housing. An imaging coupler is configured for receiving imaging information from an imaging assembly of an endoscope having a field of view (FoV) comprising of at least a portion of an end effector and a portion of a region of interest (ROI). An imaging processor is configured with instructions to process the received imaging information into pixel values representing an image of a time series and to display the image in real-time on the display, while a motion sensor is configured to detect a motion of the housing during the time series.” [0009]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with comprising receiving medical imager motion data associated with motion of the medical imager and determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient based on the medical imager motion data as taught by Patel, because there is a technological need for an imaging platform for endoscopic therapies to identify and track anatomical landmarks and therapeutic sites in vivo [0007].
Regarding Claim 3, Grasa teaches all limitations noted above except that the medical imager is an endoscopic imager and the medical imager motion data comprises data from a motion sensor system mounted to an endoscope of the endoscopic imager
However, in a similar field of endeavor, Patel teaches that the medical imager is an endoscopic imager and the medical imager motion data comprises data from a motion sensor system mounted to an endoscope of the endoscopic imager (“a motion processing unit (MPU) 120 which may include instructions or is configured to execute instructions stored on the storage device 74 to receive motion signals from a motion sensor 122. The motion sensor 122 includes at least one of a gyroscopic sensor configured to generate gyroscopic signals and an accelerometer configured to generate acceleration signals. The motion signals (e.g., the gyroscopic and acceleration signals) detect the motion of the housing 28 during the therapeutic (or another action or technique) procedure which can be used to estimate the motion of the distal tip 14 and/or the end effector 32.” [0065]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with the medical imager is an endoscopic imager and the medical imager motion data comprises data from a motion sensor system mounted to an endoscope of the endoscopic imager as taught by Patel, because there is a technological need for an imaging platform for endoscopic therapies to identify and track anatomical landmarks and therapeutic sites in vivo [0007].
Regarding Claim 14, Grasa discloses all limitations noted above except that comprising receiving medical imager motion data associated with motion of the medical imager and determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient based on the medical imager motion data.
However, in a similar field of endeavor, Patel teaches a portable endoscopic system comprising an imaging unit for an endoscopic procedure [Abstract].
Patel also teaches comprising receiving medical imager motion data associated with motion of the medical imager and determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient based on the medical imager motion data (“ an imaging unit for an endoscopic procedure is presented. The imaging unit comprises a housing and a display integrated into the housing. An imaging coupler is configured for receiving imaging information from an imaging assembly of an endoscope having a field of view (FoV) comprising of at least a portion of an end effector and a portion of a region of interest (ROI). An imaging processor is configured with instructions to process the received imaging information into pixel values representing an image of a time series and to display the image in real-time on the display, while a motion sensor is configured to detect a motion of the housing during the time series.” [0009]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with comprising receiving medical imager motion data associated with motion of the medical imager and determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient based on the medical imager motion data as taught by Patel, because there is a technological need for an imaging platform for endoscopic therapies to identify and track anatomical landmarks and therapeutic sites in vivo [0007].
Regarding Claim 15, Grasa teaches all limitations noted above except that the medical imager is an endoscopic imager and the medical imager motion data comprises data from a motion sensor system mounted to an endoscope of the endoscopic imager
However, in a similar field of endeavor, Patel teaches that the medical imager is an endoscopic imager and the medical imager motion data comprises data from a motion sensor system mounted to an endoscope of the endoscopic imager (“a motion processing unit (MPU) 120 which may include instructions or is configured to execute instructions stored on the storage device 74 to receive motion signals from a motion sensor 122. The motion sensor 122 includes at least one of a gyroscopic sensor configured to generate gyroscopic signals and an accelerometer configured to generate acceleration signals. The motion signals (e.g., the gyroscopic and acceleration signals) detect the motion of the housing 28 during the therapeutic (or another action or technique) procedure which can be used to estimate the motion of the distal tip 14 and/or the end effector 32.” [0065]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with the medical imager is an endoscopic imager and the medical imager motion data comprises data from a motion sensor system mounted to an endoscope of the endoscopic imager as taught by Patel, because there is a technological need for an imaging platform for endoscopic therapies to identify and track anatomical landmarks and therapeutic sites in vivo [0007].
Claims 4 & 16 rejected under 35 U.S.C. 103 as being unpatentable over Grasa in view of Patel as applied to Claim 3 & 15 above, and further in view of Beutter et al (US6689050B1; hereinafter referred to as Beutter)
Regarding Claim 4, Grasa in view of Patel discloses that the motion sensor system is mounted to the endoscope (“a motion processing unit (MPU) 120 which may include instructions or is configured to execute instructions stored on the storage device 74 to receive motion signals from a motion sensor 122. The motion sensor 122 includes at least one of a gyroscopic sensor configured to generate gyroscopic signals and an accelerometer configured to generate acceleration signals. The motion signals (e.g., the gyroscopic and acceleration signals) detect the motion of the housing 28 during the therapeutic (or another action or technique) procedure which can be used to estimate the motion of the distal tip 14 and/or the end effector 32.” [Patel 0065]).
Grasa in view of Patel does not specifically disclose that the housing is mounted to a light post of the endoscope.
However, in a similar field of endeavor, Beutter teaches an endoscope with an elongated shaft and a light post [Abstract].
Beutter also teaches the housing is mounted to a light post of the endoscope (“ The sensed elements may be mounted in an adapter (126) that is removably attached to the light post.” [Abstract])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa in view of Patel as outlined above with the housing is mounted to a light post of the endoscope as taught by Beutter, because it reduces the extent to which the light emitted by the light source has the potential for being a thermal hazard in a surgical suite [Pg. 35 Col. 3 Lines 19-22].
Regarding Claim 16, Grasa in view of Patel discloses that the motion sensor system is mounted to the endoscope (“a motion processing unit (MPU) 120 which may include instructions or is configured to execute instructions stored on the storage device 74 to receive motion signals from a motion sensor 122. The motion sensor 122 includes at least one of a gyroscopic sensor configured to generate gyroscopic signals and an accelerometer configured to generate acceleration signals. The motion signals (e.g., the gyroscopic and acceleration signals) detect the motion of the housing 28 during the therapeutic (or another action or technique) procedure which can be used to estimate the motion of the distal tip 14 and/or the end effector 32.” [Patel 0065]).
Grasa in view of Patel does not specifically disclose that the housing is mounted to a light post of the endoscope.
However, in a similar field of endeavor, Beutter teaches an endoscope with an elongated shaft and a light post [Abstract].
Beutter also teaches the housing is mounted to a light post of the endoscope (“ The sensed elements may be mounted in an adapter (126) that is removably attached to the light post.” [Abstract])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa in view of Patel as outlined above with the housing is mounted to a light post of the endoscope as taught by Beutter, because it reduces the extent to which the light emitted by the light source has the potential for being a thermal hazard in a surgical suite [Pg. 35 Col. 3 Lines 19-22].
Claims 5 & 17 are rejected under 35 U.S.C. 103 as being unpatentable over Grasa in view of Patel and further in view of Beutter as applied to Claim 4 & 16 above, and further in view of Ludtke et al (US6458078B1; hereinafter referred to as Ludtke)
Regarding Claim 5, Grasa in view of Patel and further in view of Beutter discloses all limitations noted above except that the motion sensor system is configured to generate electrical energy from light directed through the light post.
However, in a similar field of endeavor, Ludtke teaches A medical endoscope with sealed housing and receiving an electrically powered and/or controlled system [Abstract]
Ludtke also teaches the motion sensor system is configured to generate electrical energy from light directed through the light post (“The illuminating light guide 10 also may be used for the transmission of the light to a device within the endoscope for purposes of communication or power. As shown in the Figure, it may terminate by means of a portion of the cross-section inside the housing 4 of the endoscope 1 into a distal end surface 11′. Light radiating from the distal end surface 11′ is coupled by deflecting mirrors 38 onto an L/E transducer 39 implementing power supply to an electric apparatus (not shown).” [Pg. 4 Col. 4 Lines 14-22], “The bulb 42 is powered by a power source 45, for instance with modulated light. This modulated light for instance can be transmitted from the transducer 39 inside the endoscope housing 4 at the offset surface 11′ and be used to control an electric device (not shows). The path between the transducer 39 inside the housing 4 and the additional bulb 42 at the proximal end of the illuminating light guide also may be used in a reverse manner to transmit data from inside the endoscope to the outside.” [Pg. 4 Col. 4 Lines 32-41]).
It would have been obvious to an ordinary skilled person in the art before the effective filing date of the claimed invention to modify the system of Grasa in view of Patel and further in view of Beutter as outlined above with the motion sensor system is configured to generate electrical energy from light directed through the light post as taught by Ludtke, because serves to transmit signals very rapidly and at minimal power [Pg. 3 Col. 2 Lines 42-44].
Regarding Claim 17, Grasa in view of Patel and further in view of Beutter discloses all limitations noted above except that the motion sensor system is configured to generate electrical energy from light directed through the light post.
However, in a similar field of endeavor, Ludtke teaches A medical endoscope with sealed housing and receiving an electrically powered and/or controlled system [Abstract]
Ludtke also teaches the motion sensor system is configured to generate electrical energy from light directed through the light post (“The illuminating light guide 10 also may be used for the transmission of the light to a device within the endoscope for purposes of communication or power. As shown in the Figure, it may terminate by means of a portion of the cross-section inside the housing 4 of the endoscope 1 into a distal end surface 11′. Light radiating from the distal end surface 11′ is coupled by deflecting mirrors 38 onto an L/E transducer 39 implementing power supply to an electric apparatus (not shown).” [Pg. 4 Col. 4 Lines 14-22], “The bulb 42 is powered by a power source 45, for instance with modulated light. This modulated light for instance can be transmitted from the transducer 39 inside the endoscope housing 4 at the offset surface 11′ and be used to control an electric device (not shows). The path between the transducer 39 inside the housing 4 and the additional bulb 42 at the proximal end of the illuminating light guide also may be used in a reverse manner to transmit data from inside the endoscope to the outside.” [Pg. 4 Col. 4 Lines 32-41]).
It would have been obvious to an ordinary skilled person in the art before the effective filing date of the claimed invention to modify the system of Grasa in view of Patel and further in view of Beutter as outlined above with the motion sensor system is configured to generate electrical energy from light directed through the light post as taught by Ludtke, because serves to transmit signals very rapidly and at minimal power [Pg. 3 Col. 2 Lines 42-44].
Claims 6 & 18 is rejected under 35 U.S.C. 103 as being unpatentable over Grasa in view of Patel as applied to Claim 2 and 14 above, and further in view of Kronman (US20170119474A1)
Regarding Claim 6, Grasa in view of Patel teaches all limitations noted above except that the medical imager motion data comprises data from a camera that captures images of at least one tracking object associated with the medical imager.
However, in a similar field of endeavor, Kronman teaches systems and methods of tracking the position of an endoscope within a patient's body during an endoscopic procedure [Abstract]
Kronman also teaches that the medical imager motion data comprises data from a camera that captures images of at least one tracking object associated with the medical imager (“The present specification discloses an endoscope system having an endoscope handle and an endoscope body adapted to be inserted into a gastrointestinal tract of a patient, the system comprising: a plurality of orientation markers positioned on said endoscope handle, wherein said orientation markers are distributed around a circumference of said endoscope handle; a plurality of sensors positioned at different locations longitudinally along an external surface of the endoscope body, wherein each of said plurality of sensors is adapted to generate first orientation data; one or more cameras positioned external to said patient and adapted to detect one or more of said plurality of orientation markers and generate second orientation data; and a controller adapted to receive said first orientation data and second orientation data and generate data indicative of a position of said endoscope bod” [0011]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa in view of Patel as outlined above with the medical imager motion data comprises data from a camera that captures images of at least one tracking object associated with the medical imager as taught by Kronman, because there is need for a device and method that displays an accurate position of an endoscope within a patient's body by combining the scope's coordinates with the patient's coordinate [0009].
Regarding Claim 18, Grasa in view of Patel teaches all limitations noted above except that the medical imager motion data comprises data from a camera that captures images of at least one tracking object associated with the medical imager.
However, in a similar field of endeavor, Kronman teaches systems and methods of tracking the position of an endoscope within a patient's body during an endoscopic procedure [Abstract]
Kronman also teaches that the medical imager motion data comprises data from a camera that captures images of at least one tracking object associated with the medical imager (“The present specification discloses an endoscope system having an endoscope handle and an endoscope body adapted to be inserted into a gastrointestinal tract of a patient, the system comprising: a plurality of orientation markers positioned on said endoscope handle, wherein said orientation markers are distributed around a circumference of said endoscope handle; a plurality of sensors positioned at different locations longitudinally along an external surface of the endoscope body, wherein each of said plurality of sensors is adapted to generate first orientation data; one or more cameras positioned external to said patient and adapted to detect one or more of said plurality of orientation markers and generate second orientation data; and a controller adapted to receive said first orientation data and second orientation data and generate data indicative of a position of said endoscope bod” [0011]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa in view of Patel as outlined above with the medical imager motion data comprises data from a camera that captures images of at least one tracking object associated with the medical imager as taught by Kronman, because there is need for a device and method that displays an accurate position of an endoscope within a patient's body by combining the scope's coordinates with the patient's coordinate [0009].
Claims 8 & 11 are rejected under 35 U.S.C. 103 as being unpatentable over Grasa in view of Hufford et al (US20220020166A1; hereinafter referred to as Hufford)
Regarding Claim 8, Grasa discloses all limitations noted above except that analyzing the at least one frame of the series of medical imaging video frames to determine the position of a location of interest comprises locating the location of interest in the at least one frame based on a position of at least a portion of a tool in the at least one frame.
However, in a similar field of endeavor, Hufford teaches A system and method for measuring an area of interest within a body cavity, in which real time image data is captured at a treatment site that includes the area of interest.
Hufford also teaches that analyzing the at least one frame of the series of medical imaging video frames to determine the position of a location of interest comprises locating the location of interest in the at least one frame based on a position of at least a portion of a tool in the at least one frame (“image processing techniques are used to record the locations or movements of instrument tips or other physical markers positioned by a user in the operative site to identify to the system points between which measurements are to be taken, or to circumscribe areas that are to be measured. As one specific example, the user places the tip(s) to identify to the system points between which measurements should be taken, and image processing is used to recognize the tip(s) within the image display.” [0034]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with analyzing the at least one frame of the series of medical imaging video frames to determine the position of a location of interest comprises locating the location of interest in the at least one frame based on a position of at least a portion of a tool in the at least one frame as taught by Hufford, because ensure optimal measurement accuracy [0036].
Regarding Claim 11, Grasa discloses all limitations noted above except that comprising displaying a visualization comprising a graphical indication of the location of interest in association with the medical imaging video frames.
However, in a similar field of endeavor, Hufford teaches comprising displaying a visualization comprising a graphical indication of the location of interest in association with the medical imaging video frames (“the user might move an instrument tip to a first point and then to a second point and prompt the system to then determine the distances between pairs of points, with the process repeated until the desired area has been measured. Graphical icons or pins may be overlayed by the system at the locations on the display corresponding to those identified by the user as points to be used as reference points for measurements.” [0034]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with comprising displaying a visualization comprising a graphical indication of the location of interest in association with the medical imaging video frames as taught by Hufford, because it ensures optimal measurement accuracy [0036].
Claims 9-10, 12 & 22 is rejected under 35 U.S.C. 103 as being unpatentable over Grasa in view of Hartwig et al (R. Hartwig et al., “Constrained Visual-inertial localization with application and benchmark in laparoscopic surgery,” 2022 International Conference on Robotics and Automation (ICRA), pp. 9513–9520, May 2022; hereinafter referred to as Hartwig)
Regarding Claim 9, Grasa discloses all limitations noted above except that the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined using a first motion estimate at a rate of every M frames, wherein M is greater than one.
However, in a similar field of endeavor, Hartwig teaches a novel method to tackle the visual inertial localization problem for constrained camera movements [Abstract]
Hartwig also teaches the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined using a first motion estimate at a rate of every M frames, wherein M is greater than one (“A trade-off between gyroscope and camera-based residuals is adjusted by βk = γ for residual k belonging to gyroscope measurements and βk = (1 − γ) for reprojection residuals, with γ ∈ [0,1]. We conduct experimental runs with optimiza tion being triggered every 10 keyframes and a window size of 10. Fig. 6a and 6b show the deviation of the SLAM algorithm from the ground-truth IR tracked poses for different values of γ. Since we want to jointly optimize over all residuals, the cost emerging from different sensors should simultaneously decrease.” [Pg. 6 Col.1])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined using a first motion estimate at a rate of every M frames, wherein M is greater than one as taught by Hartwig, because it reduces the complexity of the problem and make localization feasible [Abstract].
Regarding Claim 10, Grasa discloses all limitations noted above except that the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined using a first motion estimate at a rate of every M frames, wherein M is greater than one.
However, in a similar field of endeavor, Hartwig teaches that the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined using a first motion estimate at a rate of every M frames, wherein M is greater than one (“Afterwards, in Line 3 for every incoming frame it updates the landmark projections u ∈ R2 in the 2D image plane, producing residuals of the form(6). For that, it uses the variational Lucas-Kanade-Method[37] taking two consecutive frames for every camera.This variationa lmethod performs well (small baseline, photometric consistency) if video frequency is high. For large displacements, the drift becomes large, and for fast movements, we lose track. Nevertheless, optical flow is suitable for local tracking.” [Pg. 3 Col.1])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined using a first motion estimate at a rate of every M frames, wherein M is greater than one as taught by Hartwig, because it reduces the complexity of the problem and make localization feasible [Abstract].
Regarding Claim 12, Grasa discloses all limitations noted above except that determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient comprises combining data from different medical imager motion tracking algorithms.
However, in a similar field of endeavor, Hartwig teaches that determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient comprises combining data from different medical imager motion tracking algorithms (“We use residuals from the different modalities to jointly optimize a global cost function. The residuals emerge from IMUmeasurements, stereoscopic feature points, and constraints on possible solutions in SE(3).” [Abstract]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient comprises combining data from different medical imager motion tracking algorithms as taught by Hartwig, because it reduces the complexity of the problem and make localization feasible [Abstract].
Regarding Claim 22, Grasa discloses all limitations noted above except that determining the at least one estimate of relative motion between the medical imager and the anatomy of the patient comprises combining data from different medical imager motion tracking algorithms.
However, in a similar field of endeavor, Hartwig teaches that the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined based on at least two different motion estimation methods (“We use residuals from the different modalities to jointly optimize a global cost function. The residuals emerge from IMUmeasurements, stereoscopic feature points, and constraints on possible solutions in SE(3).” [Abstract]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined based on at least two different motion estimation methods as taught by Hartwig, because it reduces the complexity of the problem and make localization feasible [Abstract].
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Grasa in view of Zhao et al (US20100168562A1;hereinafter referred to as Zhao).
Regarding Claim 21, Grasa discloses all limitations noted above except that the object of known size or geometry comprises a fiducial marker.
However, in a similar field of endeavor, Zhao teaches systems, methods, and tools for tool tracking using image-derived data from one or more tool-located reference features [Abstract].
Zhao also teaches that the object of known size or geometry comprises a fiducial marker (“A method includes: capturing a first image of a tool that includes multiple features that define a first marker, where at least one of the features of the first marker includes an identification feature; determining a position for the first marker by processing the first image; determining an identification for the first marker by using the at least one identification feature by processing the first image; and determining a tool state for the tool by using the position and the identification of the first marker.” [Abstract], “When placed on the surface of an instrument of a certain diameter, the 3-D geometry of the pattern (the 3-D coordinates of all the circles and dots in a local coordinate system) is fixed and known. If a single image is used to provide 2-D coordinates, coordinates of four points are sufficient to determine the pose of the marker (and hence the tool). If stereo images are used to provide 3-D coordinates, coordinates of three points are sufficient to determine the pose of the instrument. Accordingly, the design of these 2- D markers 150 and 170 includes four circles, thereby providing a sufficient number for either single image or stereo image processing. The dots can also be used for object pose estimation. Also, although the markers can be placed on a tool in any number of different orientations, it is presently preferred that the markers be placed so that the vertical direction aligns with the instrument axial direction.” [0107]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Grasa as outlined above with the at least one estimate of relative motion between the medical imager and the anatomy of the patient is determined based on at least two different motion estimation methods as taught by Hartwig, because it increases the rate at which highly accurate tool pose estimates are available [0014].
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN MALDONADO whose telephone number is 703-756-1421. The examiner can normally be reached 8:00 am-4:00 pm PST M-Th Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached on (571) 272-7230. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Steven Maldonado/
Patent Examiner, Art Unit 3797
/CHRISTOPHER KOHARSKI/Supervisory Patent Examiner, Art Unit 3797
Prosecution Insights