DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status:
Claim 3, 4, 11, 14, 15, and 20 are canceled.
Claims 1, 2, 5-10 and 12, 13, and 16-19 are pending and examined below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an imaging device” in claim 9,
“an indication apparatus”, in claim 9,
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Claim 9: Paragraph 0050 of the Specification discloses the imaging device is an X-ray device such as a C-arm X-ray device.
Claim 9: Paragraph 0047, display such as display 36, with a fusion image with the visual indication.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5-8, 12, 16-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent No. 6389104 to Hashemi et al. “Hashemi”, in view of U.S. Patent Application Publication No. 2020/0034969 to Isaacs et al. “Isaacs”, further in view of U.S. Patent Application Publication No. 2017/0079600 to Bracken et al. “Bracken”, and further in view of U.S. Patent Application Publication No. 2013/0243153 to Sra et al. “Sra”, and further in view of U.S. Patent Application Publication No. 2013/0011030 to Tzoumas et al. “Tzoumas”.
Regarding claim 1, Hashemi discloses a method for visual support when navigating a medical instrument (method and apparatus for providing a high-quality representation of a volume having a real-time 3-D reconstruction therein of movement of an object, Abstract, wherein the object is a catheter, Col. 3, Paragraph starting at line 21) that is introduced into a hollow organ system of a patient, into a hollow organ branch (catheter is moving through a vessel of an arterial tree of a patient, Col. 2, Paragraph starting at line 56, and Col. 3, Paragraph starting at line 21 ), the method comprising:
providing a volume image of the hollow organ system and the hollow organ branch (3D reconstruction of the arterial tree, which would include the vessels, Col. 3, first paragraph) that was recorded by an X-ray device (3D angiography apparatus that is an x-ray device, Col. 5, Paragraph starting at line 42, or a computed tomography device, Col. 3, Paragraph starting at line 41);
recording a current projection image of the medical instrument (See claim language of claim 7, acquiring a real-time 2-D projection image; Col. 2, Paragraph starting at line 8; the projection comprises the catheter, Col. 4, line 40);
recording a current projection image of the medical instrument (See claim language of claim 7, acquiring a real-time 2-D projection image; Col. 2, Paragraph starting at line 8; the projection comprises the catheter, Col. 4, line 40);
registering the volume image and the current projection image in the event that no preregistration is present ("In the catheter navigation mode, the 3-D reconstruction data of the arterial tree is merged with a 2-D image taken by an X-ray source in a fluoroscopic mode", Col. 3, Paragraph starting at line 8; wherein the 2-D projection is the real-time, Col. 3, Paragraph starting at line 8, and would read on the current projection image);
determining a current position of catheter on the current projection image based on the current projection image (Given that the catheter can be segmented in a single 2-D projection image, the 3-D position of the catheter can be constrained to pass through a set of projecting lines in 3-D space, and given that the catheter must lie inside a vessel, the reconstructed vasculature is intersected with the projecting lines. The intersection of these lines and the reconstructed vasculature yields a 3-D locus of points, wherein typically Col. 4, lines 55-64; See also Fig. 4, the current position of the catheter is determined by the 3-D points that are determined from the intersection of the projecting lines with the position of the reconstructed vessel, i.e. the 3-D points that lie in the reconstructed vessel; displaying the catheter positions/path in a 3-D rendered display, and updated incrementally as the catheter is moved, Col. 4, lines 59-65, wherein displaying and then updating would read on the determining of the catheter position and displaying the position is of the current projection, and as the catheter is moved, updated position information is determined from updated/newly acquired projection); and
determining a relative position, a relative orientation, or the relative position and the relative orientation of the medical instrument in relation to the hollow organ branch (determining the 3-D positions of the catheter based from a 3-D locus of points based on the intersection of the projecting lines in 3-D space and the vasculature depicted in the reconstruction, Col. 4, lines 55-65; since the 3-D positions are based on the positions of the vasculature, i.e. a vessel as seen in Fig. 4, this would read on the positions/path of the catheter being determined in relation to the hollow organ branch); and indicating an item of information regarding the determined relative position, the determined relative orientation, or the determined relative position and the determined relative orientation of the instrument in relation to the hollow organ branch (displaying the catheter positions/path in a 3-D rendered display, and updated incrementally as the catheter is moved, Col. 4, lines 59-65).
However, Hashemi does not explicitly disclose the 2D projection image is recorded using a cone beam x-ray device, the projection being of the catheter tip and indicating an item of information regarding the determined relative position, the determined relative orientation, or the determined relative position and the determined relative orientation of the tip of the medical instrument in relation to the hollow organ branch.
Isaacs teaches the 2D projection is recording using a cone beam x-ray device (C-arm fluoroscope, Paragraph 0244, with a cone beam x-ray, Paragraph 0203 , the projection being of the tip of a medical tool (Paragraph 0196 and Fig. 11, the X-ray image being of the tip of the tool "T").
Isaacs further teaches indicating an item of information regarding the determined relative position, the determined relative orientation, or the determined relative position and the determined relative orientation of the tip of the medical instrument in relation to the anatomy (Paragraph 0147 and Fig. 7, the tip of the effecter can be represented on the displayed x-ray image as a slug 30, wherein the position of the slug correspond to the position of the tip of the effector relative to the anatomy and varied to signify different conditions regarding the position, such as the accuracy of the position, for example the slug can be depicted as a circle when the position is lower in accuracy; the representation is updated as the tip is moved, Paragraph 0148).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Hashemi's invention, wherein the 2D projection is recorded using a cone beam x-ray device, the projection being of the tip of the of the medical device and indicating an item of information regarding the determined relative position, of the tip of the medical instrument in relation to the anatomy, as taught by Isaacs, in order to allow for accurate navigation of the medical tool to a desired position (Paragraph 0149) by indicating positions of greater accuracy in the display while navigating the medical tool in the anatomy (Paragraph 0147).
However, the modifications of Hashemi and Isaacs do not explicitly disclose providing information regarding a geometric shape of a tip of the medical instrument.
Bracken teaches in a similar field of endeavor of determining the position of a transesophageal echocardiography (TEE) probe (Paragraph 0048) in a live 2D X-ray image (Paragraph 0037) inside a hollow organ system (esophagus, Paragraph 0006).
Bracken teaches providing information regarding a geometric shape of a tip of the medical instrument ("For example, a computer model of the TEE probe head geometry is used to locate the position and orientation of the TEE probe in the live X-ray images", wherein the computer model may be a 3D CAD model, Paragraph 0021 ).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi and Isaacs, wherein the method includes providing information regarding a geometric shape of a tip of the medical instrument and determining a current position, a current orientation, or the current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image, as taught by Bracken, in order to automate searching and determination of the position of the head (i.e. tip) of the medical instrument in the current/live x-ray image (Paragraph 0021 ).
However, the modifications of Hashemi, Isaacs, and Bracken do not disclose determining a current orientation of the catheter tip comprising calculating the current orientation of the catheter tip from the current projection image using a projected mapping of the catheter tip on the current projection image, and from a beam geometry of an X-ray beam recording the current projection image, wherein determining the current orientation of the catheter tip comprises: determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, identification of an actual width of the catheter tip, determination of a projected width of the catheter tip from the projection image via an image recognition; and determination of the orientation of the catheter tip based on a ratio of the projected width of the catheter tip to the actual width of the catheter tip.
Sra teaches determining a current orientation, or a current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image (See Fig. 4, Paragraph 0091, images are generated from a single plane fluoroscopy system, Ref. 1 0; Paragraph 0096, Fig. 4, Ref. 33, wherein the 3D coordinates and orientation of a catheter tip is determined from measurements obtained from the catheter tip image),
the determining of the current orientation of the catheter tip comprises calculating the current orientation of the catheter tip from the current projection image using a projected mapping of the catheter tip on the current projection image (See Fig. 4, determining the position and orientation of the catheter tip in the image by forming clusters of pixels in the image, Fig. 4, Ref. 25, Paragraph 0094, and identifying the appropriate clusters which are determined to be the catheter tip, Fig. 4, Ref. 27, Paragraph 0094, determining measurements of the catheter tip in the selected cluster, Fig. 4, Ref. 29, Paragraph 0095, and then using the measurements to determine the 3D coordinates and orientation of the catheter tip, Fig. 4, Ref. 33, Paragraph 0096) and from a beam geometry of an X-ray beam recording the current projection image (Paragraph 0096, Fig. 4, Ref. 33, the determination of the 3D coordinates and orientation of a catheter tip also uses the geometry of the fluoroscopic system; wherein the using of the geometry of the fluoroscopic system is using the conical projection geometry, which reads on a beam geometry, of the X-ray machine 10 to determine the geometric magnification at the position of the catheter tip, Paragraph 0195, and also in applying radial elongation correction to the position and determining process, using the process defined in Paragraphs 0219-0236, which includes taking into account beam geometric parameters such as the distance of the focal spot of the X-ray source to the center of the X-ray detector, Paragraph 0227, angle difference of the ray passing through the center of the catheter tip and the ray that intercepts the center of the detector, Paragraph 0232). Sra additionally teaches in the determination of the position and orientation (Paragraphs 0213, 0215, 0237), identification of an actual width of the medical object (Paragraph 0220, ω = effective diameter of sphere 215, wherein sphere 215 is representative of the medical object of interest, See Paragraph 0215);
determination of a projected width of the projected object from the projection image via an image recognition (Paragraph 0222, WP = width of projection measured in the plane of detector); and
determination of the orientation of the medical object based on a ratio of the projected width of the medical object to the actual width of the medical object (Paragraph 0237, calculating the 3D coordinates and orientation of the medical object being tracked, using a combination of calculations in functional blocks 31 and 33 of Fig. 4, wherein functional block 31 is radial elongation correction, which is the formula,
d
=
D
*
ω
/
[
W
P
∙
sin
α
2
+
cos
α
2
∙
cos
Θ
2
1
2
]
, as described in Paragraph 0234, and wherein as stated above, ω would read on the actual width/diameter of the medical object, and Wp would read on the projected width, therefore in solving the radial elongation correction formula, which is used for determining orientation, the ratio of ω/ Wp is incorporated into the determination [Examiner notes that the limitation of identification of an actual width and a projected with of the medical object is inferred since those values would need to be identified/determined/obtained in order to use the radial elongation formula].
Sra further teaches that the position and orientation of the catheter are displayed on a display device (Paragraph 0046), and the process is performed in real time or near real time such that the method can be used contemporaneously with an interventional medical procedure (Paragraph 0118).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, and Bracken, wherein the method includes determining a current orientation, or a current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image, the determining of the current orientation of the catheter tip comprises calculating the current orientation of the catheter tip from the current projection image using a projected mapping of the catheter tip on the current projection image, and from a beam geometry of an X-ray beam recording the current projection image, wherein determining the current orientation of the catheter tip comprises: determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, identification of an actual width of the catheter tip, determination of a projected width of the catheter tip from the projection image via an image recognition; and determination of the orientation of the catheter tip based on a ratio of the projected width of the catheter tip to the actual width of the catheter tip, as taught by Sra, in order to apply conical projection and radial elongation corrections (Paragraph 0023) from keystoning effects present from the geometry of the fluoroscopic system (Paragraph 0100) in order to be able to determine the 3D position and orientation of a medical, and displaying object in a living body using a conventional single-plane fluoroscopy (Abstract, Paragraph 0015), which are cheaper and/or more readily available than an anatomical mapping system (Paragraph 0003) and biplane fluoroscopy systems (Paragraph 0006).
Further Sra teaches displaying the depth of the catheter tip relative to the viewing of the x and y image information (Paragraph 0097). Although Sra, teaches an example of a coordinate display in Paragraph 0097, Sra also teaches in Paragraph 0097 of other ways to display, such as in 3D, that includes the 3D position and orientation of the medical object, Paragraph 0047. Therefore, the determining of the current orientation of the medical object is that of the catheter tip, and must include determining the current orientation of the catheter tip relative to the planar axis, i.e. the axis that is parallel with the image plane, in order to have the option of displaying the position and orientation of the catheter in 2D or in 3D.
However, Sra does not disclose wherein the determining of the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, uses a pretrained machine learning algorithm.
Tzoumas teaches determining the current orientation of the catheter tip from the current projection image using a pretrained machine learning algorithm (device detection in a 2D medical image, Abstract; wherein the device is a pigtail catheter tip, Paragraph 0021, and the image is a 2D fluoroscopic image, Paragraph 0021; also see Fig. 5 and Fig. 7, wherein in the process starts by receiving a 2D medical image, step 502, and selecting a detected object candidate, step 506, which is then used to determine the pose of the pigtail catheter tip, Fig. 7, step 720, using Marginal space learning, Paragraphs 0026 and 0027, which would read on a machine learning algorithm). Further Tzoumas teaches the algorithm contains a classifier that classifies the image for determination of the pose of the catheter tip, based on the orientation of the tool tip relative to the projection plane of the image, such as being perpendicular to the projection plane of the image (Paragraph 0024) or parallel (Paragraph 0022). Tzoumas’ method also occurs in (Paragraph 0021), which Sra also teaches (Sra, Paragraph 0118).
Therefore, It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, and Sra, wherein the determining of the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, a taught by Sra, is performed using a pretrained machine learning algorithm, as taught by Tzoumas, in order to provide an accurate and fast detection scheme, Paragraph 0026, to detect a 3D device, such as a catheter, in a 2D medical image, Paragraph 0002, and to detect the pose of the tip of the catheter, Paragraph 0042.
Regarding claim 5, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 1 above.
As disclosed in the claim 1 rejection above, Hashemi discloses determining a relative position, a relative orientation, or the relative position and the relative orientation of the medical instrument in relation to the hollow organ branch (determining the 3-D positions of the catheter based from a 3-D locus of points based on the intersection of the projecting lines in 3-D space and the vasculature depicted in the reconstruction, Col. 4, lines 55-65; since the 3-D positions are based on the positions of the vasculature, i.e. a vessel as seen in Fig. 4, this would read on the positions/path of the catheter being determined in relation to the hollow organ branch), wherein the determining of the path is updated as the catheter is moved, Col. 4, lines 55-65, during live fluoroscopic data acquisition, Col. 3, Paragraph starting at line 21. This would read on updating the relative position at the most current “real-time” projection image.
Similarly, in the claim 1 rejection above, Sra teaches determining a current orientation, or a current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image (See Fig. 4, Paragraph 0091, images are generated from a single plane fluoroscopy system, Ref. 10; Paragraph 0096, Fig. 4, Ref. 33, wherein the 3D coordinates and orientation of a catheter tip is determined from measurements obtained from the catheter tip image). Sra teaches that the position and orientation of the catheter are displayed on a display device (Paragraph 0046), and the process is performed in real time or near real time such that the method is used contemporaneously with an interventional medical procedure (Paragraph 0118). Therefore this infers that the current orientation and position are being calculated and updated as the interventional medical procedure progresses.
Additionally, as disclosed in the claim 1 rejection above, Issacs teaches the indication can be updated as the tip is moved, Paragraph 0148.
Regarding claim 6, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 1 above, including a medical instrument that is a catheter, and the tip of the medical instrument is a catheter tip.
Isaacs teaches wherein an indication of the relative orientation of the catheter tip in relation to the hollow organ branch is formed by a mapping, a graphic symbol, a numerical indication, or a color indication (Paragraph 0147 and Fig. 7, the tip of the effecter can be represented on the displayed x-ray image as a slug 30, wherein the position of the slug correspond to the position of the tip of the effector relative to the anatomy and varied to signify different conditions regarding the position, such as the accuracy of the position, for example the slug can be depicted as a circle when the position is lower in accuracy, which would read on a graphic symbol).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein an indication of the relative orientation of the catheter tip in relation to the hollow organ branch is formed by a mapping, a graphic symbol, a numerical indication, or a color indication, as further taught by Isaacs, in order to allow for accurate navigation of the medical tool to a desired position (Paragraph 0149) by indicating positions of greater accuracy in the display while navigating the medical tool in the anatomy (Paragraph 0147).
Regarding claim 7, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 1 above.
Tzoumas further teaches wherein the pretrained machine learning algorithm is trained based on a number of projection images of catheter tips and associated orientations (Paragraph 0035, wherein the different shaped position-orientation classifiers are trained using only instances in the training data of the corresponding shape, such as circle, ellipsoid, etc., at different orientations).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein the pretrained machine learning algorithm is trained based on a number of projection images of catheter tips and associated orientations, as further taught by Tzoumas, in order to provide an accurate and fast detection scheme, Paragraph 0026, to detect a 3D device, such as a catheter, in a 2D medical image, Paragraph 0002, and to detect the pose of the tip of the catheter, Paragraph 0042.
Regarding claim 8, the modifications Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 1 above.
As disclosed in the claim 1 rejection above, Bracken teaches the geometric shape of the tip of the medical instrument is obtained from a model such as a CAD model (Paragraph 0037).
However, Bracken does not explicitly disclose the model is taken from a database.
Isaacs teaches a “data base” (Paragraph 0158) that include models of the medical instrument that is used to identify the instrument in an image (Paragraph 0139).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein the information regarding the geometric shape of the tip of the medical instrument such as the CAD model of Bracken is taken from a database, as taught by Isaacs, in order to obtain a specific model that is indicative of a type of medical instrument tip (Isaacs, Paragraph 0158).
Regarding claim 12, Hashemi discloses instructions for visual support when navigating a medical instrument (method and apparatus for providing a high-quality representation of a volume having a real-time 3-D reconstruction therein of movement of an object, Abstract, wherein the object is a catheter, Col. 3, Paragraph starting at line 21) that is introduced into a hollow organ system of a patient, into a hollow organ branch (catheter is moving through a vessel of an arterial tree of a patient, Col. 2, Paragraph starting at line 56, and Col. 3, Paragraph starting at line 21), the method comprising:
providing a volume image of the hollow organ system and the hollow organ branch (3D reconstruction of the arterial tree, which would include the vessels, Col. 3, first paragraph) that was recorded by an X-ray device (3D angiography apparatus that is an x-ray device, Col. 5, Paragraph starting at line 42, or a computed tomography device, Col. 3, Paragraph starting at line 41);
recording, a current projection image (See claim language of claim 7, acquiring a real time 2-D projection image; Col. 2, Paragraph starting at line 8);
registering the volume image and the current projection image in the event that no preregistration is present ("In the catheter navigation mode, the 3-D reconstruction data of the arterial tree is merged with a 2-D image taken by an X-ray source in a fluoroscopic mode", Col. 3, Paragraph starting at line 8; wherein the 2-D projection is the real-time, Col. 3, Paragraph starting at line 8, and would read on the current projection image);
determining a current position of the medical instrument on the current projection image based on the current projection image (Given that the catheter can be segmented in a single 2-D projection image, the 3-D position of the catheter can be constrained to pass through a set of projecting lines in 3-D space, and given that the catheter must lie inside a vessel, the reconstructed vasculature is intersected with the projecting lines. The intersection of these lines and the reconstructed vasculature yields a 3-D locus of points, wherein typically Col. 4, lines 55-64; See also Fig. 4, the current position of the catheter is determined by the 3-D points that are determined from the intersection of the projecting lines with the position of the reconstructed vessel, i.e. the 3-D points that lie in the reconstructed vessel; displaying the catheter positions/path in a 3-D rendered display, and updated incrementally as the catheter is moved, Col. 4, lines 59-65, wherein displaying and then updating would read on the determining of the catheter position and displaying the position is of the current projection, and as the catheter is moved, updated position information is determined from updated/newly acquired projection);
determining a relative position, a relative orientation, or the relative position and the relative orientation of the medical instrument in relation to the hollow organ branch (determining the 3-D positions of the catheter based from a 3-D locus of points based on the intersection of the projecting lines in 3-D space and the vasculature depicted in the reconstruction, Col. 4, lines 55-65; since the 3-D positions are based on the positions of the vasculature, i.e. a vessel as seen in Fig. 4, this would read on the positions/path of the catheter being determined in relation to the hollow organ branch); and
indicating an item of information regarding the determined relative position, the determined relative orientation, or the determined relative position and the determined relative orientation of the instrument in relation to the hollow organ branch (displaying the catheter positions/path in a 3-D rendered display, and updated incrementally as the catheter is moved, Col. 4, lines 59-65).
However, Hashemi does not explicitly disclose the 2D projection is recorded using a cone beam x-ray device, the projection being of the tip of the catheter and indicating an item of information regarding the determined relative position, the determined relative orientation, or the determined relative position and the determined relative orientation of the tip of the medical instrument in relation to the hollow organ branch.
Isaacs teaches the 2D projection is recording using a cone beam x-ray device (C-arm fluoroscope, Paragraph 0244, with a cone beam x-ray, Paragraph 0203 , the projection being of the tip of a medical tool (Paragraph 0196 and Fig. 11, the X-ray image being of the tip of the tool “T”).
Isaacs further teaches indicating an item of information regarding the determined relative position, the determined relative orientation, or the determined relative position and the determined relative orientation of the tip of the medical instrument in relation to the anatomy (Paragraph 0147 and Fig. 7, the tip of the effecter can be represented on the displayed x-ray image as a slug 30, wherein the position of the slug correspond to the position of the tip of the effector relative to the anatomy and varied to signify different conditions regarding the position, such as the accuracy of the position, for example the slug can be depicted as a circle when the position is lower in accuracy; the representation is updated as the tip is moved, Paragraph 0148).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Hashemi's invention, wherein the 2D projection is recorded using a cone beam x-ray device, the projection being of the tip of the of the medical device and indicating an item of information regarding the determined relative position, of the tip of the medical instrument in relation to the anatomy, as taught by Isaacs, in order to allow for accurate navigation of the medical tool to a desired position (Paragraph 0149) by indicating positions of greater accuracy in the display while navigating the medical tool in the anatomy (Paragraph 0147).
However, the modifications of Hashemi and Isaacs do not explicitly disclose a nontransitory computer-readable storage medium that stores instructions executable by one or more processors to execute the instructions and providing information regarding a geometric shape of a tip of the medical instrument.
Bracken teaches in a similar field of endeavor of determining the position of a transesophageal echocardiography (TEE) probe (Paragraph 0048) in a live 2D X-ray image (Paragraph 0037) inside a hollow organ system (esophagus, Paragraph 0006). Bracken teaches a computing unit that executes the computer program to perform the methods, that are presented on a CD-ROM (Paragraphs 0073, 76). This would read on a non-transitory computer-readable storage medium that stores instructions executable by one or more processors to execute the instructions. Bracken teaches providing information regarding a geometric shape of a tip of the medical instrument ("For example, a computer model of the TEE probe head geometry is used to locate the position and orientation of the TEE probe in the live X-ray images", wherein the computer model may be a 3D CAD model, Paragraph 0021).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi and Isaacs, wherein the system includes a non-transitory computer-readable storage medium that stores instructions executable by one or more processors to execute the instructions, and includes providing information regarding a geometric shape of a tip of the medical instrument, as taught by Bracken, in order to automate searching and determination of the position of the head (i.e. tip) of the medical instrument in the current/live x-ray image (Paragraph 0021).
However, the modifications of Hashemi, Isaacs, and Bracken do not disclose determining a current orientation of the catheter tip comprising calculating the current orientation of the catheter tip from the current projection image using a projected mapping of the catheter tip on the current projection image, and from a beam geometry of an X-ray beam recording the current projection image, wherein determining the current orientation of the catheter tip comprises: determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, identification of an actual width of the catheter tip, determination of a projected width of the catheter tip from the projection image via an image recognition; and determination of the orientation of the catheter tip based on a ratio of the projected width of the catheter tip to the actual width of the catheter tip.
Sra teaches determining a current orientation, or a current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image (See Fig. 4, Paragraph 0091, images are generated from a single plane fluoroscopy system, Ref. 1 0; Paragraph 0096, Fig. 4, Ref. 33, wherein the 3D coordinates and orientation of a catheter tip is determined from measurements obtained from the catheter tip image),
the determining of the current orientation of the catheter tip comprises calculating the current orientation of the catheter tip from the current projection image using a projected mapping of the catheter tip on the current projection image (See Fig. 4, determining the position and orientation of the catheter tip in the image by forming clusters of pixels in the image, Fig. 4, Ref. 25, Paragraph 0094, and identifying the appropriate clusters which are determined to be the catheter tip, Fig. 4, Ref. 27, Paragraph 0094, determining measurements of the catheter tip in the selected cluster, Fig. 4, Ref. 29, Paragraph 0095, and then using the measurements to determine the 3D coordinates and orientation of the catheter tip, Fig. 4, Ref. 33, Paragraph 0096) and from a beam geometry of an X-ray beam recording the current projection image (Paragraph 0096, Fig. 4, Ref. 33, the determination of the 3D coordinates and orientation of a catheter tip also uses the geometry of the fluoroscopic system; wherein the using of the geometry of the fluoroscopic system is using the conical projection geometry, which reads on a beam geometry, of the X-ray machine 10 to determine the geometric magnification at the position of the catheter tip, Paragraph 0195, and also in applying radial elongation correction to the position and determining process, using the process defined in Paragraphs 0219-0236, which includes taking into account beam geometric parameters such as the distance of the focal spot of the X-ray source to the center of the X-ray detector, Paragraph 0227, angle difference of the ray passing through the center of the catheter tip and the ray that intercepts the center of the detector, Paragraph 0232). Sra additionally teaches in the determination of the position and orientation (Paragraphs 0213, 0215, 0237), identification of an actual width of the medical object (Paragraph 0220, ω = effective diameter of sphere 215, wherein sphere 215 is representative of the medical object of interest, See Paragraph 0215);
determination of a projected width of the projected object from the projection image via an image recognition (Paragraph 0222, WP = width of projection measured in the plane of detector); and
determination of the orientation of the medical object based on a ratio of the projected width of the medical object to the actual width of the medical object (Paragraph 0237, calculating the 3D coordinates and orientation of the medical object being tracked, using a combination of calculations in functional blocks 31 and 33 of Fig. 4, wherein functional block 31 is radial elongation correction, which is the formula,
d
=
D
*
ω
/
[
W
P
∙
sin
α
2
+
cos
α
2
∙
cos
Θ
2
1
2
]
, as described in Paragraph 0234, and wherein as stated above, ω would read on the actual width/diameter of the medical object, and Wp would read on the projected width, therefore in solving the radial elongation correction formula, which is used for determining orientation, the ratio of ω/ Wp is incorporated into the determination [Examiner notes that the limitation of identification of an actual width and a projected with of the medical object is inferred since those values would need to be identified/determined/obtained in order to use the radial elongation formula].
Sra further teaches that the position and orientation of the catheter are displayed on a display device (Paragraph 0046), and the process is performed in real time or near real time such that the method can be used contemporaneously with an interventional medical procedure (Paragraph 0118).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, and Bracken, wherein the method includes determining a current orientation, or a current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image, the determining of the current orientation of the catheter tip comprises calculating the current orientation of the catheter tip from the current projection image using a projected mapping of the catheter tip on the current projection image, and from a beam geometry of an X-ray beam recording the current projection image, wherein determining the current orientation of the catheter tip comprises: determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, identification of an actual width of the catheter tip, determination of a projected width of the catheter tip from the projection image via an image recognition; and determination of the orientation of the catheter tip based on a ratio of the projected width of the catheter tip to the actual width of the catheter tip, as taught by Sra, in order to apply conical projection and radial elongation corrections (Paragraph 0023) from keystoning effects present from the geometry of the fluoroscopic system (Paragraph 0100) in order to be able to determine the 3D position and orientation of a medical, and displaying object in a living body using a conventional single-plane fluoroscopy (Abstract, Paragraph 0015), which are cheaper and/or more readily available than an anatomical mapping system (Paragraph 0003) and biplane fluoroscopy systems (Paragraph 0006).
Further Sra teaches displaying the depth of the catheter tip relative to the viewing of the x and y image information (Paragraph 0097). Although Sra, teaches an example of a coordinate display in Paragraph 0097, Sra also teaches in Paragraph 0097 of other ways to display, such as in 3D, that includes the 3D position and orientation of the medical object, Paragraph 0047. Therefore, the determining of the current orientation of the medical object is that of the catheter tip, and must include determining the current orientation of the catheter tip relative to the planar axis, i.e. the axis that is parallel with the image plane, in order to have the option of displaying the position and orientation of the catheter in 2D or in 3D.
However, Sra does not disclose wherein the determining of the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, uses a pretrained machine learning algorithm.
Tzoumas teaches determining the current orientation of the catheter tip from the current projection image using a pretrained machine learning algorithm (device detection in a 2D medical image, Abstract; wherein the device is a pigtail catheter tip, Paragraph 0021, and the image is a 2D fluoroscopic image, Paragraph 0021; also see Fig. 5 and Fig. 7, wherein in the process starts by receiving a 2D medical image, step 502, and selecting a detected object candidate, step 506, which is then used to determine the pose of the pigtail catheter tip, Fig. 7, step 720, using Marginal space learning, Paragraphs 0026 and 0027, which would read on a machine learning algorithm). Further Tzoumas teaches the algorithm contains a classifier that classifies the image for determination of the pose of the catheter tip, based on the orientation of the tool tip relative to the projection plane of the image, such as being perpendicular to the projection plane of the image (Paragraph 0024) or parallel (Paragraph 0022). Tzoumas’ method also occurs in (Paragraph 0021), which Sra also teaches (Sra, Paragraph 0118).
Therefore, It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, and Sra, wherein the determining of the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, a taught by Sra, is performed using a pretrained machine learning algorithm, as taught by Tzoumas, in order to provide an accurate and fast detection scheme, Paragraph 0026, to detect a 3D device, such as a catheter, in a 2D medical image, Paragraph 0002, and to detect the pose of the tip of the catheter, Paragraph 0042.
Regarding claim 16, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 12 above.
As disclosed in the claim 12 rejection above, Hashemi discloses determining a relative position, a relative orientation, or the relative position and the relative orientation of the medical instrument in relation to the hollow organ branch (determining the 3-D positions of the catheter based from a 3-D locus of points based on the intersection of the projecting lines in 3-D space and the vasculature depicted in the reconstruction, Col. 4, lines 55-65; since the 3-D positions are based on the positions of the vasculature, i.e. a vessel as seen in Fig. 4, this would read on the positions/path of the catheter being determined in relation to the hollow organ branch), wherein the determining of the path is updated as the catheter is moved, Col. 4, lines 55-65, during live fluoroscopic data acquisition, Col. 3, Paragraph starting at line 21. This would read on updating the relative position at the most current “real-time” projection image.
Similarly, in the claim 12 rejection above, Sra teaches determining a current orientation, or a current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image (See Fig. 4, Paragraph 0091, images are generated from a single plane fluoroscopy system, Ref. 10; Paragraph 0096, Fig. 4, Ref. 33, wherein the 3D coordinates and orientation of a catheter tip is determined from measurements obtained from the catheter tip image). Sra teaches that the position and orientation of the catheter are displayed on a display device (Paragraph 0046), and the process is performed in real time or near real time such that the method is used contemporaneously with an interventional medical procedure (Paragraph 0118). Therefore this infers that the current orientation and position are being calculated and updated as the interventional medical procedure progresses.
Additionally, as disclosed in the claim 12 rejection above, Issacs teaches the indication can be updated as the tip is moved, Paragraph 0148.
Regarding claim 17, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 12 above, including a medical instrument that is a catheter, and the tip of the medical instrument is a catheter tip.
Isaacs teaches wherein an indication of the relative orientation of the catheter tip in relation to the hollow organ branch is formed by a mapping, a graphic symbol, a numerical indication, or a color indication (Paragraph 0147 and Fig. 7, the tip of the effecter can be represented on the displayed x-ray image as a slug 30, wherein the position of the slug correspond to the position of the tip of the effector relative to the anatomy and varied to signify different conditions regarding the position, such as the accuracy of the position, for example the slug can be depicted as a circle when the position is lower in accuracy, which would read on a graphic symbol).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein an indication of the relative orientation of the catheter tip in relation to the hollow organ branch is formed by a mapping, a graphic symbol, a numerical indication, or a color indication, as further taught by Isaacs, in order to allow for accurate navigation of the medical tool to a desired position (Paragraph 0149) by indicating positions of greater accuracy in the display while navigating the medical tool in the anatomy (Paragraph 0147).
Regarding claim 18, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 12 above.
Tzoumas further teaches wherein the pretrained machine learning algorithm is trained based on a number of projection images of catheter tips and associated orientations (Paragraph 0035, wherein the different shaped position-orientation classifiers are trained using only instances in the training data of the corresponding shape, such as circle, ellipsoid, etc., at different orientations).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein the pretrained machine learning algorithm is trained based on a number of projection images of catheter tips and associated orientations, as further taught by Tzoumas, in order to provide an accurate and fast detection scheme, Paragraph 0026, to detect a 3D device, such as a catheter, in a 2D medical image, Paragraph 0002, and to detect the pose of the tip of the catheter, Paragraph 0042.
Regarding claim 19, the modifications Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 12 above.
As disclosed in the claim 12 rejection above, Bracken teaches the geometric shape of the tip of the medical instrument is obtained from a model such as a CAD model (Paragraph 0037).
However, Bracken does not explicitly disclose the model is taken from a database.
Isaacs teaches a “data base” (Paragraph 0158) that include models of the medical instrument that is used to identify the instrument in an image (Paragraph 0139).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein the information regarding the geometric shape of the tip of the medical instrument such as the CAD model of Bracken is taken from a database, as taught by Isaacs, in order to obtain a specific model that is indicative of a type of medical instrument tip (Isaacs, Paragraph 0158).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hashemi, in view of Isaacs, further in view of Bracken, further in view of Sra, and further in view of Tzoumas, as applied to claim 1 above, and further in view of U.S. Patent Application Publication No. 2013/0101196 to Florent et al. “Florent”.
Regarding claim 2, the modifications Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 1 above.
However, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas do not disclose wherein the volume image is segmented with regard to the hollow organ system and the hollow organ branch.
Florent teaches wherein the volume image is segmented with regard to the hollow organ system and the hollow organ branch (segmenting the tubular structure and surrounding structures, Paragraph 0061; wherein segmentation is performed with the 3D dataset or 3D volume beforehand, Paragraph 0062; the segmentation is performed on the spot using the device location in the 3D dataset as a trigger for local automatic tubular structure segmentation, Paragraph 0063).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein the volume image is segmented with regard to the hollow organ system and the hollow organ branch, as taught by Florent, in order to be able to derive information regarding the tubular structure (which reads on the hollow organ branch), such as diameter, lumen area, tissue, segment length, bifurcation positions and bifurcation angles (Florent, Paragraph 0060).
Claim(s) 9, 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hashemi, in view of Isaacs, further in view of Florent, further in view of Sra, and further in view of Tzoumas.
Regarding claim 9, Hashemi discloses a system for visual support when navigating a medical catheter for providing a high-quality representation of a volume having a real-time 3-D reconstruction therein of movement of an object, Abstract, wherein the object is a catheter, Col. 3, Paragraph starting at line 21) that is introduced into a hollow organ system of a patient, into a hollow organ branch (catheter is moving through a vessel of an arterial tree of a patient, Col. 2, Paragraph starting at line 56, and Col. 3, Paragraph starting at line 21 ), the system comprising:
an imaging device (Fig. 5, C-arm X-ray device, Col. 5, Paragraph starting at line 42, that can acquire both 3D angiographic data, and 2D live projections, Col. 1, Paragraph starting at line 26) configured to record projection images (See claim language of claim 7, acquiring a realtime 2-D projection image; Col. 2, Paragraph starting at line 8).
Hashemi further discloses a computer (Col. 7, lines 55-56) that is couple to the scanning apparatus (Col. 7, Paragraph starting at line 42) and a screen or monitor (col. 3, Paragraph starting at line 8).
Hashemi further discloses determining a relative position, a relative orientation, or the relative position and the relative orientation of the medical instrument in relation to the hollow organ branch (determining the 3-D positions of the catheter based from a 3-D locus of points based on the intersection of the projecting lines in 3-D space and the vasculature depicted in the reconstruction, Col. 4, lines 55-65; since the 3-D positions are based on the positions of the vasculature, i.e. a vessel as seen in Fig. 4, this would read on the positions/path of the catheter being determined in relation to the hollow organ branch).
However, Hashemi does not explicitly disclose a memory apparatus configured to store data and image data, wherein the computing unit is configured to determine a relative position and orientation of a catheter tip based on a projection image of the projection images, and an indication apparatus configured to indicate information regarding the determined relative position, the relative orientation, or the relative position and the relative orientation.
Isaacs teaches a memory apparatus (digital memory, Paragraph 0006) configured to store data and image data (image stored in memory, Paragraph 0222). Isaacs teaches the 2D projection is recording using a cone beam x-ray device (C-arm fluoroscope, Paragraph 0244, with a cone beam x-ray, Paragraph 0203 , the projection being of the tip of a medical tool (Paragraph 0196 and Fig. 11, the X-ray image being of the tip of the tool "T").
Isaacs further teaches an indication apparatus (display device, Fig. 1, Ref. 126, for displaying the images, Paragraph 0006) configured to indicate information regarding the determined relative position, the relative orientation, or the relative position and the relative orientation (Paragraph 0147 and Fig. 7, the tip of the effecter can be represented on the displayed x-ray image as a slug 30, wherein the position of the slug correspond to the position of the tip of the effector relative to the anatomy and varied to signify different conditions regarding the position, such as the accuracy of the position, for example the slug can be depicted as a circle when the position is lower in accuracy; the representation is updated as the tip is moved, Paragraph 0148). Further, in indicating the relative position and orientation of the tip of the effector, it is inferred that the relative position and orientation are determined, before it can be indicated.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Hashem i's invention, wherein the system includes a memory apparatus configured to store data and image data; wherein the computing unit is configured to determine a relative position and orientation of a catheter tip based on a projection image of the projection images; and an image processing apparatus and an indication apparatus configured to indicate information regarding the determined relative position, the relative orientation, or the relative position and the relative orientation, as taught by Issacs, in order to allow for accurate navigation of the medical tool to a desired position (Paragraph 0149) by indicating positions of greater accuracy in the display while navigating the medical tool in the anatomy (Paragraph 0147), using the acquired images from the x-ray system.
However, the modifications of Hashemi and Issacs do not disclose wherein the computing unit is configured to perform segmentations of medical volume images, the projection images, or the medical volume images and the projection images; and determine position and orientation of a catheter tip based on a projection image of the projection images, the determination of the orientation of the catheter tip comprising calculation of the orientation of the catheter tip from the projection image using a projected mapping of the catheter tip on the projection image; and actuate the system.
Florent teaches a computing unit (computing unit with data processor equipped to carry out the method of the invention, Paragraph 0083, which would read on a controller to actuate the system) is configured to perform segmentations of medical volume images, the projection images, or the medical volume images and the projection images (segmentation is performed with the 3D dataset or 3D volume, Paragraphs 0063-64).
Florent further teaches determining a position and orientation of a catheter tip based on a projection image of the projection images, the determination of the orientation of the catheter tip comprising calculation of the orientation of the catheter tip from the projection image using a projected mapping of the catheter tip on the projection image, from a beam geometry of an Xray beam recording the projection image, or from a combination thereof (claim language of claim 1, determine the 2D position of the interventional device in the 2D X-ray image; Fig. 6, Ref. 126 and Paragraph 0135, determining the orientation of the device 12 in relation to the surrounding tubular structure; wherein the current orientation of the catheter tip is determined using a mapping on the current projection image, See Fig. 11, Paragraph 0152 maps a device tip 314 on a 2D x-ray fluoro image 313, indicated by a white line, and the actual orientation of the tip is indicated with a depth vector 322 that is calculated from the projection image 313 in relation with a determined device plane as shown in Fig. 9, Paragraph 0152, and a small version of Fig. 9 is superimposed onto the projection image).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi and Isaacs, wherein the computing unit is configured to perform segmentations of medical volume images, the projection images, or the medical volume images and the projection images, determine position and orientation of a catheter tip based on a projection image of the projection images, the determination of the orientation of the catheter tip, comprising calculation of the orientation of the catheter tip from the projection image using a projected mapping of the catheter tip on the projection image, and actuate the system, as taught by Florent, in order to provide enhanced information, such as the orientation and position of the tip of the interventional device, to the user in an easily comprehensible manner while keeping the X-ray dose to a minimum for navigating the intervention device within a tubular structure (Florent, Abstract, Paragraph 0152).
However, the modification of Hashemi, Isaacs, and Florent do not disclose the calculation of the orientation of the catheter tip from the projection image also includes a beam geometry of an X-ray beam recording the projection image, wherein determining the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, wherein determining the current orientation of the catheter tip comprises: determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, identification of an actual width of the catheter tip, determination of a projected width of the catheter tip from the projection image via an image recognition; and determination of the orientation of the catheter tip based on a ratio of the projected width of the catheter tip to the actual width of the catheter tip.
Sra teaches determining a current orientation, or a current position and the current orientation of the tip of the medical instrument on the current projection image based on the current projection image (See Fig. 4, Paragraph 0091, images are generated from a single plane fluoroscopy system, Ref. 1 0; Paragraph 0096, Fig. 4, Ref. 33, wherein the 3D coordinates and orientation of a catheter tip is determined from measurements obtained from the catheter tip image), the determining of the current orientation of the catheter tip includes taking into account a beam geometry of an X-ray beam recording the current projection image (Paragraph 0096, Fig. 4, Ref. 33, the determination of the 3D coordinates and orientation of a catheter tip also uses the geometry of the fluoroscopic system; wherein the using of the geometry of the fluoroscopic system is using the conical projection geometry, which reads on a beam geometry, of the X-ray machine 10 to determine the geometric magnification at the position of the catheter tip, Paragraph 0195, and also in applying radial elongation correction to the position and determining process, using the process defined in Paragraphs 0219-0236, which includes taking into account beam geometric parameters such as the distance of the focal spot of the X-ray source to the center of the X-ray detector, Paragraph 0227, angle difference of the ray passing through the center of the catheter tip and the ray that intercepts the center of the detector, Paragraph 0232). Sra additionally teaches in the determination of the position and orientation (Paragraphs 0213, 0215, 0237), identification of an actual width of the medical object (Paragraph 0220, ω = effective diameter of sphere 215, wherein sphere 215 is representative of the medical object of interest, See Paragraph 0215);
determination of a projected width of the projected object from the projection image via an image recognition (Paragraph 0222, WP = width of projection measured in the plane of detector); and
determination of the orientation of the medical object based on a ratio of the projected width of the medical object to the actual width of the medical object (Paragraph 0237, calculating the 3D coordinates and orientation of the medical object being tracked, using a combination of calculations in functional blocks 31 and 33 of Fig. 4, wherein functional block 31 is radial elongation correction, which is the formula,
d
=
D
*
ω
/
[
W
P
∙
sin
α
2
+
cos
α
2
∙
cos
Θ
2
1
2
]
, as described in Paragraph 0234, and wherein as stated above, ω would read on the actual width/diameter of the medical object, and Wp would read on the projected width, therefore in solving the radial elongation correction formula, which is used for determining orientation, the ratio of ω/ Wp is incorporated into the determination [Examiner notes that the limitation of identification of an actual width and a projected with of the medical object is inferred since those values would need to be identified/determined/obtained in order to use the radial elongation formula].
Sra teaches that the position and orientation of the catheter are displayed on a display device (Paragraph 0046), and the process is performed in real time or near real time such that the method can be used contemporaneously with an interventional medical procedure (Paragraph 0118).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, and Florent, wherein the calculation of the orientation of the catheter tip from the projection image also includes a beam geometry of an X-ray beam recording the projection image, wherein determining the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, wherein determining the current orientation of the catheter tip comprises: determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, identification of an actual width of the catheter tip, determination of a projected width of the catheter tip from the projection image via an image recognition; and determination of the orientation of the catheter tip based on a ratio of the projected width of the catheter tip to the actual width of the catheter tip, as taught by Sra, in order to determine and apply conical projection and radial elongation corrections (Paragraph 0023) from keystoning effects present from the geometry of the fluoroscopic system (Paragraph 0100) in order to be able to determine the 3D position and orientation of a medical object in a living body using a conventional single-plane fluoroscopy (Abstract, Paragraph 0015), with a positional accuracy of+/- 4mm (Paragraph 0089).
Further Sra teaches displaying the depth of the catheter tip relative to the viewing of the x and y image information (Paragraph 0097). Although Sra, teaches an example of a coordinate display in Paragraph 0097, Sra also teaches in Paragraph 0097 of other ways to display, such as in 3D, that includes the 3D position and orientation of the medical object, Paragraph 0047. Therefore, the determining of the current orientation of the catheter tip must include determining the current orientation of the catheter tip relative to the planar axis, i.e. the axis that is parallel with the image plane, in order to have the option of displaying the position and orientation of the catheter in 2D or in 3D.
However, Sra does not disclose wherein the determining of the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, uses a pretrained machine learning algorithm.
Tzoumas teaches determining the current orientation of the catheter tip from the current projection image using a pretrained machine learning algorithm (device detection in a 2D medical image, Abstract; wherein the device is a pigtail catheter tip, Paragraph 0021, and the image is a 2D fluoroscopic image, Paragraph 0021; also see Fig. 5 and Fig. 7, wherein in the process starts by receiving a 2D medical image, step 502, and selecting a detected object candidate, step 506, which is then used to determine the pose of the pigtail catheter tip, Fig. 7, step 720, using Marginal space learning, Paragraphs 0026 and 0027, which would read on a machine learning algorithm). Further Tzoumas teaches the algorithm contains a classifier that classifies the image for determination of the pose of the catheter tip, based on the orientation of the tool tip relative to the projection plane of the image, such as being perpendicular to the projection plane of the image (Paragraph 0024) or parallel (Paragraph 0022). Tzoumas’ method also occurs in (Paragraph 0021), which Sra also teaches (Sra, Paragraph 0118).
Therefore, It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, and Sra, wherein the determining of the current orientation of the catheter tip comprises determining the current orientation of the catheter tip relative to an axis in an image plane or in parallel with the image plane from the current projection image, a taught by Sra, is performed using a pretrained machine learning algorithm, as taught by Tzoumas, in order to provide an accurate and fast detection scheme, Paragraph 0026, to detect a 3D device, such as a catheter, in a 2D medical image, Paragraph 0002, and to detect the pose of the tip of the catheter, Paragraph 0042.
Regarding claim 10, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas teaches all the features of claim 9 above.
As disclosed in the claim 9 rejection above, the combination of Hashemi, Isaacs, Bracken, Sra, and Tzoumas teaches the pretrained machine learning algorithm is configured to determine an orientation of the catheter tip from the projection image of the plurality of projection images.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified (XXX's invention OR the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein the current orientation of the catheter tip s determined using the pretrained machine learning algorithm, as taught by Tzoumas, since Tzoumas determines the location and orientation of any image that is chosen as the input, and if the one image from the plurality of images is the most recent image, it would detect the current position and orientation.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hashemi, in view of Isaacs, further in view of Bracken, further in view of Sra, further in view of Tzoumas, as applied to claim 12 above, and further in view of Florent.
Regarding claim 13, the modifications Hashemi, Isaacs, Bracken, Sra, and Tzoumas disclose all the features of claim 12 above.
However, the modifications of Hashemi, Isaacs, Bracken, Sra, and Tzoumas do not disclose wherein the volume image is segmented with regard to the hollow organ system and the hollow organ branch.
Florent teaches wherein the volume image is segmented with regard to the hollow organ system and the hollow organ branch (segmenting the tubular structure and surrounding structures, Paragraph 0061; wherein segmentation is performed with the 3D dataset or 3D volume beforehand, Paragraph 0062; the segmentation is performed on the spot using the device location in the 3D dataset as a trigger for local automatic tubular structure segmentation, Paragraph 0063).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the system as described by Hashemi, Isaacs, Bracken, Sra, and Tzoumas, wherein the volume image is segmented with regard to the hollow organ system and the hollow organ branch, as taught by Florent, in order to be able to derive information regarding the tubular structure (which reads on the hollow organ branch), such as diameter, lumen area, tissue, segment length, bifurcation positions and bifurcation angles (Florent, Paragraph 0060).
Response to Arguments
Applicant’s arguments, see Pages 12-16, filed 01/02/2026, with respect to the rejection(s) of claim(s) 1, 2, 5-10 and 12, 13, and 16-20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of a new interpretation of prior art to Sra. As stated above in the rejections, Sra teaches a radial elongation correction formula (Paragraph 0234) which includes amongst the terms a ratio of the actual width of the medical object with the projected width of the medical object, and the radial elongation correction is applied in the determination of the 3D coordinates and orientation of the medical object (Paragraph 0237).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Milton Truong whose telephone number is (571)272-2158. The examiner can normally be reached 9AM - 5PM, MON-FRI.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Keith Raymond can be reached at (571) 270-1790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MT/Examiner, Art Unit 3798
/KEITH M RAYMOND/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights