DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Note: all citations with respect to the specification of present application are citing the paragraph numbers in the Pre-Grant Publication US 2024/0325092 A1.
Claim Objections
Claim 1, 12 and 15 are objected to because of the following informalities:
Claim 1 line 6 – 7, limitation "an image of the fiducial marker" should read "the image of the fiducial marker".
Claim 12 line 1, limitation "the first fiducial marker" should read "the first 3D marker".
Claim 12 line 2, limitation "the second fiducial marker" should read "the second 3D marker".
Claim 15 line 8 – 9, limitation "an image of the fiducial marker" should read "the image of the fiducial marker".
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 2 and 16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “substantially aligned” in claim 2 and 16 respectively is a relative term which renders the claim indefinite. The term “substantially” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. See MPEP 2173.05 (b).
The corresponding disclosure in the specification of present application is recited as: “the fiducial marker is deployed to a surgical field and wherein at least one of the one or more fluoroscopic images is acquired at an angle that is substantially aligned to a patient bed in the surgical field” in [0010]; “In particular, the 3D fiducial marker can be clearly visualized even when the imager rotates to an angle substantially aligned with the patient bed plane” in [0062]. The specification does not specify within which range of angle to the patient bed can be considered as the “substantially aligned” pose of the fluoroscopy imager.
Thus, the above limitation is relative term which renders corresponding claim indefinite. For the purpose of examination, the above limitation is interpreted as any reasonable pose of imager aligned to the patient bed.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1, 2, 5 – 8, 13 – 16 and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Barak et al. (US 2020/0289069 A1; published on 09/17/2020) (hereinafter "Barak").
Regarding claim 1, Barak discloses a method for navigating a robotic endoscopic apparatus ("… one aspect of the present disclosure is to determine three dimensional positions of features in the fluoroscopic video, such as three dimensional catheter position ..." [0011]; "… including robotic minimally invasive surgery." [0026]) comprising:
(a) acquiring one or more fluoroscopic images using a fluoroscopic imager, wherein the one or more fluoroscopic images contain an image of a fiducial marker, a body part and a portion of the robotic endoscopic apparatus ("In a step 900, a sequence of images of the target area and of a structure of markers is acquired via a fluoroscopic imaging device." [0098]; "Thus, the sequence of images and consequently the fluoroscopic-based three-dimensional volumetric data may also include a projection of the medical device in addition to the target." [0101]) placed inside of the body part ("In an optional step 930, a medical device may be positioned in the target area prior to the acquiring of the sequence of images." [0101]), wherein the fiducial marker is a three-dimensional (3D) marker ("The structure of markers may include a plurality of at least partially radio-opaque markers arranged in a certain pattern." [0098]);
(b) estimating a pose of the fluoroscopic imager based at least in part on an image of the fiducial marker in the one or more fluoroscopic images ("In a step 910, a pose of the fluoroscopic imaging device for at least a plurality of images of the sequence of images may be estimated. The pose estimation may be performed based on detection of a possible and most probable projection of the structure of markers, as a whole, on each image of the plurality of images." [0099]); and
(c) reconstructing a 3D fluoroscopic image based at least in part on the pose estimated in (b) ("In a step 920, a fluoroscopic-based three-dimensional volumetric data of the target area may be constructed based on the estimated poses of the fluoroscopic imaging device according to the disclosed systems and methods." [0100]).
Regarding claim 2, Barak discloses all claim limitations, as applied in claim 1, and further discloses wherein the fiducial marker is deployed to a surgical field ("The structure of markers is positioned externally to the patient, e.g., under the patient, while capturing the images." [0098]) and wherein at least one of the one or more fluoroscopic images is acquired at an angle that is substantially aligned to a patient bed in the surgical field ("In step 407, the fluoroscopic imaging device is used to capture a video of about a 30° rotation of the imaging device 110 about the patient …" [0080]; see also Fig.1; the 0° position is aligned to patient bed).
Regarding claim 5, Barak discloses all claim limitations, as applied in claim 1, and further discloses wherein the robotic endoscopic apparatus is disposable ("As can be appreciated a medical instrument such as a biopsy tool or an energy device, such as a microwave ablation catheter …" [0109]; many biopsy tools are disposable with reusable handle and disposable tip to comply with biosafety regulation).
Regarding claim 6, Barak discloses all claim limitations, as applied in claim 1, and further discloses confirming the portion of the robotic endoscopic apparatus is inside a target tissue in the body part based on the 3D fluoroscopic image ("The offset (i.e., Δx, Δy and Δz) between the medical device and the target may be then determined based on the fluoroscopic-based three-dimensional volumetric data. The target may be visible or better exhibited in the generated three-dimensional volumetric data ... The medical device may be detected, automatically or manually by a user, in the sequence of images, as captured, or in the generated three-dimensional volumetric data." [0101]).
Regarding claim 7, Barak discloses all claim limitations, as applied in claim 1, and further discloses updating a location of a target tissue based on the 3D fluoroscopic image ("The target may be visible or better exhibited in the generated three-dimensional volumetric data. Therefore, the target may be detected, automatically, or manually by the user, in the three-dimensional volumetric data." [0101]).
Regarding claim 8, Barak discloses all claim limitations, as applied in claim 1, and further discloses wherein the fiducial marker comprises a first 3D marker and a second 3D marker ("In one example, four radio-opaque markers are utilized. However, less than four or more than four radio-opaque markers may be used." [0073]).
Regarding claim 13, Barak discloses all claim limitations, as applied in claim 1, and further discloses wherein the fiducial marker includes a first component located at a patient bed and a second component located on a patient ("The structure of markers is positioned externally to the patient, e.g., under the patient" [0098]; "… and with reference to FIG. 1, transmitter mat 56 may be incorporated with the structure of markers." [0103]; see Fig.1; patient is on the table and some markers are in contact with patient, and the pad 56 is located on table).
Regarding claim 14, Barak discloses all claim limitations, as applied in claim 1, and further discloses receiving a measured pose of the fluoroscopic imager based on location sensor data ("… fluoroscopic imaging device 110 includes an angle measurement device 111 which is configured to measure the angle of the fluoroscopic imaging device 110 relative to the patient “P.” Angle measurement device 111 may be an accelerometer." [0050]) and fusing the measured pose with the pose estimated in (b) ("… a method for constructing a three dimensional volume using either a single radio-opaque marker placed proximate the target … in conjunction with a fluoroscope angle measurement device." [0087]; "… the calibration data from step 501 is utilized in combination with measurements from an external angle measurement device to compute the fluoroscopic imaging device location (origin-less) in world coordinates." [0090]).
Regarding claim 15, Barak discloses a non-transitory computer-readable storage medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations ("The method of FIG. 9A, or a portion of it, may be in the form of instructions executed by a computing device, such as computing device 125 of FIG. 1. The computing device may include one or more hardware processors, one or more memories or storage devices and a display. The one or more hardware processors may be configured to execute the steps of this method. The one or more memories or storage devices may be configured to store these instruction and/or the fluoroscopic image data." [0107]) comprising:
(a) acquiring one or more fluoroscopic images using a fluoroscopic imager, wherein the one or more fluoroscopic images contain an image of a fiducial marker, a body part and a portion of the robotic endoscopic apparatus ("In a step 900, a sequence of images of the target area and of a structure of markers is acquired via a fluoroscopic imaging device." [0098]; "Thus, the sequence of images and consequently the fluoroscopic-based three-dimensional volumetric data may also include a projection of the medical device in addition to the target." [0101]) placed inside of the body part ("In an optional step 930, a medical device may be positioned in the target area prior to the acquiring of the sequence of images." [0101]), wherein the fiducial marker is a three-dimensional (3D) marker ("The structure of markers may include a plurality of at least partially radio-opaque markers arranged in a certain pattern." [0098]);
(b) estimating a pose of the fluoroscopic imager based at least in part on an image of the fiducial marker in the one or more fluoroscopic images ("In a step 910, a pose of the fluoroscopic imaging device for at least a plurality of images of the sequence of images may be estimated. The pose estimation may be performed based on detection of a possible and most probable projection of the structure of markers, as a whole, on each image of the plurality of images." [0099]); and
(c) reconstructing a 3D fluoroscopic image based at least in part on the pose estimated in (b) ("In a step 920, a fluoroscopic-based three-dimensional volumetric data of the target area may be constructed based on the estimated poses of the fluoroscopic imaging device according to the disclosed systems and methods." [0100]).
Regarding claim 16, Barak discloses all claim limitations, as applied in claim 15, and further discloses wherein the fiducial marker is deployed to a surgical field ("The structure of markers is positioned externally to the patient, e.g., under the patient, while capturing the images." [0098]) and wherein at least one of the one or more fluoroscopic images is acquired at an angle that is substantially aligned to a patient bed in the surgical field ("In step 407, the fluoroscopic imaging device is used to capture a video of about a 30° rotation of the imaging device 110 about the patient …" [0080]; see also Fig.1; the 0° position is aligned to patient bed).
Regarding claim 18, Barak discloses all claim limitations, as applied in claim 15, and further discloses wherein the fiducial marker comprises a first 3D marker and a second 3D marker ("In one example, four radio-opaque markers are utilized. However, less than four or more than four radio-opaque markers may be used." [0073]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 3, 4, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Barak, as applied in claim 1 and 15 respectively, and further in view of Grzeszczuk et al. (US 6,714,810 B2; published on 03/30/2004) (hereinafter "Grzeszczuk").
Regarding claim 3, Barak teaches all claim limitations, as applied in claim 1, except wherein the fiducial marker is an anatomical structure in the body part.
However, in the same field of endeavor, Grzeszczuk teaches wherein the fiducial marker is an anatomical structure in the body part ("... e.g., implanted fiducials or distinctive bone features, used as landmarks in the fluoroscopic images, e.g., features a, b. Alternatively, the program may automatically identify the landmark features by distinctive density values or density patterns, as above." Col.6, Ln.61 - Col.7, Ln.2; here bone features is equivalent to fiducial in coordinates registration).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to replace the fiducial markers as taught by Barak with bone features used as landmarks as taught by Grzeszczuk. Simple substitution of one known element for another to obtain predictable results would be obvious in the art. See MPEP 2141 and Sakraida v. AG Pro, Inc. at 417, 82 USPQ2d at 1395-96.
Regarding claim 4, Barak in view of Grzeszczuk teaches all claim limitations, as applied in claim 3, and Grzeszczuk further teaches wherein the fiducial marker is a bone structure ("... e.g., implanted fiducials or distinctive bone features, used as landmarks in the fluoroscopic images, e.g., features a, b. Alternatively, the program may automatically identify the landmark features by distinctive density values or density patterns, as above." Col.6, Ln.61 - Col.7, Ln.2; here bone features is equivalent to fiducial in coordinates registration).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to replace the fiducial markers as taught by Barak with bone features used as landmarks as taught by Grzeszczuk. Simple substitution of one known element for another to obtain predictable results would be obvious in the art. See MPEP 2141 and Sakraida v. AG Pro, Inc. at 417, 82 USPQ2d at 1395-96.
Regarding claim 17, Barak teaches all claim limitations, as applied in claim 1, except wherein the fiducial marker is a bone structure.
However, in the same field of endeavor, Grzeszczuk teaches wherein the fiducial marker is a bone structure ("... e.g., implanted fiducials or distinctive bone features, used as landmarks in the fluoroscopic images, e.g., features a, b. Alternatively, the program may automatically identify the landmark features by distinctive density values or density patterns, as above." Col.6, Ln.61 - Col.7, Ln.2; here bone features is equivalent to fiducial in coordinates registration).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to replace the fiducial markers as taught by Barak with bone features used as landmarks as taught by Grzeszczuk. Simple substitution of one known element for another to obtain predictable results would be obvious in the art. See MPEP 2141 and Sakraida v. AG Pro, Inc. at 417, 82 USPQ2d at 1395-96.
Claim 9, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Barak, as applied in claim 8 and 18 respectively, and further in view of Tzeisler et al. (US 2020/0046436 A1; published on 02/13/2020) (hereinafter "Tzeisler").
Regarding claim 9, Barak teaches all claim limitations, as applied in claim 8, except wherein the pose of the fluoroscopic imager is estimated based on a first pose estimated using the first 3D marker and a second pose estimated using the second 3D marker.
However, in the same field of endeavor, Tzeisler teaches wherein the pose of the fluoroscopic imager is estimated based on a first pose estimated using the first 3D marker and a second pose estimated using the second 3D marker ("... extracting a plurality of image features to estimate a relative pose change, wherein the plurality of image features comprises anatomical elements, non-anatomical elements, or any combination thereof, wherein the image features comprise: patches attached to a patient, radiopaque markers positioned in a field of view of the second imaging modality, or any combination thereof ..." Claim 3; the above listed image features are first and second 3D makers).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the pose estimation as taught by Barak with pose estimation using multiple features as taught by Tzeisler. By adding more constrains in the estimation, it is possible "to improve the accuracy and robustness of the registration method" (see Tzeisler; [0257]).
Regarding claim 12, Barak in view of Tzeisler teaches all claim limitations, as applied in claim 9, and Tzeisler further teaches wherein the first fiducial marker is an anatomical structure and the second fiducial marker is an artificial 3D marker ("... extracting a plurality of image features to estimate a relative pose change, wherein the plurality of image features comprises anatomical elements, non-anatomical elements, or any combination thereof, wherein the image features comprise: patches attached to a patient, radiopaque markers positioned in a field of view of the second imaging modality, or any combination thereof ..." Claim 3; the above listed image features are first and second 3D makers).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the pose estimation as taught by Barak with pose estimation using multiple features as taught by Tzeisler. By adding more constrains in the estimation, it is possible "to improve the accuracy and robustness of the registration method" (see Tzeisler; [0257]).
Regarding claim 19, Barak teaches all claim limitations, as applied in claim 18, except wherein the pose of the fluoroscopic imager is estimated based on a first pose estimated using the first 3D marker and a second pose estimated using the second 3D marker.
However, in the same field of endeavor, Tzeisler teaches wherein the pose of the fluoroscopic imager is estimated based on a first pose estimated using the first 3D marker and a second pose estimated using the second 3D marker ("... extracting a plurality of image features to estimate a relative pose change, wherein the plurality of image features comprises anatomical elements, non-anatomical elements, or any combination thereof, wherein the image features comprise: patches attached to a patient, radiopaque markers positioned in a field of view of the second imaging modality, or any combination thereof ..." Claim 3; the above listed image features are first and second 3D makers).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the pose estimation as taught by Barak with pose estimation using multiple features as taught by Tzeisler. By adding more constrains in the estimation, it is possible "to improve the accuracy and robustness of the registration method" (see Tzeisler; [0257]).
Claim 10, 11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Barak in view of Tzeisler, as applied in claim 9 and 19 respectively, and further in view of Fichtinger et al. (US 2008/0262345 A1; published on 10/23/2008) (hereinafter "Fichtinger").
Regarding claim 10, Barak in view of Tzeisler teaches all claim limitations, as applied in claim 9, except wherein the pose of the fluoroscopic imager is a weighted average of the first pose and the second pose.
However, in the same field of endeavor, Fichtinger teaches wherein the pose of the fluoroscopic imager is a weighted average of the first pose and the second pose ("Also, in another exemplary embodiment, the software selectively retrieves a plurality of fine pose estimations and their corresponding Euclidean distances, and calculates a weighted average of the selected fine pose estimations, wherein the respective Euclidean distance corresponds to a weighting factor or coefficient for the weighted average." [0100]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the pose estimation algorithm as taught by Barak with pose estimation algorithm as taught by Fichtinger. By iteratively performing pose estimation, it "may not only improve accuracy but may also reduce the required accuracy of the segmentation step" (see Fichtinger; [0119]).
Regarding claim 11, Barak in view of Tzeisler teaches all claim limitations, as applied in claim 9, except wherein the pose of the fluoroscopic imager is estimated using an optimization algorithm and wherein the first pose or the second pose is used as an initial solution of the optimization algorithm.
However, in the same field of endeavor, Fichtinger teaches wherein the pose of the fluoroscopic imager is estimated using an optimization algorithm and wherein the first pose is used as an initial solution of the optimization algorithm ("Referring to FIG. 4, “basic” algorithm 400 is an iterative process in which a pose estimation is iteratively computed until it converges on a solution that is within a pre-defined acceptable error threshold." [0074]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the pose estimation algorithm as taught by Barak with pose estimation algorithm as taught by Fichtinger. By iteratively performing pose estimation, it "may not only improve accuracy but may also reduce the required accuracy of the segmentation step" (see Fichtinger; [0119]).
Regarding claim 20, Barak in view of Tzeisler teaches all claim limitations, as applied in claim 19, except wherein the pose of the fluoroscopic imager is a weighted average of the first pose and the second pose.
However, in the same field of endeavor, Fichtinger teaches wherein the pose of the fluoroscopic imager is a weighted average of the first pose and the second pose ("Also, in another exemplary embodiment, the software selectively retrieves a plurality of fine pose estimations and their corresponding Euclidean distances, and calculates a weighted average of the selected fine pose estimations, wherein the respective Euclidean distance corresponds to a weighting factor or coefficient for the weighted average." [0100]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the pose estimation algorithm as taught by Barak with pose estimation algorithm as taught by Fichtinger. By iteratively performing pose estimation, it "may not only improve accuracy but may also reduce the required accuracy of the segmentation step" (see Fichtinger; [0119]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAO SHENG whose telephone number is (571)272-8059. The examiner can normally be reached Monday to Friday, 8:30 am to 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M. Kozak can be reached at (571) 270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHAO SHENG/ Primary Examiner, Art Unit 3797