DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Acknowledgement of Amendment
The following office action is in response to the applicant’s amendment filed on 06/30/2026. Claims 1-2, 4-5, 8-11, 13, 15-16, 18-22, 29, and 31-33 are pending. Claims 1, 4-5, 8-9, 11, 20, 22, 29 and 31 are amended. Claims 43-45 are newly added. Claims 1-2, 4-5, 8-11, 13, 15-16, 18-22, 29, 31-33, and 43-45 are rejected under 35 U.S.C. 103 for the reasons stated in the Response to Arguments and 35 U.S.C. 103 sections below.
Response to Arguments
Applicant’s arguments, see Remarks page 10, filed 06/30/2026, with respect to the objections to the specification have been fully considered and are persuasive. The objections to the specification in the non-final rejection of 04/01/2026 has been withdrawn.
Applicant’s arguments, see Remarks page 10-11, filed 06/30/2026, with respect to the rejection of the claims under 3 U.S.C. 112(b) have been fully considered and are persuasive.
The examiner acknowledges that claims 1, 29, and 31 have been amended to remove the term “facilitating” and instead recite steps of “generating control instructions for electronically guiding […]” and “controlling navigation of a medical tool”. Therefore, it is clear how the steps of the method are being carried out by the non-transitory computer-readable storage medium/medical targeting system.
The rejection of the claims under 35 U.S.C. 112(b) in the non-final rejection of 04/01/2026 has been withdrawn.
Applicant’s arguments, see Remarks page 11-14, filed 06/30/2026, with respect to the rejection of the claims under 35 U.S.C. 102 and 35 U.S.C. 103 have been fully considered and are persuasive.
Regarding claim 1, the claim has been amended to recite, in part, “generating control instructions for electronically guiding the real-time imaging device through one or more anatomical channels to a vicinity of the initial virtual target position using the mapping; when the real-time imaging device is at the vicinity of the initial virtual target position, detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target; controlling navigation of a medical tool to a position proximate to the physical target based at least in part on the calibrated virtual target position; and deploying a needle of the medical tool through the wall of the anatomical channel using the calibrated virtual target position”.
As discussed during the Examiner interview on June 29, 2026, the cited references do not disclose or suggest at least the steps of "detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope," "determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target," "controlling navigation of a medical tool to a position proximate to the physical target based at least in part on the calibrated virtual target position," and "deploying a needle of the medical tool through the wall of the anatomical channel using the calibrated virtual target position," as claimed.
The applicant notes, and the examiner agrees, that Averbuch generally discloses techniques for using "imagery obtained using a radial endobronchial ultrasound ("REBUS") probe to improve the clinical output of endobronchial procedures." Averbuch at [0037]. The Office Action cites disclosure in which "a REBUS probe is navigated to an area of interest near the target," and "fine registration is implemented to find the best fit between each of the features or patterns... and area of interest on the intraoperative image." Averbuch at [0047], [0050]. The Applicant argues and the examiner agrees that Averbuch does not disclose using "near-infrared fluorescent light projected by a fiberscope" to detect "distance...based on a time-of-flight measurement," nor does it disclose "determining a calibrated virtual target position...based on the distance," or "deploying a needle...through the wall of the anatomical channel using the calibrated virtual target position," as now claimed.
Therefore, the examiner agrees that Averbuch fails to anticipate claim 1 and that Averbuch does not anticipate claims 5, 8-10, 29, and 31 for at least the same reasons.
The Applicant argues that the remaining references, cited for various features of the dependent claims, fail to disclose or suggest at least the same limitations absent from Averbuch discussed above. For example, while Nie generally discloses use of "near infrared light" as one possible modality for aiding guidance in a diagnostic or therapeutic procedure, it does not disclose, alone or in combination with the other cited references, the specific claimed workflow involving "detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope," "determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target," "controlling navigation of a medical tool to a position proximate to the physical target based at least in part on the calibrated virtual target position," and "deploying a needle of the medical tool through the wall of the anatomical channel using the calibrated virtual target position," as claimed.
For at least the reasons above, the claims are patentable over the combination of references. Therefore, the rejections should be withdrawn.
The examiner respectfully acknowledges that Nie discloses the use of "near infrared light" as one possible modality for aiding guidance in a diagnostic or therapeutic procedure, it does not disclose, alone or in combination with the other cited references, the specific claimed workflow involving "detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope," "determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target," "controlling navigation of a medical tool to a position proximate to the physical target based at least in part on the calibrated virtual target position," and "deploying a needle of the medical tool through the wall of the anatomical channel using the calibrated virtual target position," as claimed.
Therefore, the rejection of the claims under 35 U.S.C. 102 and 35 U.S.C. 103 in the non-final rejection of 04/04/2026 have been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Shelton, IV et al. US 2021/0205019 A1 “Shelton” as discussed in the 35 U.S.C. 103 section below.
Applicant’s arguments, see Remarks page 14-17, filed 06/30/2026, with respect to the rejection of the claims under 35 U.S.C. 101 have been fully considered and are persuasive.
1. The Amended Claims are not Directed to an Abstract Idea Under Step 2A of the
Alice /Mayo Test:
The Applicant notes that the Office Action indicates that various limitations of the previously presented independent claims correspond to a "mental process" and therefore recite an abstract idea. Because the claims are now substantially amended from those previously characterized as an "abstract idea," the prior rejection is now moot and should be reconsidered by the Examiner.
Applicant notes that the amended claims include significant elements that cannot be properly characterized as a "mental process" because they cannot practically be performed in the human mind. See MPEP § 2106.04(a)(2)(III) ("claims do not recite a mental process when they do not contain limitations that can practically be performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations."). For example, the amended claims recite steps such as "generating control instructions for electronically guiding the real-time imaging device...", "controlling navigation of a medical tool...," and "deploying a needle of the medical tool..." These steps are inherently performed by electronic systems in a physical environment and cannot be characterized as a "mental process" that can be performed in the human mind. Furthermore, at least these steps are not limited to "generic computer components," but instead involve specialized elements such as a "a fiberscope," a "medical tool," and "a needle."
Any remaining steps that could potentially be characterized as an "abstract idea" are integrated into a practical application because they represent an improvement in the functioning of a computer, or an improvement to another technology or technical field, as discussed in MPEP §§ 2106.04(d)(1) and 2106.05(a). For example, the amended claims provide a technological improvement in the performance of electronically guided medical procedures.
This improvement is supported by the specification and reflected in the claims. For example, the specification explains several problems in the field including that the "registration process [between preprocedural and intraprocedural imaging] may suffer from inaccuracies due to anatomical changes in the patient between the preprocedural image scan and the procedure, noise in the electromagnetic field affecting the electromagnetic sensors, or other factors." Specification at [0002]. Furthermore, the specification explains that in such procedures, it is desirable to "enable discernment between a tumor and obscuring tissue, and estimation of relative distances between surfaces of the tumor and obscuring tissue," and "to enable precise deployment of the intervention tool 180 to the target." Specification at [0039], [0054]. The specification provides a solution including employing a step for "calibrating registration of the real-time images 160 and preprocedural images 150" (Specification at [0065]), performing "time-of-flight computation...to estimate the distance from the probe tip 502 to the surface of the tumor 510," (Specification at [0080]) where the target may be "on the other side of the airway wall from where the NIRF fiberscope is located," (Specification at [0045]), and "deploy[ing] [a needle] to the calibrated virtual target position" where the needle "can traverse through an anatomical channel." Specification at [0060].
The improvement is reflected in the claims. For example, the amended claims recite "detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope," "determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target," "controlling navigation of a medical tool to a position proximate to the physical target based at least in part on the calibrated virtual target position," and "deploying a needle of the medical tool through the wall of the anatomical channel using the calibrated virtual target position."
The claimed steps are therefore directed to a technical improvement that integrates the alleged abstract idea into a practical application. For at least the reasons above, the claims should be found patent eligible under Step 2A of the Alice / Mayo test.
The examiner respectfully agrees that as amended, the claims include significant elements that cannot be properly characterized as a "mental process" because they cannot practically be performed in the human mind. For example, the amended claims recite steps such as "generating control instructions for electronically guiding the real-time imaging device...", "controlling navigation of a medical tool...," and "deploying a needle of the medical tool..." These steps are inherently performed by electronic systems in a physical environment and cannot be characterized as a "mental process" that can be performed in the human mind. Furthermore, at least these steps are not limited to "generic computer components," but instead involve specialized elements such as a "a fiberscope," a "medical tool," and "a needle."
Furthermore, the examiner acknowledges that any remaining steps that could potentially be characterized as an "abstract idea" are integrated into a practical application because they represent an improvement in the functioning of a computer, or an improvement to another technology or technical field, as discussed in MPEP §§ 2106.04(d)(1) and 2106.05(a). For example, the amended claims provide a technological improvement in the performance of electronically guided medical procedures.
Thus the examiner agrees that the claimed steps are directed to a technical improvement that integrates the alleged abstract idea into a practical application.
Furthermore, the Applicant notes and the examiner agrees that analysis under Step 2B is not needed because the subject matter is patent eligible under Step 2A for the reasons above. Nevertheless, an analysis under Step 2B should find that the claims are patent eligible because they recite significantly more than an abstract idea. For the same reasons described above, the claims are directed to a technical improvement in the performance of electronically guided medical procedures. The claimed steps, when considered as an ordered combination, provide significantly more than the abstract idea and therefore also render the claims patent eligible under Step 2B of 35 U.S.C. § 101.
Therefore, the rejection of the claims under 35 U.S.C. 101 in the non-final rejection of 04/01/2026 have been withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5, 8-9, 11, 29, and 31-32 is/are rejected under 35 U.S.C. 103 as being unpatentable by Averbuch US 2020/0170623 A1 “Averbuch” and further in view of Shelton, IV et al. US 2021/0205019 A1 “Shelton”.
Regarding claims 1, 29 and 31, Averbuch teaches “A method for performing guidance to a physical target, the method comprising:” (Claim 1) (“FIG. 1 is a flowchart of an exemplary method for using endobronchial ultrasound imagery to provide localization of targets” [0014]; “FIG. 2 is a flowchart of an exemplary method for performing a portion of the method shown in FIG. 1” [0015]. As shown in FIG. 1, the method involves receiving at least one preoperative image, receiving a section of an area of interest, receiving at least one intraoperative image, highlighting a region of interest on the intraoperative image and navigating a REBUS probe to the area of interest (see steps 110-150). Therefore, the methods shown in FIGS. 1 and 2 represent a method for performing guidance to a physical target (i.e. area/region of interest within a patient).);
“A non-transitory computer-readable storage medium storing instructions for performing guidance to a physical target, the instructions when executed by one or more processors causing the one or more processors to perform steps including:” (Claim 29) (See FIGS. 1 and 2. In order to carry out the processes shown in FIGS. 1 and 2, the device must access instructions stored in a non-transitory computer-readable storage medium which are executed by one or more processors causing the one or more processors to perform guidance to a physical target (i.e. area/region of interest within a patient). Thus, Averbuch inherently includes a non-transitory computer-readable storage medium.);
“A medical targeting system comprising: a real-time imaging device; a medical tool; one or more processors; and a non-transitory computer-readable storage medium storing instructions for performing guidance to a physical target, the instructions when executed by the one or more processors causing the one or more processors to perform steps including:” (Claim 31) (See FIGS. 1 and 2 and “In an embodiment, the step of projecting the two-dimensional projection of the three-dimensional representation of the element on the at least one of the at least one intraoperative image is performed in real time” [0005]; “In an embodiment, a method includes removing the radial endobronchial ultrasound probe from the area of interest; and navigating a further endobronchial tool to the area of interest. In an embodiment, a method includes performing a procedure on the element using the further endobronchial tool” [0006]; “Referring back to FIG. 1, in step 150, a REBUS probe is navigated to an area of interest near the target. In some embodiments, navigation is accomplished through the use of enhanced imagery as generated by the steps described above. In step 160, a sequence of REBUS images is acquired as the REBUS probe is moved along the bronchial passageway. As noted above, in some embodiments, each such REBUS image represents a cross-sectional “slice” of the patient's tissue” [0050]; “The exemplary embodiments may provide the physician with REBUS images during the time when the REBUS probe is retracted and replaced by other endobronchial tool, such as biopsy forceps or ablation probe” [0063]. In order to carry out the method shown in FIGS. 1 and 2, a medical targeting system must be present. Therefore, Averbuch inherently discloses a medical targeting system comprising a real-time imaging device (i.e. REBUS probe); a medical tool (i.e. endobronchial tool, such as biopsy forceps or ablation probe); one or more processors; and a non-transitory computer-readable storage medium storing instructions for performing guidance to a physical target (i.e. area/target of interest within a patient), the instructions when executed by the one or more processors causing the one or more processors to perform multiple steps.);
“obtaining preprocedural images representing an anatomy” (Claims 1, 29, and 31) (“In some embodiments, a non-limiting example of target reconstruction can be performed in accordance with the method 100 shown in FIG. 1. In step 110, at least one preoperative image is acquired. In some embodiments, the preoperative image is a two-dimensional image. In some embodiments, the preoperative image is a three-dimensional image. In some embodiments, the preoperative image is any known suitable type of medical image (e.g., a computed tomography (“CT”) image)” [0040]. Therefore, the method carried out by the system involves obtaining preprocedural images representing an anatomy.);
“obtaining an initial virtual target position of the physical target in the preprocedural images” (Claims 1, 29, and 31) (“In step 120, a selection of an area of interest on the preoperative image is received. The area of interest may be, for example, a lesion” [0040]; “In some embodiments, the steps 100-140 are performed in accordance with the process shown in FIG. 2” [0043]; “Referring now to FIG. 2, a method 200 begins at step 210, in which a selection of an area of interest on a preoperative image, such as a CT or MRI image, is received from a user. In step 220, the volume of interest is generated the preoperative image. In some embodiments, the volume is generated in such a way that the anatomical structures in the area of interest, such as a lesion, and adjunctive anatomical structures such as bronchi or blood vessels, will be detectable on an operative image, such as fluoroscopic image” [0044]. Therefore, the method carried out by the system involves obtaining an initial virtual target position (i.e. selection of area of interest from a user) of the physical target (i.e. lesion) in the preoperative images.);
“obtaining, from a real-time imaging device, real-time intraprocedural images of the anatomy” (Claims 1, 29, and 31) (See [0005] and [0050] as discussed above and “In step 130, intraoperative images (i.e., images acquired during a procedure) are received. In some embodiments, the intraoperative images are two-dimensional images. In some embodiments, the intraoperative images are three-dimensional images. In some embodiments, the intraoperative images are any known suitable type of medical images (e.g., fluoroscope images such as X-ray images)” [0041]; “In step 230, at least one intraoperative image o is received. In an embodiment, the pose of the intraoperative modality is calculated or recorded with the at least one intraoperative image” [0044]. In this case, the intraoperative images are acquired by the REBUS probe in real-time (see [0005], [0050]). Therefore, the method carried out by the system involves obtaining, from a real-time imaging device (i.e. REBUS probe), real-time intraprocedural images of the anatomy.);
“registering the real-time intraprocedural images of the anatomy to the preprocedural images to generate a mapping between coordinates of the real-time intraprocedural images and coordinates of the preprocedural images” (Claims 1, 29, and 31) (“ Continuing to refer to FIG. 2, in step 240, coarse registration between the intraoperative and preoperative images is performed, e.g., but not limited to, fluoroscopy to DDR, to evaluate a viewpoint of DDR inside a preoperative image data, such as, but not limited to, CT volume. In some embodiments, the coarse registration of step 240 is performed by applying an iterative optimization method on a viewpoint representation vector x” [0045]. Therefore, the method carried out by the system involves registering the real-time intraprocedural images of the anatomy to the preprocedural images to generate a mapping between coordinates of the real-time intraprocedural images and coordinates of the pre-procedural images.);
“generating control instructions for electronically guiding the real-time imaging device through one or more anatomical channels to a vicinity of the initial virtual target position using the mapping” (Claims 1, 29, and 31) (“In step 140, a region of interest is highlighted in the intraoperative images” [0042]; “Referring back to FIG. 1, in step 150, a REBUS probe is navigated to an area of interest near the target. In some embodiments, navigation is accomplished through the use of enhanced imagery as generated by the steps described above” [0050]. Therefore, since the region of interest is highlighted in the intraoperative images and the REBUS probe is navigated to an area of interest near the target after the region of interest is highlighted (see steps 140 and 150 in FIG. 1) the method carried out by the system involves generating control instructions for electronically guiding the real-time imaging device through one or more anatomical channels to a vicinity of the initial virtual target position using the mapping.).; […]
“controlling navigation of a medical tool to a position proximate to the physical target based at least in part on the calibrated virtual target position; and deploying a needle of the medical tool through the wall of the anatomical channel using the calibrated virtual target position” (Claims 1, 29, and 31) (See [0063] as discussed above and “4. Position the endobronchial tool in the target center and perform an ablation or biopsy” [0070]. A biopsy is facilitated, at least in part, through the use of a needle. Therefore, the method carried out by the system involves controlling navigation of a medical tool to a position proximate to the physical target based at least in part on the calibrated virtual target position; and deploying a needle of the medical tool through the wall of the anatomical channel using the calibrated virtual target position.).
Averbuch does not teach “when the real-time imaging device is at the vicinity of the initial virtual target position, detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target” (Claims 1, 29 and 30).
Shelton is within the same field of endeavor as the claimed invention because it involves a surgical system which utilizes near-infrared light to sense distance to a critical anatomical structure (See [0026]).
Shelton teaches “when the real-time imaging device is at the vicinity of the initial virtual target position, detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target” (Claims 1, 29 and 30) (“FIG. 14 is a schematic of a near infrared (NIR) time-of-flight measurement system configured to sense distance to a critical anatomical structure, the time-of-flight measurement system including a transmitter (emitter) and a receiver (sensor) positioned on a common device, according to at least one aspect of the present disclosure” [0026]; “The foregoing application of spectral imaging to tissue can be utilized intraoperatively to measure the distance between a waveform emitter and a critical structure that is obscured by tissue. In one aspect of the present disclosure, referring now to FIGS. 14 and 15, a time-of-flight sensor system 1104 utilizing waveforms 1124, 1125 is shown. [0193]; “The waveforms 1124, 1125 are configured to penetrate obscuring tissue 1103. For example, the wavelengths of the waveforms 1124, 1125 can be in the NIR or SWIR spectrum of wavelengths” [0194]; “Based on the delay between the emitted wave 1124 and the received wave 1125, the time-of-flight sensor system 1104 is configured to determine the distance d (FIG. 14)” [0195].
Therefore, since spectral imaging is utilized to intraoperatively (i.e. in real-time) measure the distance between a waveform emitter and a critical structure that is obscured by tissue and the time-of-flight sensor system 1104 utilizes waveforms 1124, 112 which can be in the NIR (i.e. near-infrared spectrum) to determine this distance, Shelton discloses a system which performs the step of, when the real-time imaging device is at the vicinity of the initial virtual target position (i.e. corresponding to the critical structure 1101, shown in FIG. 14), detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method, non-transitory computer-readable storage medium and medical targeting system of Averbuch such that, when the real-time imaging device is at the vicinity of the initial virtual target position, they involve: 1) detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; 2) determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target as disclosed in Shelton in order to accurately determine the location of a critical structure (i.e. physical target) intraoperatively (see Shelton: [0134]). Detecting a distance to a physical target beyond a wall of an anatomical channel (i.e. embedded within an organ, for example) with a time-of-flight measurement of near-infrared fluorescent light is one of a finite number of techniques which can be used to effectively distinguish the location of a physical target (i.e. critical structure) with a reasonable expectation of success. Thus, modifying the method, non-transitory computer-readable storage medium and medical targeting system of Averbuch such that they involve: 1) detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; 2) determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target as disclosed in Shelton would yield the predictable result of accurately determining the location of a critical structure (i.e. physical target) intraoperatively (see Shelton: [0134]).
Regarding claim 5, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above, and Averbuch further teaches “wherein determining the calibrated virtual target position further comprises: receiving user inputs to control position of the real-time imaging device to capture at least two images of the physical target from different viewpoints” (“In step 160, a sequence of REBUS images is acquired as the REBUS probe is moved along the bronchial passageway. As noted above, in some embodiments, each such REBUS image represents a cross-sectional “slice” of the patient's tissue. FIGS. 3A-3H show a representative set of REBUS images.” [0050]. In this case, the REBUS probe is moved along the bronchial passageway in response to a user providing an input to control its position. Therefore, the step of determining the calibrated virtual target position comprises: receiving user inputs to control position of the real-time imaging device (i.e. REBUS probe) to capture at least two images of the physical target from different viewpoints (i.e. REBUS images shown in FIGS. 3A-3H); and
“determining the position of the physical target based at least in part on the at least two images” (“In step 170, for each REBUS image from the sequence, the target contour is extracted from the ultrasound image relatively to the REBUS probe tip which is detected both on REBUS image and in the intraoperative image” [0050]. The target contour, in this case, represents the physical target. Therefore, since the target contour is extracted from each of the REBUS images, the method involves determining a position of the physical target based at least in part on the at least two images (i.e. REBUS images acquired in the sequence, see [0050]).).
Regarding claim 8, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above, and Averbuch further teaches “wherein controlling the navigation of the medical tool to the physical target comprises: generating control signals to […] navigate the medical tool proximate to the calibrated virtual target position” (“FIGS. 7A and 7B show images generated using an embodiment of the methods of the present invention, and show a real time, localized view with augmentation using radial EBUS. The dotted circle defines the target area in FIG. 7A. The white outlined portion in FIG. 7A is the target defined using the methods described herein. The arrow points to the region which has been determined to be the target using the methods described herein” [0074]; “In an embodiment, the further endobronchial tool is a biopsy instrument or an ablation catheter” [0010]. An ablation catheter inherently requires electronic assistance (i.e. electricity, for example) in order to perform an ablation procedure. Therefore, since the image shown in FIG. 7A is displayed to the user, the step of controlling the navigation of the medical tool to the physical target comprises: generating control signals to navigate the medical tool (i.e. ablation catheter, see [0010], for example) proximate to the calibrated virtual target position (i.e. target represented by arrow, and white outlined portion in FIG. 7A).).
Shelton further teaches that navigation is performed “robotically” (“In one aspect, the surgical visualization system 100 may be incorporated into a robotic system 110. For example, the robotic system 110 may include a first robotic arm 112 and a second robotic arm 114. […] The first robotic arm 112 is configured to maneuver the surgical device 102, and the second robotic arm 114 is configured to maneuver the imaging device 120. A robotic control unit can be configured to issue control motions to the robotic arms 112, 114, which can affect the surgical device 102 and the imaging device 120, for example” [0156].; “Various surgical visualization systems disclosed herein can be configured to detect and identify one or more desired types of critical structures in a forward path of a surgical device, such as when the path of the surgical device is robotically controlled, for example” [0216]. Therefore, navigation of the surgical device is controlled by a robot.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Averbuch such that the navigation of the medical tool to the physical target is robotically controlled as disclosed in Shelton in order to automate the process of navigating the medical tool to the target position. When a medical tool is manually navigated through patient’s body to a target position, there is a risk of a medical professional making accidental movements which could negatively impact the patient. By robotically navigating a medical tool within the patient, these risks are reduced. Thus, modifying the method of Averbuch such that the navigation of the medical tool to the physical target is robotically controlled as disclosed in Shelton would yield the predictable result of automating the process of navigating the medical tool to the target position and therefore reduce potential risks to the patient.
Regarding claim 9, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above, and Averbuch further teaches “wherein controlling the navigation of the medical tool to the physical target comprises: displaying, on a display screen, the preprocedural images and a representation of a virtual target at the calibrated virtual target position overlaid on the preprocedural images” (“Referring now to FIG. 2, a method 200 begins at step 210, in which a selection of an area of interest on a preoperative image, such as a CT or MRI image, is received from a user. In step 220, the volume of interest is generated the preoperative image” [0044]. Therefore, since the volume of interest is generated and displayed on the preoperative image, the step of facilitating the guidance comprises: displaying, on a display screen, the preprocedural images and a representation of a virtual target at the calibrated virtual target position overlaid on the preprocedural images.); and
“displaying, on the display screen, a position of the medical tool overlaid on the preprocedural images; receiving control inputs to enable guidance of the medical tool proximate to the calibrated virtual target position” (“In some embodiments, the combination of the original signal from the intraoperative image, the simulated signal from the reference image, and any planning information can be displayed according to application configuration or upon the user request” [0049]. Although, FIGS. 5A and 7A represent intraoperative images in which the position of an interventional tool is overlaid, since the reference image (i.e. preoperative/preprocedural image) and planning information can be displayed upon user request, it would be obvious to display, on the display screen, a position of an intervention tool overlaid on the preprocedural images to enable guidance of the medical tool to the calibrated virtual target position. Thus, the method involves displaying, on the display screen, a position of the medical tool overlaid on the preprocedural images, and receiving control inputs to enable guidance of the medical tool proximate to the calibrated virtual target position.).
Regarding claim 11, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above and Shelton further teaches “wherein controlling the navigation of the medical tool to the physical target comprises: determining a distance from an medical tool to the physical target; and generating control signals to guide the medical tool based at least in part on the distance” (“(“FIG. 14 is a schematic of a near infrared (NIR) time-of-flight measurement system configured to sense distance to a critical anatomical structure, the time-of-flight measurement system including a transmitter (emitter) and a receiver (sensor) positioned on a common device, according to at least one aspect of the present disclosure” [0026]; “The foregoing application of spectral imaging to tissue can be utilized intraoperatively to measure the distance between a waveform emitter and a critical structure that is obscured by tissue. In one aspect of the present disclosure, referring now to FIGS. 14 and 15, a time-of-flight sensor system 1104 utilizing waveforms 1124, 1125 is shown. [0193]; “The waveforms 1124, 1125 are configured to penetrate obscuring tissue 1103. For example, the wavelengths of the waveforms 1124, 1125 can be in the NIR or SWIR spectrum of wavelengths” [0194]; “Based on the delay between the emitted wave 1124 and the received wave 1125, the time-of-flight sensor system 1104 is configured to determine the distance d (FIG. 14)” [0195]; “When the critical structure moves into range of the spectral imaging system, the system can identify the structure and, thus, communicate that the structure is within range. In such instances, an alert can be provided when a structure is initially identified and/or moved further within a predefined proximity zone. In such instances, even non-identification of a critical structure by a spectral imaging system with known bounds/ranges can provide proximity information (i.e. the lack of proximity) to the clinician” [0211].
Therefore, since spectral imaging is utilized to intraoperatively (i.e. in real-time) measure the distance between a waveform emitter and a critical structure that is obscured by tissue and the time-of-flight sensor system 1104 utilizes waveforms 1124, 112 which can be in the NIR (i.e. near-infrared spectrum) to determine this distance and the system can provide proximity information (see [0211]), Shelton discloses a system which performs the step of, controlling the navigation of the medical tool to the physical target by determining a distance from the medical tool to the physical target (i.e. through the use of NIR light, see [0193]-[0195]), and generating control signals (i.e. alerts, for example), to guide the medical tool based at least in part on the distance.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method, non-transitory computer-readable storage medium and medical targeting system of Averbuch such that, when the real-time imaging device is at the vicinity of the initial virtual target position, they involve: 1) detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; 2) determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target as disclosed in Shelton in order to accurately determine the location of a critical structure (i.e. physical target) intraoperatively (see Shelton: [0134]). Detecting a distance to a physical target beyond a wall of an anatomical channel (i.e. embedded within an organ, for example) with a time-of-flight measurement of near-infrared fluorescent light is one of a finite number of techniques which can be used to effectively distinguish the location of a physical target (i.e. critical structure) with a reasonable expectation of success. Thus, modifying the method, non-transitory computer-readable storage medium and medical targeting system of Averbuch such that they involve: 1) detecting a distance to the physical target beyond a wall of the anatomical channel based on a time-of-flight measurement of near-infrared fluorescent light projected by a fiberscope; 2) determining a calibrated virtual target position corresponding to a position of the physical target based on the distance to the physical target as disclosed in Shelton would yield the predictable result of accurately determining the location of a critical structure (i.e. physical target) intraoperatively (see Shelton: [0134]).
Regarding claim 32, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 31 above, and Shelton further teaches “wherein the medical tool comprises an electronically navigated intervention tool, and wherein the medical targeting system further comprises: a robotic control system for controlling electronic navigation of the electronically navigated intervention tool” (“In one aspect, the surgical visualization system 100 may be incorporated into a robotic system 110. For example, the robotic system 110 may include a first robotic arm 112 and a second robotic arm 114. […] The first robotic arm 112 is configured to maneuver the surgical device 102, and the second robotic arm 114 is configured to maneuver the imaging device 120. A robotic control unit can be configured to issue control motions to the robotic arms 112, 114, which can affect the surgical device 102 and the imaging device 120, for example” [0156].; “Various surgical visualization systems disclosed herein can be configured to detect and identify one or more desired types of critical structures in a forward path of a surgical device, such as when the path of the surgical device is robotically controlled, for example” [0216]. Therefore, the medical tool comprises an electronically navigated intervention tool (i.e. surgical device 102), and wherein the medical targeting system further comprises: a robotic control system (i.e. 110) for controlling electronic navigation of the electronically navigated intervention tool (i.e. surgical device 102).).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Averbuch such that the navigation of the medical tool to the physical target is robotically controlled as disclosed in Shelton in order to automate the process of navigating the medical tool to the target position. When a medical tool is manually navigated through patient’s body to a target position, there is a risk of a medical professional making accidental movements which could negatively impact the patient. By robotically navigating a medical tool within the patient, these risks are reduced. Thus, modifying the method of Averbuch such that the navigation of the medical tool to the physical target is robotically controlled as disclosed in Shelton would yield the predictable result of automating the process of navigating the medical tool to the target position and therefore reduce potential risks to the patient.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Averbuch US 2020/0170623 A1 “Averbuch” and Shelton, IV et al. US 2021/0205019 A1 “Shelton” as applied to claim 1 above, and further in view of Weinstein et al. US 2018/0185100 A1 “Weinstein”.
Regarding claim 2, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above. However, the combination does not teach “wherein determining the calibrated virtual target position further comprises: presenting, to a display device, a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images”; “receiving at least one user input for repositioning the virtual target and updating the display device to indicate the repositioning” and “determining the position of the physical target responsive to detecting alignment between a repositioned virtual target and the physical target”.
Weinstein is within a related field of endeavor to the claimed invention because it involves systems and methods for surgical navigation with a mixed reality visualization (see [Abstract]).
Weinstein teaches “wherein determining the calibrated virtual target position comprises: presenting, to a display device, a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images”; “receiving at least one user input for repositioning the virtual target and updating the display device to indicate the repositioning” and “determining the position of the physical target responsive to detecting alignment between a repositioned virtual target and the physical target” (“The method comprises registering the HMD coordinate system and the localizer coordinate system such that images displayed by the head-mounted display can be associated with real objects tracked by the localizer. Registration error is indicated to the user using a plurality of real calibration markers viewable by the user and a plurality of virtual calibration marker images displayed to the user. The virtual calibration marker images have a congruency with the real calibration markers so that the virtual calibration marker images are capable of being aligned with the real calibration markers whereby a magnitude of misalignment is indicative of the registration error. The registration of the HMD coordinate system and the localizer coordinate system is calibrated to reduce the registration error by receiving input from the user associated with adjusting positions of the virtual calibration marker images relative to the real calibration markers to better align the virtual calibration marker images with the real calibration markers” [0009]; “Once the user visually recognizes the registration error, the user can then fine tune or calibrate the registration of the HMD coordinate system and the localizer coordinate system LCLZ to reduce the registration error. In one embodiment, the user provides input, such as through an input device (e.g., keyboard, mouse, touchscreen, foot pedal, etc.) to the HMD controller 210 that essentially adjusts the positions of the virtual calibration marker images 238 relative to the real calibration markers 234 to better align the virtual calibration marker images 238 with the real calibration markers 234 so that the virtual calibration marker images 238 coincide with the real calibration markers 234.” [0122].
Therefore, the virtual calibration marker images are displayed to the user. Additionally, the user can adjust the positions of the virtual calibration marker images to better align with the real calibration markers. Thus, the method performs the step of determining the calibrated virtual target position comprises: presenting, to a display device, a representation of a virtual target (i.e. corresponding to the virtual calibration marker))at the initial virtual target position overlaid on the real-time intraprocedural images (i.e. images with virtual/real calibration markers); receiving at least one user input for repositioning the virtual target and updating the display device to indicate the repositioning (i.e. adjusting positions of virtual calibration marker images) and determining the position of the physical target (i.e. real calibration markers) responsive to detecting alignment between a repositioned virtual target and the physical target (i.e. such that registration error decreases).).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Averbuch in view of Shelton such that the step of determining the calibrated virtual target position comprises: presenting, to a display device, a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images; receiving at least one user input for repositioning the virtual target and updating the display device to indicate the repositioning and determining the position of the physical target responsive to detecting alignment between a repositioned virtual target and the physical target as disclosed in Weinstein in order to ensure that the virtual target (i.e. virtual calibration markers) accurately depict the location of the physical target (i.e. corresponding to the real calibration markers). Providing a user input to adjust the position of virtual calibration markers is one of a finite number of techniques which can be used to reduce registration error with a reasonable expectation of success. Thus, modifying the method of Averbuch, such that the step of determining the calibrated virtual target position comprises: presenting, to a display device, a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images; receiving at least one user input for repositioning the virtual target and updating the display device to indicate the repositioning and determining the position of the physical target responsive to detecting alignment between a repositioned virtual target and the physical target as disclosed in Weinstein would yield the predictable result of ensuring that the virtual target (i.e. virtual calibration markers) accurately depict the location of the physical target (i.e. corresponding to the real calibration markers).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Averbuch US 2020/0170623 A1 “Averbuch” and Shelton, IV et al. US 2021/0205019 A1 “Shelton” as applied to claim 1 above, and further in view of Bertram et al. US 2022/0031263 A1 “Bertram”.
Regarding claim 4, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above. However, the combination does not teach “wherein determining the calibrated virtual target position further comprises: generating a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images”; “automatically generating adjustments to the initial virtual target position and applying an image analysis until the virtual target substantially overlaps the physical target”; and “determining the position of the physical target based at least in part on the adjustments”.
Bertram is within a related field of endeavor to the claimed invention because it involves setting a virtual marker in a fluoroscopic image indicating a point or region of interest to be treated by surgery (see Bertram: [0012]).
Bertram teaches “wherein determining the calibrated virtual target position comprises: generating a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images”; “automatically generating adjustments to the initial virtual target position and applying an image analysis until the virtual target substantially overlaps the physical target”; and “determining the position of the physical target based at least in part on the adjustments” (“In one embodiment, the method further comprises the steps of changing the position of the virtual marker in the fluoroscopic image and of controlling the at least one laser according to the changed position of the virtual marker to adapt the position of the corresponding incision marker in real time. This means that the position of a virtual marker can be adjusted in the fluoroscopic image and the effect of the changed position of the virtual marker can immediately be visualized on the surface of the patient” [0041]; “In another embodiment, the virtual marker can be set by indicating a structure, for example by selecting a structure from one or more list(s) of structures. It is for example possible to indicate a structure like a landmark of a bone. This structure is then automatically found in the fluoroscopic image and the virtual marker is set accordingly” [0026].
Therefore, since the position of the virtual marker is changed in the fluoroscopic image to adapt the position of the corresponding incision marker in real-time and the structure (i.e. bone for example) is automatically found such that the virtual marker can be set accordingly, the step of determining the calibrated virtual target position comprises: generating a representation of a virtual target (i.e. virtual marker corresponding to a structure) at the initial virtual target position overlaid on the real-time intraprocedural images (i.e. fluoroscopic image); automatically generating adjustments (i.e. changes) to the initial virtual target position and applying an image analysis until the virtual target substantially overlaps the physical target (i.e. such that the virtual marker is immediately visualized on the surface of the patient); and determining the position of the physical target based at least in part on the adjustments.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Averbuch in view of Shelton such that the step of determining the calibrated virtual target position comprises: generating a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images; automatically generating adjustments to the initial virtual target position and applying an image analysis until the virtual target substantially overlaps the physical target; and determining the position of the physical target based at least in part on the adjustments as disclosed in Bertram in order to allow for adjustments to be made to the virtual marker such that a surgical procedure can be performed at the correct location within the patient. Automatically adjusting the position of a virtual target position (i.e. virtual marker) is one of a finite number of techniques which can be used to effectively target an anatomical structure with a reasonable expectation of success. Thus, modifying the method of Averbuch such that the step of determining the calibrated virtual target position comprises: generating a representation of a virtual target at the initial virtual target position overlaid on the real-time intraprocedural images; automatically generating adjustments to the initial virtual target position and applying an image analysis until the virtual target substantially overlaps the physical target; and determining the position of the physical target based at least in part on the adjustments as disclosed in Bertram would yield the predictable result of allowing for adjustments to be made to the virtual marker such that a surgical procedure can be performed at the correct location within the patient.
Claim(s) 13, 15-16, and 43-45 is/are rejected under 35 U.S.C. 103 as being unpatentable over Averbuch US 2020/0170623 A1 “Averbuch” and Shelton, IV et al. US 2021/0205019 A1 “Shelton” as applied to claim 1 above, and further in view of Butte et al. US 2021/0015350 A1 “Butte” and Gurevich et al. US 2021/0007687 A1 “Gurevich”.
Regarding claims 13 and 43, Averbuch discloses all features of the claimed invention as discussed with respect to claim 1 above. However, Averbuch teaches “wherein registering the real-time intraprocedural images to the preprocedural images comprises” (Claims 13 and 43) (See [0045] as discussed with respect to claim 1 above.).
However, Averbuch in view of Shelton does not teach “wherein the real-time imaging device comprises an endoscope for obtaining visible light images and wherein the real-time imaging device further includes the fiberscope for obtaining near-infrared fluorescent images” (Claims 13 and 43), […] “registering the visible light images from the endoscope to the preprocedural images” (Claims 13 and 43); and “registering the near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope” (Claims 13 and 43).
Butte is within a related field of endeavor to the claimed invention because it involves imaging systems and methods for simultaneous near-infrared light and visible light imaging of a sample (see [Abstract]).
Butte teaches “wherein the real-time imaging device comprises an endoscope for obtaining visible light images and wherein the real-time imaging device further includes a fiberscope for obtaining near-infrared fluorescent images” (Claims 13 and 43) (“The systems and methods disclosed herein are well suited for combination with many types of surgical and other procedures with minimal disruption in workflow. For example, the presently disclosed methods and apparatus are well suited for incorporation with prior operating microscopes, and other imaging devices, such as cameras, monitors, exoscopes, surgical robots, endoscopes, in order to improve the surgical work flow. In some embodiments, the systems and methods disclosed herein are capable of simultaneous capture of visible light and infrared fluorescence and can either be used stand-alone (e.g. open field or endoscopic) or as an attachment to a surgical instrument, such as an operating microscope” [0064]; “In some embodiments, the excitation light comprises infrared light. In some embodiments, the infrared light comprises near infrared light” [0012]; “FIG. 4 shows an exemplary embodiment of the imaging systems and methods capable of simultaneously acquiring both infrared or near infrared (NIR) fluorescence and visible light images; in this case, a two-camera system that can be attached to an operating microscope, in accordance with some embodiments” [0034]. In this case, since the device shown in FIG. 4 can be used to simultaneously acquire infrared or near infrared (NIR) fluorescence and visible light images, the device being an operating microscope or an endoscope, the real-time imaging device comprises an endoscope (i.e. operating microscope) for obtaining visible light images (i.e. VIS camera/lens, see FIG. 4) and wherein the real-time imaging device further includes a fiberscope (i.e. NIR camera, lens, see FIG. 4) for obtaining near-infrared fluorescent images.); […]
“registering the near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope” (Claims 13 and 43) (“In some embodiments, the viewing axis of the visible light imaging optics can be coaxial with the excitation light and fluorescent light axes in order to improve registration of the fluorescence image and the visible image over a range of distances extending between the optics and the imaged tissue” [0004]. As shown in FIG. 4, for example, the viewing axis of the visible light imaging optics are coaxial with fluorescent light axes (i.e. corresponding to the NIR camera). Therefore, the method involves registering the near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method and the non-transitory computer-readable storage medium of Averbuch in view of Shelton such that the real-time imaging device comprises an endoscope for obtaining visible light images and further includes a fiberscope for obtaining near-infrared fluorescent images and registering near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope as disclosed in Butte in order to obtain imaging data from multiple modalities for use in assessing a patient’s anatomy. Obtaining visible light images and near-infrared fluorescent images and registering them is one of a finite number of techniques which can be used to observe multiple characteristics about a patient’s anatomy with a reasonable expectation of success. Thus, modifying the method of Averbuch in view of Shelton such that the real-time imaging device comprises an endoscope for obtaining visible light images and further includes a fiberscope for obtaining near-infrared fluorescent images and registering near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope as disclosed in Butte would yield the predictable result of obtaining imaging data from multiple modalities for use in assessing a patient’s anatomy.
Averbuch in view of Shelton and Butte does not teach: “registering the visible light images from the endoscope to the preprocedural images” (Claims 13 and 43)
Gurevich is within a related field of endeavor to the claimed invention because it involves a method for visualizing tissue with two different imaging modalities (see [Abstract]).
Gurevich teaches “registering the visible light images from the endoscope to the preprocedural images” (“In some embodiments, a registration step may be performed to align images with respect to one another. In some embodiments, the registration step is performed to ensure that the stored second imaging modality frame is displayed in the correct location and/or orientation with respect to the newly acquired first imaging modality frame. For example, upon locating a stored first imaging modality frame that is similar (e.g., meets the threshold requirement) to a newly acquired first imaging modality frame, a registration process may be performed to determine whether and to the extent that the stored frame should be translated and/or rotated to align to the newly acquired frame. Then, the translation and/or rotation information can be used to register the corresponding stored second imaging modality frame for display in combination with the newly acquired first imaging modality frame” [0157]; “In any of these embodiments, the first imaging modality may include visible light imaging and second imaging modality may include fluorescence imaging” [0019]. Therefore, since the stored second imaging modality frame (i.e. preprocedural image) mis registered with the newly acquired first imaging modality frame and the first imaging modality is visible light imaging (see [0019]), the method involves registering the visible light images from the endoscope to the preprocedural images.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method and the non-transitory computer-readable storage medium of Averbuch in view of Shelton and Butte such that it involves registering the visible light images from the endoscope to the preprocedural images as disclosed in Gurevich in order to ensure that images acquired with different imaging modalities are accurately matched when performing assessment thereof. Registering a preprocedural image and a visible light image is one of a finite number of techniques which can be used to confirm that images are properly aligned/oriented such that the images can be more accurately assessed with a reasonable expectation of success. Thus, modifying the method of Averbuch in view of Shelton and Butte such that it involves registering the visible light images from the endoscope to the preprocedural images as disclosed in Gurevich would yield the predictable result of ensuring that images acquired with different imaging modalities are accurately matched when performing assessment thereof.
Regarding claims 15 and 44, Averbuch in view of Shelton, Butte and Gurevich discloses all features of the claimed invention as discussed with respect to claims 13 and 43 above, and Butte further teaches “wherein the fiberscope has a substantially fixed relative position to the endoscope, and wherein registering the near-infrared fluorescent images to the visible light images is based at least in part on the substantially fixed relative position” (Claims 15 and 44) (See [0034] and [0004] as discussed with respect to claim 13 above. As shown in FIG. 4, the NIR camera/lens (i.e. fiberscope) is located within the endoscope (i.e. 100). Therefore, the fiberscope has a substantially fixed relative position to the endoscope. Furthermore, registering the near-infrared fluorescent images to the visible light images is based at least in part on the substantially fixed relative position.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method and the non-transitory computer-readable storage medium of Averbuch in view of Shelton such that the real-time imaging device comprises a fiberscope that has a substantially fixed relative position to the endoscope and registering near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope based at least in part on the substantially fixed relative position as disclosed in Butte in order to obtain imaging data from multiple modalities for use in assessing a patient’s anatomy. Obtaining visible light images and near-infrared fluorescent images and registering them is one of a finite number of techniques which can be used to observe multiple characteristics about a patient’s anatomy with a reasonable expectation of success. Thus, modifying the method of Averbuch in view of Shelton such that the real-time imaging device comprises a fiberscope that has a substantially fixed relative position to the endoscope and registering near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope based at least in part on the substantially fixed relative position as disclosed in Butte would yield the predictable result of obtaining imaging data from multiple modalities for use in assessing a patient’s anatomy.
Regarding claims 16 and 45, Averbuch in view of Shelton, Butte and Gurevich discloses all features of the claimed invention as discussed with respect to claims 13 and 43 above, and Butte further teaches “wherein the fiberscope is positioned within a field of view of the endoscope, and wherein registering the near-infrared fluorescent images to the visible light images is based at least in part on detecting the fiberscope in the visible light images or the near-infrared fluorescent images” (Claims 16 and 45) (See [0034] and [0004] as discussed with respect to claim 13 above. As shown in FIG. 4, the NIR camera/lens (i.e. fiberscope) is located within the endoscope (i.e. 100). Thus, the fiberscope is positioned within a field of view of the endoscope. Furthermore, since the near-infrared images and registered with the visible light images, the method involves registering the near-infrared fluorescent images to the visible light images based at least in part on detecting the fiberscope in the visible light images or the near-infrared fluorescent images.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method and the non-transitory computer-readable storage medium of Averbuch such that the real-time imaging device comprises a fiberscope that is positioned within a field of view of the endoscope and registering near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope is based at least in part on detecting the fiberscope in the visible light images or the near-infrared fluorescent images as disclosed in Butte in order to obtain imaging data from multiple modalities for use in assessing a patient’s anatomy. Obtaining visible light images and near-infrared fluorescent images and registering them is one of a finite number of techniques which can be used to observe multiple characteristics about a patient’s anatomy with a reasonable expectation of success. Thus, modifying the method of Averbuch such that the real-time imaging device comprises a fiberscope that is positioned within a field of view of the endoscope and registering near-infrared fluorescent images from the fiberscope to the visible light images from the endoscope is based at least in part on detecting the fiberscope in the visible light images or the near-infrared fluorescent images as disclosed in Butte would yield the predictable result of obtaining imaging data from multiple modalities for use in assessing a patient’s anatomy.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Averbuch US 2020/0170623 A1 “Averbuch”, Shelton, IV et al. US 2021/0205019 A1 “Shelton”, Butte et al. US 2021/0015350 A1 “Butte” and Gurevich et al. US 2021/0007687 A1 “Gurevich” as applied to claim 13 above, and further in view of Nie et al. US 2012/0123205 A1 “Nie”.
Regarding claim 18, Averbuch in view of Shelton, Butte and Gurevich discloses all features of the claimed invention as discussed with respect to claim 13 above and Butte further teaches “the fiberscope” (See Butte: [0064], [0034] as discussed with respect to claim 13 above).
However, the combination does not teach “wherein the fiberscope is configured to pass through a working channel of the endoscope”.
Nie is within the same field of endeavor as the claimed invention because it involves a system and method for intra-operatively providing anatomical guidance in a diagnostic or therapeutic procedure (see [Abstract]).
Nie teaches “wherein the fiberscope is configured to pass through a working channel of the endoscope” (“In another aspect, the present disclosure relates to an imaging system using integrated bright-field imaging, near-infrared imaging, and Raman imaging and/or fluorescence imaging for intra-operatively evaluating target tissues in an area of interest of a living subject. In one embodiment, the system includes a first light source for delivering a beam of visible light to the area of interest and a second light source for delivering a beam of near-infrared light to the area of interest. The system also includes a Raman and/or fluorescence imaging means that includes a handheld probe optically coupled to the second light source, for delivering the near infrared light to illuminate target tissues of the area of interest, and for collecting scattered light and/or emitted light from a corresponding Raman probe and/or fluorescence probe that is introduced into the target tissues and illuminated by the second light source” [0008]; “This example describes endoscopes, colonoscopes, colposcopes, "fiber-optical systems" (fiberscopes), and "rigid optical systems" (borescopes) which record images from the remote end of a tube that may be flexible or rigid. […] Fiberscopes use a "coherent" fiber optic bundle (the position of each fiber is the same at each end of the bundle) to collect light at the remote end and transfer it to the local end” [0195]; “Regardless of endoscope type, a key differentiating feature is that a miniaturized electronic device or "mini-Spectropen" is either used in the working channel of the endoscope device or integrated into the endoscope device. This scope provides point excitation of fluorescence or Raman. It also has a spectroscopic return that provides spectroscopy information of the point excitation area” [0196]. In this case, since the mini-Spectropen is a scope that can be used in the working channel of the endoscope device to provide point excitation of fluorescence and the fluorescence is triggered as a result of the second light source (i.e. mini-Spectropen) delivering a beam of near-infrared light, the mini-Spectropen represents a fiberscope which is configured to pass through a working channel of the endoscope.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the fiberscope of Butte such that it is configured to pass through a working channel of the endoscope as disclosed in Nie in order to allow the fiberscope to emit near-infrared light to a specific region within a patient’s tissue. When a fiberscope is able to pass through a working channel of an endoscope it can be positioned more easily relative to a target location within a patient with a reasonable expectation of success. Thus, modifying the fiberscope of Butte such that it is configured to pass through a working channel of the endoscope as disclosed in Nie would yield the predictable result of allowing the fiberscope to emit near-infrared light to a specific region within a patient’s tissue.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Averbuch US 2020/0170623 A1 “Averbuch” and Shelton, IV et al. US 2021/0205019 A1 “Shelton” as applied to claim 1 above, and further in view of Nie et al. US 2012/0123205 A1 “Nie”.
Regarding claim 20, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above, however, the combination does not teach “wherein determining the calibrated virtual target position corresponding to the position of the physical target comprises: emitting, by the fiberscope, near-infrared light of one or more wavelengths; detecting, by the fiberscope, reflectance properties of tissue in a path of the near-infrared light; and discerning when the physical target is in the path of the near-infrared light based at least in part on the reflectance properties”.
Nie teaches “wherein determining the calibrated virtual target position corresponding to the position of the physical target comprises: emitting, by the fiberscope, near-infrared light of one or more wavelengths; detecting, by the fiberscope, reflectance properties of tissue in a path of the near-infrared light; and discerning when the physical target is in the path of the near-infrared light based at least in part on the reflectance properties” ((“FIG. 8 schematically shows a system for performing tissue penetration depth studies of near-infrared fluorescent and SERS contrast agents, according to one embodiment of the present disclosure” [0041]; “Tissue Penetration Depth Measurement: […] To determine the depth at which the handheld spectroscopic pen device can detect fluorescent dyes or SERS nanoparticles in various organs, an 8 mm.sup.3 section of the tissue was loaded with 20 .mu.L, of either 650 nM ICG or 300 pM SERS nanoparticle solution. Next, thinly sliced sections of the corresponding tissues were laid on top of the contrast agent-loaded specimen. After each tissue section was applied, fluorescent or Raman spectra were collected over 0.1-10 s with the handheld spectroscopic pen device” [0174]; “In Vivo and Intra-Operative Measurements: […] Once the tumors were approximately 4 mm in diameter, ICG was administered intravenously (i.v.) via a tail vein at a dose of 357 .mu.g/kg. […] Tumor-bearing mice undergoing bioluminescence imaging were administered i.p. 100 .mu.L, of a luciferin solution (30 mg/mL). Bioluminescent images were acquired on a Kodak In-Vivo FX Imaging System from Carestream Molecular Imaging (Rochester, N.Y.). Corresponding bright-field images were taken for anatomical reference of the bioluminescence signal. A series of spectra were acquired on tumor-bearing mice using the handheld spectroscopic pen device” [0175]; “The handheld spectroscopic pen device connects a handheld sampling head, via a fiberoptic cable, to a spectrometer that can record fluorescence and Raman signals. The ability to resolve NIR fluorescent and Raman signals from background tissue arises from the optical filtering that takes place in the handheld portion of the device, as illustrated in FIGS. 3 and 4” [0176]; “As depicted in FIG. 4, the handheld spectroscopic pen device allows for sensitive detection of both fluorescent and SERS contrast agents” [0177]; “An objective of intra-operative use of the handheld spectroscopic pen device is detection of tumor foci at the margins of the tumor mass, thereby minimizing the risk of positive margins. In practice, a real-time detection system according to aspects of the exemplary embodiment disclosed in this Example allows the surgeon to remove tumor tissue that might have gone undetected, saving the patient from repeated surgery and potentially improving survival. Sensitive tumor detection is based on the use of albumin-bound ICG or SERS nanoparticles as contrast agents” [0178]; “The ability of the handheld spectroscopic pen device to differentiate contrast agent signals from the autofluorescence and Raman scattering of major tissue/organ types (i.e. fat, liver and lung) was first examined. FIG. 4 shows representative spectra of pure ICG, animal fat, and a mixture of ICG and animal fat (ICG in fat). At 785 nm excitation, ICG has a fluorescence peak at 816 nm, while fat has a background fluorescence peak at 805 nm plus resolvable Raman signals at 862, 1070, 1297, 1439, and 1652 cm.sup.-1 (corresponding to 842, 857, 874, 885, and 902 nm in wavelength, respectively). ICG buried in fat has identifiable contributions of both ICG and fat (e.g., ICG fluorescence at 816 nm and the fat Raman peaks at 874 and 885 nm)” [0179]; “To simulate this surgical scenario, the ability of the handheld spectroscopic pen device to detect optical contrast agents below the surface of fat, liver, and lung tissues was examined, by placing contrast agent loaded tissue specimens below 1-2 mm sections of unlabeled tissue (FIG. 8). FIG. 8 schematically shows a system for performing tissue penetration depth studies of near-infrared fluorescent and SERS contrast agents” [0181]; “FIGS. 9A and 9B show the relationship between signal intensity and the depth of ICG or SERS agents deeply placed in ex vivo tissues. As suggested from light scattering, the contrast signal intensity decreased almost exponentially with tissue thickness. ICG can be detected more deeply in fat than other tissues because fat does not scatter the excitation light as strongly as lung and liver” [0182].
In this case, the handheld spectroscopic pen device represents a fiberscope because it utilizes a near-infrared diode laser (emitting at 785 nm) to perform light excitation and collection (See [0168]). In this case, since a series of spectra are acquired from tumor-bearing mice (i.e. see [0175]) using the handheld spectroscopic pen device and the handheld spectroscopic pen device is used to perform sensitive tumor detection with the use of ICG as a contrast agent (see [0178]); the step of determining the calibrated virtual target position corresponding to the detected position of the physical target (i.e. tumor, for example) comprises emitting, by a fiberscope (i.e. handheld spectroscopic pen device) of the real-time imaging device, near-infrared light of one or more wavelengths (i.e. 785 nm, for example, see [0168]); detecting, by the fiberscope reflectance propertied of tissue in a path of the near-infrared light; and discerning when the physical target (i.e. tumor) is in the path of the near-infrared light based at least in part on the reflectance properties (i.e. signal intensity corresponding to the ICG fluorescent contrast agent).).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Averbuch in view of Shelton such that the step of determining the calibrated virtual target position corresponding to the detected position of the physical target comprises: emitting, by a fiberscope of the real-time imaging device, near-infrared light of one or more wavelengths; detecting, by the fiberscope, reflectance properties of tissue in a path of the near-infrared light; and discerning when the physical target is in the path of the near-infrared light based at least in part on the reflectance properties as disclosed in Nie in order to effectively distinguish the location of a tumor prior to performing subsequent removal. Utilizing a handheld spectroscopic pen device (i.e. fiberscope) is one of a finite number of techniques which can be used to perform tumor detection based on the use of ICG (i.e. a fluorescent contrast agent) with a reasonable expectation of success. Thus, modifying the method of Averbuch such that the step of determining the calibrated virtual target position corresponding to the detected position of the physical target comprises: emitting, by a fiberscope of the real-time imaging device, near-infrared light of one or more wavelengths; detecting, by the fiberscope, reflectance properties of tissue in a path of the near-infrared light; and discerning when the physical target is in the path of the near-infrared light based at least in part on the reflectance properties as disclosed in Nie would yield the predictable result of effectively distinguishing the location of a tumor prior to performing subsequent removal (see Nie: [0178]).
Claim(s) 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Averbuch US 2020/0170623 A1 “Averbuch” and Shelton, IV et al. US 2021/0205019 A1 “Shelton” as applied to claim 1 above, and further in view of Gurevich et al. US 2021/0007687 A1 “Gurevich”.
Regarding claim 22, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 1 above, and Shelton further teaches “further comprising: determining a distance from the real-time imaging device and a surface of obscuring tissue based at least in part on a time-of-flight of […] light emitted from a […] light illuminator of the real-time imaging device and reflected by the surface of the obscuring tissue and determining a distance from the surface of obscuring tissue and the surface of physical target” (See [0026], [0193], [0194] and [0195] as discussed in claim 1 above.
Therefore, since spectral imaging is utilized to intraoperatively (i.e. in real-time) measure the distance between a waveform emitter and a critical structure that is obscured by tissue and the time-of-flight sensor system 1104 utilizes waveforms 1124, 112 which can be in the NIR (i.e. near-infrared spectrum) to determine this distance, Shelton discloses a system which performs the step of determining a distance from the real-time imaging device and a surface of obscuring tissue based at least in part on a time-of-flight of light emitted from a light illuminator of the real-time imaging device and reflected by the surface of the obscuring tissue and determining a distance from the surface of obscuring tissue and the surface of the physical target.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Averbuch such that it involves determining a distance from the real-time imaging device and a surface of obscuring tissue based at least in part on a time-of-flight of light emitted from a light illuminator of the real-time imaging device and reflected by the surface of the obscuring tissue and determining a distance from the surface of obscuring tissue and the surface of the physical target as disclosed in Shelton in order to accurately determine the location of a critical structure (i.e. physical target) intraoperatively (see Shelton: [0134]). Detecting a distance to a physical target beyond a wall of an anatomical channel (i.e. embedded within an organ, for example) with a time-of-flight measurement of near-infrared fluorescent light is one of a finite number of techniques which can be used to effectively distinguish the location of a physical target (i.e. critical structure) with a reasonable expectation of success. Thus, modifying the method of Averbuch such that it involves determining a distance from the real-time imaging device and a surface of obscuring tissue based at least in part on a time-of-flight of light emitted from a light illuminator of the real-time imaging device and reflected by the surface of the obscuring tissue and determining a distance from the surface of obscuring tissue and the surface of the physical target as disclosed in Shelton would yield the predictable result of accurately determining the location of a critical structure (i.e. physical target) intraoperatively (see Shelton: [0134]).
Averbuch in view of Shelton does not teach that the light is “white light” or that the light illuminator is a “white light illuminator”.
Gurevich teaches that the light is “white light” and that the light illuminator is a “white light illuminator” (“In any of these embodiments, the first series of first imaging modality frames may be white light frames” [0030]; “During periods when the second imaging modality is able to image the feature of interest, first and second imaging modality images are displayed together for visualization by a user—for example, a fluorescence image can be displayed as an overlay on a white light image—and are stored together in memory” [0099]; “At step 202, visible light and fluorescence light images of the tissue of the subject are captured. The tissue of the subject may be illuminated with visible light, such as white light, and visible light reflected from the tissue may be captured by an imager, which generates a visible image frame” [0131]. Therefore, since the first imaging modality frames are white light frames and visible light (i.e. white light images) are captured by an imager, the light is white light and the light illuminator is a white light illuminator.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Averbuch in view of Shelton such that the method utilizes a white light illuminator as disclosed in Gurevich in order to obtain images of tissue. White light is one of a finite number of light types which can be used to perform imaging of tissue with a reasonable expectation of success. Thus, modifying the method of Averbuch in view of Shelton such that the method utilizes a white light illuminator as disclosed in Gurevich would yield the predictable result of obtaining images of tissue.
Claim(s) 33 is/are rejected under 35 U.S.C. 103 as being unpatentable over Averbuch US 2020/0170623 A1 “Averbuch” and Shelton, IV et al. US 2021/0205019 A1 “Shelton” as applied to claim 31 above, and further in view of Butte et al. US 2021/0015350 A1 “Butte”.
Regarding claim 33, Averbuch in view of Shelton discloses all features of the claimed invention as discussed with respect to claim 31 above. However, the combination does not teach “wherein the real-time imaging device comprises an endoscope for obtaining visible light images”.
Butte teaches “wherein the real-time imaging device comprises an endoscope for obtaining visible light images” (See [0034] and [0064] as discussed with respect to claim 13 above. Therefore, the real-time imaging device comprises an endoscope for obtaining visible light images (i.e. with the VIS camera/lens, see FIG. 4).).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the medical targeting system of Averbuch in view of Shelton such that the real-time imaging device comprises an endoscope for obtaining visible light images as disclosed in Butte in order to obtain imaging data for use in assessing a patient’s anatomy. Obtaining visible light images is one of a finite number of techniques which can be used to observe characteristics about a patient’s anatomy with a reasonable expectation of success. Thus, modifying the medical targeting system of Averbuch such that the real-time imaging device comprises an endoscope for obtaining visible light images as disclosed in Butte would yield the predictable result of obtaining imaging data from for use in assessing a patient’s anatomy.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Shelton, IV et al. US 2023/0105509 A1 “Shelton-2” is pertinent to the applicant’s disclosure because it discloses “FIG. 15 illustrates one embodiment of spectral imaging to tissue being utilized intraoperatively to measure a distance between a waveform emitter and a critical structure that is obscured by tissue. FIG. 15 shows an embodiment of a time-of-flight sensor system 404 utilizing waveforms 424, 425. The time-of-flight sensor system 404 can be incorporated into a surgical visualization system, e.g., as the sensor system 104 of the surgical visualization system 100 of FIG. 1.” [0116].
Scheib et al. US 2020/0015905 A1 “Scheib” is pertinent to the applicant’s disclosure because it discloses “FIG. 19 is a schematic of a near infrared (NIR) time-of-flight measurement system configured to sense distance to a critical anatomical structure, the time-of-flight measurement system including a transmitter (emitter) and a receiver (sensor) positioned on a common device, according to at least one aspect of the present disclosure” [0027].
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN E SEBASTIAN whose telephone number is (571)272-6190. The examiner can normally be reached Mon.- Fri. 7:30-4:30 (Alternate Fridays Off).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M Kozak can be reached at (571) 270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KAITLYN E SEBASTIAN/Examiner, Art Unit 3797
Prosecution Insights