DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Objections
The following claims are objected to because of the following informalities and should recite:
Claim 1: line 13, “a viewing”. Remove the extra space.
Appropriate correction is needed.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
Such claim limitation(s) is/are:
“input unit” in claims 1-2 for receiving user input for controlling a viewing area and a viewing angle of the reconstructed 3D image invokes 35 U.S.C. 112(f).
“imaging device” in claim 11 for receiving one or more secondary images invokes 35 U.S.C. 112(f).
The term “unit” & “device’ is a non-structural generic placeholder that does not include any specific structure for performing the accompany functions. See MPEP 2181.I.A: The following is a list of non-structural generic placeholders that may invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, paragraph 6: "mechanism for," "module for," "device for," "unit for," "component for," "element for," "member for," "apparatus for," "machine for," or "system for." Welker Bearing Co., v. PHD, Inc., 550 F.3d 1090, 1096, 89 USPQ2d 1289, 1293-94 (Fed. Cir. 2008); Massachusetts Inst. of Tech. v. Abacus Software, 462 F.3d 1344, 1354, 80 USPQ2d 1225, 1228 (Fed. Cir. 2006); Personalized Media, 161 F.3d at 704, 48 USPQ2d at 1886–87; Mas-Hamilton Group v. LaGard, Inc., 156 F.3d 1206, 1214-1215, 48 USPQ2d 1010, 1017 (Fed. Cir. 1998).
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
“input unit” refers to a processor in specification paragraph [0018], ‘an alphanumeric input device 812 (e.g., a keyboard),’ Therefore, the “input unit” refers to a generic keyboard.
“imaging device” refers to a processor in specification paragraph [0069], ‘a device configured to engage tissue and collect and store a portion of that tissue and through which an imaging device (e.g., a camera) can view target tissue via inclusion of optically enhanced materials and components. The control unit 16 can be configured to activate an imaging device (e.g., a camera) at the functional section of the endoscope 14 to view target tissue distal of surgical instrument 200 and endoscopy system 10,’ Therefore, the “imaging device” refers to a generic camera.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 14 & 16-17 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 14: recites:
“wherein the processor is further configured to: generate the integrated reconstructed 3D image by applying the reconstructed 3D image and the secondary image to a trained machine-learning model”
Claim 16: recites:
“the trained machine-learning model being trained to establish a relationship between (i) one or more images or image features representing variants of the anatomical target, and (ii) one or more endoscope navigation plans for the variants of the anatomical target.”
Claim 17: recites:
“wherein the processor is configured to train the machine-learning model using a training dataset comprising procedure data from past endoscopic procedures on a plurality of patients, the procedure data including (i) one or more images of anatomical targets of the plurality of patients and (ii) one or more corresponding endoscope navigation plans.”
An algorithm is defined, for example, as "a finite sequence of steps for solving a logical or mathematical problem or performing a task." Microsoft Computer Dictionary (5th ed., 2002). Applicant may "express that algorithm in any understandable terms including as a mathematical formula, in prose, or as a flow chart, or in any other manner that provides sufficient structure." Finisar Corp. v. DirecTV Grp., Inc., 523 F.3d 1323, 1340 (Fed. Cir. 2008) (internal citation omitted). This can occur when the algorithm or steps/procedure for performing the computer function are not explained at all or are not explained in sufficient detail (simply restating the function recited in the claim is not necessarily sufficient). In other words, the algorithm or steps/procedure taken to perform the function must be described with sufficient detail so that one of ordinary skill in the art would understand how the inventor intended the function to be performed. It is not enough that one skilled in the art could write a program to achieve the claimed function because the specification must explain how the inventor intends to achieve the claimed function to satisfy the written description requirement. See, e.g., Vasudevan Software, Inc. v. MicroStrategy, Inc., 782 F.3d 671, 681-683, 114 USPQ2d 1349, 1356, 1357 (Fed. Cir. 2015), see MPEP § 2161(I).
The claim is rejected under 35 USC § 112(a) for a lack of written description. Proper written description cannot be identified in the specification, claims, and drawings directed to the computer implemented steps of the trained machine learning model being trained to establish a relationship between one or more images or image features representing variants of the anatomical target and one or more endoscope navigation plans the variants of the anatomical targets; and wherein the processor is configured to train the machine-learning model using a training dataset comprising procedure data from past endoscopic procedures on a plurality of patients, the procedure data including (i) one or more images of anatomical targets of the plurality of patients and (ii) one or more corresponding endoscope navigation plans. Specifically, the specification does not provide and lacks detailed a step-by-step description, any algorithmic or flowchart-based disclosure, specific functions, and/or weights for the machine learning techniques or convolutional networks used for the analysis.
These limitations are computer/processor-implemented functional claim limitation as it is directed to a processor-controlled algorithm configured to determine a location. Yet the specification does not disclose the computer and the algorithm (e.g., the necessary steps and/or flowcharts) that perform the claimed functions of claim 14 & 16-17, in sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor possessed the claimed subject matter at the time of filing. It is not enough to disclose that one skilled in the art could write a program to achieve the claimed function because the specification must explain how the inventor intends to achieve the claimed function to satisfy the written description requirement. See, e.g., Vasudevan Software, Inc. v. MicroStrategy, Inc., 782 F.3d 671, 681-683, 114 USPQ2d 1349, 1356, 1357 (Fed. Cir. 2015). As the specification does not provide a disclosure of the computer and algorithm in sufficient detail to demonstrate to one of ordinary skill in the art that the inventor possessed the invention, these claims are rejected for lack of written description. For more information regarding the written description requirement, see MPEP §§ 2161, 2162-2163.07(b).
Indeed, the specification paragraph is directed to a mere example of generalized machine learning models tantamount to a black box, rather than showing procession of a particular implementation. Without the level of detail regarding the weights used, parameters used, and operations of the analysis using machine learning models, the written description requirement is not satisfied. The mere use of stating relationship analysis is achieved by machine learning techniques is not sufficient. One of ordinary skill in the art would not be able to implement the described process without disclosure of said weights, parameters, and/or operations of the trained machine learning model are achieved in a step-by-step manner. In addition, an assertion that could be derived using simulations or test (i.e., prophetic examples) does not demonstrate that the inventors actual did so or had possession of the specific functional relationships and constraints to obviate the lack of written description requirement.
Consequently, one of ordinary skill in the art would not deem the instant specification having sufficient detail so that they could understand how the inventor intended to achieve the aforementioned step. Since the instant specification fails to provide a finite sequence of steps for performing step, the aforementioned claim fails to meet the written description requirement under 35 U.S.C. 112(a).
Dependent claims are rejected by virtue of their dependency to abovementioned claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5, 7-13, 15, & 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wong et al (US 20190298451 A1) in view of Mintz et al (US 20170084027 A1).
Claim 1: Wong discloses, An image-guided endoscopic system (100), comprising: (¶Abstract, ¶0022, ¶0027, ¶0065-0066)
an endoscope (102/202) configured to be positioned and navigated in a patient anatomy; (¶0020-0022, ¶0027, ¶0033)
a display (110/810, ¶0022, ¶0048, ¶0066); and
a processor (112/812, ¶0022, ¶0066) configured to:
receive at least two images of an anatomical target; (¶0027, ¶0029, ¶0037, the imaging sensor 230 captures images including video images that are processed by the system)
reconstruct a three-dimensional (3D) image of the anatomical target using the at least two received images; (¶0035-0038, ¶0041, the processor generates a 3D model (reconstructed 3D image/map) of the patient anatomy and tissue target. The model is build and updated utilizing data from the imaging sensor 230 using the imaging based methods and positional data.)
generate an endoscope navigation plan for positioning and navigating the endoscope based at least on the reconstructed 3D image of the anatomical target; (¶0042-0046, ¶0060, the model is used to plan the medical procedure including the navigation path to an anatomic or ablation target. This navigation path is saved within the processor and displayed in the model to guide the device to the anatomical target.)
control a user input received via an input unit; and (¶0039, ¶0066, master assembly 806 allows the operator to provide input to control the assembly.)
Wong fails to disclose:
control the display to automatically adjust at least one of a viewing area or a viewing angle of the display of the reconstructed 3D image in accordance with at least one of: i) the endoscope navigation plan or ii) a position or direction of a distal end of the endoscope relative to the anatomical target;
control the display to automatically zoom in on a portion of the reconstructed 3D image as the distal portion of the endoscope gets closer to the anatomical target.
However, Mintz in the context of navigation methods of tubular networks with endoscope discloses: control the display to automatically adjust at least one of a viewing area or a viewing angle of the display of the reconstructed 3D image in accordance with at least one of: i) the endoscope navigation plan or ii) a position or direction of a distal end of the endoscope relative to the anatomical target; (¶0061, ¶0180, the control adjust the viewing angle based on the tip’s position along the path “The display modules 202 may automatically display different views of the model of the endoscope 118 depending on user settings and a particular surgical procedure.”, ¶0061. The system generates a virtual view (1450) from a third-person viewpoint to the rear of the tip and this virtual viewpoint may project ahead of the endoscope tip location to show the next bifurcation and highlight the intended path to guide the user.)
control a user input received via an input unit; (¶0055)
control the display to automatically zoom in on a portion of the reconstructed 3D image as the distal portion of the endoscope gets closer to the anatomical target. (Under the broadest reasonable interpretation, Mintz discloses that the display module can “automatically display different views”, ¶0061, and specifically describing that as the tool gets closer to the anatomical target, a zoomed view is presented. Mintz teaches that when the endoscope tip “has been navigated to nearly the end of path 1441, the system presents view 1460 which, “shows a map view of the 3D model, zoomed in around the target location 1443”, ¶0181. The system utilizes interface controls capable of map pan and zoom, ¶0178, FIG. 14B.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the control of the display of Wong to incorporate the teachings of Mintz. The motivation to do this yield predictable results such as improving the navigation through tubular networks, as suggested by Mintz, ¶Abstract.
Claim 2: Wong as modified discloses all the elements above in claim 1, Wong fails to explicitly disclose: further comprising: the input unit, wherein the input unit is configured to receive user input for controlling the viewing area and the viewing angle of the reconstructed 3D image.
However, Mintz is relied upon above discloses: further comprising: the input unit, wherein the input unit is configured to receive user input for controlling the viewing area and the viewing angle of the reconstructed 3D image. (¶0054-0055, ¶0059, ¶0061, ¶0143-0144, ¶0172-0173, ¶0178-0179)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the input unit of modified Wong to incorporate the teachings of Mintz. The motivation to do this yield predictable results such as improving the navigation through tubular networks, as suggested by Mintz, ¶Abstract.
Claim 3: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the endoscope is configured to be positioned and navigated in a pancreaticobiliary system of the patient (¶0055).
Claim 4: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the endoscope includes an imaging sensor, (“Elongate device 202 [endoscope may include imaging sensor 230” [0027]) and wherein the at least two received images include at least one endoscopic image of the anatomical target generated by the imaging sensor (“the imaging sensor, receive data [images] for identifying anatomic features of interest within the patient anatomy” [0005]).
Claim 5: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the at least two received images include at least one fluoroscopic image of the anatomical target (“the initial model [the generated 3D image generated from the received images] is generated from pre-operative scans from imaging data obtained from a CT, pet CT, MRI, DICOM, ultrasound or fluoroscopic images.” [0037]).
Claim 7: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the at least two received images include first and second two-dimensional (2D) images (“the initial model [the generated 3D image generated from the received images] is generated from pre-operative scans from imaging data obtained from a CT, pet CT, MRI, DICOM, ultrasound or fluoroscopic images.” [0037]; As interpretated as the different modalities believed to include 2D images.).
Claim 8: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the at least two received images include a first two-dimensional (2D) image and a second three- dimensional (3D) image (“the initial model [the generated 3D image generated from the received images] is generated from pre-operative scans from imaging data obtained from a CT, pet CT, MRI, DICOM, ultrasound or fluoroscopic images.” [0037]; As interpretated as the different modalities believed to include at least one 2D image and one 3D image.).
Claim 9: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the at least two received images include first and second three-dimensional (3D) images (“the initial model [the generated 3D image generated from the received images] is generated from pre-operative scans from imaging data obtained from a CT, pet CT, MRI, DICOM, ultrasound or fluoroscopic images.” [0037]; As interpretated as the different modalities believed to include 3D images.).
Claim 10: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the at least two received images include images from different sources or with different modalities (“the initial model [the generated 3D image generated from the received images] is generated from pre-operative scans from imaging data obtained from a CT, pet CT, MRI, DICOM, ultrasound or fluoroscopic images.” [0037]).
Claim 11: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the processor is configured to: receive one or more secondary images of the anatomical target generated by an imaging device other than the endoscope (“the target areas of interest identified in process 330 may then be displayed or rendered on the model of the anatomy generated from process 320 to create an updated model [integrating ultrasound images to update 3D model [3D image]. In some embodiments, if ultrasound is used to identify depth of target tissue [ultrasound images], the target tissue can be displayed at a measured location within the model” [0041]); integrate the reconstructed 3D image with the one or more secondary images; (“the target areas of interest identified in process 330 may then be displayed or rendered on the model of the anatomy generated from process 320 to create an updated model [integrating ultrasound images to update 3D model [3D image]. In some embodiments, if ultrasound is used to identify depth of target tissue [ultrasound images], the target tissue can be displayed at a measured location within the model” [0041]) and generate the endoscope navigation plan based at least on the integrated reconstructed 3D image of the anatomical target (“the updated model [integrated reconstructed 3D image] may be used to plan a medical procedure including determining a navigational path to an anatomic or ablation target and/or creating a treatment plan.” [0042]).
Claim 12: Wong as modified discloses all the elements above in claim 11, Wong discloses, wherein the secondary image includes one or more of: a computer-tomography (CT) scan image; a magnetic resonance imaging (MRI) scan image; a magnetic resonance cholangiopancreatography (MRCP) image; or an endoscopic ultrasonography (EUS) image (“the target areas of interest identified in process 330 may then be displayed or rendered on the model of the anatomy generated from process 320 to create an updated model [integrating ultrasound images to update 3D model [3D image]. In some embodiments, if ultrasound is used to identify depth of target tissue [ultrasound images], the target tissue can be displayed at a measured location within the model” [0041]).
Claim 13: Wong as modified discloses all the elements above in claim 11, Wong discloses, wherein the processor is further configured to: generate an integrated reconstructed 3D image by superimposing the reconstructed 3D image over the one or more secondary images (“the target areas of interest identified in process 330 may then be displayed or rendered on the model of the anatomy generated from process 320 to create an updated model [superimposed ultrasound images to update 3D model [3D image]. In some embodiments, if ultrasound is used to identify depth of target tissue [ultrasound images], the target tissue can be displayed at a measured location within the model” [0041]).
Claim 15: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein to generate an endoscope navigation plan includes to automatically recognize the anatomical target, (“identification of the landmark can be vision based, where the image (e.g. the endoscopic image) is identified automatically” [0039]) and automatically recognize the anatomical target, and to estimate one or more navigation parameters including: a distance of an endoscope distal portion relative to an anatomical target; a heading direction of the endoscope distal portion relative to the anatomical target; an angle of cannula or a surgical element; a protrusion amount of a cannula or a surgical element; a speed or force applied to the endoscope distal portion or a surgical element; a rotational direction or a cutting area of a surgical element; or a projected navigation path toward the anatomical target (“a shape sensor 108 [on endoscope] coupled to the tracking system 130 for receiving and processing sensor data and information for determining the position, orientation, speed, velocity, pose, and/or shape of distal end 118} [0023]; “The model can indicate the location and depth of target tissue, indicators (such as arrows or lines) can be displayed to guide the user from a current position of the device to the target tissue” [0050]).
Claim 18: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the display is configured to adjust the display of the reconstructed 3D image according to the endoscope navigation plan (“initial model may then be altered, supplemented, or merged with data collected while the elongate body is navigated through anatomy [navigation plan], e.g. positional/location data gathered during navigation, endoscopic camera data correlated to positional data, and/or intravascular or endoluminal ultrasound data, to more accurately display the patient anatomy as previously described above.” [0037]).
Claim 19: Wong as modified discloses all the elements above in claim 1, Wong fails to disclose: wherein to adjust the display includes to automatically zoom a portion of the reconstructed 3D image based on a position or a direction of a distal portion of the endoscope relative to an anatomical target.
However, Mintz is relied upon above disclose: wherein to adjust the display includes to automatically zoom a portion of the reconstructed 3D image based on a position or a direction of a distal portion of the endoscope relative to an anatomical target. (¶0061, ¶0181)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the control of the display of modified Wong to incorporate the teachings of Mintz. The motivation to do this yield predictable results such as improving the navigation through tubular networks, as suggested by Mintz, ¶Abstract.
Claim 20: Wong as modified discloses all the elements above in claim 1, Wong discloses, wherein the display is further configured to display one or more visual indications overlaid upon the reconstructed 3D image, (“The model can indicate the location and depth of target tissue, indicators (such as arrows or lines) can be displayed to guide the user from a current position of the device to the target tissue” [0050]) the one or more visual indications including: an anatomical target; a projected navigation path toward the anatomical target; or a progress of the endoscope advancing toward the anatomical target along the projected navigation path (“The model can indicate the location and depth of target tissue, indicators (such as arrows or lines) can be displayed to guide the user from a current position of the device to the target tissue [including anatomical target, projected navigation path, and progress of navigation path” [0050]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Wong et al (US 20190298451 A1) in view of Mintz et al (US 20170084027 A1), as applied to claim 1, in further view of Anderson (US 20100168561 A1).
Claim 6: Wong as modified discloses all the elements above in claim 1, Wong discloses: reconstructing the 3D image of the anatomical target using the inferred anatomical information (“the target areas of interest identified in process 330 may then be displayed or rendered on the model of the anatomy generated from process 320 to create an updated model [integrating ultrasound images to update 3D model [3D image]. In some embodiments, if ultrasound is used to identify depth of target tissue [ultrasound images], the target tissue can be displayed at a measured location within the model” [0041]).
Wong fails to disclose: wherein: the at least two received images include at least one of at least one electrical potential map or an electrical impedance map of the anatomical target; and the processor is configured to infer anatomical information from the electrical potential map or an electrical impedance map,
However, in the same field of endeavor, Anderson teaches wherein: wherein: the at least two received images include at least one of at least one electrical potential map or an electrical impedance map of the anatomical target; (“the tissue of interest may be characterizable by a measure of electrical impedance, for example, between the at least one excitation element and the at least one sensing element.” [0019]; “Procedures include, for example, surgical procedures, minimally invasive procedures, endoscopic procedures” [0059]) and the processor is configured to infer anatomical information from the electrical potential map or an electrical impedance map, (“The tissue characterization system can further comprise an excitation element and a sensing element.” [0028]; “the tissue of interest may be characterizable by a measure of electrical impedance, for example, between the at least one excitation element and the at least one sensing element.” [0019]; “Procedures include, for example, surgical procedures, minimally invasive procedures, endoscopic procedures” [0059]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of modified Wong to be configured such that wherein: the at least two received images include at least one of at least one electrical potential map or an electrical impedance map of the anatomical target; and the processor is configured to infer anatomical information from the electrical potential map or an electrical impedance map, as taught by Anderson, because electrical impedance map “signal from the excitation element may be modulated in ways to improve detectability” ¶0061 of Anderson.
Claim 14, & 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Wong et al (US 20190298451 A1) in view of Mintz et al (US 20170084027 A1), as applied to claim 1 & 11 respectively, in further view of Ye et al (US 20210196398 A1).
Claim 14: Wong as modified discloses all the elements above in claim 11, Wong discloses, wherein the processor is further configured to: generate the integrated reconstructed 3D image by applying the reconstructed 3D image and the secondary image (“the target areas of interest identified in process 330 may then be displayed or rendered on the model of the anatomy generated from process 320 to create an updated model [integrating ultrasound images to update 3D model [3D image]. In some embodiments, if ultrasound is used to identify depth of target tissue [ultrasound images], the target tissue can be displayed at a measured location within the model” [0041]);
Wong fails to disclose: wherein the processor is further configured to: generate the integrated reconstructed 3D image by applying the reconstructed 3D image and the secondary image to a trained machine-learning model.
However, in the same field of endeavor, Ye teaches, wherein the processor is further configured to: generate the integrated reconstructed 3D image by applying the reconstructed 3D image and the secondary image to a trained machine-learning model. (“The transform circuitry 720 may be trained according to known anatomical images 712 and target labels 732 corresponding to the respective images 712 as input/output pairs” [0126]; “the machine learning framework may be configured to execute the learning/training in any suitable or desirable manner.” [0126]; As interpretated as believed to use machine learning for integrating images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the processor of modified Wong to be configured to: generate the integrated reconstructed 3D image by applying the reconstructed 3D image and the secondary image to a trained machine-learning model, as taught by Ye, because machine learning “advantageously provide[s] mechanisms for automatic target detection, tracking, and/or three-dimensional positioned estimation to assist physicians (e.g., urologists) to achieve relatively efficient and accurate percutaneous access for various surgical operations” [0039 of Ye].
Claim 16: Wong as modified discloses all the elements above in claim 1, Wong fails to disclose: wherein the processor is configured to generate the endoscope navigation plan by applying the reconstructed 3D image to a trained machine-learning model, the trained machine-learning model being trained to establish a relationship between (i) one or more images or image features representing variants of the anatomical target, and (ii) one or more endoscope navigation plans for the variants of the anatomical target.
However, in the same field of endeavor, Ye teaches, wherein the processor is configured to generate the endoscope navigation plan by applying the reconstructed 3D image to a trained machine-learning model (“certain image data may be collected and used for identifying target anatomical features. Such identification may be achieved at least in part using a machine learning framework.” [0101]), the trained machine-learning model being trained to establish a relationship between (i) one or more images or image features representing variants of the anatomical target, (“certain image data may be collected and used for identifying target anatomical features. Such identification may be achieved at least in part using a machine learning framework.” [0101]), and (ii) one or more endoscope navigation plans for the variants of the anatomical target (“Such identification may be achieved at least in part using a machine learning framework. For example, systems, devices, and methods of the present disclosure may provide for identification of target anatomical features in real-time endoscope images, wherein identification of a target anatomical features in an image may prompt certain responsive action [navigation plans]” [0101]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the processor of modified Wong to be configured to generate the endoscope navigation plan by applying the reconstructed 3D image to a trained machine-learning model, the trained machine-learning model being trained to establish a relationship between (i) one or more images or image features representing variants of the anatomical target, and (ii) one or more endoscope navigation plans for the variants of the anatomical target, as taught by Ye, because machine learning “advantageously provide[s] mechanisms for automatic target detection, tracking, and/or three-dimensional positioned estimation to assist physicians (e.g., urologists) to achieve relatively efficient and accurate percutaneous access for various surgical operations” [0039 of Ye].
Claim 17: Wong as modified discloses all the elements above in claim 16, Wong fails to disclose, wherein the processor is configured to train the machine-learning model using a training dataset comprising procedure data from past endoscopic procedures on a plurality of patients, the procedure data including (i) one or more images of anatomical targets of the plurality of patients and (ii) one or more corresponding endoscope navigation plans.
However, Ye as relied upon above, teaches, wherein the processor is configured to train the machine-learning model using a training dataset comprising procedure data from past endoscopic procedures on a plurality of patients (“the method is performed by control circuitry of a medical system and the target position data and the first image are received from an endoscope of the medical system. The target anatomical feature can be an exposed portion of a papilla within a calyx of a kidney of the patient. The artificial neural network can be pretrained based on known image and label data.” [0011]), the procedure data including (i) one or more images of anatomical targets of the plurality of patients (“the method is performed by control circuitry of a medical system and the target position data and the first image are received from an endoscope of the medical system. The target anatomical feature can be an exposed portion of a papilla within a calyx of a kidney of the patient. The artificial neural network can be pretrained based on known image and label data.” [0011]) and (ii) one or more corresponding endoscope navigation plans (“Potential pre-operative data on the touchscreen 26 may include pre-operative plans, navigation and mapping data derived from pre-operative computerized tomography (CT) scans, and/or notes from pre-operative patient interviews.” [0062]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the processor of modified Wong to be configured to train the machine-learning model using a training dataset comprising procedure data from past endoscopic procedures on a plurality of patients, the procedure data including (i) one or more images of anatomical targets of the plurality of patients and (ii) one or more corresponding endoscope navigation plans, as taught by Ye, because machine learning “advantageously provide[s] mechanisms for automatic target detection, tracking, and/or three-dimensional positioned estimation to assist physicians (e.g., urologists) to achieve relatively efficient and accurate percutaneous access for various surgical operations” [0039 of Ye].
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over Claims 1-19 of patent US12478433B2 (U.S. Application 18/047,555). Although the claims at issue are not identical, they are not patentably distinct from each other.
Claim 1: US12478433B2 teaches, An image-guided endoscopic system, comprising: an endoscope configured to be positioned and navigated in a patient anatomy; a display; and a processor configured to: receive at least two images of an anatomical target; reconstruct a three-dimensional (3D) image of the anatomical target using the at least two received images; generate an endoscope navigation plan for positioning and navigating the endoscope based at least on the reconstructed 3D image of the anatomical target; (Claim 1, ‘An image-guided endoscopic system, comprising: an endoscope configured to be positioned and navigated in a patient anatomy; a processor configured to: receive at least two images of an anatomical target; reconstruct a three-dimensional (3D) image of the anatomical target using the at least two received images; and generate an endoscope navigation plan for positioning and navigating the endoscope based at least on the reconstructed 3D image of the anatomical target;”)
control the display to automatically adjust at least one of a viewing area or a viewing angle of the display of the reconstructed 3D image in accordance with at least one of: i) the endoscope navigation plan or ii) a position or direction of a distal end of the endoscope relative to the anatomical target; control a user input received via an input unit; and (Claim 1, ‘an input unit configured to receive user input for controlling a viewing area and a viewing angle of the reconstructed 3D image, wherein the display is configured to automatically adjust at least one of the viewing area or the viewing angle of the display of the reconstructed 3D image in accordance with at least one of: i) the endoscope navigation plan or ii) a position or direction of a distal end of the endoscope relative to the anatomical target, and the user input received via the input unit, wherein the display is configured to automatically zoom in on a portion of the reconstructed 3D image as the distal end of the endoscope gets closer to the anatomical target, and wherein the processor is further configured to:’)
control the display to automatically zoom in on a portion of the reconstructed 3D image as the distal portion of the endoscope gets closer to the anatomical target. (Claim 1, ‘wherein the display is configured to automatically zoom in on a portion of the reconstructed 3D image as the distal end of the endoscope gets closer to the anatomical target,’)
Claim 2: US12478433B2 teaches, The image-guided endoscopic system of claim 1, further comprising: the input unit, wherein the input unit is configured to receive user input for controlling the viewing area and the viewing angle of the reconstructed 3D image. (Claim 1, ‘an input unit configured to receive user input for controlling a viewing area and a viewing angle of the reconstructed 3D image, wherein the display is configured to automatically adjust at least one of the viewing area or the viewing angle of the display of the reconstructed 3D image in accordance with at least one of: i) the endoscope navigation plan or ii) a position or direction of a distal end of the endoscope relative to the anatomical target, and the user input received via the input unit, wherein the display is configured to automatically zoom in on a portion of the reconstructed 3D image as the distal end of the endoscope gets closer to the anatomical target, and wherein the processor is further configured to:’)
Claim 3: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the endoscope is configured to be positioned and navigated in a pancreaticobiliary system of the patient. (Claim 11, ‘a magnetic resonance cholangiopancreatography (MRCP) image;’)
Claim 4: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the endoscope includes an imaging sensor, and wherein the at least two received images include at least one endoscopic image of the anatomical target generated by the imaging sensor. (Claim 3, ‘wherein the endoscope includes an imaging sensor, and the at least two received images include at least one endoscopic image of the anatomical target generated by the imaging sensor.’)
Claim 5: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the at least two received images include at least one fluoroscopic image of the anatomical target. (Claim 4, ‘wherein the at least two received images include at least one fluoroscopic image of the anatomical target.’)
Claim 6: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein: the at least two received images include at least one of at least one electrical potential map or an electrical impedance map of the anatomical target; and the processor is configured to infer anatomical information from the electrical potential map or an electrical impedance map, and to reconstruct the 3D image of the anatomical target using the inferred anatomical information. (Claim 5, ‘wherein: the at least two received images include at least one of at least one electrical potential map or an electrical impedance map of the anatomical target; and the processor is configured to infer anatomical information from the electrical potential map or an electrical impedance map, and to reconstruct the 3D image of the anatomical target using the inferred anatomical information.’)
Claim 7: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the at least two received images include first and second two-dimensional (2D) images. (Claim 6, ‘wherein the at least two received images include first and second two-dimensional (2D) images.’)
Claim 8: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the at least two received images include a first two-dimensional (2D) image and a second three- dimensional (3D) image. (Claim 7, ‘wherein the at least two received images include a first two-dimensional (2D) image and a second three-dimensional (3D) image.’)
Claim 9: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the at least two received images include first and second three-dimensional (3D) images. (Claim 8, ‘wherein the at least two received images include first and second three-dimensional (3D) images.’)
Claim 10: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the at least two received images include images from different sources or with different modalities. (Claim 9, ‘wherein the at least two received images include images from different sources or with different modalities.’)
Claim 11: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the processor is configured to: receive one or more secondary images of the anatomical target generated by an imaging device other than the endoscope; integrate the reconstructed 3D image with the one or more secondary images; and generate the endoscope navigation plan based at least on the integrated reconstructed 3D image of the anatomical target. (Claim 10, ‘wherein the processor is configured to: receive one or more secondary images of the anatomical target generated by an imaging device other than the endoscope; integrate the reconstructed 3D image with the one or more received secondary images; and generate an updated endoscope navigation plan based at least on the integrated reconstructed 3D image of the anatomical target.’)
Claim 12: US12478433B2 teaches, The image-guided endoscopic system of claim 11, wherein the secondary image includes one or more of: a computer-tomography (CT) scan image; a magnetic resonance imaging (MRI) scan image; a magnetic resonance cholangiopancreatography (MRCP) image; or an endoscopic ultrasonography (EUS) image. (Claim 11, ‘wherein the one or more received secondary images includes one or more of: a computer-tomography (CT) scan image; a magnetic resonance imaging (MRI) scan image; a magnetic resonance cholangiopancreatography (MRCP) image; or an endoscopic ultrasonography (EUS) image.’)
Claim 13: US12478433B2 teaches, The image-guided endoscopic system of claim 11, wherein the processor is further configured to: generate an integrated reconstructed 3D image by superimposing the reconstructed 3D image over the one or more secondary images. (Claim 12, ‘wherein the processor is further configured to: generate the integrated reconstructed 3D image by superimposing the reconstructed 3D image over the one or more received secondary images, wherein the superimposing aligns corresponding features in the reconstructed 3D image and at least one of the one or more received secondary images.’)
Claim 14: US12478433B2 teaches, The image-guided endoscopic system of claim 11, wherein the processor is further configured to: generate the integrated reconstructed 3D image by applying the reconstructed 3D image and the secondary image to a trained machine-learning model. (Claim 13, ‘wherein the processor is further configured to: generate the integrated reconstructed 3D image by applying the reconstructed 3D image and the one or more received secondary images to a trained machine-learning model.’)
Claim 15: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein to generate an endoscope navigation plan includes to automatically recognize the anatomical target, and automatically recognize the anatomical target, and to estimate one or more navigation parameters including: a distance of an endoscope distal portion relative to an anatomical target; a heading direction of the endoscope distal portion relative to the anatomical target; an angle of cannula or a surgical element; a protrusion amount of a cannula or a surgical element; a speed or force applied to the endoscope distal portion or a surgical element; a rotational direction or a cutting area of a surgical element; or a projected navigation path toward the anatomical target. (Claim 14, ‘wherein to generate the endoscope navigation plan includes to automatically recognize the anatomical target and to estimate one or more navigation parameters including: a distance of the distal end of the endoscope relative to an anatomical target; a heading direction of the distal end of the endoscope relative to the anatomical target; an angle of cannula or a surgical element; a protrusion amount of a cannula or a surgical element; a speed or force applied to the distal end of the endoscope or a surgical element; a rotational direction or a cutting area of a surgical element; or a projected navigation path toward the anatomical target.’)
Claim 16: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the processor is configured to generate the endoscope navigation plan by applying the reconstructed 3D image to a trained machine-learning model, the trained machine-learning model being trained to establish a relationship between (i) one or more images or image features representing variants of the anatomical target, and (ii) one or more endoscope navigation plans for the variants of the anatomical target. (Claim 15, ‘wherein the processor is configured to generate the endoscope navigation plan by applying the reconstructed 3D image to a trained machine-learning model, the trained machine-learning model being trained to establish a relationship between (i) one or more images or image features representing variants of the anatomical target, and (ii) one or more endoscope navigation plans for the variants of the anatomical target.’)
Claim 17: US12478433B2 teaches, The image-guided endoscopic system of claim 16, wherein the processor is configured to train the machine-learning model using a training dataset comprising procedure data from past endoscopic procedures on a plurality of patients, the procedure data including (i) one or more images of anatomical targets of the plurality of patients and (ii) one or more corresponding endoscope navigation plans. (Claim 16, ‘wherein the processor is configured to train the machine-learning model using a training dataset comprising procedure data from past endoscopic procedures on a plurality of patients, the procedure data including (i) one or more images of anatomical targets of the plurality of patients and (ii) one or more corresponding endoscope navigation plans.’)
Claim 18: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the display is configured to adjust the display of the reconstructed 3D image according to the endoscope navigation plan. (Claim 17, ‘wherein the display is configured to adjust the display of the reconstructed 3D image according to the endoscope navigation plan.’)
Claim 19: US12478433B2 teaches, The image-guided endoscopic system of claim 18, wherein to adjust the display includes to automatically zoom a portion of the reconstructed 3D image based on a position or a direction of a distal portion of the endoscope relative to an anatomical target. (Claim 18, ‘wherein to adjust the display includes to automatically zoom a portion of the reconstructed 3D image based on a position or a direction of the distal end portion of the endoscope relative to an anatomical target to reveal a structural characteristic of the anatomical target, wherein the structural characteristic includes at least one of a shape, depth, or geometric feature of the anatomical target.’)
Claim 20: US12478433B2 teaches, The image-guided endoscopic system of claim 1, wherein the display is further configured to display one or more visual indications overlaid upon the reconstructed 3D image, the one or more visual indications including: an anatomical target; a projected navigation path toward the anatomical target; or a progress of the endoscope advancing toward the anatomical target along the projected navigation path. (Claim 19, ‘wherein the display is further configured to display one or more visual indications overlaid upon the reconstructed 3D image, the one or more visual indications including: the anatomical target; a projected navigation path toward the anatomical target; or a progress of the endoscope advancing toward the anatomical target along the projected navigation path.’)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nicholas Robinson whose telephone number is (571)272-9019. The examiner can normally be reached M-F 9:00AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.A.R./Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights