DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/3/25 has been entered.
Response to Arguments
Response: 35 U.S.C. § 101
1. Applicants argue:
The applicant argues that the claims are not directed towards an abstract idea, where the claims as a whole integrate the alleged judicial exception to improve the field of intraoperative imaging and applies that improvement to the practical application of accurately determining an optimum projection geometry and displaying a planning trajectory in a 2D X-ray image. The applicant points to paragraphs [0018] – [0019] and [0024] – [0026] of the specification for support as to why the claims as a whole integrate the alleged judicial exception to improve the field of intraoperative imaging. (Remarks: pages 10-12)
2. Examiner Response:
The examiner respectfully disagrees. The examiner notes that the determining limitation that states “determining an optimum projection geometry defining a registration between the 3D image data set and the unmasked 2D X-ray image” doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Also, the limitation of claim 1 that states “and displaying the unmasked 2D X-ray image including the at least one foreign object on the display device” amounts to insignificant extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process, see MPEP 2106.05(g).
Also, the limitation of “wherein the planning trajectory is superimposed on the unmasked 2D X-ray image on the display device using the registration defined by the optimum projection geometry” doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Further, the examiner notes that the limitations of claim 1 do not improve a technology or technical field, but rather improves the abstract idea. The limitations of claim 1 does not integrate the abstract idea into a practical application. The current claims are not eligible under 35 U.S.C. 101.
3. Applicants argue:
The applicant argues that the additional elements amount to significantly more than the identified judicial exception, and are therefore eligible under Step 2B. (Remarks: pages 12-13)
4. Examiner Response:
The examiner respectfully disagrees. The examiner notes that even with the recent amendment to claim 1, the claim does not include the additional element of a processor. However, if written with a processor as shown above, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of the processor amounts no more than mere instructions to apply the exception using a generic computer component that does not impose any meaningful limits on practicing the abstract idea and therefore cannot provide an inventive concept (See MPEP 2106.05(b).
5. Applicants argue:
The applicant argues that the current claims as a whole amount to significantly more than an abstract idea. The applicant points to the generating and calculating limitation of claim 1 that states “generating a masked 2D X-ray image by masking out regions of the 2D X- ray image containing the at least one identified foreign object”; “calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image” (Remarks: pages 12-13)
6. Examiner Response:
The examiner notes that the limitation of “generating a masked 2D X-ray image by masking out regions of the 2D X- ray image containing the at least one identified foreign object” amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the masking out of regions of the 2D X-ray image are being conducted. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Also, the limitation of “calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image” is calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image. Therefore, under MPEP 2106.04(a)(2), this limitation covers a mathematical concept, which falls in the “Mathematical Concept” grouping of abstract ideas.
Further, the examiner notes that even with the recent amendments to the claims, the claims are still not eligible under 35 U.S.C. 101.
Response: 35 U.S.C. § 103
7. Applicants argue:
The applicant argues that the prior art of record doesn’t teach the limitation of claim 1 that states and displaying the unmasked 2D X-ray image including the at least one foreign object on the display device, wherein the planning trajectory is superimposed on the unmasked 2D X-ray image on the display device using the registration defined by the optimum projection geometry” (Remarks: pages 8-10)
8. Examiner Response:
The examiner respectfully disagrees. In paragraph [0051] of the Rai et al. reference it teaches displaying the device location, where the surgical device is superimposed on model projection 2D fluoroscopy view. Also, this paragraph teaches additional markers, routes, indicia and planning information may be displayed with model or real images of the organ. This demonstrates that the Rai et al. reference teaches the limitation of “and displaying the unmasked 2D X-ray image including the at least one foreign object on the display device” [Rai (paragraph [0051] “Finally, method 100 shows step 150 displaying the device location. Displaying the device location provides tracking and guidance information to the user or physician. In one embodiment, a virtual surgical device is superimposed onto a 3D view of the body organ. Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ.”)];
Also, in paragraph [0037] of the Rai et al. reference it teaches a wide range of information such as planning information, region of interests, virtual target markers, vessels, virtual obstacles, real devices, virtual devices, routes to a target, notes and indicia provided by the user can be superimposed on the different types of views. The display can display a number of types of images that include 3D model views, 2D model fluoroscopy views, real fluoroscopy views, real endoscopic views and model endoscopic views. This demonstrates that the Rai et al. reference teaches the limitation of “wherein the planning trajectory is superimposed on the unmasked 2D X-ray image on the display device using the registration defined by the optimum projection geometry” [Rai (paragraph [0037] “FIG. 1 also illustrates a display 60 showing a plurality of images. As will be described in greater detail herein, workstation 50 is configured to send to the display a number of types of images including 3D model views, 2D model fluoroscopy views, real fluoroscopy views, real endoscopic views, model endoscopic views, and a wide range of information superimposed on the views such as without limitation planning information, region of interests, virtual target markers, vessels, virtual obstacles, real devices, virtual devices, routes to a target, notes and indicia provided by the user, etc.”)];
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Under the broadest reasonable interpretation, the claims covers performance of the limitation in the mind or by pencil and paper and as a mathematical concept.
Claim 1
Regarding step 1, claim 1 is directed towards a system, method and medium, which has the claims fall within the eligible statutory categories of processes, machines, manufactures and composition of matter under 35 U.S.C. 101.
Claim 1
Regarding step 2A, prong 1, claim 1 recites “inputting a planning trajectory into at least one generated layer of the 3D image data set”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 1 recites “wherein the examination region contains the at least one foreign object and the unmasked 2D X-ray image includes the at least one foreign object”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 1 recites “identifying the at least one foreign object in the unmasked 2D X-ray image that is not contained in the 3D image data set”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 1 recites “determining an optimum projection geometry defining a registration between the 3D image data set and the unmasked 2D X-ray image”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 1 recites “calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image”. This limitation is calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image. Therefore, under MPEP 2106.04(a)(2), this limitation covers a mathematical concept, which falls in the “Mathematical Concept” grouping of abstract ideas.
Claim 1 recites “and identifying the optimum projection geometry as a registration between the 3D image data set and the masked 2D X-ray image corresponding to a local similarity optimum of the measure of similarity” This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 1 recites “wherein the planning trajectory is superimposed on the unmasked 2D X-ray image on the display device using the registration defined by the optimum projection geometry.” This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Regarding step 2A, prong 2, the limitation of “providing a 3D image data set” amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process, see MPEP 2106.05(g).
Also, the limitation of “providing a 3D image data set” amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the 3D image data set is being provided. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Also, the limitation of “recording an unmasked 2D X-ray image of an examination region by means of the X-ray apparatus” amounts to insignificant extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process, see MPEP 2106.05(g).
Also, the limitations of “and displaying at least one layer generated from the 3D image data set on a display device” and “and displaying the unmasked 2D X-ray image including the at least one foreign object on the display device” amounts to insignificant extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process, see MPEP 2106.05(g).
Also, the limitation of “generating a masked 2D X-ray image by masking out regions of the 2D X- ray image containing the at least one identified foreign object” amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the masking out of regions of the 2D X-ray image are being conducted. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Further, the claim language also does not include a computer or components of a computer, but if written with, for example, a processor, the claim language would still not be eligible under 35 U.S.C. 101. For example, adding the phrase “by a processor” to the claim language, would encompass the processor be recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, the additional element of a processor does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding Step 2B, the limitation of providing a 3D image data set is also shown to reflect the court decisions of Versata Dev. Group, Inc. v. SAP Am., Inc. iv. Storing and retrieving information in memory, shown in MPEP 2106.05(d) (II).
Also, the limitation of “providing a 3D image data set” amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the 3D image data set is being provided. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Also, the limitations of “recording an unmasked 2D X-ray image of an examination region by means of the X-ray apparatus”, and displaying at least one layer generated from the 3D image data set on a display device” and “and displaying the unmasked 2D X-ray image including the at least one foreign object on the display device” are also shown to reflect the court decisions of Versata Dev. Group, Inc. v. SAP Am., Inc. iv. Storing and retrieving information in memory, shown in MPEP 2106.05(d) (II).
Also, the limitation of “generating a masked 2D X-ray image by masking out regions of the 2D X- ray image containing the at least one identified foreign object” amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the masking out of regions of the 2D X-ray image are being conducted. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Also, the claim does not include the additional element of a processor. However, if written with a processor as shown above, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of the processor amounts no more than mere instructions to apply the exception using a generic computer component that does not impose any meaningful limits on practicing the abstract idea and therefore cannot provide an inventive concept (See MPEP 2106.05(b).
Claim 2
Dependent claim 2 recites “wherein the unmasked 2D X-ray image is a live image X-ray image recording”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 3
Dependent claim 3 recites “wherein the determination of the optimum projection geometry takes place by using an iterative and/or parallel optimization method”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 4
Dependent claim 4 recites “wherein the projection geometry is determined on a fixed grid by using a parallel method on a multiprocessor architecture”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 5
Dependent claim 5 recites “wherein the optimum projection geometry must satisfy a configurable threshold value of the similarity measure”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 6
Dependent claim 6 recites “wherein a subset of available geometric degrees of freedom is used to determine the projection geometry”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 7
Dependent claim 7 recites “wherein the planning trajectory is represented in a second display plane different from a first display plane used to display the at least one layer”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 7 recites “and wherein an intersection point of the planning trajectory is displayed in a third display plane”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 8
Dependent claim 8 recites “wherein, when a plurality of planning trajectories are represented, they are identified differently from one another and/or individual planning trajectories are masked off.”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 9
Dependent claim 9 recites “wherein movements of the X-ray apparatus and/or of an operating table are detected and included in the determination of the optimum projection geometry”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 10
Dependent claim 10 recites “wherein positions to be approached which facilitate an assessment of an intermediate operation result are determined by a criterion based on the planning trajectories”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 11
Dependent claim 11 recites “before recording the unmasked 2D X-ray image, calculating a virtual forward projection from the 3D image data set”. This limitation is calculating a virtual forward projection from the 3D image data set. Therefore, under MPEP 2106.04(a)(2), this limitation covers a mathematical concept, which falls in the “Mathematical Concept” grouping of abstract ideas.
Claim 12
Dependent claim 12 recites “after successful determination of an optimum projection geometry, superimposing the forward projection of the 3D image data set with the unmasked 2D X-ray image” amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the superimposing is being conducted. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Claim 13
Dependent claim 13 recites “wherein a new determination of the optimum projection geometry is triggered by operating a hand or foot switch, by changing an X-ray geometry, or by comparing a live image recording to the unmasked 2D X-ray image, wherein a new determination is triggered in the event of an excessive difference.”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 14
Dependent claim 14 recites “wherein the unmasked 2D X-ray image is recorded before the input of the planning trajectory and/or registration is determined before the input of the planning trajectory”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 15
Dependent claim 15 recites “wherein the display of the planning trajectory is no longer updated, or is hidden, if no projection geometry is generated which changes or improves the similarity value the previous projection geometry by a fixed relative or absolute value”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claim 16
Dependent claim 16 recites “a memory unit in which a recorded 3D image data set of X-rays is stored”. This limitation recites the additional element of memory. The memory is recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 16 recites “reconstruct the 3D image data set from X- rays to form a 3D volume”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 16 recites “permit determination of an optimum projection geometry between a forward projection of the 3D image data set and a recorded 2D X-ray image”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 16 recites “generate a 3D view of the 3D X-ray image data set having variable 3D views”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 16 recites “and for defining sectional planes for sectional plane image representations”. Under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 16 recites “and communicate with a GUI configured to output images and to receive input for inputting and changing the sectional planes and planning trajectories.”. This limitation amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process, see MPEP 2106.05(g).
Claim 17
Dependent claim 17 recites “A tangible, non-transitory computer-readable storage medium having stored thereon a computer program which can be loaded directly into a memory unit of a control unit for a conical beam computer tomograph, in particular a C-arm X-ray device, with program sections that cause the conical beam computer tomograph to perform the method according to Claim 1 when the computer program is executed in the control unit of the conical beam computer tomograph”. The claim language includes the additional element of medium. The medium is recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Also, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of the medium amounts no more than mere instructions to apply the exception using a generic computer component that does not impose any meaningful limits on practicing the abstract idea and therefore cannot provide an inventive concept (See MPEP 2106.05(b).
Claim 18
Dependent claim 18 recites “A tangible, non-transitory computer-readable storage medium having stored thereon program sections which can be read in and executed by a computer unit in order to perform the method according to Claim 1 when the program sections are executed by the computer unit.” The claim language includes the additional element of a computer and medium. The computer and medium are recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a computer and/or a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Claim 19
Dependent claim 19 recites “monitoring, by one or more processors in communication with the X-ray apparatus, an imaging geometry of the X-ray apparatus using signals received from one or more position or angle sensors of the X-ray apparatus”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 19 recites “and in response to detecting a change in the imaging geometry: discontinuing display of the planning trajectory superimposed on the unmasked 2D X-ray image displayed on the display device”. This limitation amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the discontinuing of displaying the planning trajectory superimposed on the unmasked 2D X-ray image is being conducted. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Dependent claim 19 recites “determining a new optimum projection geometry”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 19 recites “and restoring display of the planning trajectory superimposed on the unmasked 2D X-ray image using the new optimum projection geometry.”. This limitation amounts to mere instructions to apply an exception, where it recites an idea of a solution. The limitation doesn’t indicate how the restoring is being conducted. See MPEP 2106.05 (f) (1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
Claim 20
Dependent claim 20 recites “recording a subsequent 2D X-ray image of the examination region by means of the X-ray apparatus”. This limitation amounts to insignificant extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process, see MPEP 2106.05(g).
Dependent claim 20 recites “calculating an updated optimum projection geometry using the measure of similarity between the 3D image data set and the subsequent 2D X-ray image”. This limitation is calculating an updated optimum projection geometry using the measure of similarity between the 3D image data set and the subsequent 2D X-ray image. Therefore, under MPEP 2106.04(a)(2), this limitation covers a mathematical concept, which falls in the “Mathematical Concept” grouping of abstract ideas.
Dependent claim 20 recites “wherein the updated optimum projection geometry is calculated using a subset of available geometric degrees of freedom restricted to translations and rotations within an image plane of the X- ray apparatus”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Dependent claim 20 recites “and displaying the planning trajectory in the subsequent 2D X-ray image on the display device”. This limitation amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process, see MPEP 2106.05(g).
Dependent claim 20 recites “wherein registration of the planning trajectory with the subsequent 2D X-ray image on the display device is defined by the updated optimum projection geometry.”. This limitation doesn’t distinguish itself from being able to be conducted in the human mind or with pencil and paper. Therefore, under the broadest reasonable interpretation, this limitation is a process step that covers performance in the human mind or with the aid of pencil and paper. As such, this limitation falls within the “Mental Process” grouping of abstract ideas.
Claims 1-20 are therefore not drawn to eligible subject matter as they are directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35
U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2 and 4-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over
Rai et al. (U.S. PGPub 2012/0289825) in view of Gemmel et al. (U.S. PGPub 2018/0250076).
With respect to claim 1, Rai et al. discloses “A method for monitoring images by means of an X-ray apparatus during a surgical procedure by means of 3D-2D registration using at least one foreign object in an examination region” as [Rai et al. (paragraph [0052] “FIG. 4 is flow chart diagram illustrating a registration method 120 in accordance with the present invention for assisting a physician to track a surgical device during a live procedure. The steps may be carried out on a computer or system and include: step 122 creating a 3D model of a body organ; step 124 receiving at least one real-time fluoroscopy image of the body organ; 126 registering a 3D point from the model to a 2D fluoroscopy point in the at least one fluoroscopy image; Rai et al. paragraph [0034] “While surgical device 30 shown in FIG. 1 is intended to represent an endoscope, namely, a bronchoscope, the invention is not so limited. The surgical device may be a wide range of devices, instruments, implants, and markers which are visible under fluoroscopy, have a portion which is visible under fluoroscopy, or be modifiable such that it is visible under fluoroscopy. Examples, without limitation, include catheters, sheaths, needles, ablation devices, stents, valves, fiducial markers, seeds, coils, etc.”, The examiner considers the instrument to be the foreign object, since a foreign object can be an instrument, see paragraph [0023] of the specification)];
“providing a 3D image data set and displaying at least one layer generated from the 3D image data set on a display device” as [Rai et al. (paragraph [0011] “The method includes receiving at least one 3D image or 3D image data set of the non-rigid body organ in a first position and computing a 3D model of the body organ”, Rai et al. paragraph [0012] “In another embodiment of the present invention, the method further comprises the step of sending to a display at least one information for display fused with the 3D image and also fused with the 2D fluoroscopic images.”)];
“inputting a planning trajectory into at least one generated layer of the 3D image data set” as [Rai et al. (paragraph [0044] “The user input device allows a user such as the physician to add or input data and information as well as modify planning information and to make notes in the files and records.”, Rai et al. paragraph [0066] “The 3D model of the body organ is created from input including available image data from the subject patient such as high resolution computed tomography (HRCT) scans.”)];
“recording a 2D X-ray image of an examination region by means of the X-ray apparatus, wherein the examination region contains the at least one foreign object and the unmasked 2D X-ray image includes the at least one foreign object” as [Rai et al. (paragraph [0068] “Step 230 recites fluoroscopic camera pose. The real time camera pose (e.g., a 2D X-RAY unit, C-ARM, or fluoroscope camera) is an input to step 252 described more below. Camera pose may be obtained variously including, for example, with an external sensor, such as the Polaris Spectra tracker manufactured by NDI in Waterloo, Ontario, Canada. The position of fluoroscope camera is initially recorded and the fluoroscope image is initially registered with an image of the 3D model. The initial registration may be performed based on a rigid anatomical feature or another mark that is in a known location. The initial camera pose is recorded.”, Rai et al. paragraph [0034] “While surgical device 30 shown in FIG. 1 is intended to represent an endoscope, namely, a bronchoscope, the invention is not so limited. The surgical device may be a wide range of devices, instruments, implants, and markers which are visible under fluoroscopy, have a portion which is visible under fluoroscopy, or be modifiable such that it is visible under fluoroscopy. Examples, without limitation, include catheters, sheaths, needles, ablation devices, stents, valves, fiducial markers, seeds, coils, etc.”)];
“identifying the at least one foreign object in the unmasked 2D X-ray image that is not contained in the 3D image data set” as [Rai et al. (paragraph [0008] “using a 3D image data set of the area of examination and generating a 3D reconstructed image of the area of examination, taking at least one 2D X-ray image of the area of examination in which the instrument is visualized, registering the 3D reconstructed image relative to the 2D X-ray image”, Rai et al. paragraph [0051] “Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ.”)];
“determining an optimum projection geometry using a measure of similarity between the 3D image data set and the unmasked 2D X-ray image” as [Rai et al. (paragraph [0051] “Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ. Another surgical-guidance method may be provided without executing step 140. Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
“wherein determining the optimum projection geometry comprises: generating a masked 2D X-ray image by masking out regions of the 2D X- ray image containing the at least one identified foreign object” as [Rai et al. (paragraph [0051] “Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”, The examiner considers only the planned location of the surgical device being projected and superimposed on the 2D fluoroscopy view to be the generating a masked 2D X-ray image, since masking can be a marking out or omission or extraction of the at least one foreign object or an image region containing the foreign object during the calculation of the similarity measure, see paragraph [0025] of the specification)];
“and displaying the unmasked 2D X-ray image including the at least one foreign object on the display device” as [Rai et al. (paragraph [0051] “Finally, method 100 shows step 150 displaying the device location. Displaying the device location provides tracking and guidance information to the user or physician. In one embodiment, a virtual surgical device is superimposed onto a 3D view of the body organ. Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ.”)];
“wherein the planning trajectory is superimposed on the unmasked 2D X-ray image on the display device using the registration defined by the optimum projection geometry.” as [Rai et al. (paragraph [0037] ““FIG. 1 also illustrates a display 60 showing a plurality of images. As will be described in greater detail herein, workstation 50 is configured to send to the display a number of types of images including 3D model views, 2D model fluoroscopy views, real fluoroscopy views, real endoscopic views, model endoscopic views, and a wide range of information superimposed on the views such as without limitation planning information, region of interests, virtual target markers, vessels, virtual obstacles, real devices, virtual devices, routes to a target, notes and indicia provided by the user, etc.”)];
While Rai et al. teaches generating a masked 2D X-ray image by masking out regions of the 2D X- ray image containing the at least one identified foreign object, Rai et al. doesn’t explicitly disclose “calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image; and identifying the optimum projection geometry as a registration between the 3D image data set and the masked 2D X-ray image corresponding to a local similarity optimum of the measure of similarity”
Gemmel et al. discloses “calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image” as [Gemmel et al. (paragraph [0016] “If the X-ray images, from which the three-dimensional model data set is calculated, are therefore recorded with the instrument already located in the procedure site, the instrument also appears in the three-dimensional model data set, if the instrument is visible as a result of X-ray radiation, and this is undesirable since it can irritate the user to see a plurality of instruments.”)];
“and identifying the optimum projection geometry as a registration between the 3D image data set and the masked 2D X-ray image corresponding to a local similarity optimum of the measure of similarity” as [Gemmel et al. (paragraph [0016] “If the X-ray images, from which the three-dimensional model data set is calculated, are therefore recorded with the instrument already located in the procedure site, the instrument also appears in the three-dimensional model data set, if the instrument is visible as a result of X-ray radiation, and this is undesirable since it can irritate the user to see a plurality of instruments.”, Two-dimensional X-ray images are recorded during a procedure in order to determine an inaccurate or incomplete three-dimensional model data set of the underlying anatomy in the procedure site. During the recording of an instrument, it’s segmented in the X-ray images and the model data set and also removed from the model data set. The X-ray images are two-dimensional X-ray images that are used to determine the three-dimensional data set. The examiner considers the determination of whether the associating of the two-dimensional images recorded during a procedure to the three-dimensional model data set to be the measure of similarity, since the three-dimensional model data set is calculated from the two-dimensional images where an instrument can be removed. The measure of similarity is between the 2D X-ray images and the three-dimensional image data set, where the quality of a two-dimensional projection is analyzed, see paragraphs [0018] and [0025] – [0026] of the specification.)];
Rai et al. and Gemmel et al. are analogous art because they are from the same field endeavor of analyzing an X-ray image of a procedure site for a surgical procedure.
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to modify the teachings of Rai et al. of generating a masked 2D X-ray image by masking out regions of the 2D X- ray image containing the at least one identified foreign object by incorporating calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image; and identifying the optimum projection geometry as a registration between the 3D image data set and the masked 2D X-ray image corresponding to a local similarity optimum of the measure of similarity as taught by Gemmel et al. for the purpose of having image support for a person carrying out a minimally invasive procedure with an instrument in a procedure site of a patient.
Rai et al. in view of Gemmel et al. teaches calculating a measure of similarity between the 3D image data set and the masked 2D X-ray image; and identifying the optimum projection geometry as a registration between the 3D image data set and the masked 2D X-ray image corresponding to a local similarity optimum of the measure of similarity.
The motivation for doing so would have been because Gemmel et al. teaches that by having image support for a person carrying out a minimally invasive procedure with an instrument, the ability to have the user change the recording geometry until the position of the instrument is in an accurate position, can be accomplished. This allows the image to be viewed accurately by the user (Gemmel et al. (paragraph [0004])).
With respect to claim 2, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein the unmasked 2D X-ray image is a live image X-ray image recording.” as [Rai et al. (paragraph [0068] “Step 230 recites fluoroscopic camera pose. The real time camera pose (e.g., a 2D X-RAY unit, C-ARM, or fluoroscope camera) is an input to step 252 described more below. Camera pose may be obtained variously including, for example, with an external sensor, such as the Polaris Spectra tracker manufactured by NDI in Waterloo, Ontario, Canada. The position of fluoroscope camera is initially recorded and the fluoroscope image is initially registered with an image of the 3D model. The initial registration may be performed based on a rigid anatomical feature or another mark that is in a known location. The initial camera pose is recorded.”)];
With respect to claim 4, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein the projection geometry is determined on a fixed grid by using a parallel method on a multiprocessor architecture.” as [Rai et al. (paragraph [0051] “Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ. Another surgical-guidance method may be provided without executing step 140. Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
With respect to claim 5, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein the optimum projection geometry must satisfy a configurable threshold value of the similarity measure” as [Rai et al. (paragraph [0058] “The 3D deformation model is desired because the 3D model of the subject in the first position may not match that of the subject in the second real-time position. The 3D deformation model may either be available before hand or is estimated by step 128. The 3D deformation model applied to the 3D image generates a modified set of 3D locations which are then used for registration.”, Rai et al. paragraph [0059] “In one embodiment a loop 129 between step 126 and step 128 is continued until a maximum estimated deformation in step 128 is less than a threshold value. This computation is shown in step 130 of FIG. 4. An example of a threshold value ranges from 0.5 and 5 mm and can be set to about 1 mm.”)];
With respect to claim 6, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein a subset of available geometric degrees of freedom is used to determine the projection geometry” as [Rai et al. (paragraph [0085] “The plane in which the left/right arcs lie correspond to the anatomy depicted in a transverse CT slice. Likewise the plane in which the cranial/caudal rotates corresponds to a sagittal slice. Exemplary ranges of subsets include -60 to 60 degrees left/right in 5 degree increments as well as -45 to 45 cranium/caudal in 5 deg increments with both deviations measured from AP.”)];
With respect to claim 7, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein the planning trajectory is represented in a second display plane different from a first display plane used to display the at least one layer, and wherein an intersection point of the planning trajectory is displayed in a third display plane.” as [Rai et al. (paragraph [0037] “workstation 50 is configured to send to the display a number of types of images including 3D model views, 2D model fluoroscopy views, real fluoroscopy views, real endoscopic views, model endoscopic views, and a wide range of information superimposed on the views such as without limitation planning information, region of interests”, By there being different regions of interest that are displayed, demonstrates that a second display is different from a first display)];
With respect to claim 8, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein, when a plurality of planning trajectories are represented, they are identified differently from one another and/or individual planning trajectories are masked off.” as [Rai et al. (paragraph [0044] “The user input device allows a user such as the physician to add or input data and information as well as modify planning information and to make notes in the files and records.”)];
With respect to claim 9, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein movements of the X-ray apparatus and/or of an operating table are detected and included in the determination of the optimum projection geometry.” as [Rai et al. (paragraph [0081] “In one embodiment the computation step is carried out so that the length of surgical device appears as long as possible throughout the rotation of the C-arm along an arc.”)];
With respect to claim 10, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein positions to be approached which facilitate an assessment of an intermediate operation result are determined by a criterion based on the planning trajectories.” as [Rai et al. (paragraph [0079] “The information may also be provided from a surgical planning module or workstation adapted to identify surgical plans. Planning information may include pre-guidance knowledge of the planned location and orientation of the surgical device. Such planning information may include fusing the surgical device with segmented anatomical structures and/or the CT data set.”)];
With respect to claim 11, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “before recording the unmasked 2D X-ray image, calculating a virtual forward projection from the 3D image data set” as [Rai et al. (paragraph [0051] “Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
With respect to claim 12, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “after successful determination of an optimum projection geometry, superimposing the forward projection of the 3D image data set with the unmasked 2D X-ray image.” as [Rai et al. (paragraph [0051] “Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
With respect to claim 13, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein a new determination of the optimum projection geometry is triggered by operating a hand or foot switch, by changing an X-ray geometry, or by comparing a live image recording to the unmasked 2D X-ray image, wherein a new determination is triggered in the event of an excessive difference.” as [Rai et al. (paragraph [0051] “Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
With respect to claim 14, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein the unmasked 2D X-ray image is recorded before the input of the planning trajectory and/or registration is determined before the input of the planning trajectory.” as [Rai et al. (paragraph [0068] “Step 230 recites fluoroscopic camera pose. The real time camera pose (e.g., a 2D X-RAY unit, C-ARM, or fluoroscope camera) is an input to step 252 described more below. Camera pose may be obtained variously including, for example, with an external sensor, such as the Polaris Spectra tracker manufactured by NDI in Waterloo, Ontario, Canada. The position of fluoroscope camera is initially recorded and the fluoroscope image is initially registered with an image of the 3D model. The initial registration may be performed based on a rigid anatomical feature or another mark that is in a known location. The initial camera pose is recorded”)];
With respect to claim 15, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “wherein the display of the planning trajectory is no longer updated, or is hidden, if no projection geometry is generated which changes or improves the similarity value the previous projection geometry by a fixed relative or absolute value.” as [Rai et al. (paragraph [0047] “FIG. 3 is a flow chart illustrating an overview of a surgical procedure 100 for tracking a surgical device and displaying information about the surgical device in real time.”, Fig. 3)];
With respect to claim 16, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “A device for recording image data sets of X-ray images, in particular a C-arm X- ray apparatus, configured to carry out the method of Claim 1” as [Rai et al. (paragraph [0068] “Step 230 recites fluoroscopic camera pose. The real time camera pose (e.g., a 2D X-RAY unit, C-ARM, or fluoroscope camera) is an input to step 252 described more below. Camera pose may be obtained variously including, for example, with an external sensor, such as the Polaris Spectra tracker manufactured by NDI in Waterloo, Ontario, Canada. The position of fluoroscope camera is initially recorded and the fluoroscope image is initially registered with an image of the 3D model.”, Fig. 5)];
“the device comprising: a memory unit in which a recorded 3D image data set of X-rays is stored; and one or more processors” as [Rai et al. (paragraph [0020] “The workstation includes a processor operable to receive a 3D image of the non-rigid body organ in a first position and computing a 3D model of the body organ”)];
“reconstruct the 3D image data set from X- rays to form a 3D volume” as [Rai et al. (paragraph [0052] “FIG. 4 is flow chart diagram illustrating a registration method 120 in accordance with the present invention for assisting a physician to track a surgical device during a live procedure. The steps may be carried out on a computer or system and include: step 122 creating a 3D model of a body organ; step 124 receiving at least one real-time fluoroscopy image of the body organ; 126 registering a 3D point from the model to a 2D fluoroscopy point in the at least one fluoroscopy image)];
“permit determination of an optimum projection geometry between a forward projection of the 3D image data set and a recorded 2D X-ray image” as [Rai et al. (paragraph [0020] “The workstation includes a processor operable to receive a 3D image of the non-rigid body organ in a first position and computing a 3D model of the body organ”)];
“generate a 3D view of the 3D X-ray image data set having variable 3D views and for defining sectional planes for sectional plane image representations” as [Rai et al. (paragraph [0020] “The workstation includes a processor operable to receive a 3D image of the non-rigid body organ in a first position and computing a 3D model of the body organ; to receive a real-time fluoroscopy image from a fluoroscopy unit of the body organ and showing a reference mark in the non-rigid body organ; and to deform the 3D model of the non-rigid body organ to match the body organ of the subject in a second position and based on the reference mark. In one embodiment the reference mark is part of the surgical device enabling tracking of the surgical device through the body organ in real time based on the reference mark. In another embodiment the workstation is operable to estimate a deforming field or deformation model of the 3D model for matching any point of the real time fluoroscopy image to a point in the body organ. In another embodiment the system further includes a display for presenting the surgical device in combination or fused with the 3D model. In another embodiment the system further comprises a catheter or instrument which includes a reference mark for facilitating fluoroscopy based tracking during a real time procedure.”
“and communicate with a GUI configured to output images and to receive input for inputting and changing the sectional planes and planning trajectories.” as [Rai et al. (paragraph [0020] “The workstation includes a processor operable to receive a 3D image of the non-rigid body organ in a first position and computing a 3D model of the body organ; to receive a real-time fluoroscopy image from a fluoroscopy unit of the body organ and showing a reference mark in the non-rigid body organ; and to deform the 3D model of the non-rigid body organ to match the body organ of the subject in a second position and based on the reference mark. In one embodiment the reference mark is part of the surgical device enabling tracking of the surgical device through the body organ in real time based on the reference mark. In another embodiment the workstation is operable to estimate a deforming field or deformation model of the 3D model for matching any point of the real time fluoroscopy image to a point in the body organ. In another embodiment the system further includes a display for presenting the surgical device in combination or fused with the 3D model. In another embodiment the system further comprises a catheter or instrument which includes a reference mark for facilitating fluoroscopy based tracking during a real time procedure.”)];
With respect to claim 17, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “A tangible, non-transitory computer-readable storage medium having stored thereon a computer program which can be loaded directly into a memory unit of a control unit for a conical beam computer tomograph, in particular a C-arm X-ray device, with program sections that cause the conical beam computer tomograph to perform the method according to Claim 1 when the computer program is executed in the control unit of the conical beam computer tomograph.” as [Rai et al. (paragraph [0039] “FIG. 2 illustrates a surgical device tracking system 90 including a workstation or specially programmed computer 50. The workstation 50 shown in FIG. 2 includes a processor 70 operable to determine the 3D location of the surgical device in real time based on, amongst other things, the real-time fluoroscopy images as will be described in more detail herein.”)];
With respect to claim 18, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “A tangible, non-transitory computer-readable storage medium having stored thereon program sections which can be read in and executed by a computer unit in order to perform the method according to Claim 1 when the program sections are executed by the computer unit.” as [Rai et al. (paragraph [0020] “The workstation includes a processor operable to receive a 3D image of the non-rigid body organ in a first position and computing a 3D model of the body organ; to receive a real-time fluoroscopy image from a fluoroscopy unit of the body organ and showing a reference mark in the non-rigid body organ; and to deform the 3D model of the non-rigid body organ to match the body organ of the subject in a second position and based on the reference mark. In one embodiment the reference mark is part of the surgical device enabling tracking of the surgical device through the body organ in real time based on the reference mark. In another embodiment the workstation is operable to estimate a deforming field or deformation model of the 3D model for matching any point of the real time fluoroscopy image to a point in the body organ. In another embodiment the system further includes a display for presenting the surgical device in combination or fused with the 3D model. In another embodiment the system further comprises a catheter or instrument which includes a reference mark for facilitating fluoroscopy based tracking during a real time procedure.”)];
With respect to claim 19, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “monitoring, by one or more processors in communication with the X-ray apparatus, an imaging geometry of the X-ray apparatus using signals received from one or more position or angle sensors of the X-ray apparatus” as [Rai et al. (paragraph [0035] “A fluoroscope 40 takes real time fluoroscopic video of the subject. Video frames of the video or images are collected or received by workstation 50 for processing. Real time images may also be displayed on a video monitor 62. The location and pose of the fluoroscopy camera 42 is tracked with a tracking sensor 44. In the fluoroscopy unit shown in FIG. 1, an optically visible symbol 46 is observed by the optical tracking sensor 44 to provide to workstation 50 information about the location, orientation and pose of the fluoroscopy camera 42 in real time.”
“and in response to detecting a change in the imaging geometry: discontinuing display of the planning trajectory superimposed on the unmasked 2D X-ray image displayed on the display device” as [Rai et al. (paragraph [0035] “A fluoroscope 40 takes real time fluoroscopic video of the subject. Video frames of the video or images are collected or received by workstation 50 for processing. Real time images may also be displayed on a video monitor 62. The location and pose of the fluoroscopy camera 42 is tracked with a tracking sensor 44. In the fluoroscopy unit shown in FIG. 1, an optically visible symbol 46 is observed by the optical tracking sensor 44 to provide to workstation 50 information about the location, orientation and pose of the fluoroscopy camera 42 in real time”, Rai et al. (paragraph [0038] “As described herein, the information and location may be displayed in a number of ways to the physician to assist tracking the surgical device in real time, and in the event planning information has been provided to the workstation, to guide the physician to a target.”, The examiner considers the information and location being displayed to a physician to be detecting a change and discontinuing to display, since real time video of the subject is collected and sent to the workstation, where different views of this information is displayed)];
“determining a new optimum projection geometry” as [Rai et al. (paragraph [0035] “A fluoroscope 40 takes real time fluoroscopic video of the subject. Video frames of the video or images are collected or received by workstation 50 for processing. Real time images may also be displayed on a video monitor 62. The location and pose of the fluoroscopy camera 42 is tracked with a tracking sensor 44. In the fluoroscopy unit shown in FIG. 1, an optically visible symbol 46 is observed by the optical tracking sensor 44 to provide to workstation 50 information about the location, orientation and pose of the fluoroscopy camera 42 in real time”, Rai et al. (paragraph [0038] “As described herein, the information and location may be displayed in a number of ways to the physician to assist tracking the surgical device in real time, and in the event planning information has been provided to the workstation, to guide the physician to a target.”, Rai et al. (paragraph [0051] “Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ. Another surgical-guidance method may be provided without executing step 140. Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
“and restoring display of the planning trajectory superimposed on the unmasked 2D X-ray image using the new optimum projection geometry.” as [Rai et al. (paragraph [0035] “A fluoroscope 40 takes real time fluoroscopic video of the subject. Video frames of the video or images are collected or received by workstation 50 for processing. Real time images may also be displayed on a video monitor 62. The location and pose of the fluoroscopy camera 42 is tracked with a tracking sensor 44. In the fluoroscopy unit shown in FIG. 1, an optically visible symbol 46 is observed by the optical tracking sensor 44 to provide to workstation 50 information about the location, orientation and pose of the fluoroscopy camera 42 in real time”, Rai et al. (paragraph [0038] “As described herein, the information and location may be displayed in a number of ways to the physician to assist tracking the surgical device in real time, and in the event planning information has been provided to the workstation, to guide the physician to a target.”)];
With respect to claim 20, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above, and Rai et al. discloses “recording a subsequent 2D X-ray image of the examination region by means of the X-ray apparatus” as [Rai et al. (paragraph [0068] “Step 230 recites fluoroscopic camera pose. The real time camera pose (e.g., a 2D X-RAY unit, C-ARM, or fluoroscope camera) is an input to step 252 described more below. Camera pose may be obtained variously including, for example, with an external sensor, such as the Polaris Spectra tracker manufactured by NDI in Waterloo, Ontario, Canada. The position of fluoroscope camera is initially recorded and the fluoroscope image is initially registered with an image of the 3D model. The initial registration may be performed based on a rigid anatomical feature or another mark that is in a known location. The initial camera pose is recorded.”, Rai et al. paragraph [0034] “While surgical device 30 shown in FIG. 1 is intended to represent an endoscope, namely, a bronchoscope, the invention is not so limited. The surgical device may be a wide range of devices, instruments, implants, and markers which are visible under fluoroscopy, have a portion which is visible under fluoroscopy, or be modifiable such that it is visible under fluoroscopy. Examples, without limitation, include catheters, sheaths, needles, ablation devices, stents, valves, fiducial markers, seeds, coils, etc.”)];
“calculating an updated optimum projection geometry using the measure of similarity between the 3D image data set and the subsequent 2D X-ray image” as [Rai et al. (paragraph [0051] “Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ. Another surgical-guidance method may be provided without executing step 140. Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
“wherein the updated optimum projection geometry is calculated using a subset of available geometric degrees of freedom restricted to translations and rotations within an image plane of the X- ray apparatus” as [Rai et al. (paragraph [0051] “Additionally, because the 3D location of the surgical device is computed from step 140, the surgical device may be superimposed on model projection 2D fluoroscopy view. Indeed, a number of views showing the surgical device, as well as any additional markers, routes, indicia and planning information may be displayed with model or real images of the organ. Another surgical-guidance method may be provided without executing step 140. Instead of estimating location of surgical device, only the "planned" location of the surgical device and ROI is projected and superimposed on 2D fluoroscopy view. The physician may then visually compare the "planned" projection and actual image of the surgical device to navigate to the ROI.”)];
“and displaying the planning trajectory in the subsequent 2D X-ray image on the display device, wherein registration of the planning trajectory with the subsequent 2D X-ray image on the display device is defined by the updated optimum projection geometry.” as [Rai et al. (paragraph [0035] “A fluoroscope 40 takes real time fluoroscopic video of the subject. Video frames of the video or images are collected or received by workstation 50 for processing. Real time images may also be displayed on a video monitor 62. The location and pose of the fluoroscopy camera 42 is tracked with a tracking sensor 44. In the fluoroscopy unit shown in FIG. 1, an optically visible symbol 46 is observed by the optical tracking sensor 44 to provide to workstation 50 information about the location, orientation and pose of the fluoroscopy camera 42 in real time”, Rai et al. (paragraph [0038] “As described herein, the information and location may be displayed in a number of ways to the physician to assist tracking the surgical device in real time, and in the event planning information has been provided to the workstation, to guide the physician to a target.”)];
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over
Rai et al. in view of Gemmel et al. in further view of Carlson et al. (U.S. PGPub 2017/0249725).
With respect to claim 3, the combination of Rai et al. and Gemmel et al. discloses the method of claim 1 above.
While the combination of Rai et al. and Gemmel et al. teaches determining an optimum projection geometry using a measure of similarity between the 3D image data set and the 2D X-ray image, Rai et al. and Gemmel et al. do not explicitly disclose “wherein the determination of the optimum projection geometry takes place by using an iterative and/or parallel optimization method.”
Carlson et al. discloses “wherein the determination of the optimum projection geometry takes place by using an iterative and/or parallel optimization method.” as [Carlson et al. (paragraph [0034] “Alternatively, the entire surface in the two models can be matched by scaling, rotation, and translation through various well-known optimization techniques such as simulated annealing.”, Carlson et al. paragraph [0048] “The optimization routine of FIG. 10 is based on the concept of simulated annealing. The routine of FIG. 10 randomly tests small alterations of the orientation/scale of the surface model to see if the “fit” can be improved. If the small alteration would improve the fit, the routine will always accept the fit (i.e., because the acceptance probability will be greater than one and the random number generated in step 1011 must be equal to or less than one). However, in order to avoid becoming stuck on a local maxima of a fit score, the mechanism of FIG. 10 will also sometimes accept an alteration that results in a worse fit score (as long as the calculated acceptance probability (step 1009) is greater than the random number generated in step 1011). As a result, the alignment optimization technique can move beyond a “local” optimized solution and locate an even better alignment.”)];
Rai et al., Gemmel et al. and Carlson et al. are analogous art because they are from the same field endeavor of analyzing X-ray images.
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to modify the teachings of Rai et al. and Gemmel et al. of determining an optimum projection geometry sing a measure of similarity between the 3D image data set and the 2D X-ray image by incorporating wherein the determination of the optimum projection geometry takes place by using an iterative and/or parallel optimization method as taught by Carlson et al. for the purpose of reconstructing a 3D image of a patient’s teeth.
Rai et al. in view of Gemmel et al. in further view of Carlson et al. teaches wherein the determination of the optimum projection geometry takes place by using an iterative and/or parallel optimization method.
The motivation for doing so would have been because Carlson et al. teaches that by reconstructing a 3D image of a patient’s teeth, the ability to remove artifacts from x-ray data can be accomplished, where the user can see accurate x-ray data (Carlson et al. (paragraph [0008] – [0009])).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BERNARD E COTHRAN whose telephone number is (571)270-5594. The examiner can normally be reached 9AM -5:30PM EST M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan F Pitaro can be reached at (571)272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BERNARD E COTHRAN/Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188