DETAILED ACTIONNotice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant Response to Official Action
The response filed on 3/4/2026 has been entered and made of record.
Acknowledgment
Claims 1-26, canceled on 3/4/2026, are acknowledged by the examiner.
Claims 27-34, 36-37, 39-41, 43, and 45-46, amended on 3/4/2026, are acknowledged by the examiner.
Response to Arguments
Applicant’s arguments with respect to claims 27, 28, 46, and their dependent claims have been considered but they are moot in view of the new grounds of rejection necessitated by amendments initiated by the applicant. Examiner addresses the main arguments of the Applicant as below.
Regarding the claim objection, the amendment filed on 3/4/2026 addresses the issue. As a result, the claim objection is withdrawn.
Regarding the drawing objection, the argument filed on 3/4/2026 is not persuasive. For instance, in paragraph 4 on page 1 of the Remarks, the Applicant stated that “As a first matter, the Specification teaches that "the invention may be implemented in a computer program for running on a computer system”, at least including code portions for performing steps of a method according to the invention when run on a programmable apparatus, such as a computer system or enabling a programmable apparatus to perform functions of a device or system according to the invention." However, there is no drawing for “eye-tracking hardware”, “users gaze”, “atlas image”, “pre-existing contours”, “deformable registration”, “affine registration”, “rigid registration”, “machine learning”. As a result, the drawing objection is maintained.
Regarding the 35 U.S.C. 112(f) interpretation, the Remarks filed on 3/4/2026 acknowledged the 35 U.S.C. 112(f) interpretation. As a result, the 35 U.S.C. 112(f) interpretation is maintained.
Regarding the 35 U.S.C. 112(b) rejection, the amendment filed on 3/4/2026 addresses the issue. As a result, the 35 U.S.C. 112(b) rejection is withdrawn.
Regarding the U.S.C. 102/103 rejection, the Applicant amended the claim then argued that, “Applicant respectfully disagrees that the reference identically discloses every element of the claims in the same detail as claimed. For example, with respect to independent claim 27, the reference fails to teach at least "wherein the guidance comprises clinical guidelines for contouring the structure or at least one atlas image showing at least one example contour for the structure."” [Paragraph 5 on page 7 of the Remarks]. Examiner respectfully disagrees with the Applicant’s argument. The cited references teaches argued limitations as follow.
wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] comprises clinical guidelines for contouring the structure ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19] ((Module 7 is an image annotation module that includes image processing algorithms or advanced Deep Learning based techniques for detecting anatomical landmarks in a medical image and identifying contours or boundaries of anatomical objects in a medical image, such as bone or soft tissue boundaries. Anatomical Landmark detection stands for the identification of key elements of an anatomical body part that potentially have a high level of similarity with the same anatomical body part of other patients. The Deep Learning algorithm encompasses various conventional layers and its final output layer provides self-driven data, including, but not limited to, the system coordinates of important points in the image. In the current invention, landmark detection can be also applied to determine some key positions of anatomical parts in the body, for example, left/right of the femur, and left/right of the shoulder.) [Boddington: col. 11, line 51-66; Figs. 5A- 12B]; (a visual display configured to provide the intra-operative surgical guidance to the orthopedic surgeon conducting an alignment or fixation procedure) [Boddington: col. 3, line 65-67; Figs. 5A- 12B] – Note: Please see contours and guidance displayed in Figs. 5A- 12B; (The computing platform 100 includes a plurality of software modules 103 to receive and process medical image data, including modules for image distortion correction, image feature detection, image annotation and segmentation, image to image registration, three-dimensional estimation from two-dimensional images, medical image visualization, and one or more surgical guidance modules that use artificial intelligence models to classify images as predictive of optimal or suboptimal surgical outcomes) [Boddington: col. 9, line 1-9; Figs. 5A- 31]) or at least one atlas image showing at least one example contour for the structure ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]).
Accordingly, the Examiner respectfully maintains the rejections and applicability of the arts used.
Objections
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, “eye-tracking hardware”, “users gaze”, “atlas image”, “pre-existing contours”, “deformable registration”, “affine registration”, “rigid registration”, “machine learning” must be shown or the feature(s) must be canceled from the claims 27-46. No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) ELEMENT IN CLAIM FOR A COMBINATION.—An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as "configured to" or "so that"; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 27-29, 32-33, 36, 40, 43, and 45-46 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Piper (US Patent 10,930,002 B2), (“Piper”).
Regarding claim 27, Piper meets the claim limitations as follows:
A method for contouring a medical image in a contouring system (systems and methods for contouring a set of medical images) [Piper: col. 1, line 30-31; Title, Abstract] comprising (An example method may include the following steps) [Piper: col. 1, line 51-52]:providing at least one medical image ((receiving a source image from the set of medical images) [Piper: col. 1, line 52-53]; (The contour transformation engine may be configured to receive source contour data that identifies the one or more objects within the source image) [Piper: col. 1, line 41-44]) to be contoured by a user (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]; determining that the user (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13] has initiated contouring ((For instance, the initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]) of a structure on the medical image ((contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]; (At decision step 410, a user input is received to either accept or modify the automatically generated target contour. If the automatically generated contour is accepted, then the method proceeds to step 412. Otherwise, if the user chooses to modify the automatically generated target contour, then the contour is manually edited at step 414 before the method proceeds to step 412. The dotted line between steps 414 and 404 signifies that if a target contour is manually edited, then during the next iteration of the contouring method (if any), the manually edited contour may be used as the source contour for the next target image) [Piper: col. 6, line 26-36]);
determining the structure (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55] that is being contoured (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19];displaying (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9], in response to the determination of the structure being contoured (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55], guidance for the contouring of the structure that is being contoured ((receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7]; (With reference again to FIG. 1, the image and contour reslicing block 104 generates resliced image and contour
data, which is received by the seed grid generator 106. At the seed grid generator 106, seed grid template data is received from the seed grid template database 114 along with an input for selecting one or more of the seed grid templates from the
seed grid template database 114. The seed grid template database 114 includes seed grid templates that may be used to create a seed grid 402 on the resliced image, as shown in FIG. 4. FIG. 4 illustrates a seed grid 402 overlaid on a resliced
contoured image slice, such as the example from FIG. 3B. The image contour slices portrayed in FIG. 3B are also represented in FIG. 4. For example, a clinical target volume contour 404, a planning target volume contour 406, and a chest wall contour 408 are each illustrated. The seed grid 402 is centered at the isocenter 410. Each image slice that has been resliced may include a seed grid 402. Including a seed grid 402 at each possible image slice allows for a treatment plan to be accurately planned at different depth levels of a target volume) [Piper: col. 4, line 45-65; Figs 1, 4]; (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device. FIG. 4 is a flow diagram of another example method 400 for contouring a set of medical images. In step 402, the method is initialized with an initial source image along with associated source contour data. Then, in step 404, a target image is identified for contouring based on the source image and source contour. In step 406, a deformation algorithm is applied to the source and target images to generate deformation field data that indicates how one or more objects within the images have changed from the source image to the target image. The deformation field data is then applied to the source contour data in step 408 to automatically generate contour data for the target image by transforming the source contour to match the changes from the source image to the target image) [Piper: col. 6, line 5-15; Figs. 2, 4, 6-7] – Note: Figs 4, as well as Figs. 2, 6-7, display the guidance for contouring the image); contouring the structure that is being contoured in accordance with the guidance (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64],wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] comprises clinical guidelines for contouring the structure ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) or at least one atlas image showing at least one example contour for the structure ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]).
Regarding claim 28, Piper meets the claim limitations as follows:
A method for reviewing contours on a contoured medical image ((systems and methods for contouring a set of medical images) [Piper: col. 1, line 30-31; Title, Abstract]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]) comprising (An example method may include the following steps) [Piper: col. 1, line 51-52]:providing at least one medical image ((receiving a source image from the set of medical images) [Piper: col. 1, line 52-53]; (The contour transformation engine may be configured to receive source contour data that identifies the one or more objects within the source image) [Piper: col. 1, line 41-44]) annotated with one or more pre-existing contours (a set of contoured images has been generated) [Piper: col. 5, line 4];
displaying the at least one medical image annotated with the one or more pre-existing contours ((the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]); determining that a user has initiated reviewing ((The initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]; (a user input is received to either accept or modify the automatically generated target contour) [Piper: col. 6, line 26-27]; (the source image and contour data may be selected from a database of stored images and contours or may otherwise be provided or selected by the user) [Piper: col. 5, line 12-14]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31]) a one of the one or more pre-existing contour on the at least one medical image ((Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (At decision step 410, a user input is received to either accept or modify the automatically generated target contour. If the automatically generated contour is accepted, then the method proceeds to step 412. Otherwise, if the user chooses to modify the automatically generated target contour, then the contour is manually edited at step 414 before the method proceeds to step 412. The dotted line between steps 414 and 404 signifies that if a target contour is manually edited, then during the next iteration of the contouring method (if any), the manually edited contour may be used as the source contour for the next target image) [Piper: col. 6, line 26-36]);
determine a structure associated with the one of the one or more pre-existing contour ((The initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]; (the source image and contour data may be selected from a database of stored images and contours or may otherwise be provided or selected by the user) [Piper: col. 5, line 12-14]); displaying (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9], in response to the determination of the structure being contoured (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55], guidance for the contouring of the structure ((receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7]; (With reference again to FIG. 1, the image and contour reslicing block 104 generates resliced image and contour data, which is received by the seed grid generator 106. At the seed grid generator 106, seed grid template data is received from the seed grid template database 114 along with an input for selecting one or more of the seed grid templates from the seed grid template database 114. The seed grid template database 114 includes seed grid templates that may be used to create a seed grid 402 on the resliced image, as shown in FIG. 4. FIG. 4 illustrates a seed grid 402 overlaid on a resliced contoured image slice, such as the example from FIG. 3B. The image contour slices portrayed in FIG. 3B are also represented in FIG. 4. For example, a clinical target volume contour 404, a planning target volume contour 406, and a chest wall contour 408 are each illustrated. The seed grid 402 is centered at the isocenter 410. Each image slice that has been resliced may include a seed grid 402. Including a seed grid 402 at each possible image slice allows for a treatment plan to be accurately planned at different depth levels of a target volume) [Piper: col. 4, line 45-65; Figs 1, 4]; (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device. FIG. 4 is a flow diagram of another example method 400 for contouring a set of medical images. In step 402, the method is initialized with an initial source image along with associated source contour data. Then, in step 404, a target image is identified for contouring based on the source image and source contour. In step 406, a deformation algorithm is applied to the source and target images to generate deformation field data that indicates how one or more objects within the images have changed from the source image to the target image. The deformation field data is then applied to the source contour data in step 408 to automatically generate contour data for the target image by transforming the source contour to match the changes from the source image to the target image) [Piper: col. 6, line 5-15; Figs. 2, 4, 6-7]; (receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7] – Note: Figs 4, as well as Figs. 2, 6-7, display the guidance for contouring the image);
wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] comprises clinical guidelines for contouring the structure ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) or at least one atlas image showing at least one example contour for the structure ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]).
Regarding claim 29, Piper meets the claim limitations as set forth in claim 28. Piper further meets the claim limitations as follow.
further comprising completing the review of the one of the one or more pre-existing contour or editing the one of the one or more pre-existing contour in accordance with the displayed guidance ((Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11] ; (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31]).
Regarding claim 32, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
outputting (The image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9], responsive to contouring the structure that is being contoured (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55], the user-initiated a contour structure ((Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (The image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9]).
Regarding claim 33, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein the determination that the user has initiated contouring or reviewing of a contour is determined ((Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]) when the user starts to draw or edit a contour of the structure on the at least one medical image (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31].
Regarding claim 36, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein the determination of the structure (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55] being reviewed or contoured is made ((Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11] ; (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31]) by finding a mapping between the at least one medical image and at least one atlas image; wherein the mapping is done using image registration (The image deformation engine 102 may, for example, utilize a free-form intensity-based deformable registration algorithm that maps similar tissues from the source image to the target image by matching intensity values from one image to the next. More specifically, a free-form intensity-based deformable registration algorithm is a type of deformation algorithm that generates deformation field data in the form of an x-y mapping for each pixel in the target image. For each pixel in the target image, the x-y mapping identifies the pixel in the source image that most likely corresponds to similar anatomy in the target image based on the pixel intensities. In this way, pixel positions along the edge of an object ( e.g., a piece of anatomy) in the source image may be mapped to corresponding pixel positions along the edge of the same object in the target image, showing how the object (e.g., anatomy) has changed from the source image to the target image) [Piper: col. 3, line 4-20]; wherein the image registration comprises one or more of: deformable registration (The image deformation engine 102 may, for example, utilize a free-form intensity-based deformable registration algorithm that maps similar tissues from the source image to the target image by matching intensity values from one image to the next. More specifically, a free-form intensity-based deformable registration algorithm is a type of deformation algorithm that generates deformation field data in the form of an x-y mapping for each pixel in the target image. For each pixel in the target image, the x-y mapping identifies the pixel in the source image that most likely corresponds to similar anatomy in the target image based on the pixel intensities. In this way, pixel positions along the edge of an object ( e.g., a piece of anatomy) in the source image may be mapped to corresponding pixel positions along the edge of the same object in the target image, showing how the object (e.g., anatomy) has changed from the source image to the target image) [Piper: col. 3, line 4-20], affine registration or rigid registration.
Regarding claim 40, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein the contouring of the determined structure by the user ((the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55]; (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) is one of manual (if the user chooses to
modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31], semi-automatic or automatic contouring (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64].
Regarding claim 43, Piper meets the claim limitations as set forth in claim 28. Piper further meets the claim limitations as follow.
wherein the one or more pre-existing contours displayed to the user are either manual contours from a previous contouring session (At decision step 410, a user input is received to either accept or modify the automatically generated target contour. If the automatically generated contour is accepted, then the method proceeds to step 412. Otherwise, if the user chooses to modify the automatically generated target contour, then the contour is manually edited at step 414 before the method proceeds to step 412. The dotted line between steps 414 and 404 signifies that if a target contour is manually edited, then during the next iteration of the contouring method (if any), the manually edited contour may be used as the source contour for the next target image) [Piper: col. 6, line 26-36] or automatically produced contours (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64].
Regarding claim 45, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein the at least one medical image is one a CT scan (FIGS. 8-12 are illustrations of contoured image slices from a CT scan of a patient's pelvic region) [Piper: col. 8, line 59-60; Figs. 8-12], an MRI scan, a PET scan, a SPECT scan or an ultrasound scan (Medical images, such CT (computed tomography), MR (magnetic resonance), US (ultrasound), or PET (positron emission tomography) scans, are regularly contoured to identify certain pieces of anatomy within the image.) [Piper: col. 1, line 13-17].
Regarding claim 46, Piper meets the claim limitations as follows:
A system for contouring at least one medical image (systems and methods for contouring a set of medical images) [Piper: col. 1, line 30-31; Title, Abstract] comprising:
a display (a display device) [Piper: col. 6, line 2; Fig. 3] for displaying the at least one medical image (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device) [Piper: col. 6, line 5-8; Figs. 2, 4, 6-7] to be contoured by a user (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]; a processor (a processor to perform the methods' operations and implement the systems described herein) [Piper: col. 11, line 58-59; Fig. 3] for determining that the user has initiated contouring (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13] of a structure on the at least one medical image (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]; the processor (a processor to perform the methods' operations and implement the systems described herein) [Piper: col. 11, line 58-59; Fig. 3] determining the structure that is being contoured ((the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55]; (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]); and
in response to the determination of the structure, (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55] the system displaying guidance to the user for the contouring of the determined structure ((receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7]; (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device. FIG. 4 is a flow diagram of another example method 400 for contouring a set of medical images. In step 402, the method is initialized with an initial source image along with associated source contour data. Then, in step 404, a target image is identified for contouring based on the source image and source contour. In step 406, a deformation algorithm is applied to the source and target images to generate deformation field data that indicates how one or more objects within the images have changed from the source image to the target image. The deformation field data is then applied to the source contour data in step 408 to automatically generate contour data for the target image by transforming the source contour to match the changes from the source image to the target image.) [Piper: col. 6, line 5-15; Figs. 2, 4, 6-7] – Note: Figs 4, as well as Figs. 2, 6-7, display the guidance for contouring the image); so that the determined structure can be contoured in accordance with the displayed guidance (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64];wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] comprises clinical guidelines for contouring the structure ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) or at least one atlas image showing at least one example contour for the structure ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102(e), (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 27-34, 36-37, 39-41, 43, 45-46 are rejected under 35 U.S.C. 103 as being unpatentable over Piper (US Patent 10,930,002 B2), (“Piper”), in view of Boddington et al. (US Patent 10,973,590 B2), (“Boddington”).
Regarding claim 27, Piper meets the claim limitations as follows:
A method for contouring a medical image in a contouring system (systems and methods for contouring a set of medical images) [Piper: col. 1, line 30-31; Title, Abstract] comprising (An example method of contouring a set of medical images may include the following steps) [Piper: col. 1, line 51-52]:
providing at least one medical image ((receiving a source image from the set of medical images) [Piper: col. 1, line 52-53]; (The contour transformation engine may be configured to receive source contour data that identifies the one or more objects within the source image) [Piper: col. 1, line 41-44]) to be contoured by a user (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]; determining that the user (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13] has initiated contouring ((For instance, the initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]) of a structure on the medical image (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19];
determining the structure (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55] that is being contoured (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19];displaying (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9], in response to the determination of the structure being contoured (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55], guidance for the contouring of the structure that is being contoured ((receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7]; (With reference again to FIG. 1, the image and contour reslicing block 104 generates resliced image and contour
data, which is received by the seed grid generator 106. At the seed grid generator 106, seed grid template data is received from the seed grid template database 114 along with an input for selecting one or more of the seed grid templates from the
seed grid template database 114. The seed grid template database 114 includes seed grid templates that may be used to create a seed grid 402 on the resliced image, as shown in FIG. 4. FIG. 4 illustrates a seed grid 402 overlaid on a resliced
contoured image slice, such as the example from FIG. 3B. The image contour slices portrayed in FIG. 3B are also represented in FIG. 4. For example, a clinical target volume contour 404, a planning target volume contour 406, and a chest wall contour 408 are each illustrated. The seed grid 402 is centered at the isocenter 410. Each image slice that has been resliced may include a seed grid 402. Including a seed grid 402 at each possible image slice allows for a treatment plan to be accurately planned at different depth levels of a target volume) [Piper: col. 4, line 45-65; Figs 1, 4]; (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device. FIG. 4 is a flow diagram of another example method 400 for contouring a set of medical images. In step 402, the method is initialized with an initial source image along with associated source contour data. Then, in step 404, a target image is identified for contouring based on the source image and source contour. In step 406, a deformation algorithm is applied to the source and target images to generate deformation field data that indicates how one or more objects within the images have changed from the source image to the target image. The deformation field data is then applied to the source contour data in step 408 to automatically generate contour data for the target image by transforming the source contour to match the changes from the source image to the target image) [Piper: col. 6, line 5-15; Figs. 2, 4, 6-7] – Note: Figs 4, as well as Figs. 2, 6-7, display the guidance for contouring the image); contouring the structure that is being contoured in accordance with the guidance (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64],wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] comprises clinical guidelines for contouring the structure ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) or at least one atlas image showing at least one example contour for the structure ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]).
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
displaying, …, guidance for the contouring of the determined structure that is being contoured ((Module 7 is an image annotation module that includes image processing algorithms or advanced Deep Learning based techniques for detecting anatomical landmarks in a medical image and identifying contours or boundaries of anatomical objects in a medical image, such as bone or soft tissue boundaries. Anatomical Landmark detection stands for the identification of key elements of an anatomical body part that potentially have a high level of similarity with the same anatomical body part of other patients. The Deep Learning algorithm encompasses various conventional layers and its final output layer provides self-driven data, including, but not limited to, the system coordinates of important points in the image. In the current invention, landmark detection can be also applied to determine some key positions of anatomical parts in the body, for example, left/right of the femur, and left/right of the shoulder.) [Boddington: col. 11, line 51-66; Figs. 5A- 12B]; (a visual display configured to provide the intra-operative surgical guidance to the orthopedic surgeon conducting an alignment or fixation procedure) [Boddington: col. 3, line 65-67; Figs. 5A- 12B] – Note: Please see contours and guidance displayed in Figs. 5A- 12B; (The computing platform 100 includes a plurality of software modules 103 to receive and process medical image data, including modules for image distortion correction, image feature detection, image annotation and segmentation, image to image registration, three-dimensional estimation from two-dimensional images, medical image visualization, and one or more surgical guidance modules that use artificial intelligence models to classify images as predictive of optimal or suboptimal surgical outcomes) [Boddington: col. 9, line 1-9; Figs. 5A- 31]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Regarding claim 28, Piper meets the claim limitations as follows:
A method for reviewing contours on a contoured medical image ((systems and methods for contouring a set of medical images) [Piper: col. 1, line 30-31; Title, Abstract]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]) comprising (An example method may include the following steps) [Piper: col. 1, line 51-52]:providing at least one medical image ((receiving a source image from the set of medical images) [Piper: col. 1, line 52-53]; (The contour transformation engine may be configured to receive source contour data that identifies the one or more objects within the source image) [Piper: col. 1, line 41-44]) annotated with one or more pre-existing contours (a set of contoured images has been generated) [Piper: col. 5, line 4];
displaying the at least one medical image annotated with the one or more pre-existing contours ((the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]); determining that a user has initiated reviewing ((The initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]; (a user input is received to either accept or modify the automatically generated target contour) [Piper: col. 6, line 26-27]; (the source image and contour data may be selected from a database of stored images and contours or may otherwise be provided or selected by the user) [Piper: col. 5, line 12-14]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31]) a one of the one or more pre-existing contour on the at least one medical image (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11];
determine a structure associated with the one of the one or more pre-existing contour ((The initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]; (the source image and contour data may be selected from a database of stored images and contours or may otherwise be provided or selected by the user) [Piper: col. 5, line 12-14]); displaying (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9], in response to the determination of the structure being contoured (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55], guidance for the contouring of the structure ((receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7]; (With reference again to FIG. 1, the image and contour reslicing block 104 generates resliced image and contour data, which is received by the seed grid generator 106. At the seed grid generator 106, seed grid template data is received from the seed grid template database 114 along with an input for selecting one or more of the seed grid templates from the seed grid template database 114. The seed grid template database 114 includes seed grid templates that may be used to create a seed grid 402 on the resliced image, as shown in FIG. 4. FIG. 4 illustrates a seed grid 402 overlaid on a resliced contoured image slice, such as the example from FIG. 3B. The image contour slices portrayed in FIG. 3B are also represented in FIG. 4. For example, a clinical target volume contour 404, a planning target volume contour 406, and a chest wall contour 408 are each illustrated. The seed grid 402 is centered at the isocenter 410. Each image slice that has been resliced may include a seed grid 402. Including a seed grid 402 at each possible image slice allows for a treatment plan to be accurately planned at different depth levels of a target volume) [Piper: col. 4, line 45-65; Figs 1, 4]; (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device. FIG. 4 is a flow diagram of another example method 400 for contouring a set of medical images. In step 402, the method is initialized with an initial source image along with associated source contour data. Then, in step 404, a target image is identified for contouring based on the source image and source contour. In step 406, a deformation algorithm is applied to the source and target images to generate deformation field data that indicates how one or more objects within the images have changed from the source image to the target image. The deformation field data is then applied to the source contour data in step 408 to automatically generate contour data for the target image by transforming the source contour to match the changes from the source image to the target image) [Piper: col. 6, line 5-15; Figs. 2, 4, 6-7]; (receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7] – Note: Figs 4, as well as Figs. 2, 6-7, display the guidance for contouring the image);
wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] comprises clinical guidelines for contouring the structure ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) or at least one atlas image showing at least one example contour for the structure ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]).
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
displaying, …, guidance for the contouring of the determined structure that is being contoured ((Module 7 is an image annotation module that includes image processing algorithms or advanced Deep Learning based techniques for detecting anatomical landmarks in a medical image and identifying contours or boundaries of anatomical objects in a medical image, such as bone or soft tissue boundaries. Anatomical Landmark detection stands for the identification of key elements of an anatomical body part that potentially have a high level of similarity with the same anatomical body part of other patients. The Deep Learning algorithm encompasses various conventional layers and its final output layer provides self-driven data, including, but not limited to, the system coordinates of important points in the image. In the current invention, landmark detection can be also applied to determine some key positions of anatomical parts in the body, for example, left/right of the femur, and left/right of the shoulder.) [Boddington: col. 11, line 51-66; Figs. 5A- 12B]; (a visual display configured to provide the intra-operative surgical guidance to the orthopedic surgeon conducting an alignment or fixation procedure) [Boddington: col. 3, line 65-67; Figs. 5A- 12B] – Note: Please see contours and guidance displayed in Figs. 5A- 12B; (The computing platform 100 includes a plurality of software modules 103 to receive and process medical image data, including modules for image distortion correction, image feature detection, image annotation and segmentation, image to image registration, three-dimensional estimation from two-dimensional images, medical image visualization, and one or more surgical guidance modules that use artificial intelligence models to classify images as predictive of optimal or suboptimal surgical outcomes) [Boddington: col. 9, line 1-9; Figs. 5A- 31]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Regarding claim 29, Piper meets the claim limitations as set forth in claim 28. Piper further meets the claim limitations as follow.
further comprising the step of completing the review of the pre-existing contour or editing the pre-existing contour in accordance with the displayed guidance ((Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11] ; (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31]).
Regarding claim 30, Piper meets the claim limitations as set forth in claim 29. Piper further meets the claim limitations as follow.
wherein completing the review the one or more pre-existing contours comprises (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11] providing feedback on the one or more pre-existing contours (a user input is received to either accept or modify the automatically generated target contour) [Piper: col. 6, line 26-27].
Regarding claim 31, Piper meets the claim limitations as set forth in claim 30. Piper further meets the claim limitations as follow.
wherein the feedback is provided as comments (a user input is received to either accept or modify the automatically generated target contour) [Piper: col. 6, line 26-27] or annotations.
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
annotations (accepting or rejecting the preoperative image based on quality score generated by a Pose Guide Module, if the at least one preoperative image is accepted; correcting for distortion in the at least one preoperative image; annotating an at least one anatomical landmark in the preoperative image using an Image Annotation Module to provide an at least one annotated preoperative image) [Boddington: col. 3, line 1-7]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Regarding claim 32, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
further comprising the step of outputting the user-initiated contour (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11].
Regarding claim 33, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein determining that the user has initiated contouring of the structure on the medical image comprises ((The initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]) determining the user to draw or edit a contour of the structure on the at least one medical image (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31].
Regarding claim 34, Piper meets the claim limitations as set forth in claim 28. Piper further meets the claim limitations as follow.
wherein determining that a user has initiated reviewing ((The initial contour data may be provided by manually contouring one of the image slices 108 using one of a variety of know manual contouring software applications) [Piper: col. 3, line 42-45; Fig. 1]; (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]; (a user input is received to either accept or modify the automatically generated target contour) [Piper: col. 6, line 26-27]; (the source image and contour data may be selected from a database of stored images and contours or may otherwise be provided or selected by the user) [Piper: col. 5, line 12-14]; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]; (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31]) the one of the one or more pre-existing contour on the at least one medical image (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11] comprises using eye-tracking hardware and/or software to identify a location of the users gaze on the at least one medical image.
Piper does not explicitly disclose the following claim limitations (Emphasis added).
using eye-tracking hardware and/or software to identify a location of the users gaze on the at least one medical image.
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
using eye-tracking hardware and/or software to identify a location of the users gaze on the at least one medical image (A user such as a surgeon, selects at least one
anatomical landmark in the anatomical image on the graphical user interface 151. Anatomical landmark selection can be accomplished by a various methods including but not limited to: auto-segmentation where the software of the computing platform 100 uses feature/pattern recognition process to auto-detect known and targeted anatomical landmarks; use of a remote infrared device, such as a gyro mouse; voice command; air gestures; gaze (surgeon uses gaze and direction of eye or head motion to control targeting) or touching the visualization screen at the selected anatomical landmarks) [Boddington: col. 18, line 60 – col. 19, line 4]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Regarding claim 36, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] for the contouring of the structure comprises that at least one atlas image ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]) and wherein determining the structure (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55] that is being contoured comprises ((Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11] (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31]) finding a mapping between the at least one medical image ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) and the at least one atlas image ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]); wherein the mapping is done using image registration (The image deformation engine 102 may, for example, utilize a free-form intensity-based deformable registration algorithm that maps similar tissues from the source image to the target image by matching intensity values from one image to the next. More specifically, a free-form intensity-based deformable registration algorithm is a type of deformation algorithm that generates deformation field data in the form of an x-y mapping for each pixel in the target image. For each pixel in the target image, the x-y mapping identifies the pixel in the source image that most likely corresponds to similar anatomy in the target image based on the pixel intensities. In this way, pixel positions along the edge of an object ( e.g., a piece of anatomy) in the source image may be mapped to corresponding pixel positions along the edge of the same object in the target image, showing how the object (e.g., anatomy) has changed from the source image to the target image) [Piper: col. 3, line 4-20]; andwherein the image registration comprises one or more of: deformable registration (The image deformation engine 102 may, for example, utilize a free-form intensity-based deformable registration algorithm that maps similar tissues from the source image to the target image by matching intensity values from one image to the next. More specifically, a free-form intensity-based deformable registration algorithm is a type of deformation algorithm that generates deformation field data in the form of an x-y mapping for each pixel in the target image. For each pixel in the target image, the x-y mapping identifies the pixel in the source image that most likely corresponds to similar anatomy in the target image based on the pixel intensities. In this way, pixel positions along the edge of an object ( e.g., a piece of anatomy) in the source image may be mapped to corresponding pixel positions along the edge of the same object in the target image, showing how the object (e.g., anatomy) has changed from the source image to the target image) [Piper: col. 3, line 4-20], affine registration or rigid registration.
Regarding claim 37, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
using machine learning to predict the structure that is being contoured (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19].
Piper does not explicitly disclose the following claim limitations (Emphasis added).
machine learning.
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
machine learning ((These automated artificial intelligence models include: Deep Learning, machine learning and reinforcement learning based techniques. For example, a Convolutional Neural Network (CNN) is trained using annotated/labeled images which include good and bad images to learn local image features linked to low-resolution, presence of noise/artifact, contrast/lighting conditions, etc. The CNN model uses the learning features to make predictions about a new image) [Boddington: col. 10, line 19 – 26]; (When an image is acquired, the artificial intelligence intra-operative surgical guidance system 1 uses the hip application segmentation machine learning module to identify the relevant anatomical, instrument, and implant features such as the nail alignment jig, and nail and lag-screw) [Boddington: col. 26, line 9-14]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Regarding claim 39, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein determining of the structure that is being contoured comprises (a user input is received to either accept or modify the automatically generated target contour) [Piper: col. 6, line 26-27] ; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]) using eye-tracking hardware and/or software to identify a location of the users gaze on the at least one medical image.
Piper does not explicitly disclose the following claim limitations (Emphasis added).
using eye-tracking hardware and/or software to identify a location of the users gaze on the at least one medical image.
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
using eye-tracking hardware and/or software to identify the location of the users gaze on the at least one medical image (A user such as a surgeon, selects at least one
anatomical landmark in the anatomical image on the graphical user interface 151. Anatomical landmark selection can be accomplished by a various methods including but not limited to: auto-segmentation where the software of the computing platform 100 uses feature/pattern recognition process to auto-detect known and targeted anatomical landmarks; use of a remote infrared device, such as a gyro mouse; voice command; air gestures; gaze (surgeon uses gaze and direction of eye or head motion to control targeting) or touching the visualization screen at the selected anatomical landmarks) [Boddington: col. 18, line 60 – col. 19, line 4]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Regarding claim 40, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein contouring the structure comprises: ((the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55]; (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) manual contouring (if the user chooses to modify the automatically generated target contour, then the contour is manually edited) [Piper: col. 6, line 29-31], semi-automatic contouring, or automatic contouring (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64].
Regarding claim 41, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
receiving (a user input is received to either accept or modify the automatically generated target contour) [Piper: col. 6, line 26-27] ; (Then, once the target contour 314 is generated, it may be displayed as an overlay on the target image for the user to review and possibly edit) [Piper: col. 6, line 9-11]) using a contour name (At decision step 410, a user input is received to either accept or modify the automatically generated target contour. If the automatically generated contour is accepted, then the method proceeds to step 412. Otherwise, if the user chooses to modify the automatically generated target contour, then the contour is manually edited at step 414 before the method proceeds to step 412. The dotted line between steps 414 and 404 signifies that if a target contour is manually edited, then during the next iteration of the contouring method (if any), the manually edited contour may be used as the source contour for the next target image) [Piper: col. 6, line 26-36] (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]; and
identifying, using a look-up table ((The computer components, software modules, functions, data stores and data structures described herein may be connected directly or indirectly to each other in order to allow the flow of data needed for their operations.) [Piper: col. 11, line 60-63] – Note: Look up table is one form of the data structure) a standardized name of the structure.
Piper does not explicitly disclose the following claim limitations (Emphasis added).
a standardized name of the structure.
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
a standardized name of the structure (Please see look up tables with standardized names in Figs. 12A-B) [Boddington: Figs. 12A-B]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Regarding claim 43, Piper meets the claim limitations as set forth in claim 28. Piper further meets the claim limitations as follow.
wherein the pre-existing contours displayed to the user are either manual contours from a previous contouring session (At decision step 410, a user input is received to either accept or modify the automatically generated target contour. If the automatically generated contour is accepted, then the method proceeds to step 412. Otherwise, if the user chooses to modify the automatically generated target contour, then the contour is manually edited at step 414 before the method proceeds to step 412. The dotted line between steps 414 and 404 signifies that if a target contour is manually edited, then during the next iteration of the contouring method (if any), the manually edited contour may be used as the source contour for the next target image) [Piper: col. 6, line 26-36], or automatically produced contours (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64].
Regarding claim 45, Piper meets the claim limitations as set forth in claim 27. Piper further meets the claim limitations as follow.
wherein the at least one medical image is one a CT scan (FIGS. 8-12 are illustrations of contoured image slices from a CT scan of a patient's pelvic region) [Piper: col. 8, line 59-60; Figs. 8-12], an MRI scan, a PET scan, a SPECT scan or an ultrasound scan (Medical images, such CT (computed tomography), MR (magnetic resonance), US (ultrasound), or PET (positron emission tomography) scans, are regularly contoured to identify certain pieces of anatomy within the image) [Piper: col. 1, line 13-17].
Regarding claim 46, Piper meets the claim limitations as follows:
A system for contouring at least one medical image (systems and methods for contouring a set of medical images) [Piper: col. 1, line 30-31; Title, Abstract] comprising:
a display (a display device) [Piper: col. 6, line 2; Fig. 3] for displaying the at least one medical image (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device) [Piper: col. 6, line 5-8; Figs. 2, 4, 6-7] to be contoured by a user ((the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13]; (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device, along with an overlay of the source contour data 302 on the displayed source image) [Piper: col. 6, line 5-9]); a processor (a processor to perform the methods' operations and implement the systems described herein) [Piper: col. 11, line 58-59; Fig. 3] for determining that the user has initiated contouring (the user may use the source and target identifications 120, 122 to dictate which source contour is applied to which target image) [Piper: col. 4, line 11-13] of a structure on the at least one medical image ((contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19] (At decision step 410, a user input is received to either accept or modify the automatically generated target contour. If the automatically generated contour is accepted, then the method proceeds to step 412. Otherwise, if the user chooses to modify the automatically generated target contour, then the contour is manually edited at step 414 before the method proceeds to step 412. The dotted line between steps 414 and 404 signifies that if a target contour is manually edited, then during the next iteration of the contouring method (if any), the manually edited contour may be used as the source contour for the next target image) [Piper: col. 6, line 26-36]); the processor (a processor to perform the methods' operations and implement the systems described herein) [Piper: col. 11, line 58-59; Fig. 3] determining the structure that is being contoured ((the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55]; (contoured to identify certain pieces of anatomy within the image. For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]); and
in response to the determination of the structure (the source contour data identifying one or more objects within the source image) [Piper: col. 1, line 54-55], the system displaying guidance to the user for the contouring of the determined structure ((receiving instructions identifying a target image in the set of medical images to contour; using a deformation algorithm to generate deformation field data from the source image and the target image, the deformation field data indicative of changes between the one or more objects from the source image to the target image) [Piper: col. 1, line 57-61; Figs 2, 4, 6-7]; (the image rendering engine 322 may display one or both of the selected source and target images 318, 320 on the display device. FIG. 4 is a flow diagram of another example method 400 for contouring a set of medical images. In step 402, the method is initialized with an initial source image along with associated source contour data. Then, in step 404, a target image is identified for contouring based on the source image and source contour. In step 406, a deformation algorithm is applied to the source and target images to generate deformation field data that indicates how one or more objects within the images have changed from the source image to the target image. The deformation field data is then applied to the source contour data in step 408 to automatically generate contour data for the target image by transforming the source contour to match the changes from the source image to the target image.) [Piper: col. 6, line 5-15; Figs. 2, 4, 6-7] – Note: Figs 4, as well as Figs. 2, 6-7, display the guidance for contouring the image); so that the determined structure can be contoured in accordance with the displayed guidance (using the deformation field data and the source contour data to generate automatic target contour data, the automatic target contour data identifying the one or more objects within the target image) [Piper: col. 1, line 61-64];wherein the guidance (a virtual guide) [Piper: col. 5, line 5-6] comprises clinical guidelines for contouring the structure ((Each insertion point 412 on the seed grid 402 may represent a location for a seed needle during the radiation therapy procedure. Thus, providing a seed grid 402 on a medical image slice that corresponds to a physical seed template allows a medical practitioner to have a virtual guide to each needle insertion point) [Piper: col. 5, line 1-6]; (For example, a radiologist or oncologist may contour a medical image to identify a tumor within the image) [Piper: col. 1, line 16-19]) or at least one atlas image showing at least one example contour for the structure ((a common atlas, which could then be registered to a patient planning image) [Piper: col. 11, line 8-9]; (One such task is transforming contours from a source image to a target image. These images could be individual slices or 3D medical images (e.g., for atlas-based segmentation)) [Piper: col. 9, line 28-31]).
In the same field of endeavor, Boddington further discloses the claim limitations as follows:
displaying, …, guidance for the contouring of the determined structure that is being contoured ((Module 7 is an image annotation module that includes image processing algorithms or advanced Deep Learning based techniques for detecting anatomical landmarks in a medical image and identifying contours or boundaries of anatomical objects in a medical image, such as bone or soft tissue boundaries. Anatomical Landmark detection stands for the identification of key elements of an anatomical body part that potentially have a high level of similarity with the same anatomical body part of other patients. The Deep Learning algorithm encompasses various conventional layers and its final output layer provides self-driven data, including, but not limited to, the system coordinates of important points in the image. In the current invention, landmark detection can be also applied to determine some key positions of anatomical parts in the body, for example, left/right of the femur, and left/right of the shoulder.) [Boddington: col. 11, line 51-66; Figs. 5A- 12B]; (a visual display configured to provide the intra-operative surgical guidance to the orthopedic surgeon conducting an alignment or fixation procedure) [Boddington: col. 3, line 65-67; Figs. 5A- 12B] – Note: Please see contours and guidance displayed in Figs. 5A- 12B; (The computing platform 100 includes a plurality of software modules 103 to receive and process medical image data, including modules for image distortion correction, image feature detection, image annotation and segmentation, image to image registration, three-dimensional estimation from two-dimensional images, medical image visualization, and one or more surgical guidance modules that use artificial intelligence models to classify images as predictive of optimal or suboptimal surgical outcomes) [Boddington: col. 9, line 1-9; Figs. 5A- 31]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Piper with Boddington to program the system to implement of Boddington’s method.
Therefore, the combination of Piper with Boddington will enable the system to assist a surgeon to make predictive of optimal or sub-optimal surgical outcomes. [Boddington: col. 9, line 1-10].
Reference Notice
Additional prior arts, included in the Notice of Reference Cited, made of record and not relied upon is considered pertinent to applicant's disclosure.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Philip Dang whose telephone number is (408) 918-7529. The examiner can normally be reached on Monday-Thursday between 8:30 am - 5:00 pm (PST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached on 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Philip P. Dang/Primary Examiner, Art Unit 2488
Prosecution Insights