DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to the Applicants’ Amendment/Remark filed on December 19, 2024. Claims 21, and 30 have been amended; claims 23, 31-32 and 34 have been canceled. Claims 21-22, 24-30, 33, and 35-37 are still pending in this application.
Response to Arguments
Regarding claims 21, 22, 30 and 35:
Applicant’s arguments with respect to claims 21 and 30 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claims 22, 24-25, 33 and 35-37 depend either from claims 21 or 30, therefore, they have been considered, but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Regarding claims 26, 28, and 29:
Applicant’s arguments: Applicant argues on page 8 of the remark that “Claim 26 recites, inter alia, "graphically representing data sufficiency of the three- dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies." The Examiner acknowledges that this aspect of the claims cannot be found in Razzaque. Office action, p.30. According to the Examiner, however, it can be found in Altmann. Id. This is error.” The Examiner's position is that Altmann discloses the use of pseudocolor or greyscale to "indicate the areas that were imaged[.]"Id. at 31. Even accepting this as true, neither pseudocolor nor greyscale is voxel opacity as recited. Both vary the color of a voxel, not its opacity.
Examiner’s response: Examiner respectfully disagrees with the argument, because Altman fairly discloses the highlight claim invention. Altman discloses the color changes (adjusting opacities) from gray to red (opacity) at every point and the operator receives a clear indication of the current data coverage (data efficiencies) (Altmann, see par. [0068]). Altmann’s pseudocolor is additional option to use to modify the grayscale (Altmann, see par. [0079]). The combination of Razzaque and Altmann fairly discloses the highlight claim invention. The claim invention does not claim how to adjust opacities of the plurality of voxel according to their respective data sufficiencies. Therefore, the argument is not persuasive.
Claims 27-29 depend on claim 26, for the reasons stated above and the detail the rejection below. Therefore, claims 26-29 are maintain.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims (21 and 30), (22 and 35), 27, (24, 28 and 36), (25, 29 and 37), 26, and 33 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims (1 and 13), (2 and 15), 4, (8 and 14), 12, 9 and 7, respectively of U.S. Patent No. US 11941754 in view of Vincent et al. (US 20130044928 A1).
Table 1 illustrates the conflicting claim pairs:
Present Application
21, and 30
22,
and 35
27
24, 28, and
36
25, 29
and 37
26,
33
Patent No. 11941754
1, and 13
2 and 15
4
8 and 14
12
9
7
Table 2 illustrates example of claims (21 and 30) and 26 of the present Application compared against claims 1 and 9 of U.S Patent No. 11941754.
Present Application
US. Patent No. 11941754
Claim 21,
Claim 1,
A method of generating a three-dimensional geometric model of an anatomical region, the method comprising:
A method of generating a three-dimensional geometric model of an anatomical region, the method comprising:
defining a three-dimensional voxel space comprising a plurality of voxels;
defining a three-dimensional voxel space comprising a plurality of voxels;
receiving a plurality of two-dimensional echographic image slices of the anatomical
region, wherein each image slice of the plurality of image slices is associated with localization information;
receiving a plurality of two-dimensional echographic image slices of the anatomical
region, wherein each image slice of the plurality of image slices is associated with localization information;
assembling the plurality of two-dimensional echographic image slices of the anatomical region into a three-dimensional geometric model of the anatomical region using the associated localization information;
assembling the plurality of two-dimensional echographic image slices of the anatomical region into a three-dimensional geometric model of the anatomical region using the associated localization information;
graphically outputting the three-dimensional geometric model of the anatomical region in the three-dimensional voxel space; and
graphically outputting the three-dimensional geometric model of the anatomical region in the three-dimensional voxel space; and
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency.
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region, wherein graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region comprises outputting a one-dimensional illustration of the data sufficiency of the three-dimensional geometric model.
Claim 26,
A method of generating a three-dimensional geometric model of a heart, the method comprising:
Claim 9,
A method of generating a three-dimensional geometric model of a heart, the method comprising:
receiving a plurality of two-dimensional image slices of the heart from an intracardiac echocardiography ("ICE") catheter, wherein each image slice of the plurality of image slices is associated with localization information of the ICE catheter;
receiving a plurality of two-dimensional image slices of the heart from an intracardiac echocardiography (“ICE”) catheter, wherein each image slice of the plurality of image slices is associated with localization information of the ICE catheter;
graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart; and
graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart;
graphically representing data sufficiency of the three-dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies.
graphically representing data sufficiency of the three-dimensional geometric model of the heart, wherein graphically representing data sufficiency of the three-dimensional geometric model of the heart comprises one or more of: representing data sufficiency of the three-dimensional geometric model of the heart using a one-dimensional illustration of data sufficiency relative to angular rotation of the ICE catheter; and representing data sufficiency of the three-dimensional geometric model of the heart using a two-dimensional illustration of data sufficiency relative to localization of the ICE catheter.
Claim 30,
A method of representing data sufficiency of a three-dimensional geometric model of an anatomical region, the method comprising:
Claim 1,
A method of generating a three-dimensional geometric model of an anatomical region, the method comprising:
defining a three-dimensional voxel space comprising a plurality of voxels;
defining a three-dimensional voxel space comprising a plurality of voxels;
assembling the plurality of two-dimensional echographic image slices of the anatomical region into a three-dimensional geometric model of the anatomical region using the associated localization information, wherein the three-dimensional geometric model of the anatomical region occupies a three-dimensional voxel space;
assembling the plurality of two-dimensional echographic image slices of the anatomical region into a three-dimensional geometric model of the anatomical region using the associated localization information;
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using one or more of: a three-dimensional graphical representation of data sufficiency; a two-dimensional graphical representation of data sufficiency; and a one-dimensional graphical representation of data sufficiency.
graphically outputting the three-dimensional geometric model of the anatomical region in the three-dimensional voxel space; and graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region, wherein graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region comprises outputting a one-dimensional illustration of the data sufficiency of the three-dimensional geometric model.
Although the claims at issued are not identical, they are patentably distinct for each other because:
Comparing of table 2 above shows the claim 1 of Patent No. US 11941754 discloses claim 21 of present application, but does not explicitly discloses graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency. However,
Vincent discloses:
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency (Vincent, see FIGs. [0063-0064]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Claim 1 of U.S Patent 11087554, to have graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency, as suggested by Vincent. The modification provide an improved system and method for generating a three-dimensional geometric model of medical image and improve quality of the image data, thereby to reduce manual workloads in determining properties of images. Image analysis is used in a variety of different fields to identify features of images. One area in which image analysis techniques are used is to process medical images where the identification of anatomical features in an image can provide useful information to medical practitioners. (Vincent, see par. [0004]).
Comparing of table 2 above shows the claim 9 of Patent No. US 11941754 discloses claim 26 of present Application, but does not explicitly disclose graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart; and graphically representing data sufficiency of the three-dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies. However, Altmann discloses:
graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart (Altmann, see pars. [0052]); and
graphically representing data sufficiency of the three-dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies (Altmann, see pars. [0061], [0063], [0068] and [0079]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of claimed invention, to modify the method and apparatus of Claim 10 of U.S Patent No. 11087554, and have graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart; and graphically representing data sufficiency of the three-dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies, as taught by Altmann. The modification provides an improved system and method for generating a three- dimensional geometric model of medical image, in order to assist the physician in performing a real time cardiac imaging procedure, a three-dimensional image of the heart may be rendered during acquisition and helps the user to determine when adequate image data have been acquired or whether details are still missing (Altmann, see par. [0007]).
Comparing of table 2 above shows the claim 1 of Patent No. US 11941754 discloses claim 30 of present Application, but does not explicitly disclose
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional graphical representation of data sufficiency. However,
Vincent discloses:
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional graphical representation of data sufficiency (Vincent, see at least par. [0064]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Claim 1 of U.S Patent 11087554, to have graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency, as suggested by Vincent. The modification provide an improved system and method for generating a three-dimensional geometric model of medical image and improve quality of the image data, thereby to reduce manual workloads in determining properties of images. Image analysis is used in a variety of different fields to identify features of images. One area in which image analysis techniques are used is to process medical images where the identification of anatomical features in an image can provide useful information to medical practitioners. (Vincent, see par. [0004]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Ohbuchi et al. (1992) “Incremental volume reconstruction and rendering for 3-D ultrasound imaging.” Proc. SPIE 1808, Visualization in Biomedical Computing, pp 312-323. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/1808/0000/Incremental-volume-reconstruction-and-rendering-for-3-D-ultrasound-imaging/10.1117/12.131087.pdf in view of Vincent et al. (US 20130044928 A1).
Regarding claim 21. Ohbuchi discloses a method of generating a three-dimensional geometric model of an anatomical region (Ohbuchi, see abstract,
defining a three-dimensional voxel space comprising a plurality of voxels (p. 315, sec. 3.1 paragraph 1, “regular 3D volume”, paragraph 4, “Here (u,v,n) is the coordinate of the voxel in the 3D coordinate system attached to the input image plane,” which is converted to (x,y,z) coordinates in the 3D coordinate system, “(x,y,z) is the coordinate of the reconstruction buffer”);
receiving a plurality of two-dimensional echographic image slices of the anatomical region (Fig. 3, showing multiple 2D slices being reconstructed into the 3D volume), wherein each image slice of the plurality of image slices is associated with localization information (e.g., (u,v,n); see also the transformation matrix Ti, “which is derived from the locations and orientations of the transducer and other geometric information”, i.e., for each slice);
assembling the plurality of two-dimensional echographic image slices of the anatomical region into a three-dimensional geometric model of the anatomical region using the associated localization information (Fig. 3, showing the placement of each slice into the 3D buffer; the equations in Sec. 3.1 describe how each slice is converted using its localization information (u,v,n) into the 3D buffer);
graphically outputting the three-dimensional geometric model of the anatomical region in the three-dimensional voxel space (Ohbuchi, see Fig. 12, Fig. 13, showing a rendered display of the 3D model; Sec. 3.2 describes a rendering technique based on ray-casting).
Ohbuchi does not explicitly disclose graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency. However,
Vincent discloses:
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency (Vincent, see at least par. [0063] It is described above that two images are combined at step S5 of FIG. 3 based upon respective acquisition data to generate data values for each voxel of a volume which represents an anatomical region of interest. However it will be appreciated that data vales for each voxel of a volume which represents an anatomical region of interest can be generated from a single image based upon acquisition data. In particular, each voxel in the volume that lies on a path as indicated by the acquisition data may take a value based upon a pixel of the image in the manner described above with reference to FIG. 6. In this way a three-dimensional representation of the anatomical region of interest may be generated from a single two-dimensional image and that three-dimensional representation may be processed using the system of FIG. 7 to generate a parameterisation indicating features of the anatomical region of interest represented by the single image. [0064] Where the three-dimensional representation of an anatomical region of interest is generated from a single two-dimensional image by the system of FIG. 7, the model 32 of FIG. 7 is a model trained on three-dimensional representations generated in the same way, and includes an indication of features of the anatomical region of interest in three-dimensions rather than the features visible in the two-dimensional projection. As such, it is still possible to identify features that cannot be visually identified in the two-dimensional image using model fitting.).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Ohbuchi in view of Vincent, to have graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional representation of data sufficiency, as provided by Vincent. The modification provides an improved system and method for generating a three-dimensional geometric model of medical image and improve quality of the image data, thereby to reduce manual workloads in determining properties of images. Image analysis is used in a variety of different fields to identify features of images. One area in which image analysis techniques are used is to process medical images where the identification of anatomical features in an image can provide useful information to medical practitioners. (Vincent, see par. [0004]).
Regarding claim 30, Ohbuchi discloses a method of representing data sufficiency of a three-dimensional geometric model of an anatomical region (Ohbuchi, see abstract, “reconstructs a regular 3D volume”, and Fig. 13, “left thigh”), the method comprising:
defining a three-dimensional voxel space comprising a plurality of voxels (Ohbuchi, see p. 315, sec. 3.1 paragraph 1, “regular 3D volume”, paragraph 4, “Here (u,v,n) is the coordinate of the voxel in the 3D coordinate system attached to the input image plane,” which is converted to (x,y,z) coordinates in the 3D coordinate system, “(x,y,z) is the coordinate of the reconstruction buffer”);
receiving a plurality of two-dimensional echographic image slices of the anatomical region (Ohbuchi, see Fig. 3, showing multiple 2D slices being reconstructed into the 3D volume), wherein each image slice of the plurality of image slices is associated with localization information (e.g., (u,v,n); see also the transformation matrix Ti, “which is derived from the locations and orientations of the transducer and other geometric information”, i.e., for each slice), wherein each image slice of the plurality of image slices is associated with localization information (e.g., (u,v,n); see also the transformation matrix Ti, “which is derived from the locations and orientations of the transducer and other geometric information”, i.e., for each slice);
assembling the plurality of two-dimensional echographic image slices of the anatomical region into a three-dimensional geometric model of the anatomical region using the associated localization information (Ohbuchi, see Fig. 3, showing the placement of each slice into the 3D buffer; the equations in Sec. 3.1 describe how each slice is converted using its localization information (u,v,n) into the 3D buffer);
Ohbuchi does not explicitly disclose wherein the three-dimensional geometric model of the anatomical region occupies a three-dimensional voxel space; and graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional graphical representation of data sufficiency. However,
Vincent discloses:
wherein the three-dimensional geometric model of the anatomical region occupies a three-dimensional voxel space (Vincent, see at least par. [0063] It is described above, that two images are combined at step S5 of FIG. 3 based upon respective acquisition data to generate data values for each voxel of a volume which represents an anatomical region of interest. However it will be appreciated that data vales for each voxel of a volume which represents an anatomical region of interest can be generated from a single image based upon acquisition data. In particular, each voxel in the volume that lies on a path as indicated by the acquisition data may take a value based upon a pixel of the image in the manner described above with reference to FIG. 6. In this way a three-dimensional representation of the anatomical region of interest may be generated from a single two-dimensional image and that three-dimensional representation may be processed using the system of FIG. 7 to generate a parameterisation indicating features of the anatomical region of interest represented by the single image);
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional graphical representation of data sufficiency (Vincent, see at least par. [0064] Where the three-dimensional representation of an anatomical region of interest is generated from a single two-dimensional image by the system of FIG. 7, the model 32 of FIG. 7 is a model trained on three-dimensional representations generated in the same way, and includes an indication of features of the anatomical region of interest in three-dimensions rather than the features visible in the two-dimensional projection. As such, it is still possible to identify features that cannot be visually identified in the two-dimensional image using model fitting.).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Ohbuchi in view of Vincent, to have wherein the three-dimensional geometric model of the anatomical region occupies a three-dimensional voxel space; and graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region using a two-dimensional graphical representation of data sufficiency, as provided by Vincent. The modification provide an improved system and method for generating a three-dimensional geometric model of medical image and improve quality of the image data, thereby to reduce manual workloads in determining properties of images. Image analysis is used in a variety of different fields to identify features of images. One area in which image analysis techniques are used is to process medical images where the identification of anatomical features in an image can provide useful information to medical practitioners. (Vincent, see par. [0004]).
Claims 22 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Ohbuchi et al. (1992) “Incremental volume reconstruction and rendering for 3-D ultrasound imaging.” Proc. SPIE 1808, Visualization in Biomedical Computing, pp 312-323. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/1808/0000/Incremental-volume-reconstruction-and-rendering-for-3-D-ultrasound-imaging/10.1117/12.131087.pdf in view of Vincent et al. (US 20130044928 A1) as applied claim 21 above and further in view of Altmann et al. (US 20080137927 A1).
Regarding claim 22. Ohbuchi in view of Vincent discloses the method of claim 21 (as rejected above), but Ohbuchi in view of Vincent does not disclose further comprising outputting guidance to a practitioner to increase data sufficiency of at least a portion of the three-dimensional geometric model. However,
Altmann discloses:
further comprising outputting guidance to a practitioner to increase data sufficiency of at least a portion of the three-dimensional geometric model (Altmann, see at least par. [0068] Referring again to FIG. 4, as the data are acquired in successive iterations of step 70, the electroanatomical map, and optionally vessels, which may be shown diagrammatically on the electroanatomical map as contours or cylindrical structures, are progressively colored to indicate the areas that were imaged, as shown on the image 94 (FIG. 5). For example, the map may start with a gray color, as on the image 92 (FIG. 5), and the color may then change from gray to red at every point on the map that corresponds to points where ultrasound image data were acquired. In this manner, the operator receives a clear indication of the current data coverage.).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Ohbuchi, to have further comprising outputting guidance to a practitioner to increase data sufficiency of at least a portion of the three-dimensional geometric model, as provided by Altmann. The modification provides an improved system and method for generating a three-dimensional geometric model of medical image and improve quality of the image data, in order to assist the physician in performing a real time cardiac imaging procedure, a three-dimensional image of the heart may be rendered during acquisition and help the user to determine when adequate image data have been acquired or whether details are still missing (Altmann, see par. [0007]).
Regarding claim 35, The method according to claim 35 performs same step of claim 22. Therefore, claim 35 is further rejected based on the same rationale as claim 22 set forth above and incorporated herein.
Claim 24-25, 33 and 36-37 are rejected under 35 U.S.C. 103 as being unpatentable over Ohbuchi et al. (1992) “Incremental volume reconstruction and rendering for 3-D ultrasound imaging.” Proc. SPIE 1808, Visualization in Biomedical Computing, pp 312-323. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/1808/0000/Incremental-volume-reconstruction-and-rendering-for-3-D-ultrasound-imaging/10.1117/12.131087.pdf in view of Vincent et al. (US 20130044928 A1), applied claim 21 above, and further in view of Razzaque (US 20110137156 A1).
Regarding claim 24. Ohbuchi in view of Vincent discloses the method of claim 21 (as rejected above), but Ohbuchi in view of Vincent does not disclose wherein graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region further comprises outputting a data collection cue. However, Razzaque teaches:
graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region further comprises outputting a data collection cue (Razzaque, see at least [0135], The rings may be spaced at regular (e.g., 0.5, 1, or 2 cm) intervals to provide the physician with visual cues regarding the distance from the needle tip to the targeted anatomy. In some embodiments, the spacing of the rings may indicate other aspects of the data, such as the drive speed of the needle, the density of the tissue, the distance to a landmark, such as the ultrasound data, or any other appropriate guidance data or property).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Ohbuchi in view of Altmann, and to have graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region further comprises outputting a data collection cue, as provided by Razzaque. The modification provides an improved system and method for generating a three-dimensional geometric model of medical image; thereby to aid the surgeon to perform more effective or more accurate surgery. As merely one example of such image-guided surgery, to aid the surgeon to identify important features in the medical image. In addition, when image guidance cues are co-located with medical display object on the display, the medical device system may switch between displaying the image guidance cue and the medical display object, resulting in a flicker (Razzaque, see par. [0004]).
Regarding claim 25. Ohbuchi in view of Vincent discloses claim 24 (as rejected above), but Ohbuchi in view of Vincent does not disclose wherein the data collection cue is responsive to a rotational speed of an echographic imaging medical device. However,
Razzaque discloses:
wherein the data collection cue is responsive to a rotational speed of an echographic imaging medical device (Razzaque, see at least par. [0055] The rings can be spaced at regular (e.g., 0.5, 1, or 2 cm) intervals to provide the healthcare provider with visual or guidance cues regarding the distance from the medical device tip to the targeted anatomy. In some embodiments, the spacing of the rings can indicate other aspects of the data, such as the drive speed of the medical device, the density of the tissue, the distance to a landmark, such as the ultrasound data, or any other appropriate guidance data or property. In some embodiments, the rings or other trajectory indicators can extend beyond the medical device tip, by a distance equal to the length of the medical device-shaft. This way, the user knows if the medical device is long enough to reach the target--even before the tip enters the patient. That is, in some embodiments, if the rings do not reach the target with the tip still outside the body, then the tip won't reach the target even when the entire length shaft is inserted into the body.
[0056] Other display markers can be used to show trajectory, such as a dashed, dotted, or solid line, transparent medical device shaft, point cloud, wire frame, etc. In some embodiments, three-dimensional rings can be used and provide depth cues and obscure little of the ultrasound image. Virtual rings or other virtual markers can be displayed semi-transparently, so that they obscure less of the ultrasound image than an opaque marker would.).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Ohbuchi in view of Altmann, and to have wherein the data collection cue is responsive to a rotational speed of an echographic imaging medical device, as provided by Razzaque. The modification provide an improved system and method for generating a three-dimensional geometric model of medical image; thereby to aid the surgeon to identify important features in the medical image. In addition, when image guidance cues are co-located with medical display object on the display, the medical device system may switch between displaying the image guidance cue and the medical display object, resulting in a flicker (Razzaque, see par. [0004]).
Regarding claim 33. Ohbuchi in view of Vincent discloses claim 30 (as rejected above), but Ohbuchi in view of Vincent does not disclose wherein the two-dimensional graphical representation of data sufficiency represents data sufficiency of the three-dimensional geometric model relative to localization of the plurality of echographic image slices. However, Razzaque discloses:
wherein the two-dimensional graphical representation of data sufficiency represents data sufficiency of the three-dimensional geometric model relative to localization of the plurality of echographic image slices (Razzaque, see FIG. 19 and at least par. [0187], Doppler information can be collected over time as the doctor sweeps a 2D ultrasound probe over a volume. Since the ultrasound probe is tracked, the image guidance system can determine the relative locations of the collected ultrasound slices and locate them in 3D space. From this data, in some embodiments, the image guidance data can approximate 3D flow information in various ways. For example, in some embodiments, in order to observe the progression of the ablation with a 2D transducer, the physician may continually sweep the ultrasound transducer back and forth over the general area of tissue that contains the lesion and ablation needle. Some of the tissue being ablated contains may expand into micro bubbles that can be detected in ultrasound. The image guidance system may extract those pixels and represent the area of Doppler flow (e.g., "a Doppler slice"), relative to the latest 2D ultrasound image ("the ultrasound slice"). For example, as depicted in FIG. 19, as 2D ultrasound probe 1955 is swept across the volume of interest, Doppler slices 1921 may be collected and displayed on display 1920. Additionally, the ultrasound slice 1922 may also be displayed. Optionally, ultrasound needle 1945 may also be displayed on display 1920).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Ohbuchi, and to have wherein the two-dimensional graphical representation of data sufficiency represents data sufficiency of the three-dimensional geometric model relative to localization of the plurality of echographic image slices, as provided by Razzaque. The modification provide an improved system and method for generating a three-dimensional geometric model of medical image; thereby to aid the surgeon to perform more effective or more accurate surgery. As merely one example of such image-guided surgery, to aid the surgeon to identify important features in the medical image. In addition, when image guidance cues are co-located with medical display object on the display, the medical device system may switch between displaying the image guidance cue and the medical display object, resulting in a flicker (Razzaque, see par. [0004]).
Regarding claim 36, The method according to claim 36 performs same step of claim 24. Therefore, claim 36 is further rejected based on the same rationale as claim 24 set forth above and incorporated herein.
Regarding claim 37, The method according to claim 37 performs same step of claim 25. Therefore, claim 37 is further rejected based on the same rationale as claim 25 set forth above and incorporated herein.
Claims 26 and 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over in view of Razzaque et al. (US 20110137156 A1) in view of Altmann et al. (US 20080137927 A1, hereinafter Altmann).
Regarding claim 26. Razzaque discloses a method of generating a three-dimensional geometric model of a heart, the method comprising:
receiving a plurality of two-dimensional image slices of the heart from an intracardiac echocardiography ("ICE") catheter, wherein each image slice of the plurality of image slices is associated with localization information of the ICE catheter (Razzaque, see at least par [0187] In some embodiments, Doppler information can be collected over time as the doctor sweeps a 2D ultrasound probe over a volume. Since the ultrasound probe is tracked, the image guidance system can determine the relative locations of the collected ultrasound slices and locate them in 3D space. From this data, in some embodiments, the image guidance data can approximate 3D flow information in various ways. For example, in some embodiments, in order to observe the progression of the ablation with a 2D transducer, the physician may continually sweep the ultrasound transducer back and forth over the general area of tissue that contains the lesion and ablation needle. Some of the tissue being ablated contains may expand into micro bubbles that can be detected in ultrasound. The image guidance system may extract those pixels and represent the area of Doppler flow (e.g., "a Doppler slice"), relative to the latest 2D ultrasound image ("the ultrasound slice"). For example, as depicted in FIG. 19, as 2D ultrasound probe 1955 is swept across the volume of interest, Doppler slices 1921 may be collected and displayed on display 1920. Additionally, the ultrasound slice 1922 may also be displayed. Optionally, ultrasound needle 1945 may also be displayed on display 1920.);
graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter (Razzaque, see at least par. [0186] For example, in some embodiments, the image guidance system may have a Doppler 3D mode, in which volumetric images (such as a 3D ultrasound) are sampled, and then those volumetric images are displayed such that only those voxels which contain flow (above some threshold velocity) are opaque, while all other voxels (without sufficient flow) are made transparent. By displaying only the portions of the image that have Doppler-detected motion, the image guidance system may provide an easy-to-decipher 3D image of the progress of the ablation. For example, FIG. 18 illustrates a needle 1845 ablating tissue while a 3D ultrasound probe 1855 is collecting ultrasound data. The Doppler data is collected from 3D ultrasound probe 1855 and only the progress of the ablation 1804 is shown on display 1820. One or more slices of the collected ultrasound data may also be shown on display 1820 (not pictured)), thereby creating a three-dimensional model of the anatomical region (Razzaque, see at least par. [0200] As another example, consider an embodiment in which the volume is divided into parallel slices. These slices are parallel to the display screen, or parallel to the ultrasound transducer. The slices are drawn from back to front. The transparency of the slice is determined by some function whose input is its distance from the region of interest (such as the ultrasound transducer). Similarly the blurriness may be determined by a similar function. The transparency may be passed as the alpha parameter to the 3D rendering library (OpenGL, DirectX, etc.). Blur may be passed to the 3D rendering library as a maximum MIPMAP level.);
Razzaque does not disclose graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart; and graphically representing data sufficiency of the three-dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies. However,
Altmann discloses:
graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart (Altmann, see at least par.[0052], At initial step 80, a three-dimensional model of the structure is acquired and displayed. This can be an image of the heart, obtained with a system such as the above-noted CARTO XP EP Navigation and Ablation System. However any three-dimensional model can be used, for example a tomographic image. It is important to display the topography of the heart or other structure, and the functional data, for example electrical potentials that may be shown on the model are incidental.);
graphically representing data sufficiency of the three-dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies (Altmann, see pars. [0061], [0063],
the display enables the operator to see where sufficient ultrasound data have been captured and is useful to guide the operator to areas of the heart chamber where additional data collection is still needed; [0068] Referring again to FIG. 4, as the data are acquired in successive iterations of step 70, the electroanatomical map, and optionally vessels, which may be shown diagrammatically on the electroanatomical map as contours or cylindrical structures, are progressively colored to indicate the areas that were imaged, as shown on the image 94 (FIG. 5). For example, the map may start with a gray color, as on the image 92 (FIG. 5), and the color may then change from gray to red at every point on the map that corresponds to points where ultrasound image data were acquired. In this manner, the operator receives a clear indication of the current data coverage; [0079] In additional application of pseudocolor itself can be modified according to the gray scale level of each voxel using a corresponding lookup table. This enables the user to see if the acquired data corresponds to a wall tissue or to a vessel or valve opening in the chamber.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of claimed invention to combine the Razzaque disclosed invention, and have graphically outputting the plurality of two-dimensional image slices in a plurality of voxels according to the associated localization information of the ICE catheter, thereby creating a three-dimensional geometric model of the heart, thereby creating a three-dimensional geometric model of the heart graphically representing data sufficiency of the three-dimensional geometric model of the heart by adjusting opacities of the plurality of voxels according to their respective data sufficiencies, as taught by Altmann. The modification provides an improved system and method for generating a three- dimensional geometric model of medical image, in order to assist the physician in performing a real time cardiac imaging procedure, a three-dimensional image of the heart may be rendered during acquisition and helps the user to determine when adequate image data have been acquired or whether details are still missing (Altmann, see par. [0007]).
Regarding claim 28. Razzaque in view of Altmann discloses the method of claim 26 (as rejected above) Razzaque in view of Altmann further discloses:
graphically representing data sufficiency of the three-dimensional geometric model of the heart region further comprises outputting a data collection cue (Razzaque, see at least [0135], The rings may be spaced at regular (e.g., 0.5, 1, or 2 cm) intervals to provide the physician with visual cues regarding the distance from the needle tip to the targeted anatomy. In some embodiments, the spacing of the rings may indicate other aspects of the data, such as the drive speed of the needle, the density of the tissue, the distance to a landmark, such as the ultrasound data, or any other appropriate guidance data or property).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Razzaque in view of Altmann, and to have wherein graphically representing data sufficiency of the three-dimensional geometric model of the anatomical region comprises outputting a data collection cue, as provided by Razzaque. The modification provides an improved system and method for generating a three-dimensional geometric model of medical image; thereby to aid the surgeon to identify important features in the medical image. In addition, when image guidance cues are co-located with medical display object on the display, the medical device system may switch between displaying the image guidance cue and the medical display object, resulting in a flicker (Razzaque, see par. [0004]).
Regarding claim 29. Razzaque in view of Altmann discloses the method of claim 28 (as rejected above), and Razzaque in view of Altmann further discloses:
wherein the data collection cue is responsive to a rotational speed of the ICE catheter (Razzaque, see at least par. [0055] The rings can be spaced at regular (e.g., 0.5, 1, or 2 cm) intervals to provide the healthcare provider with visual or guidance cues regarding the distance from the medical device tip to the targeted anatomy. In some embodiments, the spacing of the rings can indicate other aspects of the data, such as the drive speed of the medical device, the density of the tissue, the distance to a landmark, such as the ultrasound data, or any other appropriate guidance data or property. In some embodiments, the rings or other trajectory indicators can extend beyond the medical device tip, by a distance equal to the length of the medical device-shaft. This way, the user knows if the medical device is long enough to reach the target--even before the tip enters the patient. That is, in some embodiments, if the rings do not reach the target with the tip still outside the body, then the tip won't reach the target even when the entire length shaft is inserted into the body. [0056] Other display markers can be used to show trajectory, such as a dashed, dotted, or solid line, transparent medical device shaft, point cloud, wire frame, etc. In some embodiments, three-dimensional rings can be used and provide depth cues and obscure little of the ultrasound image. Virtual rings or other virtual markers can be displayed semi-transparently, so that they obscure less of the ultrasound image than an opaque marker would.).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Razzaque in view of Altmann, and to have wherein the data collection cue is responsive to a rotational speed of the ICE catheter, as provided by Razzaque. The modification provide an improved system and method for generating a three-dimensional geometric model of medical image; thereby to aid the surgeon to identify important features in the medical image. In addition, when image guidance cues are co-located with medical display object on the display, the medical device system may switch between displaying the image guidance cue and the medical display object, resulting in a flicker (Razzaque, see par. [0004]).
Claim 27 is rejected under 35 U.S.C. 103 as being unpatentable over in view of Razzaque et al. (US 20110137156 A1) in view of Altmann et al. (US 20080137927 A1, hereinafter Altmann), applied claim 26 above, and further in view of further in view of KIM (US 20140330121 A1 hereinafter KIM).
Regarding claim 27. Razzaque in view of Altmann discloses the method of claim 26, but Razzaque in view of Altmann does not disclose wherein, for a selected voxel of the plurality of voxels, the opacity of the selected voxel increases as data sufficiency at the selected voxel increases. However,
KIM discloses:
wherein, for a selected voxel of the plurality of voxels, an opacity of the selected voxel increases as data sufficiency at the selected voxel increases (KIM, see at least par. [0166] When the operator wants to obtain information regarding the surface of the target region rather than the inside of the target region, the opacity value may be established as a 2D function with respect to the elasticity value and the voxel value, while only increasing proportionally to the voxel value. Thus, when the elasticity value is 0, the opacity value may be established as a 1D increasing function with respect to the voxel value.).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method and apparatus of Razzaque in view of Altmann, and to have wherein, for a selected voxel of the plurality of voxels, the opacity of the selected voxel increases as data sufficiency at the selected voxel increases, as provided by KIM. The modification provide an improved system and method for generating a three-dimensional geometric model of medical image and provide an ultrasonic imaging apparatus to output a multi-dimensional ultrasonic image of a target region of an object to be diagnosed in which lesion areas are separated from non-lesion tissues using elasticity data of the object and a method of controlling the ultrasonic imaging apparatus. (KIM, see par. [0009]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIM THANH THI TRAN whose telephone number is (571)270-1408. The examiner can normally be reached Monday-Friday 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KEE M TUNG can be reached on 571-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KIM THANH T TRAN/Examiner, Art Unit 2612
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights