DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 12, and 16 are objected to because of the following informalities:
In claim 1, lines 13-14, it appears ‘the one or more echogenic boundaries of the intermediate image data’ is not previously set forth as the prior recitation refers to the boundaries of the “first image data”. The same issue is present in claim 12.
In claim 1, line 15, it appears ‘the’ should be inserted before ‘registration’. The same issue is present in claim 12.
In claim 16, line 4, it appears ‘includes’ should be deleted
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-13, and 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gross et al (US Pub 2013/0345545 -cited by applicant) in view of Weiss et al (US Pub 2020/0104878 -cited by applicant) and Maraghoosh et al (US Pub 2016/0217560 -cited by applicant).
Re claims 1, 2, 12: Gross discloses a system and method of registering a first image data to a second image data via an intermediate image data, the method comprising:
acquiring a first MR image data (fig 1, 0026; see step 32 where MR data is acquired);
acquiring a second image data (fig 1, 0017; see step 30 where ultrasound data is acquired);
identifying one or more echogenic boundaries of in the first image data [0043, 0088; see the feature based registration which initially identifies echogenic boundaries in both images, such as bone features];
generating the intermediate image data that is correlatable to the second image data from the first image data using the echogenic boundaries [0043, 0088; see the MR data that is converted to emulate ultrasound data, wherein the MR image includes bone features which are inherently used and converted to emulated data];
registering the second image to the intermediate image based on correlating the intermediate image data to the second image data by matching the echogenic boundaries of the intermediate and second images [0043; see the registration that uses correlation and see the feature based registration which matches features such as bone in the images]; and
registering the first image data to the second image data using the registration of the second image to the intermediate image (fig 1, 0041, 0043; see step 40 where the ultrasound frames are spatially registered with the MR frames and see that the synthesized ultrasound data is registered with the acquired ultrasound data, wherein both the registration of the US to MR image and the US to the emulated image apply the same registration, whether it be the feature-based registration, correlation, translation, scale, etc).
Further, the method is performed with a processor module of the system to execute instructions [0083; see the hardware and software].
Gross discloses all features including extracting features from MR data or from the processed MR data (i.e. the intermediate image) and that the MR data include bone structures [0043, 0088], but does not disclose that generating the intermediate image includes identifying an echogenic boundary of a vertebral body in the first image. However, Weiss teaches of a method and device wherein generating the intermediate image includes identifying an echogenic boundary of a vertebral body in the first image [0203; see the first image or 3D model that is generated from the first image wherein a vertebrae contour is identified and then used for subsequent alignment/correlation]. It would have been obvious to the skilled artisan to modify Gross, to identify vertebral bodies as taught by Weiss, in order to accurately alignment image features in the correct position and orientation.
Gross/Weiss disclose all features except for segmenting the intermediate image data into one or more portions corresponding to the boundaries. However, Maraghoosh teaches of a method and system for image registration wherein image segmentation is performed corresponding to echogenic boundaries [0026, 0034; see the anatomical bone structure wherein the segmentation of the anatomical feature is implemented]. It would have been obvious to the skilled artisan to modify Gross/Weiss, to perform segmentation on the intermediate image as taught by Gross/Weiss, because the image processing technique is very well known and would facilitate a subsequent registration/correlation [0001 of Maraghoosh].
Re claims 3, 13: Gross discloses generating the intermediate image data that is correlatable to the second image data from the first image data includes generating a mock computed tomography image data based on the first image data and includes identifying echogenic features in the first image data [0043; see that the emulated ultrasound data is based on the first MR data and such is considered a mock CT because it is not actually a CT image while having similarities to a CT image; the emulated ultrasound includes identified echogenic features of which ultrasound is based upon].
Re claim 5: Gross discloses generating the intermediate image data that is correlatable to the second image data from the first image data is automatically generated by executing instructions of an algorithm with a processor module [0056, 0083; see the hardware and software as well as the automatic processing].
Re claims 6, 15: Gross/Weiss discloses all features including generating the intermediate image data that is correlatable to the second image data from the first image data [0043; see the MR data that is converted to emulate ultrasound data] but does not disclose that it is generated by processing the first image data with a trained model stored in memory. However, Maraghoosh teaches of a method and system that includes memory wherein a trained model is used for registration [0048; see the training datasets for the deformable registration]. It would have been obvious to the skilled artisan to modify Gross/Weiss, to use a trained model as taught by Maraghoosh, in order to improve live registration of images [see 0009 of Maraghoosh].
Re claims 7, 8, 16: Gross discloses correlating the intermediate image data to the second image data includes relating features identified in both the second image data and the intermediate image data and registering the first image data to the second image data includes defining a translation between a first image coordinate system and a second image coordinate system based on the correlation of the intermediate image data to the second image data [0043; see the registration that uses correlation of a measure of similarity which is a relating of features of the image data and see the “translation” of the registration that is found for the best match].
Re claims 9-11, 17-20: Gross discloses the method and instructions include registering the first image coordinate system to a subject coordinate system, and include tracking an imaging system in the subject coordinate system while the system is acquiring the second image data while being tracked; registering the first image data to the subject coordinate system based on the registration of the first image data to the second image data; and where the image data is acquired by a first and second imaging system and the tracking is performed by a tracking navigation system [0034, 0043, 0045, 0048; fig 4; see the coordinate system of the subject represented by the coordinate location of the patient and the coordinate system of the first and second image data and see the MR and ultrasound imaging systems that are used for tracking and registration; also see the MR and ultrasound systems as well as the processor that tracks and/or navigates using fiducials or position sensors].
Response to Arguments
Applicant's arguments filed 8/5/26 have been fully considered but they are not persuasive. Applicant argues that the limitations of identifying echogenic boundaries and then using those boundaries to generate the intermediate image is not met by the prior art. Respectfully, the Examiner disagrees. Gross is found to disclose to disclose echogenic boundaries as the reference describes feature based registration and that features of the first MR image include bone. Since the synthesized image is generated from the first MR image (i.e. all of the pixels/features of the first MR image are processed), it follows that the echogenic features are used to generated the synthesized image. Weiss is applied to teach that the echogenic boundaries are those of a “vertebral body” as obvious to meet the recited limitation. The combination of Gross and Weiss meets the limitation and Weiss is not relied upon solely to teach generation of an intermediate image. It is noted that the claimed ‘echogenic boundaries’ are present in Gross even though the image is an MR image because the bone features are echogenic. Furthermore, Gross is relied upon to disclose registration (or correlating/matching) of the second image to the intermediate image and of the first image to the second image. The registration of the second image to the intermediate image used a feature based registration and also specifically applies correlation, translation, scale, etc. As the embodiment described in [0043] spatially registers the first to the second image as well as the second image to the intermediate image, the same registration technique is applied. It follows that the registration of the first image to the second image uses the registration of the second image to the intermediate image, as claimed.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL T ROZANSKI whose telephone number is (571)272-1648. The examiner can normally be reached Mon - Fri 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached at 571-272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL T ROZANSKI/Primary Examiner, Art Unit 3797
Prosecution Insights