DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wen et al. (“A Novel Ultrasound Probe Spatial Calibration Method Using a Combined Phantom and Stylus”, 2020), as cited by Applicant, and as evidenced by “Optical Navigation Technology” (online: https://www.ndigital.com/optical-navigation-technology/, Accessed on 5/9/25), in view of Fichtinger et al. (US Pub No. 2014/0121501).
With regards to claims 1, 12 and 17, Wen et al. disclose a method and system comprising:
an imaging system comprising an imaging device (i.e. US probe) (pg. 2081, left column, 2nd paragraph – right column, 2nd paragraph, referring to the 4.5 MHz abdominal curvilinear transducer/”US probe” and the corresponding US scanner; Figures 1-2);
a tracking system (i.e. “optical tracking system”, pg. 2081, left column, 2nd paragraph, referring to the real-time tracking being performed by the optical tracking system; pg. 2083, right column, 2nd paragraph, referring to the tracking system which tracks a position/orientation sensor that is attached to the stylus) comprising:
a transmission device (pg. 2081, left column, 2nd paragraph, referring to the use of the optical tracking system “Polaris Spectra, NDI”, which, as evidenced by Optical Navigation Technology comprises using NIR light to detect and track navigation markers); and
a tracked device (“tracked stylus”) (pg. 2081, right column, 2nd-3rd paragraphs, pg. 2083, right column, 2nd paragraph, referring to the stylus which is referred to as the pointer or 3D localizer, wherein on one end, the stylus is attached to a position/orientation sensor that can be tracked by the tracking system; Figures 2, 4-5);
a calibration phantom (pg. 2081, right column, 2nd paragraph, pg. 2083, left column, 2nd paragraph, referring to the calibration phantom);
a processor (“Data acquisiton and processing system”) (pg. 2083, left column, 1st paragraph; Figures 3 and 6-7); and
a memory storing data that, when processed by the processor (pg. 2083, left column, 1st paragraph; Figures 3 and 6-7, referring to the “data acquisition and processing system”, which would inherently include a memory for executing the functions), cause the processor to:
generate a navigation space (“world coordinate system W” or “position/orientation sensor coordinate system S”) based on one or more tracking signals (pg. 2081, right column, 3rd paragraph, referring to the position/orientation sensor coordinate system S and the world coordinate system (W) (i.e. the optical tracking system); Figures 2, 5;
generate a virtual space (i.e. “image coordinate system I”) comprising at least a portion of the calibration phantom based on one or more images generated by the imaging system, wherein the one or more images (i.e. US B-scan image planes) are generated in response to one or more signals transmitted by the imaging device (pg. 2081, right column, last paragraph-pg. 2083, left column, 1st paragraph, referring to B-scan image coordinate system (I), wherein the stylus tip coincides with the center of the phantom slot, and thus coincides with the US B-scan image plane; Figures 2-4);
track, based on electromagnetic signals, the tracked device moving within the calibration phantom (pg. 2081, right column, 2nd-3rd paragraphs, pg. 2083, left column, 2nd paragraph-right column, 2nd paragraph, pg. 2084, right column, last paragraph, referring to the stylus being attached to a position/orientation sensor that can be tracked by the tracking system, wherein the stylus is placed in the calibration phantom; pg. 2081, left column, 2nd paragraph, referring to the use of the optical tracking system “Polaris Spectra, NDI”, which, as evidenced by Optical Navigation Technology comprises using NIR light to detect and track navigation markers, and thus electromagnetic (i.e. NIR is within the electromagnetic range) signals are used to track the tracked device; Figures 2, 4-5);
detect an event when a portion of the tracked device intersects with a point within a field of view of the imaging device (pg. 2084, left column, last paragraph-right column, last paragraph, referring to the “intersection point” which is between the virtual plane and the stylus tip and yields a strong echo spot in the B-scan image, wherein a user draws a circle around the echo spot of the feature point as the region of interest, which acts as the initial search window for the CAMSHIFT algorithm which is used to semi-automatically extract and continuously track the pixel location of the feature point, wherein for each extraction of a feature point, a simple circle drawing of the echo spot is only required and the tracking algorithm is capable of reporting the accurate pixel location of the feature point continuously, and therefore the “circle drawing” serves as the detected event when a portion of the tracked device intersects with a point (i.e. “feature point”) within a field of view of the imaging device);
identify, in response to detecting the event (i.e. “circle drawing” initiates the identification of set of coordinates, registration, etc.) (pg. 2084, left column, last paragraph-right column, last paragraph):
a set of coordinates (i.e. 2D pixel coordinate (u, v) in the B-scan image coordinate system) in a first coordinate system of the virtual space (pg. 2081, right column, last paragraph-pg. 2082, right column, first full paragraph; pg. 2084, left column, last paragraph-right column, 3rd paragraph, referring to the identification of the feature point/pixel location pI in the B-mode image; Figures 2-6); and
register the first coordinate system (i.e. image coordinate system) associated with the virtual space with a second coordinate system (i.e. world coordinate system) associated with the tracked device based on the set of coordinates (pg. 2081, right column, last paragraph-pg.2086, right column, 1st paragraph, referring to the pixel location of the feature point corresponding to the stylus tip in the B-scan being extracted and continuously tracked, wherein during the acquisition of the pixel location of the feature point, the spatial position of the stylus tip in the world coordinate is recorded simultaneously, and wherein, once the corresponding data sets are obtained, a solution for the probe calibration is found by mapping/registering one set to the other; Figures 2-6).
However, Wen et al. do not specifically disclose that the registration is further based on “temporal information”, wherein the temporal information indicates when the portion of the tracked device intersected with the point.
Fichtinger et al. disclose a calibration method that uses a calibration phantom and provides automatic, intraoperative calibration of ultrasound imaging systems, wherein a temporal calibration is performed to synchronize individual ultrasound image frames with their respective positions prior to using a closed-form formula or an iterative solver to calculate calibration parameters (Abstract; paragraph [0026], [0066]; Figure 4). Since the images and the tracked probe positions are generated by separate hardware (i.e. TRUS scanner and the stepper tracking system), proper synchronization between the two must be established to correctly associate each acquired TRUS image with its corresponding positional data, wherein this process is referred to as the “temporal calibration”) (paragraph [0084]). When the TRUS image is acquired and its corresponding stepper position are recorded, both data can be time-stamped, wherein the time-stamping process itself introduces some delay, and therefore the goal of temporal calibration is to determine this delay (“latency”) (paragraphs [0084]-[0085]; note that the “temporal information” (i.e. time-stamp information and/or determined delay/latency) in Wen et al. would also serve as an indication of when the portion of the tracked device intersected with the point as stylus/tracked device position in Wen et al. is acquired due to the detection of the event when a portion of the tracked device intersect with the point within the field of view of the imaging device (i.e. see pg. 2084, detection of the “bright echo spot/feature point which is due to the intersection).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the registration of Wen et al. to be further based on “temporal information”, wherein the temporal information indicates when the portion of the tracked device intersected with the point, as taught by Fichtinger et al., in order to synchronize individual ultrasound image frames with their respective positions and thus correctly associate each acquired TRUS image with its corresponding positional data (paragraphs [0026], [0066]; [0084]).
With regards to claims 2, 13 and 18, Wen et al. disclose that the point is within a plane that correspond to the field of view of the imaging device (pg. 2081, right column, 2nd-3rd paragraphs, referring to “…verifying the intersection point between the needle tip and stylus tip in the B-scan image…We use the stylus tip to directly measure the spatial position of intersection point in 3-D space by pointing the stylus at the target in our calibration system” and pg. 2084, left column, bottom paragraph-right column, top paragraph, referring to “ The intersection point between the virtual plane and the stylus tip yields a strong echo spot in the B-scan image..The position p1 of stylus tip in the B-scan is extracted…”, and thus, if the point is in the “B-scan image”, the point is within a plane that corresponds to the field of view of the imaging devce).
With regards to claims 3, 14 and 19, Wen et al. disclose re-registering the first coordinate system with the second coordinate system in response to one or more additional occurrences of the event (pg. 2084, right column, 2nd paragraph and 4th paragraph; pg. 2086, right column, 2nd paragraph, referring to the continuous tracking which would thus encompass one or more occurrences of the event); and calibrating the first coordinate system with respect to the second coordinate system is absent pausing a surgical procedure (pg. 2084, right column, 2nd paragraph and 4th paragraph; pg. 2086, right column, 2nd paragraph, wherein the calibration occurs when intra-operative US images are acquired, and thus calibration would be absent when the intraoperative US images are not being acquired, which would occur when a surgical procedure/operation is paused).
With regards to claims 4 and 16, Wen et al. disclose registering the first coordinate system with the second coordinate system is based on: beam thickness, beam shape, or both of the one or more signals transmitted by the imaging device [which provide the B-scan scans]; pose information (position/orientation) of the portion of the tracked device; and one or more properties (i.e. shape of the stylus) of the portion of the tracked device (pg. 2081, right column, last paragraph-pg.2086, right column, 1st paragraph, referring to the pixel location of the feature point corresponding to the stylus tip in the B-scan being extracted and continuously tracked, wherein during the acquisition of the pixel location of the feature point, the spatial position (i.e. pose information) of the stylus tip in the world coordinate is recorded simultaneously (i.e. temporal information), and wherein, once the corresponding data sets are obtained, a solution for the probe calibration is found by mapping one set to the other; further referring to the size of the holes in the phantom, which is used for the calibration, being designed according to the shape of the positioning stylus; Figures 2-6).
With regards to claims 5, 15 and 20, Wen et al. disclose that the calibration phantom comprises: ultrasound conductive material (i.e. water); or a tissue phantom comprised in a body of a subject, wherein the tissue phantom and the subject are associated with a surgical procedure (pg. 2083, left column, 2nd paragraph, referring to the slot in the center of the phantom is filled with water; Figure 2).
With regards to claim 6, Wen et al. disclose that the instructions are further executable by the processor to: output guidance information associated with positioning the imaging device, the tracked device, or both (pg. 2081, left column, 1st paragraph, referring to the integrated design of the needle-guide with the marker frame resulting in the B-scan image being capable of providing 3D image guidance for surgical procedures, Figures 2 and 7, wherein the B-scan images are output/displayed and provide a depiction of the puncture needle and intersection point between the needle tip and the stylus tip).
With regards to claim 7, Wen et al. disclose that the tracked device is comprised in at least a portion of surgical instrument (i.e. stylus/needle), and the instructions are further executable by the processor to: detect, in the one or more images, one or more landmarks (i.e. stylus tip) of the surgical instrument, wherein registering the first coordinate system with the second coordinate system is based on the one or more landmarks (pg. 2084, left column, last paragraph-right column, first paragraph, referring to the “The intersection point between the virtual plane and the stylus tip yields a strong echo spot in the B-scan image…”; Figure 2, in particular, see the image of Figure 2b, wherein the intersection point between the needle tip and the stylus tip are visible as a landmark in the image).
With regards to claim 8, Wen et al. disclose that the instructions are further executable by the processor to verify accuracy of the registration between the first coordinate system and the second coordinate system (pg. 2086, right column, 2nd paragraph -pg. 2087, right column, 2nd paragraph, referring to the evaluation of the calibration accuracy of the probe, wherein the index of point reconstruction accuracy (PRA) is used to estimate the calibration accuracy and is defined by Eq. 9).
With regards to claim 9, Wen et al. disclose that the instructions are further executable by the processor to: detect one or more discrepancies (i.e. see Equation 9 which finds the difference/discrepancy between the spatial position of the stylus tip and the transformed pixel location of the stylus tip in the B-scan image) between first tracking data of the tracked device generated based on the electromagnetic signals and second tracking data of the tracked device generated based on output of the imaging device; and generate a notification (see Tables 1, 2, 3 and Figure 7) associated with the one or more discrepancies, perform one or more operations associated with compensating for the one or more discrepancies, or both (pg. 2086, right column, 2nd paragraph -pg. 2087, right column, 2nd paragraph, referring to the evaluation of the calibration accuracy of the probe, wherein the index of point reconstruction accuracy (PRA) is used to estimate the calibration accuracy and is defined by Eq. 9; Tables 1-3, Figure 7).
With regards to claim 10, Wen et al. disclose that the imaging device comprise an ultrasound imaging device (pg. 2081, left column, 2nd paragraph, referring to the US probe; Figures 1-2).
With regards to claim 11, Wen et al. disclose that the system further comprises the imaging device (pg. 2081, left column, 2nd paragraph, referring to the US probe; Figures 1-2).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Fichtinger has been introduced to teach the limitations of identifying temporal information indicating when the portion of the tracked device intersected with the point, etc..
With regards to Wen, Applicant argues that Wen is silent with regards to tracking a device based on electromagnetic signals and then detecting an event when a portion of the tracked device intersects with a point within a field of view of an imaging device.
Examiner respectfully disagrees and, with regards to the tracking of a device based on “electromagnetic signals”, points to pg. 2081, left column, 2nd paragraph of Wen et al., which discloses the use of the optical tracking system “Polaris Spectra, NDI”, which, as evidenced by Optical Navigation Technology comprises using NIR light to detect and track navigation markers, and thus electromagnetic (i.e. NIR is within the electromagnetic range) signals are used to track the tracked device.
With regards to detecting an event when a portion of the tracked device intersects with a point within a field of view of an imaging device, Examiner points to pg. 2084, left column, last paragraph-right column, last paragraph of Wen et al., referring to the “intersection point” which is between the virtual plane and the stylus tip and yields a strong echo spot in the B-scan image, wherein a user draws a circle around the echo spot of the feature point as the region of interest, which acts as the initial search window for the CAMSHIFT tracking algorithm which is used to semi-automatically extract and continuously track the pixel location of the feature point, wherein for each extraction of a feature point, a simple circle drawing of the echo spot is only required and the tracking algorithm is capable of reporting the accurate pixel location of the feature point continuously. Therefore, the “circle drawing” serves as the detected event when a portion of the tracked device intersects with a point (i.e. “feature point”) within a field of view of the imaging device as the circle is drawn around the “bright/strong echo point” which corresponds to the claimed intersection. The detection of the event of the “circle drawing” initiates the CAMSHIFT tracking algorithm, which leads to the identification of the set of coordinates, etc..
Wen et al. is therefore maintained in the rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATHERINE L FERNANDEZ whose telephone number is (571)272-1957. The examiner can normally be reached Monday-Friday 9:00 AM - 5:30 PM (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHERINE L FERNANDEZ/ Primary Examiner, Art Unit 3798
Prosecution Insights