DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 16 is objected to because of the following informalities:
Claim 16 line 8, limitation “the oodometer data” should read “the odometer data”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 8 – 15 and 23 – 30 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 8 recites the limitation "the output data" in line 2 – 3. There is insufficient antecedent basis for this limitation in the claim.
In addition, it is also unclear the above “output data” is the data output from which element or by which process, since there are position data fusion process, position data acquisition process, image data acquisition process from various structural elements.
Thus, the above limitation renders claim indefinite. For the purpose of examination, the above “output data” is interpreted as any reasonable data.
Claim 13 recites the limitation "the x-axis, y-axis, and z-axis" in line 2 – 3. There is insufficient antecedent basis for this limitation in the claim.
Claim 23 recites the limitation "the output data" in line 2 – 3. There is insufficient antecedent basis for this limitation in the claim.
In addition, it is also unclear the above “output data” is the data output from which element or by which process, since there are position data fusion process, position data acquisition process, image data acquisition process from various structural elements.
Thus, the above limitation renders claim indefinite. For the purpose of examination, the above “output data” is interpreted as any reasonable data.
Claim 28 recites the limitation "the x-axis, y-axis, and z-axis" in line 2 – 3. There is insufficient antecedent basis for this limitation in the claim.
Therefore, claim 8, 13, 23, 28 and all corresponding dependent claim 9 – 12, 14, 15, 24 – 27, 29 and 30 are rejected under 35 U.S.C. 112(b), as being indefinite.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 1 – 4, 6 – 8, 11, 12, 16 – 19, 21 – 23, 26 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over dos Santos Mendonca et al. (US 2016/0317122 A1; published on 11/03/2016) (hereinafter "dos Santos Mendonca") in view of Hirota (US 2015/0335252 A1; published on 11/26/2015).
Regarding claim 1, dos Santos Mendonca teaches an apparatus for three-dimensional (3D) image reconstruction ("... relates to an apparatus for ultrasonography ..." [0006]; "… the ultrasound image data may include a series of 2-D image frames and the processed ultrasound image data may include a 3-D image." [0007]), the apparatus comprising:
a housing ("FIG. 1 illustrates a hand-held ultrasonic imaging probe, according to an implementation. The apparatus 100 …" [0028]; see Fig.1) configured for freehand movement within a 3D reference frame defined by an Xw-axis, Yw-axis, and Zw-axis ("... to calculate, in real time, the probe's X, Y and Z location as well as the probe's pitch, yaw, and roll orientation with respect to these landmarks." [0029]; "… to measure translational and rotational motion of the apparatus 100." [0030]), the housing comprising:
an ultrasound probe configured to generate scans of the sample ("… the ultrasound image data may include a series of 2-D image frames …" [0008]; "… outputs of an ultrasonic scan performed by the probe may be processed …" [0035]);
a visual odometer configured to track movement of the ultrasound probe in the 3D reference frame to generate odometer data ("… the hand-held ultrasonic imaging probe may be configured to make a real-time determination of its spatial position with respect to an arbitrary coordinate system using a combination of optical and inertial sensors." [0028]; here the optical and inertial sensors are the visual odometer); and
a processor in communication with the ultrasound probe and the visual odometer ("… a processor 140 communicatively coupled with the ultrasonic transducer 110, the inertial sensor 120, and the optical sensor 130." [0028]), configured to:
receive the scans and odometer data ("… the processor 140 may be configured to receive data inputs from the inertial sensor 120 and the optical sensor 130 and/or the ultrasonic transducer …" [0031]), wherein the scans and odometer data each include timestamps ("For example where the estimated 6-DoF spatial position at a time corresponding to a 2-D image frame (i) …" [0051]; in order to perform this operation, timestamp of position data and image frame must be known),
synchronize the scans and odometer data ("… the processor may be configured to adjust one or more of the 2-D image frames in view of the estimated 6-DoF spatial position at the time of obtaining each respective 2-D image ... one or both of the respective 2-D image frames may be adjusted to compensate for the difference." [0051]; compensation for the difference is a synchronization), and
construct a 3D image of the scans based on the synchronization ("As a result, temporal series of 2-D images may be more accurately combined to compute 3-D image data 560." [0051]).
dos Santos Mendonca fails to explicitly teach a laser source configured to transmit an electromagnetic (EM) wave into a sample to produce a photoacoustic effect therein. However, it is well-known in the art that an ultrasound probe is capable of performing photoacoustic imaging in combination with additional light source.
In the same field of endeavor, Hirota teaches an apparatus for three-dimensional (3D) image reconstruction ("A photoacoustic measurement device 10 according to this embodiment …" [0040]; "… when a plurality of two-dimensional photoacoustic images are acquired by the probe 11 including a two-dimensional transducer array or probe scanning, the display control unit 30 may create volume data on the basis of the photoacoustic images and display a composite image as a three-dimensional image on the display unit 14." [0064]), the apparatus comprising:
a housing ("… a housing 45 that contains these components." [0043]), the housing comprising:
a laser source configured to transmit an electromagnetic (EM) wave into a sample ("The laser unit 13 is a light source unit that emits the laser light L to be radiated to the subject M as measurement light." [0050]) to produce a photoacoustic effect therein ("When the subject M absorbs the laser light, photoacoustic waves are generated in the subject M." [0050]);
an ultrasound probe configured to generate scans of the sample ("The ultrasonic transducer 20 a has a function of converting a received signal into an electric signal when the acoustic wave U is received." [0043]; "For example, when a plurality of two-dimensional photoacoustic images are acquired by the probe 11 including a two-dimensional transducer array or probe scanning …" [0064]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the ultrasound apparatus as taught by dos Santos Mendonca with the additional photoacoustic imaging elements as taught by Hirota. Doing so would make it possible to "accurately generate an ultrasonic image and a photoacoustic image as a reflected acoustic image in the same imaging range" (see Hirota; [0048]).
Regarding claim 2, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 1, and Hirota further teaches wherein the ultrasound probe includes one or more one- dimensional (1D) or two-dimensional (2D) array transducers ("The transducer array 20 is constituted of, for example, a plurality of ultrasonic transducers 20 a (acoustic wave detection elements) that are one-dimensionally or two-dimensionally arranged." [0043]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the ultrasound apparatus as taught by dos Santos Mendonca with the additional photoacoustic imaging elements as taught by Hirota. Doing so would make it possible to "accurately generate an ultrasonic image and a photoacoustic image as a reflected acoustic image in the same imaging range" (see Hirota; [0048]).
Regarding claim 3, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 1, and dos Santos Mendonca further teaches wherein the visual odometer includes at least one optical sensor and an inertial measurement unit (IMU) ("… the hand-held ultrasonic imaging probe may be configured to make a real-time determination of its spatial position with respect to an arbitrary coordinate system using a combination of optical and inertial sensors." [0028]; here the optical and inertial sensors are the visual odometer).
Regarding claim 4, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 3, and dos Santos Mendonca further teaches wherein the at least one optical sensor includes a visible light camera, or infrared camera ("… the term “optical sensor” refers to a device configured to optically detect visible, infrared and or ultraviolet light and/or images thereof, and includes any kind of camera or photodetector." [0028]).
Regarding claim 6, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 3, and dos Santos Mendonca further teaches wherein the optical sensor is configured to be directed away from the sample ("… the processor 140 may be configured to collect image data of an environment (for example, an examining room) in which a subject is to be examined using the apparatus." [0029]; the optical sensor 130 is within the same housing of transducer and sensing the external environment, which means the optical sensor is facing away from where the transducer is contacting).
Regarding claim 7, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 3, and dos Santos Mendonca further teaches wherein the IMU includes a tri-axial gyroscope, a tri-axial accelerometer ("The inertial sensor 120 may be configured as or include an accelerometer, a gyroscope …" [0030]).
Regarding claim 8, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 7, and dos Santos Mendonca further teaches wherein the processor is further configured to fuse data acquired by the optical sensor, gyroscope, accelerometer, into the output data including a six degrees of freedom (6 DOF) pose and an orientation of the optical sensor ("For example, the processor may be configured to estimate a 6-DoF spatial position of the apparatus using a combination of outputs from two or more of the ultrasonic transducer 110, the inertial sensor 120 and the optical sensor 130." [0031]).
Regarding claim 11, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 8, and dos Santos Mendonca further teaches wherein the processor is further configured to trim the odometer data to match the scans ("For example where the estimated 6-DoF spatial position at a time corresponding to a 2-D image frame (i) is different from the estimated 6-DoF spatial position at a time corresponding to a 2-D image frame (i+1), one or both of the respective 2-D image frames may be adjusted to compensate for the difference." [0051]).
Regarding claim 12, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 11, and dos Santos Mendonca further teaches wherein the processor is further configured to apply a smoothing filter to the output data to generate smooth data with reduced noise ("... the integration of optical with inertial measurements may include use of an extended Kalman filter (EKF) which would optimally combine measurements from each type of sensor into an overall coherent estimation of the probes position and orientation." [0045]; Kalman filter has the same function to smooth signal to reduce noise).
Regarding claim 16, dos Santos Mendonca teaches a method for three-dimensional (3D) image reconstruction ("… a method for ultrasonography …" [0012]; "… the ultrasound image data may include a series of 2-D image frames and the processed ultrasound image data may include a 3-D image." [0007]), the method comprising:
receiving, using a processor ("… a processor 140 communicatively coupled with the ultrasonic transducer 110, the inertial sensor 120, and the optical sensor 130." [0028]; this citation applies to all features that using the processor in later sections), ultrasound scans from an ultrasound probe ("… the ultrasound image data may include a series of 2-D image frames …" [0008]; "… outputs of an ultrasonic scan performed by the probe may be processed …" [0035]; it is well-known in the art that any ultrasound probe is capable of receiving photoacoustic scans, because the photoacoustic waves are also ultrasound waves) within a housing ("FIG. 1 illustrates a hand-held ultrasonic imaging probe, according to an implementation. The apparatus 100 includes an ultrasonic transducer 110 …" [0028]; see Fig.1) configured for freehand movement within a 3D reference frame defined by an Xwv-axis, Yw-axis, and Zw-axis ("... to calculate, in real time, the probe's X, Y and Z location as well as the probe's pitch, yaw, and roll orientation with respect to these landmarks." [0029]; "… to measure translational and rotational motion of the apparatus 100." [0030]);
receiving, using the processor, odometer data from a visual odometer within the housing configured to track movement of the ultrasound probe in the 3D reference frame ("… the hand-held ultrasonic imaging probe may be configured to make a real-time determination of its spatial position with respect to an arbitrary coordinate system using a combination of optical and inertial sensors." [0028]; here the optical and inertial sensors are the visual odometer; see Fig.1, the optical sensor 130 and inertial sensor 120 are all within the same handheld device 100);
synchronizing the scans and the odometer data ("… the processor may be configured to adjust one or more of the 2-D image frames in view of the estimated 6-DoF spatial position at the time of obtaining each respective 2-D image ... one or both of the respective 2-D image frames may be adjusted to compensate for the difference." [0051]; compensation for the difference is a synchronization) based on timestamps associated with each of the scans and the odometer data ("For example where the estimated 6-DoF spatial position at a time corresponding to a 2-D image frame (i) …" [0051]; in order to perform this operation, timestamp of position data and image frame must be known);
constructing a 3D image of the scans based on the synchronization ("As a result, temporal series of 2-D images may be more accurately combined to compute 3-D image data 560." [0051]).
Although the ultrasound probe, as taught by dos Santos Mendonca, is capable of receiving photoacoustic ultrasound scans, such method step is not explicitly taught by dos Santos Mendonca.
However, in the same field of endeavor, Hirota teaches the step of receiving, using a processor ("... a photoacoustic image generation unit 24 ..." [0056]), photoacoustic ultrasound scans from an ultrasound probe ("That is, the probe 11 can emit (transmit) ultrasonic waves to the subject M and detect (receive) ultrasonic waves (reflected acoustic waves) reflected from the subject M. In addition, the probe 11 can detect photoacoustic waves which are generated in the subject M by the absorption of laser light by an absorber 65 in the subject M." [0042]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the ultrasound apparatus as taught by dos Santos Mendonca with the additional photoacoustic imaging elements as taught by Hirota. Doing so would make it possible to "accurately generate an ultrasonic image and a photoacoustic image as a reflected acoustic image in the same imaging range" (see Hirota; [0048]).
Regarding claim 17, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 16, and Hirota further teaches wherein the ultrasound probe includes one or more one- dimensional (1D) or two-dimensional (2D) array transducers ("The transducer array 20 is constituted of, for example, a plurality of ultrasonic transducers 20 a (acoustic wave detection elements) that are one-dimensionally or two-dimensionally arranged." [0043]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the ultrasound apparatus as taught by dos Santos Mendonca with the additional photoacoustic imaging elements as taught by Hirota. Doing so would make it possible to "accurately generate an ultrasonic image and a photoacoustic image as a reflected acoustic image in the same imaging range" (see Hirota; [0048]).
Regarding claim 18, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 16, and dos Santos Mendonca further teaches wherein the visual odometer includes at least one optical sensor and an inertial measurement unit (IMU) ("… the hand-held ultrasonic imaging probe may be configured to make a real-time determination of its spatial position with respect to an arbitrary coordinate system using a combination of optical and inertial sensors." [0028]; here the optical and inertial sensors are the visual odometer).
Regarding claim 19, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 18, and dos Santos Mendonca further teaches wherein the at least one optical sensor includes a visible light camera, or infrared camera ("… the term “optical sensor” refers to a device configured to optically detect visible, infrared and or ultraviolet light and/or images thereof, and includes any kind of camera or photodetector." [0028]).
Regarding claim 21, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 18, and dos Santos Mendonca further teaches wherein the optical sensor is configured to be directed away from the sample ("… the processor 140 may be configured to collect image data of an environment (for example, an examining room) in which a subject is to be examined using the apparatus." [0029]; the optical sensor 130 is within the same housing of transducer and sensing the external environment, which means the optical sensor is facing away from where the transducer is contacting).
Regarding claim 22, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 18, and dos Santos Mendonca further teaches wherein the IMU includes a tri-axial gyroscope, a tri-axial accelerometer ("The inertial sensor 120 may be configured as or include an accelerometer, a gyroscope …" [0030]).
Regarding claim 23, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 22, and dos Santos Mendonca further teaches using the processor, fusing data acquired by the optical sensor, gyroscope, accelerometer, into the output data including a six degrees of freedom (6 DOF) pose and an orientation of the optical sensor ("For example, the processor may be configured to estimate a 6-DoF spatial position of the apparatus using a combination of outputs from two or more of the ultrasonic transducer 110, the inertial sensor 120 and the optical sensor 130." [0031]).
Regarding claim 26, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 23, and dos Santos Mendonca further teaches using the processor, trimming the odometer data to match the scans ("For example where the estimated 6-DoF spatial position at a time corresponding to a 2-D image frame (i) is different from the estimated 6-DoF spatial position at a time corresponding to a 2-D image frame (i+1), one or both of the respective 2-D image frames may be adjusted to compensate for the difference." [0051]).
Regarding claim 27, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 26, and dos Santos Mendonca further teaches using the processor, applying a smoothing filter to the output data to generate smooth data with reduced noise ("... the integration of optical with inertial measurements may include use of an extended Kalman filter (EKF) which would optimally combine measurements from each type of sensor into an overall coherent estimation of the probes position and orientation." [0045]; Kalman filter has the same function to smooth signal to reduce noise).
Claim 5 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over dos Santos Mendonca in view of Hirota, as applied in claim 3 and 18 respectively, and further in view of Casas (US 2020/0221060 A1; published on 07/09/2020).
Regarding claim 5, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 3, except wherein the at least one optical sensor includes a fisheye camera.
However, in the same field of endeavor, Casas teaches wherein the at least one optical sensor includes a fisheye camera ("… the cameras composing the stereoscopic camera system 114 may have special lenses (e.g. fisheye lenses, or wide-angle lenses) and they are arranged e.g. vertically side by side, to obtain a wider view …" [0068]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the tracking camera as taught by dos Santos Mendonca with the fisheye lens as taught by Casas. Doing so would make it possible "to obtain a wider view" (see Casas; [0068]).
Regarding claim 20, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 18, except wherein the at least one optical sensor includes a fisheye camera.
However, in the same field of endeavor, Casas teaches wherein the at least one optical sensor includes a fisheye camera ("… the cameras composing the stereoscopic camera system 114 may have special lenses (e.g. fisheye lenses, or wide-angle lenses) and they are arranged e.g. vertically side by side, to obtain a wider view …" [0068]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the tracking camera as taught by dos Santos Mendonca with the fisheye lens as taught by Casas. Doing so would make it possible "to obtain a wider view" (see Casas; [0068]).
Claim 9, 10, 24 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over dos Santos Mendonca in view of Hirota, as applied in claim 8 and 23 respectively, and further in view of Erpelding et al. (WO 2014/076674 A2; published on 05/22/2014) (hereinafter "Erpelding").
Regarding claim 9, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 8, except wherein the processor is further configured to identify a motion of a feature in the scans.
However, in the same field of endeavor, Erpelding teaches wherein the processor is further configured to identify a motion of a feature in the scans ("The point target within the B-mode image can be estimated by speckle tracking … which can be done from a few preliminary frame to frame pattern matching comparisons. From the spatial relationship between the point target and the landmark, the in-plane displacement, if any, is determinined." Page 7 Line 17 - 29).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with estimation of point target displacement as taught by Erpelding. "By combining the determined rotational angle 216 and the determined point target displacement, a rigid transformation model can be derived for application to each PA image 208 to register it with respect to the reference imaging geometry which is usually set at the central transducer position corresponding to the PA image" (see Erpelding; Page 6 Line 33 - Page 8 Line 10).
Regarding claim 10, dos Santos Mendonca in view of Hirota and Erpelding teaches all claim limitations, as applied in claim 9, and Erpelding further teaches wherein the feature includes a speckle pattern ("The point target within the B-mode image can be estimated by speckle tracking … which can be done from a few preliminary frame to frame pattern matching comparisons. From the spatial relationship between the point target and the landmark, the in-plane displacement, if any, is determinined." Page 7 Line 17 – 29).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with estimation of point target displacement as taught by Erpelding. "By combining the determined rotational angle 216 and the determined point target displacement, a rigid transformation model can be derived for application to each PA image 208 to register it with respect to the reference imaging geometry which is usually set at the central transducer position corresponding to the PA image" (see Erpelding; Page 6 Line 33 - Page 8 Line 10).
Regarding claim 24, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 23, except using the processor, identifying a translational motion of a feature in the scans.
However, in the same field of endeavor, Erpelding teaches using the processor ("… has instructions executable by a processor for performing a plurality of acts …" Page 2 Line 29 - Page 3 Line 3), identifying a translational motion of a feature in the scans ("The point target within the B-mode image can be estimated by speckle tracking … which can be done from a few preliminary frame to frame pattern matching comparisons. From the spatial relationship between the point target and the landmark, the in-plane displacement, if any, is determinined." Page 7 Line 17 - 29).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with estimation of point target displacement as taught by Erpelding. "By combining the determined rotational angle 216 and the determined point target displacement, a rigid transformation model can be derived for application to each PA image 208 to register it with respect to the reference imaging geometry which is usually set at the central transducer position corresponding to the PA image" (see Erpelding; Page 6 Line 33 - Page 8 Line 10).
Regarding claim 25, dos Santos Mendonca in view of Hirota and Erpelding teaches all claim limitations, as applied in claim 24, and Erpelding further teaches wherein the feature includes a speckle pattern ("The point target within the B-mode image can be estimated by speckle tracking … which can be done from a few preliminary frame to frame pattern matching comparisons. From the spatial relationship between the point target and the landmark, the in-plane displacement, if any, is determinined." Page 7 Line 17 – 29).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with estimation of point target displacement as taught by Erpelding. "By combining the determined rotational angle 216 and the determined point target displacement, a rigid transformation model can be derived for application to each PA image 208 to register it with respect to the reference imaging geometry which is usually set at the central transducer position corresponding to the PA image" (see Erpelding; Page 6 Line 33 - Page 8 Line 10).
Claim 13 – 15 and 28 – 30 are rejected under 35 U.S.C. 103 as being unpatentable over dos Santos Mendonca in view of Hirota, as applied in claim 12 and 27 respectively, and further in view of Mahfouz (US 2017/0340447 A1; published on 11/30/2017).
Regarding claim 13, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 12, except wherein the processor is further configured to divide the smoothed output data by a spatial resolution of the ultrasound probe in each of the x-axis, y-axis, and z-axis to determine a pixel shift.
However, in the same field of endeavor, Mahfouz teaches wherein the processor ("… As part of the registration process, the software of the computer utilizes …" [0391]) is further configured to divide the smoothed output data by a spatial resolution of the ultrasound probe in each of the x-axis, y-axis, and z-axis ("The normalization process may involve scaling one or both of the anatomical 3D model and the template 3D model to have a common unit scale." [0249]) to determine a pixel shift ("… the multi-resolution 3D registration process uses the scale space extracted features to carry out an affine registration calculation between the anatomical 3D model and template 3D model in order to register the two models." [0251]; registration is a process to determine pixel shaft and alignment).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with multi-scale space mapping and registration as taught by Mahfouz. It is important to ensure two different data have common unit scale "so that shape features can be utilized independent of scale" (see Mahfouz; [0249]).
Regarding claim 14, dos Santos Mendonca in view of Hirota and Mahfouz teaches all claim limitations, as applied in claim 13, and dos Santos Mendonca further teaches wherein the processor is further configured to up-sample the scans to match a frame rate of the optical sensor ("For example, a series of sequential 2-D image frames (obtained for example at a rate of 30 frames per second or higher) may be collated to form a 3-D image, after appropriate adjustment of each 2-D image in view of the respective spatial position of the apparatus 100 at the time of obtaining each respective 2-D image." [0031]; here the spatial position data is sampled at the same frame rates as the 2-D image frames).
Regarding claim 15, dos Santos Mendonca in view of Hirota and Mahfouz teaches all claim limitations, as applied in claim 14, and Mahfouz further teaches wherein the processor ("… As part of the registration process, the software of the computer utilizes …" [0391]) is further configured to spatially align the scans in a 3D space based on a transformation, wherein the transformation includes the pixel shift ("… the multi-resolution 3D registration process uses the scale space extracted features to carry out an affine registration calculation between the anatomical 3D model and template 3D model in order to register the two models. In particular, the anatomical 3D model and template 3D model are processed via a rigid registration process." [0251]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with multi-scale space mapping and registration as taught by Mahfouz. It is important to ensure two different data have common unit scale "so that shape features can be utilized independent of scale" (see Mahfouz; [0249]).
Regarding claim 28, dos Santos Mendonca in view of Hirota teaches all claim limitations, as applied in claim 27, except using the processor, dividing the smoothed output data by a spatial resolution of the ultrasound probe in each of the x-axis, y-axis, and z-axis to determine a pixel shift.
However, in the same field of endeavor, Mahfouz teaches using the processor ("… As part of the registration process, the software of the computer utilizes …" [0391]), dividing the smoothed output data by a spatial resolution of the ultrasound probe in each of the x-axis, y-axis, and z-axis ("The normalization process may involve scaling one or both of the anatomical 3D model and the template 3D model to have a common unit scale." [0249]) to determine a pixel shift ("… the multi-resolution 3D registration process uses the scale space extracted features to carry out an affine registration calculation between the anatomical 3D model and template 3D model in order to register the two models." [0251]; registration is a process to determine pixel shaft and alignment).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with multi-scale space mapping and registration as taught by Mahfouz. It is important to ensure two different data have common unit scale "so that shape features can be utilized independent of scale" (see Mahfouz; [0249]).
Regarding claim 29, dos Santos Mendonca in view of Hirota and Mahfouz teaches all claim limitations, as applied in claim 13, and dos Santos Mendonca further teaches using the processor, up-sampling the scans to match a frame rate of the optical sensor ("For example, a series of sequential 2-D image frames (obtained for example at a rate of 30 frames per second or higher) may be collated to form a 3-D image, after appropriate adjustment of each 2-D image in view of the respective spatial position of the apparatus 100 at the time of obtaining each respective 2-D image." [0031]; here the spatial position data is sampled at the same frame rates as the 2-D image frames).
Regarding claim 30, dos Santos Mendonca in view of Hirota and Mahfouz teaches all claim limitations, as applied in claim 29, and Mahfouz further teaches using the processor ("… As part of the registration process, the software of the computer utilizes …" [0391]), spatially aligning the scans in a 3D space based on a transformation, wherein the transformation includes the pixel shift ("… the multi-resolution 3D registration process uses the scale space extracted features to carry out an affine registration calculation between the anatomical 3D model and template 3D model in order to register the two models. In particular, the anatomical 3D model and template 3D model are processed via a rigid registration process." [0251]).
It would have been prima facie obvious to one ordinary skilled in the art before the effective filing date of the invention to modify the 3D image reconstruction as taught by dos Santos Mendonca with multi-scale space mapping and registration as taught by Mahfouz. It is important to ensure two different data have common unit scale "so that shape features can be utilized independent of scale" (see Mahfouz; [0249]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mihailescu et al. (US 2013/0237811 A1; published on 09/12/2013) teach a method and system for tracking handheld instrument with both optical sensor and inertial measurement unit.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAO SHENG whose telephone number is (571)272-8059. The examiner can normally be reached Monday to Friday, 8:30 am to 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M. Kozak can be reached at (571) 270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHAO SHENG/ Primary Examiner, Art Unit 3797
