DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) ) 1-3 and 5-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sprung et al. (US 2020/0196984; hereinafter Sprung) in view of Zheng (US 2024/0201371).
Sprung shows a method and system of determining a motion path characterizing the motion of a movable ultrasound probe across a skin surface of a body portion during acquisition of an ultrasound image of the body portion by the ultrasound probe (abstract), the method comprising: receiving a stream of optical flow data from an optical flow sensor fixedly mounted to the ultrasound probe, wherein the optical flow data is indicative of a two-dimensional movement of the optical flow sensor over the skin surface wherein the optical flow sensor is directed to observe the skin surface so that the optical flow data is indicative of a two-dimensional movement of the optical flow sensor over the skin surface ([0031], [0067]); receiving measurement data from a sensor fixedly mounted to the ultrasound probe (sensor may comprise an optical device such as a camera or laser based motion detecting system; [0090]-[0095], [0103]); inputting the optical flow data and the measurement data into a pathing model ([0090]); and determining, by the pathing model, the motion path based on the optical flow data and the measurement data ([0024]-[0032], [0100]).
Sprung also shows wherein the pathing model includes at least one of the group consisting of a linear scaling model, a non-linear scaling model, an empirical model based on empirical data, and a machine learning mode ([0024]-[0028]); wherein the motion path includes at least one of the group comprising: a scanning direction of the ultrasound probe with respect to the skin surface; a scanning distance of the ultrasound probe with respect to the skin surface; a yaw motion of the ultrasound probe with respect to the skin surface; a scanning depth with respect to the body portion; and a forward-backward directedness ([0090]-[0095]); receiving inertial data from an inertial measurement unit fixedly mounted to the ultrasound probe, wherein the inertial data is indicative of a rotation (Ox, Oy, Oz) of the ultrasound probe, wherein the determining, by the pathing model, of the motion path is further based on the inertial data ([0090]-[0095]); receiving a stream of ultrasound image data from the ultrasound probe while the ultrasound probe is moved along the skin surface of the body portion ([0024]); inputting at least a subset of the ultrasound image data representing a plurality of ultrasound image frames into a machine learning module, wherein the machine learning module has been trained to determine a relative motion between ultrasound image frames ([0024]); determining, by the machine learning module, an image motion path indicating the relative motion between the ultrasound image frames ([0026]); and merging the motion path determined by the pathing model and the image motion path determined from the ultrasound image frames ([0031]); a probe input interface for receiving a stream of ultrasound image data from the ultrasound probe while the ultrasound probe is moved along the skin surface; a machine-learning module having (a) an input section adapted for receiving, as an input, at least a sub-set of the ultrasound image data representing a plurality of ultrasound image frames,(b) a training data section containing training data having been trained to determine the relative motion between ultrasound image frames, wherein the machine-learning module is adapted for determining, from the input and using the training data, an image motion path indicating the relative motion between the ultrasound image frames; and a merging system configured to merge the motion path determined by the pathing model and the image motion path determined from the ultrasound image frames ([0024]-[0028]); inputting at least a subset of the ultrasound image data representing a plurality of ultrasound image frames into a processor; inputting at least a subset of the probe motion data representing the motion of the ultrasound probe into the processor; and generating, by the processor, the 3D tomographic image based on the inputted ultrasound image data and the inputted probe motion data ([0105]); wherein the processor includes a machine learning module which has been trained to generate 3D tomographic images based on the ultrasound image data and the probe motion data ([0024]).
While Sprung describes utilizing additional sensor data related to the position of the ultrasound probe ([0090]-[0095], Sprung fails to show receiving distance measurement data from a distance sensor fixedly mounted to the ultrasound probe, wherein the distance measurement data is indicative of a distance (D) between the distance sensor and the skin surface. Sprung also fails to show wherein the distance sensor is a time-of-flight sensor, particularly an optical time-of- flight sensor, more particularly a laser distance sensor, and wherein the distance sensor is preferably arranged for measuring the distance (D) with respect to a portion of the skin surface within a field of view (FOV) of the optical flow sensor; a mounting structure for fixedly mounting the optical flow sensor and the distance sensor to the ultrasound probe.
Zheng discloses three-dimensional ultrasound imaging methods and systems based on LIDAR. Zheng teaches receiving distance measurement data from a distance sensor fixedly mounted to the ultrasound probe, wherein the distance measurement data is indicative of a distance (D) between the distance sensor and the skin surface ([0004], [0063], [0070]-[0074], [0080] [0082]). Zheng also teaches wherein the distance sensor is a time-of-flight sensor, particularly an optical time-of- flight sensor, more particularly a laser distance sensor, and wherein the distance sensor is preferably arranged for measuring the distance (D) with respect to a portion of the skin surface within a field of view (FOV) of the optical flow sensor ([0004], [0063], [0070]-[0074], [0080] [0082]); a mounting structure for fixedly mounting the optical flow sensor and the distance sensor to the ultrasound probe ([0070]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Sprung to utilize an optical distance sensor such as a LIDAR type sensor as taught by Zheng, in order to more accurately determine the position and orientation of the ultrasound probe in relation to the patient’s body surface, thereby producing more accurate three-dimensional ultrasound images. It would have been obvious to include an optical distance sensor as a substitute for, or in addition to, the additional sensors described by Sprung ([0090]).
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sprung et al. (US 2020/0196984; hereinafter Sprung) in view of Zheng (US 2024/0201371) as applied to claim 1 above, and further in view of Guterman et al. (US 2013/0278776; hereinafter Guterman).
Sprung fails to explicitly state that the optical flow data includes position differential data (Ax, Ay) and summation data (Ex, Ey) relative to the skin surface.
Guterman discloses methods for motion detection in diagnostic ultrasound imaging. Guterman teaches wherein the optical flow data includes position differential data (Ax, Ay) and summation data (Ex, Ey) relative to the skin surface ([0118]-[0119]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the combined invention of Sprung and Zheng to utilize optical flow data including position differential and summation data relative to the skin surface as taught by Guterman, in order to more accurately track the motion of the anatomical object in the ultrasound images as described by Guterman ([0118]-[0119]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Aase (US 2023/0218265) discloses systems and methods for displaying a position of an ultrasound probe including motion detection (abstract).
Atlas (US 2019/0254584) discloses systems and methods for pulsatility measurements including using time of flight or LIDAR type sensors ([0033]).
Boctor (US 2013/0218024) discloses image guidance techniques including fusing ultrasound and video, including optical flow techniques ([0218]).
Imai (US 2021/0137492) discloses ultrasound diagnostic apparatus including tracking movement of an ultrasound probe using optical flow methods ({0091]).
Kotoku (US 2021/0065370) discloses medical imaging processing techniques including using optical flow to track subject movement ([0110]).
Matsumoto (US 2019/0200963) discloses ultrasound diagnostic apparatus including using optical flow image analysis techniques ([0064]).
Mihailescu (US 2016/0242744) discloses methods and systems for tracking and guiding instruments, including time of flight cameras and LIDAR systems ([0129]).
Wang (US 2007/0232908) discloses systems and methods to improve clarity in ultrasound images including using optical flow techniques ([0025]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN CWERN whose telephone number is (571)270-1560. The examiner can normally be reached Monday - Friday, 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached at (571) 272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN CWERN/ Primary Examiner, Art Unit 3797
