DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 07/23/2026 has been entered. Claims 1-21 remain pending in the application. Applicant’s amendments to the drawings and claims have overcome each and every objection and 112(b) rejections previously set forth in the Non-Final Office Action mailed 04/23/2026.
Response to Arguments
Applicant’s arguments with respect to the pending claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
EXAMINER’S AMENDMENT
An examiner’s amendment to the record appears below. Should the changes and/or additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the payment of the issue fee.
The application has been amended as follows:
IN THE CLAIMS:
In claim 3 last line, delete “a trackable feature” and replace with –the trackable feature–.
In claim 16 line 12, delete “ids” and replace with –is–.
In claim 18 last line, delete “a trackable feature” and replace with –the trackable feature–.
Authorization for this examiner’s amendment was given in an interview with Yee Jasmine Lau on 08/10/2026.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 16, and 20 similarly recite “detect a movement, or receive an information representative of the detected movement, of the imaging interface device derived from content of the ultrasound image by defining one or more planes in the ultrasound image relative to the imaging interface device”. However, the specification filed 05/08/2024 provides no support that describes that movement of the imaging interface device, i.e. ultrasound probe, is derived from content of the ultrasound image by defining one or more planes in the ultrasound image relative to the imaging interface device. At most, the movement of the ultrasound probe is derived from content of the ultrasound image, but not by defining one or more planes in the ultrasound image. The relevant sections from the specifications that describe how movement of the ultrasound probe is detected is Page 6 line 34 – Page 7 line 2, Page 9 lines 28-36, Page 12 lines 31-37, Page 18 lines 30-36, and page 19 lines 30-32. These sections describe that movement of the imaging interface device (ultrasound probe) may be detected by image processing (thereby derived from content of an ultrasound image) and/or supplemented by data provided by inertial measurement units (IMUs). However, these sections fail to disclose that the movement is detected by defining one or more planes in the ultrasound image. In fact, the specification on page 19 lines 8-35 describes that the planes, e.g. image plane B and reference plane R, are defined after detection of movement. While the examiner acknowledges applicant has amended the claims according to statements made by the examiner in the previous Office Action dated 04/23/2026 regarding allowable subject matter, upon further consideration, the statements were found to be incorrect in describing applicant’s invention.
Claims 2-15, 17-19, and 21 are rejected by virtue of dependency on the rejected claims above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Berke (US20120265071) in view of Lorraine (US20200375571), Mansi (US20210145412), Meral (US20200337673), and Neubach (US20110112549). Berke is cited in the IDS filed 05/08/2024.
Regarding claim 1, Berke teaches a controller (S) for controlling a robotic device (K) configured for insertion into an anatomical structure of a subject (P) (Fig. 4, [0032], [0052-0053], “whereby the control device S is enabled to move the robot arm M in such a way that the cannula K automatically follows the manual motion of the ultrasonic transducer 2, in particular follows it constantly”),
receiving ultrasound image data from an imaging interface device (2), the ultrasound image data corresponding to an ultrasound image (B) showing the anatomical structure and a portion of the robotic device (K) within in the anatomical structure ([0032], [0038], “Accordingly, the ultrasound images B depict online the tissue layer of the person P…The doctor can thereby be enabled to see on the monitor 4 relatively exactly in which tissue layer of the person P the cannula K is located at the moment”), wherein the imaging interface device (2) is maneuverable for providing the ultrasound image data ([0053], “The ultrasonic transducer 2, by contrast, is guided manually by the doctor”);
detecting a movement, or receive an information representative of the detected movement of the imaging interface device (2) ([0053], “On the basis of the signals coming from the navigation system N, the position and possibly the orientation of the ultrasonic transducer 2 in space is detected… follows the manual motion of the ultrasonic transducer 2”); and
outputting at least one command for controlling movement of the robotic device (K) from a first position to a second position in response to the detected movement of the imaging interface device (2) ([0053], “the position and possibly the orientation of the ultrasonic transducer 2 in space is detected…the cannula K automatically follows the manual motion of the ultrasonic transducer 2, in particular follows it constantly”).
However, Berke fails to teach wherein the detected movement is derived from content of the ultrasound image data.
In an analogous ultrasound imaging field of endeavor, Lorraine teaches such a feature. Lorraine teaches techniques for determining ultrasound probe motion (Title, Abstract, [0032]). Lorraine teaches wherein movement of an ultrasound probe may be detected based on and thus derived from acquired image data ([0032-0033], [0044]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to derive motion of the ultrasound imaging device from content of the ultrasound images as taught by Lorraine ([0007], [0032-0033], [0044]). By having probe motion be derived from the ultrasound images themselves, separate probe tracking systems may not be required. Alternatively, additionally deriving motion from ultrasound images may further improve robustness of probe tracking.
However, modified combination noted above fails to explicitly teach the controller comprising: at least one processor; and a non-transitory memory for storing machine executable instructions that, when executed by the at least one processor, cause the at least one processor to perform the methods described above.
In an analogous control of a robotic device field of endeavor, Mansi teaches such a feature. Mansi teaches a robotically steered catheter (108) (Title, Fig. 1, [0022-0023]). Moreover, Mansi teaches a controller (118) configured to control automatic steering of the catheter and may be implemented using a computer (802) ([0024], [0026], [0039]). Mansi teaches the systems and methods described herein may be implemented using a processor and corresponding non-transitory machine-readable storage device or memory having instructions therein executable by the processor (Claim 15, [0098], [0101]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the controller include a processor and memory storing instructions for controlling the robotic device as taught by Mansi (Claim 15, [0024], [0026], [0039], [0098], [0101]). Including generic computer components for implementing the methods of an invention is well-understood, routine, and conventional. Moreover, having the methods be performed by a processor and corresponding memory predictably improves automation of the method or steps.
However, the modified combination noted above fails to teach wherein the movement of the imaging interface device derived from content of the ultrasound image data is detected by defining one or more planes in the ultrasound image relative to the imaging interface device.
In an analogous ultrasound imaging field of endeavor, Meral teaches such a feature. Meral teaches ultrasound probe tracking using integrated ultrasound image-based and inertial tracking data ([0001]). Meral teaches an imaging interface device comprising a hand held ultrasound imaging probe (10) (Fig. 1, [0036]). Meral teaches wherein pose estimation uses images to derive motion ([0041]). Meral teaches calculating the orientation of the ultrasound probe and the imaging plane normal to the probe to determine out-of-plane translation (detecting movement) ([0057]). Meral teaches the final imaging plane (FIP) is known and the final imaging plane (FIP) having a minimum RMS error defines the out-of-plane translation and may be used together with image based in-plane translations and in-plane rotation to define the final pose ([0058]). Meral therefore teaches wherein movement of an imaging interface device (ultrasound probe) derived from content of an ultrasound image is detected by defining one or more planes.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to use planes to help detect movement of the ultrasound probe as taught by Meral ([0041], [0057-0058]). Tracking of probe movement may be optimized or simplified by using the imaging planes known through inertial sensor data as recognized by Meral ([0056-0057]).
However, the modified combination noted above fails to teach wherein the at least one command is based on a calculation associated with the one or more planes relative to the imaging interface device or a trackable feature of the robotic device.
In an analogous ultrasound imaging field of endeavor, Neubach teaches such a feature. Neubach teaches a robotic system for steering a flexible needle under ultrasound imaging (Abstract). Neubach teaches ascertaining a position of a tip of a needle using an ultrasound imaging system ([0022]). Moreover, Neubach teaches a robot for maneuvering the tissue and a control system to minimize differences between the ascertained position of the tip of the needle and a desired position of the tip according to a predetermined trajectory ([0021], [0024]). Neubach teaches the control system may determine a deviation of a position of a tip determined by image processing of an ultrasound image and calculates a motion to be applied to the robot to reduce the deviation ([0038]). Neubach teaches wherein the robot (25) controls movement of the needle (24) while the insertion of the needle is monitored by an ultrasound imaging system including an ultrasound probe (22) (Fig. 2, [0081-0083]). Neubach teaches the deviation of the position of the tip from the desired path is used in the next insertion step, with the magnitude and angle of insertion being determined by the controller ([0088], [0096]). Neubach therefore teaches wherein at least one command for controlling movement of a robotic device (needle controlled by robot) from a first position to a second position is based on a calculation associated with a trackable feature of the robotic device (needle tip).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the command be based on a trackable feature such as a tip of the robotic device as taught by Neubach ([0038], [0088], [0096]). By having the command be based on a tracked tip of the robotic device, more accurate placement or insertion of the tip may predictably achieved, thereby lowering misplacement and improving safety, while also only having to rely on ultrasound images of the tip.
Regarding claim 2, Berke in view of Lorraine, Mansi, Meral, and Neubach teaches the invention as claimed above in claim 1.
However, Berke fails to explicitly teach wherein the movement of the imaging interface device comprises at least one of a translational movement or a rotational movement.
In an analogous ultrasound imaging field of endeavor, Lorraine teaches such a feature. Lorraine teaches techniques for determining ultrasound probe motion (Title, Abstract, [0032]). Lorraine teaches wherein movement of an ultrasound probe may be detected based on and thus derived from acquired image data ([0032-0033], [0044]). Lorraine further teaches wherein motions of the probe may include translational motion ([0072]) and rotational (twisting) motion ([0054-0055]). Lorraine teaches a sonographer may move an ultrasound probe (14) on a patient to acquire ultrasound data, and wherein the movement of the probe may comprise lateral (e.g. translational), tilting, or rotating motions ([0040]). Lorraine therefore teaches wherein the movement of an imaging interface device may comprise either a translational movement or a rotational movement.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the movements comprise either translational or rotational movements as taught by Lorraine ([0040], [0054-0055], [0072]). Performing translational and rotational movements of an ultrasound probe or imaging device may predictably allow for an operator to reach a desired or optimal image of an anatomic region of interest.
Regarding claim 20, Berke teaches controlling a robotic device (K) configured for insertion into an anatomical structure of a subject (P) (Fig. 4, [0032], [0052-0053] “whereby the control device S is enabled to move the robot arm M in such a way that the cannula K automatically follows the manual motion of the ultrasonic transducer 2, in particular follows it constantly”),
receiving ultrasound image data from an imaging interface device (2), the ultrasound image data corresponding to an ultrasound image (B) showing the anatomical structure and a portion of the robotic device (K) within in the anatomical structure ([0032], [0038], “Accordingly, the ultrasound images B depict online the tissue layer of the person P…The doctor can thereby be enabled to see on the monitor 4 relatively exactly in which tissue layer of the person P the cannula K is located at the moment”), wherein the imaging interface device (2) is maneuverable for providing the ultrasound image data ([0053], “The ultrasonic transducer 2, by contrast, is guided manually by the doctor”);
detecting a movement, or receive an information representative of the detected movement of the imaging interface device (2) ([0053], “On the basis of the signals coming from the navigation system N, the position and possibly the orientation of the ultrasonic transducer 2 in space is detected… follows the manual motion of the ultrasonic transducer 2”); and
outputting at least one command for controlling movement of the robotic device (K) from a first position to the second position in response to the detected movement of the imaging interface device (2) ([0053], “the position and possibly the orientation of the ultrasonic transducer 2 in space is detected…the cannula K automatically follows the manual motion of the ultrasonic transducer 2, in particular follows it constantly”).
However, Berke fails to teach wherein the detected movement is derived from content of the ultrasound image data.
In an analogous ultrasound imaging field of endeavor, Lorraine teaches such a feature. Lorraine teaches techniques for determining ultrasound probe motion (Title, Abstract, [0032]). Lorraine teaches wherein movement of an ultrasound probe may be detected based on and thus derived from acquired image data ([0032-0033], [0044]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to derive motion of the ultrasound imaging device from content of the ultrasound images as taught by Lorraine ([0007], [0032-0033], [0044]). By having probe motion be derived from the ultrasound images themselves, separate probe tracking systems may not be required. Alternatively, additionally deriving motion from ultrasound images may further improve robustness of probe tracking.
However, the modified combination noted above fails to teach a non-transitory computer readable medium storing instructions that, the instruction, when executed by at least one processor, cause the at least one processor to: perform the method/steps above.
In an analogous control of a robotic device field of endeavor, Mansi teaches such a feature. Mansi teaches a robotically steered catheter (108) (Title, Fig. 1, [0022-0023]). Moreover, Mansi teaches a controller (118) configured to control automatic steering of the catheter and may be implemented using a computer (802) ([0024], [0026], [0039]). Mansi teaches the systems and methods described herein may be implemented using a processor and corresponding non-transitory machine-readable storage device or memory having instructions therein executable by the processor (Claim 15, [0098], [0101]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the controller include a processor and memory storing instructions for controlling the robotic device as taught by Mansi (Claim 15, [0024], [0026], [0039], [0098], [0101]). Including generic computer components for implementing the methods of an invention is well-understood, routine, and conventional. Moreover, having the methods be performed by a processor and corresponding memory predictably improves automation of the method or steps.
However, the modified combination noted above fails to teach wherein the movement of the imaging interface device derived from content of the ultrasound image data is detected by defining one or more planes in the ultrasound image relative to the imaging interface device.
In an analogous ultrasound imaging field of endeavor, Meral teaches such a feature. Meral teaches ultrasound probe tracking using integrated ultrasound image-based and inertial tracking data ([0001]). Meral teaches an imaging interface device comprising a hand held ultrasound imaging probe (10) (Fig. 1, [0036]). Meral teaches wherein pose estimation uses images to derive motion ([0041]). Meral teaches calculating the orientation of the ultrasound probe and the imaging plane normal to the probe to determine out-of-plane translation (detecting movement) ([0057]). Meral teaches the final imaging plane (FIP) is known and the final imaging plane (FIP) having a minimum RMS error defines the out-of-plane translation and may be used together with image based in-plane translations and in-plane rotation to define the final pose ([0058]). Meral therefore teaches wherein movement of an imaging interface device (ultrasound probe) derived from content of an ultrasound image is detected by defining one or more planes.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to use planes to help detect movement of the ultrasound probe as taught by Meral ([0041], [0057-0058]). Tracking of probe movement may be optimized or simplified by using the imaging planes known through inertial sensor data as recognized by Meral ([0056-0057]).
However, the modified combination noted above fails to teach wherein the at least one command is based on a calculation associated with the one or more planes relative to the imaging interface device or a trackable feature of the robotic device.
In an analogous ultrasound imaging field of endeavor, Neubach teaches such a feature. Neubach teaches a robotic system for steering a flexible needle under ultrasound imaging (Abstract). Neubach teaches ascertaining a position of a tip of a needle using an ultrasound imaging system ([0022]). Moreover, Neubach teaches a robot for maneuvering the tissue and a control system to minimize differences between the ascertained position of the tip of the needle and a desired position of the tip according to a predetermined trajectory ([0021], [0024]). Neubach teaches the control system may determine a deviation of a position of a tip determined by image processing of an ultrasound image and calculates a motion to be applied to the robot to reduce the deviation ([0038]). Neubach teaches wherein the robot (25) controls movement of the needle (24) while the insertion of the needle is monitored by an ultrasound imaging system including an ultrasound probe (22) (Fig. 2, [0081-0083]). Neubach teaches the deviation of the position of the tip from the desired path is used in the next insertion step, with the magnitude and angle of insertion being determined by the controller ([0088], [0096]). Neubach therefore teaches wherein at least one command for controlling movement of a robotic device (needle controlled by robot) from a first position to a second position is based on a calculation associated with a trackable feature of the robotic device (needle tip).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the command be based on a trackable feature such as a tip of the robotic device as taught by Neubach ([0038], [0088], [0096]). By having the command be based on a tracked tip of the robotic device, more accurate placement or insertion of the tip may predictably achieved, thereby lowering misplacement and improving safety, while also only having to rely on ultrasound images of the tip.
Claims 12 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Berke (US20120265071) in view of Lorraine (US20200375571), Mansi (US20210145412), and Meral (US20200337673), and Neubach (US20110112549) as applied to claims 1 and 20 respectively above, and further in view of Savitsky (US20120238875).
Regarding claim 12, Berke in view of Lorraine, Mansi, Meral, and Neubach teaches the invention as claimed above in claim 1.
However, Berke fails to teach wherein the imaging interface device is selectively operable in an imaging mode and one or more control modes, wherein the instruction, when executed by the at least one processor, further cause the at least one processor to: receive the ultrasound imaging data from the imaging interface device in the imaging mode, and detect the movement or receive information representative of the movement of the imaging interface device in the one or more control modes.
In an analogous ultrasound imaging field of endeavor, Savitsky teaches such a feature. Savitsky teaches an ultrasound system (100) including an ultrasound probe (104) (Fig. 1, [0017], wherein the ultrasound system 100 and/or ultrasound probe is an imaging interface device). Savitsky teaches the ultrasound system (100) may operate in a standard mode (i.e. imaging mode) which allows a user to use the probe to scan and thus image patients (Claim 13, [0024]). Savitsky further teaches wherein the ultrasound system (100) may operate in a training mode (i.e. control mode) in which the ultrasound probe (104) acts as a motion sensing peripheral, thus sensing or detecting probe motion (Claim 13, [0024]). Savitsky teaches in the training mode (control mode), the ultrasound probe is employed as a motion sensing peripheral to navigate existing patient cases ([0007], [0024-0025]). Savitsky therefore teaches an imaging mode in which ultrasound images are received and a control mode in which movement of the imaging interface device (ultrasound probe) is detected.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the imaging interface device be selectively operable in an imaging mode and control mode as taught by Savitsky (Claim 13, [0007], [0024-0025]). By having the two modes, a user may navigate through existing patient cases which may help the user expand and refine their knowledge of ultrasound imaging as recognized by Savitsky ([0024-0025]).
Regarding claim 21, Berke in view of Lorraine, Mansi, Meral, and Neubach teaches the invention as claimed above in claim 20.
However, Berke fails to teach wherein the instructions, when executed by the at least one processor, further cause the at least one processor to: receive selection of an imaging mode or a control mode from the imaging interface device, wherein the ultrasound imaging data is received from the imaging interface device in the imaging mode, and the movement of the imaging interface device is detected in the control mode.
In an analogous ultrasound imaging field of endeavor, Savitsky teaches such a feature. Savitsky teaches an ultrasound system (100) including an ultrasound probe (104) (Fig. 1, [0017], wherein the ultrasound system 100 and/or ultrasound probe is an imaging interface device). Savitsky teaches the ultrasound system (100) may operate in a standard mode (i.e. imaging mode) which allows a user to use the probe to scan and thus image patients (Claim 13, [0024]). Savitsky further teaches wherein the ultrasound system (100) may operate in a training mode (i.e. control mode) in which the ultrasound probe (104) acts as a motion sensing peripheral, thus sensing or detecting probe motion (Claim 13, [0024]). Savitsky teaches in the training mode (control mode), the ultrasound probe is employed as a motion sensing peripheral to navigate existing patient cases ([0007], [0024-0025]). Savitsky therefore teaches an imaging mode in which ultrasound images are received and a control mode in which movement of the imaging interface device (ultrasound probe) is detected.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the imaging interface device be configured to select an imaging mode and control mode as taught by Savitsky (Claim 13, [0007], [0024-0025]). By having the two modes, a user may navigate through existing patient cases which may help the user expand and refine their knowledge of ultrasound imaging as recognized by Savitsky ([0024-0025]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Berke (US20120265071) in view of Lorraine (US20200375571), Mansi (US20210145412), Meral (US20200337673), Neubach (US20110112549), and Savitsky (US20120238875) as applied to claim 12 above, and further in view of Diolaiti (US20100274087).
Regarding claim 13, Berke in view of Lorraine, Mansi, Meral, Neubach, and Savitsky teaches the invention as claimed above in claim 12.
However, Berke fails to explicitly teach wherein the controller is further configured to select the imaging mode and the control mode.
In an analogous robotic system field of endeavor, Diolaiti teaches such a feature. Diolaiti teaches a robot-assisted surgical system (2100) including a surgeon’s console (2102) (Fig. 1, [0095]). Diolaiti teaches wherein the surgeon’s console (2102) is configured to control or manipulate surgical instruments and imaging systems, and thus comprises a controller ([0096]). Diolaiti teaches wherein the console (2102) includes a graphical user interface (GUI) (2291) configured to allow user selection of a plurality of control modes including an “imaging system” mode and a “tool following” mode ([0137-0138], [0152], [0157]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the controller include a graphical user interface (GUI) configured for selecting modes as taught by Diolaiti ([0137-0138], [0152], [0157]). Having the controller include a GUI for mode selection may predictably make mode selection more convenient.
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Berke (US20120265071) in view of Lorraine (US20200375571), Mansi (US20210145412), Meral (US20200337673), Neubach (US20110112549), and Savitsky (US20120238875) as applied to claim 12 above, and further in view of Toporek (US20200188041) and Schatzle (US5928169).
Regarding claim 14, Berke in view of Lorraine, Mansi, Meral, Neubach, and Savitsky teaches a system for controlling a robotic device (K) configured for insertion into an anatomical structure of a subject (P), comprising: the controller of claim 12 (See claim 12 above).
Berke further teaches the system comprising: a movable imaging interface device (2) ([0053]), the ultrasound imaging data corresponding to an ultrasound image (B) showing the anatomical structure and a portion of the robotic device (K) within the anatomical structure ([0032], [0038], “Accordingly, the ultrasound images B depict online the tissue layer of the person P…The doctor can thereby be enabled to see on the monitor 4 relatively exactly in which tissue layer of the person P the cannula K is located at the moment”).
However, Berke fails to explicitly teach wherein the movable imaging interface device is arranged to send ultrasound image data to the controller.
In an analogous robotic device field of endeavor, Toporek teaches such a feature. Toporek teaches a robotic system (10) including an acoustic probe (20) and an instrument guide (40) mounted on the probe (20) (Fig. 6, [0044], [0074]) Toporek teaches the acoustic probe controller (30a) generates ultrasound volumetric image data (34) and sends the ultrasound image data (34) to the robotic instrument guide controller (50a) (Figs. 1 & 6, [0073]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to send the ultrasound image data to the robotic controller as taught by Toporek (Figs. 1 & 6, [0073-0074]). Using the ultrasound image data, the controller may generate robot actuation commands as needed to actuate the transition of the end effector to a desired pose as recognized by Toporek ([0073]).
However, the modified combination noted above fails to teach wherein the imaging interface device comprises an actuator operable by the user while holding imagining interface device to interactively select the imaging mode and the control mode.
In an analogous ultrasound probe field of endeavor, Schatzle teaches such a feature. Schatzle teaches an ultrasound probe (1) including a button (6) on its handle (3) (Fig. 1, Column 5 lines 1-25). Schatzle teaches wherein a therapy mode may be activated by pressing the button (6) and wherein a locating mode (i.e. imaging mode) may be activated when the button (6) is not actuated (Fig. 1, Column 5 lines 15-25, Column 6 lines 48-56). Schatzle therefore teaches wherein an imaging interface device (1) comprises an actuator (6) operable by a user while holding the imaging interface device (1) to interactively select between modes.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the imaging interface device include a button on its handle for switching or selecting modes as taught by Schatzle (Fig. 1, Column 5 lines 15-25). Having the mode selection be configured as a button on the handle of the probe predictably allows for ease of mode selection during operation of the probe or ultrasound imaging.
Regarding claim 15, Berke in view of Lorraine, Mansi, Meral, Neubach, Savitsky, Toporek, and Schatzle teaches the invention as claimed above in claim 14.
However, Berke fails to teach wherein the imaging interface device comprises an ultrasound probe, and the actuator comprises at least one of a push button, a slider button or a touch sensitive button on a probe handle of the ultrasound probe.
In an analogous ultrasound probe field of endeavor, Schatzle teaches such a feature. Schatzle teaches an ultrasound probe (1) including a button (6) on its handle (3) (Fig. 1, Column 5 lines 1-25). Schatzle teaches wherein a therapy mode may be activated by pressing the button (6) and wherein a locating mode (i.e. imaging mode) may be activated when the button (6) is not actuated (Fig. 1, Column 5 lines 15-25, Column 6 lines 48-56). Schatzle therefore teaches wherein an imaging interface device (1) comprises an ultrasound probe, and the actuator (6) comprises a push button on a probe handle of the ultrasound probe.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the imaging interface device be an ultrasound probe and the actuator comprise a push button on its handle for switching or selecting modes as taught by Schatzle (Fig. 1, Column 5 lines 15-25). Having the mode selection be configured as a push button on the handle of the probe predictably allows for ease of mode selection during operation of the probe or ultrasound imaging. Moreover, ultrasound probes are conventional imaging devices for performing diagnostic ultrasound imaging.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Berke (US20120265071) in view of Lorraine (US20200375571), Meral (US20200337673), and Neubach (US20110112549).
Regarding claim 16, Berke teaches a method of controlling a robotic device (K) configured for insertion into an anatomical structure of a subject (P) (Fig. 4, [0032], [0052-0053], “whereby the control device S is enabled to move the robot arm M in such a way that the cannula K automatically follows the manual motion of the ultrasonic transducer 2, in particular follows it constantly”), the method comprising:
receiving ultrasound image data from an imaging interface device (2), the ultrasound image data corresponding to an ultrasound image (B) showing the anatomical structure and a portion of the robotic device (K) within in the anatomical structure ([0032], [0038], “Accordingly, the ultrasound images B depict online the tissue layer of the person P…The doctor can thereby be enabled to see on the monitor 4 relatively exactly in which tissue layer of the person P the cannula K is located at the moment”), wherein the imaging interface device (2) is maneuverable for providing the ultrasound image data ([0053], “The ultrasonic transducer 2, by contrast, is guided manually by the doctor”);
detecting a movement, or receiving an information representative of the detected movement of the imaging interface device ([0053], “On the basis of the signals coming from the navigation system N, the position and possibly the orientation of the ultrasonic transducer 2 in space is detected… follows the manual motion of the ultrasonic transducer 2”); and
outputting to a robot controller (S) at least one command for controlling movement of the robotic device (K) from a first position to a second position in response to the detected movement of the imaging interface device (2) ([0053], “the position and possibly the orientation of the ultrasonic transducer 2 in space is detected…the cannula K automatically follows the manual motion of the ultrasonic transducer 2, in particular follows it constantly”).
However, Berke fails to teach wherein the detected movement is derived from content of the ultrasound image data.
In an analogous ultrasound imaging field of endeavor, Lorraine teaches such a feature. Lorraine teaches techniques for determining ultrasound probe motion (Title, Abstract, [0032]). Lorraine teaches wherein movement of an ultrasound probe may be detected based on and thus derived from acquired image data ([0032-0033], [0044]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to derive motion of the ultrasound imaging device from content of the ultrasound images as taught by Lorraine ([0007], [0032-0033], [0044]). By having probe motion be derived from the ultrasound images themselves, separate probe tracking systems may not be required. Alternatively, additionally deriving motion from ultrasound images may further improve robustness of probe tracking.
However, the modified combination noted above fails to teach wherein the movement of the imaging interface device derived from content of the ultrasound image data is detected by defining one or more planes in the ultrasound image relative to the imaging interface device.
In an analogous ultrasound imaging field of endeavor, Meral teaches such a feature. Meral teaches ultrasound probe tracking using integrated ultrasound image-based and inertial tracking data ([0001]). Meral teaches an imaging interface device comprising a hand held ultrasound imaging probe (10) (Fig. 1, [0036]). Meral teaches wherein pose estimation uses images to derive motion ([0041]). Meral teaches calculating the orientation of the ultrasound probe and the imaging plane normal to the probe to determine out-of-plane translation (detecting movement) ([0057]). Meral teaches the final imaging plane (FIP) is known and the final imaging plane (FIP) having a minimum RMS error defines the out-of-plane translation and may be used together with image based in-plane translations and in-plane rotation to define the final pose ([0058]). Meral therefore teaches wherein movement of an imaging interface device (ultrasound probe) derived from content of an ultrasound image is detected by defining one or more planes.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to use planes to help detect movement of the ultrasound probe as taught by Meral ([0041], [0057-0058]). Tracking of probe movement may be optimized or simplified by using the imaging planes known through inertial sensor data as recognized by Meral ([0056-0057]).
However, the modified combination noted above fails to teach wherein the at least one command is based on a calculation associated with the one or more planes relative to the imaging interface device or a trackable feature of the robotic device.
In an analogous ultrasound imaging field of endeavor, Neubach teaches such a feature. Neubach teaches a robotic system for steering a flexible needle under ultrasound imaging (Abstract). Neubach teaches ascertaining a position of a tip of a needle using an ultrasound imaging system ([0022]). Moreover, Neubach teaches a robot for maneuvering the tissue and a control system to minimize differences between the ascertained position of the tip of the needle and a desired position of the tip according to a predetermined trajectory ([0021], [0024]). Neubach teaches the control system may determine a deviation of a position of a tip determined by image processing of an ultrasound image and calculates a motion to be applied to the robot to reduce the deviation ([0038]). Neubach teaches wherein the robot (25) controls movement of the needle (24) while the insertion of the needle is monitored by an ultrasound imaging system including an ultrasound probe (22) (Fig. 2, [0081-0083]). Neubach teaches the deviation of the position of the tip from the desired path is used in the next insertion step, with the magnitude and angle of insertion being determined by the controller ([0088], [0096]). Neubach therefore teaches wherein at least one command for controlling movement of a robotic device (needle controlled by robot) from a first position to a second position is based on a calculation associated with a trackable feature of the robotic device (needle tip).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the command be based on a trackable feature such as a tip of the robotic device as taught by Neubach ([0038], [0088], [0096]). By having the command be based on a tracked tip of the robotic device, more accurate placement or insertion of the tip may predictably achieved, thereby lowering misplacement and improving safety, while also only having to rely on ultrasound images of the tip.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Berke (US20120265071) in view of Lorraine (US20200375571), Meral (US20200337673), and Neubach (US20110112549) as applied to claim 16 above, and further in view of Savitsky (US20120238875).
Regarding claim 17, Berke in view of Lorraine, Meral, and Neubach teaches the invention as claimed above in claim 16.
However, Berke fails to teach the invention further comprising: receiving selection of an imaging mode or a control mode from the imaging interface device, wherein the ultrasound image data is received from the imaging interface device in the imaging mode, and the movement of the imaging interface device is detected in the control mode.
In an analogous ultrasound imaging field of endeavor, Savitsky teaches such a feature. Savitsky teaches an ultrasound system (100) including an ultrasound probe (104) (Fig. 1, [0017], wherein the ultrasound system 100 and/or ultrasound probe is an imaging interface device). Savitsky teaches the ultrasound system (100) may operate in a standard mode (i.e. imaging mode) which allows a user to use the probe to scan and thus image patients (Claim 13, [0024]). Savitsky further teaches wherein the ultrasound system (100) may operate in a training mode (i.e. control mode) in which the ultrasound probe (104) acts as a motion sensing peripheral, thus sensing or detecting probe motion (Claim 13, [0024]). Savitsky teaches in the training mode (control mode), the ultrasound probe is employed as a motion sensing peripheral to navigate existing patient cases ([0007], [0024-0025]). Savitsky therefore teaches an imaging mode in which ultrasound images are received and a control mode in which movement of the imaging interface device (ultrasound probe) is detected.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Berke to have the imaging interface device be configured to select an imaging mode and control mode as taught by Savitsky (Claim 13, [0007], [0024-0025]). By having the two modes, a user may navigate through existing patient cases which may help the user expand and refine their knowledge of ultrasound imaging as recognized by Savitsky ([0024-0025]).
Allowable Subject Matter
Claims 3-5, 6-11, and 18-19 are objected to as being dependent upon a rejected base claim but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and if the pending 112(a) issues are resolved. The following is a statement of reasons for the indication of allowable subject matter:
Within the context of claim 3, the prior arts cited fails to teach “define an image plane B in the ultrasound image relative to the imaging interface device, the image plane B including a longitudinal axis of the robotic device; define a reference plane R in the ultrasound image relative to the imaging interface device that intersects the image plane B at the first position of the robotic device, wherein the reference plane R moves with the imaging interface device; and output the at least one command for controlling the robotic device based on a distance between a trackable feature of the robotic device and the translated reference plane R”.
Within the context of claim 6, the prior arts cited fail to teach “define a reference plane R in the ultrasound image relative to the imaging interface device that is referenced to anatomical features; detect a direction, or receive an information representative of a direction and angle of the rotational movement, of the imaging interface device; and output the at least one command for controlling the robotic device in response to an amount the reference plane R rotates relative to the imaging interface device”.
Within the context of claim 18, the prior arts cited fail to teach “defining an image plane B in the ultrasound image relative to the imaging interface device, the image plane B including a longitudinal axis of the robotic device; defining a reference plane R in the ultrasound image relative to the imaging interface device that intersects the image plane B at the first position of the robotic device; translating the reference plane R in association with the translational movement of the imaging interface device; and outputting the at least one command for controlling the robotic device based on a distance between a trackable feature of the robotic device and the translated reference plane R”.
Within the context of claim 19, the prior arts cited fail to teach “defining a reference plane R in the ultrasound image, wherein the reference plane R intersects a rotational axis of the imaging interface device; detecting a direction and angle of the rotational movement of the imaging interface device; and outputting the at least one command for controlling the robotic device in response to an amount of rotation of the reference plane R relative to the imaging interface device”.
The most relevant prior arts cited are Berke (US20120265071), Lorraine (US20200375571), Meral (US20200337673), and Neubach (US20110112549). Berke teaches receiving ultrasound image data from an ultrasound probe (2) ([0032], [0038]), detecting movement of the ultrasound probe (2) (Fig. 4, [0053]), and outputting commands for controlling movement of a robotic device (K) in response to the detected movement of the ultrasound probe (2) ([0053]). Lorraine teaches wherein the detected movement of the ultrasound probe may be derived from content of ultrasound images ([0032-0033], [0044]). Meral teaches wherein movement of the probe may be detected by defining a plane in the ultrasound image ([0041], [0057-0058]). Neubach teaches wherein a command for controlling movement of a robotic device may be based on a trackable feature, e.g. a tip, of a robotic device ([0038], [0088], [0096]). However, the prior arts fail to teach defining reference planes R and/or B as recited by and in the context of claims 3, 6, 18, and 19 above and also fail to teach outputting a command based on “a distance between a trackable feature of the robotic and the translated reference plane R”, “an amount the reference plane R rotates relative to the imaging interface device”, a distance between a trackable feature of the robotic device and the translated reference plane R”, and “an amount of rotation of the reference plane R relative to the imaging interface device” as recited in claims 3, 6, 18, and 19 respectively.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TOMMY T LY whose telephone number is (571) 272-6404. The examiner can normally be reached M-F 12:00pm-8:00pm eastern time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at 571-272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TOMMY T LY/ Examiner, Art Unit 3797
/SERKAN AKAR/ Primary Examiner, Art Unit 3797
Prosecution Insights