DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
a pre-treatment relative relationship calculation unit of claims 1 and 8.
current relative relationship calculation unit of claims 1 and 8.
probe control support unit of claims 1 and 8.
The original specification filed 09/15/2025 discloses a processor in [0046] for the means-plus-function of a pre-treatment relative relationship calculation unit and current relative relationship calculation unit, and a processor in [0021] for the means-plus-function of probe control support unit.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3-4 and 8 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shahedi, et al., US 20240260945 A1.
Regarding claim 1, Shahedi teaches an operation support system for an ultrasound probe (see abstract), comprising:
a pre-treatment relative relationship calculation unit that (processor 320 of [0092]) calculates, based on a pre-treatment camera image obtained by imaging a probe detection mark attached to an ultrasound probe and a target detection mark attached to a treatment target with a camera before a treatment on a subject ( [0018] a 4D ultrasound imaging system comprises an ultrasound scanner; an ultrasound transducer (e.g., a linear or phased-array ultrasound transducer); a position tracker; two sets of tracking markers (one for the transducer and one for patient's body)), a pre-treatment relative relationship that is a relative position and orientation relationship between the ultrasound probe and the treatment target ([0030] Another object of the invention is to compute an estimate of the optimal location and angle of the ultrasound transducer for imaging and to mitigate the airways' effects on the ultrasound images based on anatomical information obtained from the pre-operative image) in a case in which ultrasound is transmitted to and received from the subject for forming an ultrasound image before the treatment on the subject ([0048] describes image acquisition using the probe which conventionally includes transmitting ultrasound into the body and receiving ultrasound from the body);
a current relative relationship calculation unit (processor 320 of [0092]) that calculates, based on a current camera image obtained by imaging the probe detection mark and the target detection mark with the camera at a current time after a start of the treatment on the subject( [0018]a 4D ultrasound imaging system comprises an ultrasound scanner; an ultrasound transducer (e.g., a linear or phased-array ultrasound transducer); a position tracker; two sets of tracking markers (one for the transducer and one for patient's body)), a current relative relationship that is a relative position and orientation relationship between the ultrasound probe and the treatment target at the current time ([0030] Another object of the invention is to compute an estimate of the optimal location and angle of the ultrasound transducer for imaging and to mitigate the airways' effects on the ultrasound images based on anatomical information obtained from the pre-operative image); and
a probe control support unit (processor 320 of [0092]) that executes control of bringing the current relative relationship closer to the pre-treatment relative relationship ([0048]-[0049] describe moving the probe to scan the subject and [0030] discloses doing so to an optimal location of the probe).
Regarding claim 3, Shahedi further teaches wherein the probe control support unit notifies an operator who operates the ultrasound probe of a position and an orientation of the ultrasound probe for achieving the current relative relationship ([0015] discloses that the system is operable to display the suggested location and angle, and optionally alert the physician if the probe is off-track or off-angle. Examples of types of alerts include visual and audible).
Regarding claim 4, Shahedi further teaches wherein the probe control support unit displays, on a display unit [0086], a camera image that is obtained by imaging the ultrasound probe and the treatment target with the camera and that shows the position and the orientation of the ultrasound probe for achieving the current relative relationship ([0014] states the processor is further programmed and operable to compute a suggested location, and optionally a suggested angle, for the ultrasound probe for generating the 2D image slices. In embodiments of the invention, the processor computes the suggested location and angle based on location of the targets (e.g., pulmonary nodules) and airways in the lung, lung motion, and position of the ribs or other bony structures and [0015] discloses that the system is operable to display the suggested location and angle, and optionally alert the physician if the probe is off-track or off-angle).
Regarding claim 8, Shahedi teaches a non-transitory computer-readable storage medium storing an operation support program for an ultrasound probe ([0093] discloses a storage or memory device 330 which can hold or store information including imaging, device, marker, and procedural data as well as one or more of the software modules 340, described herein. The memory device may be a non-transitory solid state storage device or hard drive), causing a computer to function as:
a pre-treatment relative relationship calculation unit that (processor 320 of [0092]) calculates, based on a pre-treatment camera image obtained by imaging a probe detection mark attached to an ultrasound probe and a target detection mark attached to a treatment target with a camera before a treatment on a subject ( [0018] a 4D ultrasound imaging system comprises an ultrasound scanner; an ultrasound transducer (e.g., a linear or phased-array ultrasound transducer); a position tracker; two sets of tracking markers (one for the transducer and one for patient's body)), a pre-treatment relative relationship that is a relative position and orientation relationship between the ultrasound probe and the treatment target ([0030] Another object of the invention is to compute an estimate of the optimal location and angle of the ultrasound transducer for imaging and to mitigate the airways' effects on the ultrasound images based on anatomical information obtained from the pre-operative image) in a case in which ultrasound is transmitted to and received from the subject for forming an ultrasound image before the treatment on the subject ([0048] describes image acquisition using the probe which conventionally includes transmitting ultrasound into the body and receiving ultrasound from the body);
a current relative relationship calculation unit (processor 320 of [0092]) that calculates, based on a current camera image obtained by imaging the probe detection mark and the target detection mark with the camera at a current time after a start of the treatment on the subject ( [0018]a 4D ultrasound imaging system comprises an ultrasound scanner; an ultrasound transducer (e.g., a linear or phased-array ultrasound transducer); a position tracker; two sets of tracking markers (one for the transducer and one for patient's body)), a current relative relationship that is a relative position and orientation relationship between the ultrasound probe and the treatment target at the current time ([0030] Another object of the invention is to compute an estimate of the optimal location and angle of the ultrasound transducer for imaging and to mitigate the airways' effects on the ultrasound images based on anatomical information obtained from the pre-operative image); and
a probe control support unit (processor 320 of [0092]) that executes control of bringing the current relative relationship closer to the pre-treatment relative relationship ([0048]-[0049] describe moving the probe to scan the subject and [0030] discloses doing so to an optimal location of the probe).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Shahedi.
Regarding claim 2, the embodiment of Shahedi relied upon above teaches all the limitations of claim 1 above.
The embodiment of Shahedi relied upon above fails to teach wherein the probe control support unit brings the current relative relationship closer to the pre-treatment relative relationship by changing a position or an orientation of a robot arm that holds the ultrasound probe.
However, in a separate embodiment, Shahedi further teaches wherein the probe control support unit brings the current relative relationship closer to the pre-treatment relative relationship by changing a position or an orientation of a robot arm that holds the ultrasound probe ([0111] discloses that robotic arms or automated mechanical assemblies control the ultrasound probe motion. For example, in an embodiment, a fixture adapted to hold the transducer is controlled by a linear motor and rail to move the ultrasound transducer across (or otherwise, e.g. tilt) the patient's chest. The linear motor & rail apparatus is controlled by the workstation to move according to a predetermined motion profile, e.g., a constant speed. The position information of the motor is used for localization and probe tracking as described above in addition to (or in lieu of) an optical tracker. The robotic motion can have some advantage where precise and repeatable motion is desired versus manual motion. Optionally, the mechanical and robotic assemblies are adapted to move the ultrasonic probe in several directions and/or multiple degrees of freedom including XYZ, as well as rotation, tilt, etc).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to the embodiment of Shahedi relied upon above in the rejection of claim 1, wherein the probe control support unit brings the current relative relationship closer to the pre-treatment relative relationship by changing a position or an orientation of a robot arm that holds the ultrasound probe, as taught by the embodiment in [0111], for precise movement of the transducer to the desire region of interest [0111], with a reasonable expectation of success, as Shahedi also strives to provide accurate estimates of locations of the region of interest [0003].
Claims 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Shahedi in view of Cameron, et al., US 20200297430 A1.
Regarding claim 5, Shahedi teaches all the limitations of claim 1 above.
Shahedi does not teach wherein the pre-treatment relative relationship calculation unit converts, in accordance with an arm-probe position and orientation relationship that is a fixed position and orientation relationship between a robot arm and the ultrasound probe held by the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in a camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information, in a robot coordinate system, recognized by a robot control device that controls the robot arm, converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system, and calculates the current relative relationship in the robot coordinate system.
However, within the same field of endeavor, Cameron teaches an improved system and computer product for robotic brain surgery in which brain deformation during surgery caused by tools or pressure changes is tracked, allowing for improved accuracy in targeting structures for robotic surgical procedures (see abstract), wherein the pre-treatment relative relationship calculation unit converts, in accordance with an arm-probe position and orientation relationship that is a fixed position and orientation relationship between a robot arm and the ultrasound probe held by the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in a camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information, in a robot coordinate system, recognized by a robot control device that controls the robot arm, converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system ([0053] FIG. 4 is a diagram illustrating a data flow 400 for a multiple coordinate transformation system, to enable determining a position and orientation of an anatomical feature of a patient with respect to a robot arm of a surgical robot, according to some embodiments. In this example, data from a plurality of exam image spaces 402, based on a plurality of exam images, may be transformed and combined into a common exam image space 404. The data from the common exam image space 404 and data from a verification image space 406, based on a verification image, may be transformed and combined into a registration image space 408. Data from the registration image space 408 may be transformed into patient fiducial coordinates 410, which is transformed into coordinates for a DRB 412. A tracking camera 414 may detect movement of the DRB 412 (represented by DRB 412′) and may also detect a location of a probe tracker 416 to track coordinates of the DRB 412 over time. A robotic arm tracker 418 determines coordinates for the robot arm based on transformation data from a Robotics Planning System (RPS) space 420 or similar modeling system, and/or transformation data from the tracking camera 414. [0054] It should be understood that these and other features may be used and combined in different ways to achieve registration of image space, i.e., coordinates from image volume, into tracking space, i.e., coordinates for use by the surgical robot in real-time. As will be discussed in detail below, these features may include fiducial-based registration such as stereotactic frames with CT localizer, preoperative CT or MRI registered using intraoperative fluoroscopy, calibrated scanner registration where any acquired scan's coordinates are pre-calibrated relative to the tracking space, and/or surface registration using a tracked probe, for example. The registration is performed for both the preoperative ultrasound data [0008]-[0009] and live ultrasound data[0007]-[0009]), and
calculates the current relative relationship in the robot coordinate system ([0036]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to configure Shahedi wherein the pre-treatment relative relationship calculation unit converts, in accordance with an arm-probe position and orientation relationship that is a fixed position and orientation relationship between a robot arm and the ultrasound probe held by the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in a camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information, in a robot coordinate system, recognized by a robot control device that controls the robot arm, converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the arm-probe position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system, and calculates the current relative relationship in the robot coordinate system, as taught by Cameron, to provide an accurate tracking of the region of interest ([0005]), with a reasonable expectation of success, as Shahedi also strives to provide accurate estimates of locations of the region of interest ([0003]).
Regarding claim 6, Shahedi teaches all the limitations of claim 1 above.
Shahedi does not teach wherein the pre-treatment relative relationship calculation unit converts, in accordance with an arm-camera position and orientation relationship that is a fixed position and orientation relationship between a robot arm and the camera held by the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in a camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information, in a robot coordinate system, recognized by a robot control device that controls the robot arm, converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system, and calculates the current relative relationship in the robot coordinate system.
However, Cameron further teaches wherein the pre-treatment relative relationship calculation unit converts, in accordance with an arm-camera position and orientation relationship that is a fixed position and orientation relationship between a robot arm and the camera held by the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in a camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information, in a robot coordinate system, recognized by a robot control device that controls the robot arm, converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system, and calculates the current relative relationship in the robot coordinate system ([0053] FIG. 4 is a diagram illustrating a data flow 400 for a multiple coordinate transformation system, to enable determining a position and orientation of an anatomical feature of a patient with respect to a robot arm of a surgical robot, according to some embodiments. In this example, data from a plurality of exam image spaces 402, based on a plurality of exam images, may be transformed and combined into a common exam image space 404. The data from the common exam image space 404 and data from a verification image space 406, based on a verification image, may be transformed and combined into a registration image space 408. Data from the registration image space 408 may be transformed into patient fiducial coordinates 410, which is transformed into coordinates for a DRB 412. A tracking camera 414 may detect movement of the DRB 412 (represented by DRB 412′) and may also detect a location of a probe tracker 416 to track coordinates of the DRB 412 over time. A robotic arm tracker 418 determines coordinates for the robot arm based on transformation data from a Robotics Planning System (RPS) space 420 or similar modeling system, and/or transformation data from the tracking camera 414. [0054] It should be understood that these and other features may be used and combined in different ways to achieve registration of image space, i.e., coordinates from image volume, into tracking space, i.e., coordinates for use by the surgical robot in real-time. As will be discussed in detail below, these features may include fiducial-based registration such as stereotactic frames with CT localizer, preoperative CT or MRI registered using intraoperative fluoroscopy, calibrated scanner registration where any acquired scan's coordinates are pre-calibrated relative to the tracking space, and/or surface registration using a tracked probe, for example. The registration is performed for both the preoperative ultrasound data [0008]-[0009] and live ultrasound data[0007]-[0009]), and
calculates the current relative relationship in the robot coordinate system ([0036]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to configure Shahedi wherein the pre-treatment relative relationship calculation unit converts, in accordance with an arm-camera position and orientation relationship that is a fixed position and orientation relationship between a robot arm and the camera held by the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in a camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information, in a robot coordinate system, recognized by a robot control device that controls the robot arm, converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the arm-camera position and orientation relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system, and calculates the current relative relationship in the robot coordinate system, as taught by Cameron, to provide an accurate tracking of the region of interest ([0005]), with a reasonable expectation of success, as Shahedi also strives to provide accurate estimates of locations of the region of interest ([0003]).
Regarding claim 7, Shahedi teaches all the limitations of claim 1 above.
Shahedi fails to teach wherein the pre-treatment camera image and the current camera image are obtained by imaging the probe detection mark, the target detection mark, and an arm detection mark attached to a robot arm with the camera, the pre-treatment relative relationship calculation unit converts, in accordance with an inter-coordinate system relationship indicating a relationship between camera coordinates and robot coordinates and obtained based on a position and an orientation of the robot arm, in a camera coordinate system, specified based on an image of the arm detection mark included in the pre-treatment camera image and a position and an orientation of the robot arm, in a robot coordinate system, recognized by a robot control device that controls the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system, and calculates the current relative relationship in the robot coordinate system.
However, Cameron further teaches wherein the pre-treatment camera image and the current camera image are obtained by imaging the probe detection mark, the target detection mark, and an arm detection mark attached to a robot arm with the camera ([0053]),
the pre-treatment relative relationship calculation unit converts, in accordance with an inter-coordinate system relationship indicating a relationship between camera coordinates and robot coordinates and obtained based on a position and an orientation of the robot arm, in a camera coordinate system, specified based on an image of the arm detection mark included in the pre-treatment camera image and a position and an orientation of the robot arm, in a robot coordinate system, recognized by a robot control device that controls the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system ([0053] FIG. 4 is a diagram illustrating a data flow 400 for a multiple coordinate transformation system, to enable determining a position and orientation of an anatomical feature of a patient with respect to a robot arm of a surgical robot, according to some embodiments. In this example, data from a plurality of exam image spaces 402, based on a plurality of exam images, may be transformed and combined into a common exam image space 404. The data from the common exam image space 404 and data from a verification image space 406, based on a verification image, may be transformed and combined into a registration image space 408. Data from the registration image space 408 may be transformed into patient fiducial coordinates 410, which is transformed into coordinates for a DRB 412. A tracking camera 414 may detect movement of the DRB 412 (represented by DRB 412′) and may also detect a location of a probe tracker 416 to track coordinates of the DRB 412 over time. A robotic arm tracker 418 determines coordinates for the robot arm based on transformation data from a Robotics Planning System (RPS) space 420 or similar modeling system, and/or transformation data from the tracking camera 414. [0054] It should be understood that these and other features may be used and combined in different ways to achieve registration of image space, i.e., coordinates from image volume, into tracking space, i.e., coordinates for use by the surgical robot in real-time. As will be discussed in detail below, these features may include fiducial-based registration such as stereotactic frames with CT localizer, preoperative CT or MRI registered using intraoperative fluoroscopy, calibrated scanner registration where any acquired scan's coordinates are pre-calibrated relative to the tracking space, and/or surface registration using a tracked probe, for example. The registration is performed for both the preoperative ultrasound data [0008]-[0009] and live ultrasound data[0007]-[0009]), and
calculates the current relative relationship in the robot coordinate system ([0036]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to configure Shahedi wherein the pre-treatment camera image and the current camera image are obtained by imaging the probe detection mark, the target detection mark, and an arm detection mark attached to a robot arm with the camera, the pre-treatment relative relationship calculation unit converts, in accordance with an inter-coordinate system relationship indicating a relationship between camera coordinates and robot coordinates and obtained based on a position and an orientation of the robot arm, in a camera coordinate system, specified based on an image of the arm detection mark included in the pre-treatment camera image and a position and an orientation of the robot arm, in a robot coordinate system, recognized by a robot control device that controls the robot arm, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the pre-treatment camera image into position and orientation information in the robot coordinate system, and calculates the pre-treatment relative relationship in the robot coordinate system, and the current relative relationship calculation unit converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the ultrasound probe, in the camera coordinate system, specified based on an image of the probe detection mark included in the current camera image into position and orientation information in the robot coordinate system, converts, in accordance with the inter-coordinate system relationship, position and orientation information indicating a position and an orientation of the treatment target, in the camera coordinate system, specified based on an image of the target detection mark included in the current camera image into position and orientation information in the robot coordinate system, and calculates the current relative relationship in the robot coordinate system, as taught by Cameron, to provide an accurate tracking of the region of interest ([0005]), with a reasonable expectation of success, as Shahedi also strives to provide accurate estimates of locations of the region of interest ([0003]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Farouk A Bruce whose telephone number is (408)918-7603. The examiner can normally be reached Mon-Fri 8-5pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached at (571) 272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FAROUK A BRUCE/ Examiner, Art Unit 3797
Prosecution Insights