DETAILED ACTION
This Office action is responsive to communications filed on 03/13/2026. Claims 1, 6, 9-10, 12, 19, & 21 have been amended. Claim 4, 14, & 18 are canceled. Claims 22-23 is newly added. Presently, Claims 1-3, 5-13, 15-17, & 19-23 remain pending and are hereinafter examined on the merits.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/13/2026 has been entered.
Response to Arguments
Previous claim objections are withdrawn in view of the amendments filed on 03/13/2026.
The Applicant’s arguments with respect to rejections under 35 USC § 101 have been fully, considered, but are not persuasive.
The Examiner maintains that the claims recite a mental process. The Applicant argues that the claimed stimulation, including multiple robotic arms each having seven or more degrees of freedom, is too complex to be performed in the human mind. However, the claims do not recite any specific computational technique, algorithm, or technological constraint that would preclude performance as a mental process under the broadest reasonable interpretation. The recited stimulating, mapping trajectories, identifying collisions, selecting steps remain expressed at a high level of generality. As claimed, these steps amount to evaluating possible configurations, comparing outcomes, and selected a preferred result. These are indeed operations that could conceptually be performed mentally. The mere inclusions of the multiple robotic arms with multiple degrees of freedom (DOF) does not change the fundamental character of the recited steps into something other than a mental process. The claims do not recite any particular simulation model, numerical method, or specifically, an implementation that would render the process non-mental.
The assertion that the complexity of the system and the alleged impracticality of performing such simulation mentally rely on limitations and assumptions not present in the claims. The claims do not impose bounds on the number of starting states, the resolution of the trajectories, or the computational techniques used and/or non-mental. Accordingly, the asserted complexity is not commensurate in scope with the claims.
With respect to Step 2A, Prong Two, the applicants contends that the claims integrate the alleged abstract idea into practical application by improving operation of a robotic medical system, particularly, by reducing collisions. However, the claims do not recite any technological mechanism, particularly reducing collisions, by placing the abstract idea into a practical application that solves a technological solution in a meaningful way. The claimed steps of obtaining imaging data and procedural information remain as data gathering, and the process and memory are recited at a high level of generality. The simulation and selection steps are part of the abstract idea itself and do not provide integration into a practical application.
The step of “perform[ing] the medical procedure with the plurality of robotic arms initially in the respective starting poses” is properly characterized as post-solution activity. This step merely applies the result of the abstract evaluation (i.e., the selected starting state without recite any change to how the robotic system operates or any specific technique that improves the functioning of the robotic arms. The claims do not recite how the robotic arms are controlled differently, how collisions are actively prevented during execution, or any modification to the robotic system itself. As such this step does not meaningfully limit the abstract idea.
The Applicant’s reliance on improvements described in the specification, such a minimizing collisions, is not persuasive because the improvements are not reflect in the claim language. In addition, to merely or generically assert a desired outcome or advantage does not define “a specific technological mechanism” that achieves that outcome. The claim recites identifying and selecting based on a number of collisions, but do not recite any specific manner of achieving the collision avoidance beyond the abstract evaluation itself. Accordingly, the asserted advantageous and alleged improvements are not supported by the claim language and cannot establish eligibility.
The Applicant argument that performing the medical procedure using the selected starting poses constitutes a technological improvement is not commensurate with the scope of the claim. The claims do define what the medical procedure is. If the medical procedure involves a plurality of sequential steps in which at least a subset of the sequential steps have corresponding required robotic arm positions, and wherein the mapping maps the trajectories as the plurality of robotic arms move from respective starting poses to the required robotic arm positions for at least the subset of the sequential steps, as recited at claim 22, then the medical procedure is also directed to the abstract idea. This is a further mental step that does not integrate the abstract idea into practical application. This alleged benefit is therefore an intended result rather than a claimed technical implementation.
For these reasons, the 35 USC § 101 rejection remains.
Applicant’s arguments with respect to claim(s) rejected under 35 USC § 102 have been considered but are moot because the new ground of rejection does not rely on Coste-Maniere et al (US 2007/0293734 A1) alone applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The new grounds of rejection now relies on Coste-Maniere et al (US 2007/0293734 A1), in view of Andre ("A Motion Planning and Velocity Collision Avoidance Framework for Bilateral Manipulation")(December 30, 2021) (Year: 2021), in view of Ho et al (US 20190105776 A1) as rejected under 35 USC § 103.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 5-13, 15-17, 19-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 of the subject matter eligibility test (see MPEP 2106.03).
Claim 1-3, 5-11, & 22-23 is directed to an “apparatus” which describes one of the four statutory categories of patentable subject matter, i.e., a machine.
Claim 12-13, 15-17, 19-20 are directed to a “method” which describes one of the four statutory categories of patentable subject matter, i.e., a process.
Claim 21 is drawn to a “system” which describes one of the four statutory categories, i.e., a machine.
Step 2A of the subject matter eligibility test (see MPEP 2106.04).
Prong One:
Claim 1 recite (“sets forth” or “describes”) the abstract idea of “a mental process” (MPEP 2106.04(a)(2).III.), substantially as follows:
“ for each respective starting state of a plurality of starting states; simulate the medical procedure from the respective starting state using the imaging data and the information on the medical procedure, the simulating comprising mapping trajectories for each robotic arm of the plurality of robotic arms during the medical procedure,
identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
record the respective number of the collisions resulting from the respective starting state;
select a first starting state of the plurality of starting states that has a least number of the collisions among the plurality of starting states based on the respective number of the collisions identified and recorded for each respective starting state of the plurality of starting states, wherein the first starting state includes respective starting poses corresponding to the plurality of robotic arms;
provide pose information for the first starting state to an operator of the robotic medical system, the pose information identifying the respective starting poses corresponding to the plurality of robotic arms; and ”
Claim 12 recite (“sets forth” or “describes”) the abstract idea of “a mental process” (MPEP 2106.04(a)(2).III.), substantially as follows:
“ for each respective starting state of a plurality of starting states: simulating the medical procedure from the respective starting state using the imaging data and the information on the medical procedure, the simulating comprising mapping trajectories for each robotic arm of the plurality of robotic arms during the medical procedure,
identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
record the respective number of the collisions resulting from the respective starting state;
selecting a first starting state of the plurality of starting states that has a least number of the collisions among the plurality of starting states based on the respective number of the collisions identified and recorded for each respective starting state of the plurality of starting states, wherein the first starting state includes respective starting poses corresponding to the plurality of robotic arms;
providing pose information for the first starting state to an operator of the robotic system, the pose information identifying the respective starting poses corresponding to the plurality of robotic arms; ”
Claim 21 recite (“sets forth” or “describes”) the abstract idea of “a mental process” (MPEP 2106.04(a)(2).III.), substantially as follows:
“ for each respective starting state of a plurality of starting states: simulate the medical procedure from the respective starting state using the 3-D scan data and the information on the medical procedure, the simulating comprising mapping trajectories for each robotic arm of the plurality of robotic arms during the medical procedure,
identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
record the respective number of the collisions resulting from the respective starting state;
select a first starting state of the plurality of starting states that has a least number of the collisions among the plurality of starting states based on the respective number of the collisions identified and recorded for each respective starting state of the plurality of starting states, wherein the first starting state includes respective starting poses corresponding to the plurality of robotic arms;
provide pose information for the first starting state to an operator of the robotic medical system, the pose information identifying the respective starting poses corresponding to the plurality of robotic arms; ”
For each claim (1, 12, 21), the above recited steps can be practically performed in the human mind, with the aid of a pen and paper to perform the steps. Specifically, a human mind could mentally visualize/envision different possible starting positions for the robotic arms, and using the imaging data and general procedural knowledge, mentally simulate how the arms would move through the procedure. In doing so, the person could mentally trace the paths of each arm, recognize where collisions would occur, and keep a mental tally of such collisions for each scenario. These abstract limitations as generically recited amount to this. The person could then compare these outcomes across different predicted starting states, determine which configuration results in the fewest collisions and select that configuration. Finally, the person could chose a starting pose, reflecting a mental evaluation. Thus the recited steps when viewed as a whole, describe operations, that under the broadest reasonably interpretation, amount to evaluations, comparisons, and communicating the results that can be practically performed in the human mind. There is nothing recited in the claim to suggest an undue level of complexity in simulation is done. Therefore, a person would be able to perform selecting and simulation and providing mentally and/or with aid of pen and paper.
Prong Two: Claims (1, 12, 21) do not include additional elements that integrate the mental process into a practical application.
This judicial exception is not integrated into a practical application. In particular, the claims recites [1] additional:
i) “a plurality of robotic arms, each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints; one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: obtain imaging data of a patient; obtain information on a medical procedure for the patient;”-(claim 1),
ii) “a plurality of robotic arms, each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints, the method comprising: obtaining imaging data of a patient; obtaining information on a medical procedure for patient;”-(claim 12),
ii) “a plurality of robotic arms, each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints; one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: obtain three-dimensional (3-D) scan data that includes a view of a patient and one or more objects surrounding the patient; obtain information on a medical procedure for the patient;”-(claim 21);
[2] further an addition step of
i) “perform the medical procedure with the plurality of robotic arms initially in the respective starting poses of the first starting state.”-(claim 1),
ii) “performing the medical procedure with the plurality of robotic arms initially in the respective starting poses of the first starting state.”-(claim 12),
iii) “perform the medical procedure with the plurality of robotic arms initially in the respective starting poses of the first starting state.”-(claim 21).
The steps in [1] represent merely data gathering or pre-solution activities that are necessary for use of the recited judicial exception and are recited at a high level of generality with conventionally used tools (see below Step IIB for further details). Data gathering and mere instructions to implement an abstract idea on a computer do not integrate a judicial exception into a practical application (MPEP 2106.05 (f and g)). Regarding the processor language written at such a high level of generality of structural limitations, the processor language amounts to a generic computer component with mere instructions to implement the abstract idea on a computer.
The step in [2] represents merely post-solution activity and is recited at a high level of generality. Specifically, this step does not provide a practical application. This step merely places the system into the selected starting poses and then proceeds to carry out the medical procedure itself, which is post-solution activity following the abstract idea of evaluating, simulating, and selecting configurations. The claim does not recite any technological improvement to how the robotic procedure is performed, nor does it integrate the abstract selection/simulation process into a specific technological solution. As such, this step is insufficient to improve a meaningful limit on the judicial exception and does not transform the claim into a practical application.
As a whole, the additional elements merely serve to gather and feed information to the abstract idea and to output a notification based on the abstract idea, while generically implementing it on conventionally used tools. There is no practical application because the abstract idea is not applied, relied on, or used in a meaningful way. No improvement to the technology is evident, and the estimated bio-information is not outputted in any way such that a practical benefit is realized. Therefore, the additional elements, alone or in combination, do not integrate the abstract idea into a practical application.
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Further, there is no evidence of record that would support the assertion that this step is an improvement to a computer or technological solution to a technological problem. Ultimately, the Applicant’s describe improvement in the process robotic techniques, but this is not an improvement in the function of a computer or other technology (See MPEP 2106.05(a)(ii); “the court determined that the claimed user interface simply provided a trader with more information to facilitate market trades, which improved the business process of market trading but did not improve computers or technology”; See MPEP 2106.04(d)(1); 2106.05(a); and 2106.05(f)). The claims are directed to the abstract idea. Also, there does not appear to be any particular structure or machine, treatment or prophylaxis, transformation, or any other meaningful application that would render the claim eligible at step 2A, prong 2.
Step 2B of the subject matter eligibility test (see MPEP 2106.05).
Claims (1, 12, 21) do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the claims recite additional steps of obtaining imaging data of the patient and obtaining information on a medical procedure for the patient. These steps represents mere data gathering, data outputting or pre-extra-solution activities that are necessary for use of the recited judicial exception and are recited at a high level of generality. Furthermore, as discussed above, limitations with respect to the processor languages/terms, respectively, amount to mere instructions to implement the abstract idea on a computer. As discussed with respect to Step 2A Prong Two, the additional elements in the claims amount to no more than insignificant extra solution activity and mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B and does not provide an inventive concept. The data gathering steps that were considered insignificant extra-solution activity in Step 2A Prong Two, have been re-evaluated in Step 2B and determined to be well-understood, routine, conventional activity in the field.
As an evidence, Patriciu et al (US 2020/0375675 A1) discloses:
¶0012, ‘the term “medical procedure” broadly encompasses all diagnostic, surgical and interventional procedures, as known in the art of the present disclosure or hereinafter conceived, for an imaging, a diagnosis and/or a treatment of a patient anatomy;’; ¶0041, ‘medical robotic system 40 of the present disclosure provides robotic guidance for one or more medical tools 20 utilized to conduct an imaging, a diagnosis and/or a treatment of a patient anatomy in accordance with a medical procedure as known in the art of the present disclosure.’
As an evidence, Leung (US 2013/0293578 A1) discloses:
¶0023, ‘As is known in the art, prior to performing the surgical procedure, the physician may obtain pre-operative planning images of the patient to aid in planning the surgical procedure. These pre-operative planning images are referred to herein as patient planning images and may be obtained using the same imaging device that obtains the real-time patient image. Further, the physician may annotate a patient planning image to describe aspects of the planned surgical procedure. For example, the physician may indicate a line on the patient planning image depicting where an incision or cut is to be made on the patient. This line can be generated in different ways, including by the physician using a computer aided graphics tool for indicating the line over the patient planning image. This line, excluding the patient planning image, is one form of what is termed herein as a pre-operative planning image.’
As an evidence, Bachmann et al (US 20220016781 A1) discloses:
¶0003, ‘Multi-axis robotic arms are basically known from prior art. Robotic arms with six or seven rotary drives are often used for industrial applications, as the high number of degrees of freedom enables very flexible positioning.’;
¶0052, ‘This is a robotic arm with seven robotic joints J1 to J7, each of which enables a rotation about an associated axis of rotation R1 to R7. This is therefore a robotic arm with seven rotational degrees of freedom. “Innermost” joint J1 is connected to a base B which serves as a superordinate mechanical mass. “Outermost” joint J7 can carry an end effector (not shown in detail) at location TCP. A drive device each is arranged within individual joints J1 to J7. These are rotary drives for rotating the individual joints, the basic structure of which and their mechanical mode of operation are known from prior art’
For these reasons, there is no inventive concept. The claim is not patent eligible. Even when viewed as a whole, nothing in the claim adds significantly more to the abstract idea.
Dependent Claims
The following dependent claims merely further define the abstract idea and, therefore, recite an abstract idea for similar reasons:
defining wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to identify an anatomic target for the medical procedure, wherein the simulating further uses the identified anatomic target / further comprising identifying an anatomic target for the medical procedure, wherein the simulating further uses the identified anatomic target. (claim 2, 13)
defining wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to identify the anatomical target using the imaging data of the patient. (claim 3)
defining wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to select a recommended port location based on the simulating / further comprising selecting a recommended port location based on the simulating. (claim 5, 15)
defining wherein the first starting state includes, for each robotic arm of the plurality of robotic arms, positioning information for each joint of the plurality of joints. (claim 6)
defining wherein the medical procedure requires a medical instrument, and wherein the pose information includes information regarding to which of the plurality of robotic arms the medical instrument should be mounted.. (claim 9)
defining wherein the medical procedure involves a plurality of sequential steps in which at least a subset of the sequential steps have corresponding required robotic arm positions, and wherein the mapping maps the trajectories as the plurality of robotic arms move from respective starting poses to the required robotic arm positions for at least the subset of the sequential steps. (claim 22)
The following dependent claims merely further describe the extra-solution activities and therefore, do not amount to significantly more than the judicial exception or integrate the abstract idea into a practical application for similar reasons:
describing wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to obtain imaging data of environs of the robotic medical system, wherein the simulating further uses the imaging data of the environs / further comprising obtaining imaging data of environs of the robotic system, wherein the simulating further uses the imaging data of the environs. (claim 7, 16). The phrase, “obtain imaging data for environs of the robotic medical system” is directed to data gathering steps and pre-solution activity are conventional and recited at high level of generality. As such, the abstract idea is not applied, relied on, or used in a meaningful way. No improved to the technology is evident, and the determined visualization of context is not outputted in any way such that the practical benefit is realized. Whereas the simulating further uses the imaging data of the enviorns recites an abstract idea for similar reasons as identified above.
describing obtain positional information for one or more objects in a vicinity of the robotic medical system, is directed to data gathering steps and pre-solution activity are conventional and recited at high level of generality. As such, the abstract idea is not applied, relied on, or used in a meaningful way. No improved to the technology is evident, and the determined visualization of context is not outputted in any way such that the practical benefit is realized. “wherein the simulating further uses the positional information.” recites an abstract idea for similar reasons as identified above. (claim 8, 17)
describing to automatically move at least one arm of the plurality of robotic arms into the respective starting pose prior to docking the plurality of robotic arms to a plurality of respective ports, (Claim 10, 19), is directed to data gathering steps and post-solution activity are conventional and recited at high level of generality. As such, the abstract idea is not applied, relied on, or used in a meaningful way. No improved to the technology is evident, and the determined visualization of context is not outputted in any way such that the practical benefit is realized.
describing wherein the imaging data for the patient includes images of the patient after the patient is prepared for the medical procedure. (Claim 11, 20)
describing wherein, for each robotic arm of the plurality of robotic arms, the plurality of joints includes a subset of joints that lock in place during the medical procedure, and wherein the respective starting poses include locking positions for the subset of joints. (claim 23)
Taken alone and in combination, the additional elements do not integrate the judicial exception into a practical application at least because the abstract idea is not applied, relied on, or used in a meaningful way. They also do not add anything significantly more than the abstract idea. Their collective functions merely provide computer/electronic implementation and processing, and no additional elements beyond those of the abstract idea. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements individually. There is no indication that the combination of elements improves the functioning of a computer, output device, improves technology other than the technical field of the claimed invention, etc. Therefore, the claims are rejected as being directed to non-statutory subject matter.
Claim Objections
The following claims are objected to because of the following informalities and should recite:
Claim 22: line 4, “the respective starting poses”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5-9, 12-13, 15-17, 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Coste-Maniere et al (US 2007/0293734 A1), in view of Andre ("A Motion Planning and Velocity Collision Avoidance Framework for Bilateral Manipulation")(December 30, 2021) (Year: 2021), in view of Ho et al (US 20190105776 A1).
Claim 1: Coste-Maniere discloses, A robotic medical system comprising: (¶Abstract)
a plurality of robotic arms, each robotic arm of the plurality of robotic arms having a plurality of degrees of freedom provided by a plurality of links and a plurality of joints; (¶0045, ‘Cart 300 here includes three robotic manipulator arm assemblies 302, each manipulator supporting an instrument 100.’; Claim 53, “a plurality of robotic arms individually manipulatable in a plurality of degrees of freedom movement”; ¶0064, ‘number of robot arms and number of degrees of freedom for each arm, potential collisions between the robot arms, potential collisions between an arm and the patient, other potential collisions (e.g. with anesthesia equipment or operating room table), and/or miscellaneous constraints (e.g. endoscope orientation for assistant surgeon).’; ¶0116, ‘active manipulator links and joints (servo-operated and passively responding joints which move during tissue treatment operation).’)
one or more processors; and (Claim 61: “a processor configured to determine whether any two or more of the plurality of robotic arms may collide during the surgical procedure using information of a target area in which the surgical procedure is to be performed within a patient, and information mechanically characterizing the plurality of robotic arms.’)
memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: (¶0046, ‘processor 400 coupled with master control station 200 and cart 300 and a tangible medium 410 embodying machine readable code, or software. The software typically includes instructions which enable various embodiments of the methods of the present invention. The tangible medium 410 may be coupled with the processor 400 for use. Generally, the software may be used with any suitable hardware, such as a personal computer work station with graphics capabilities, such as but not limited to a PENTIUM Ill: or equivalent processor with a GEFORCE2® graphics card. Other hardware which may be used with software of the present invention includes a display monitor, such as a 17″ monitor, a processor with 256 Mbytes of RAM and a 10 Gigabytes hard disk. Input devices will typically include a mouse and may also include a 3D mouse or a PHANTOM® arm.’; see also ¶0047-0048; ¶0113, ‘Recorded procedure history and computer data, including surgeon inputs, tool motions and robotic arm movements, may be used to refine models, optimization criteria, feasibility criteria and/or cost function terms.’)
obtain imaging data of a patient; (¶0052, ‘Data acquisition 112 generally involves acquiring any data regarding a volume which is to be operated upon, such as a portion of a patient's body, as well as, in some embodiments, data regarding a robot, surgical tools, and the like, to be used in performing the operation. Data may include, for example, CT scan data, with or without contrast, MRI data, coronary artery angiograms, conventional radiographs, digital representations of conventional radiographs, and/or the like. In a totally endoscopic coronary artery bypass graft (TECAB) operation, for example, CT scan data is typically used. This generally involves acquiring helical CT scans of a patient, with 3 mm spacing, from approximately the neck region to the hip region of the patient. Slice size is often decreased to 1 cm in the area of the heart, to acquire more image information, and often a dye is injected to better visualize the heart and aorta. Additionally, such CT data acquisition will often be synchronized with electrocardiogram (ECG) data acquisition. Coronary angiograms may also be acquired, to enable an accurate diagnosis of the state of heart vessels. Data from multiple types of imaging studies, such as CT scans and angiograms, may be used together in various embodiments to enhance planning of port placement.’; ¶0081, ‘Modeling the patient. Various exemplary embodiments include the use of patient-specific data to characterize the body portion being treated. In some embodiments and surgical procedures, port optimization planning on a representative sample of patients will have sufficient generality to be useful as a generic port placement plan. Modeling a patient may involve several sub-steps, such as: 1. acquiring patient-specific data for at least a portion of patent's body via such modalities as CT, MRI, and or arterial angiograms; 2. segmenting acquired data to distinguish organ, bone, vessel and other tissue structures (may be automated, manual or a combination of these); 3. reconstructing segmented, acquired data to construct a 3D model for at least a portion of patent's body. Optionally, such a model may include additional overlaid patient data, a body cavity insufflation space model, and/or the like.’)
obtain information on a medical procedure for the patient; (¶Abstract, ‘Methods and apparatus for enhancing surgical planning’; The determination of the port locations and robot positioning considers “factors such as patient anatomy, surgeon performance, the surgical procedure to be performed”, ¶0011. “Surgeon preferences” incorporate including specifying “target points within the patient” and “preferred ‘attack directions’”, ¶0058. Step 6, “Step 6: Defining operative motion prediction algorithms. This step involves defining a predictive model of expected range of surgeon-commanded operational tool movements during surgical task (generic task, specific procedural and/or tool-types).”, ¶0080. Simulations, as well as validation, involves, ‘applying the predictive model of expected surgeon-commanded operational instrument movements for a surgical procedure during manipulations at a surgical target site within the body.’, ¶0097.)
for each respective starting state of a plurality of starting states:
simulate the medical procedure from the respective starting state using the image data and the information on the medical procedure, the simulating comprising mapping trajectories for each arm of the plurality of robotic arms during the medical procedure;
-Coste-Maniere simulating the medical procedure from various starting states to identify resulting collisions, particularly when determining the advantageous position for the robotic system. The core of Coste-Maniere is to configure a planning process for robotic surgery involving the determining of an advantageous position for one or more arms of the plurality of robotic arms relative to the patient based on entry port placement, ¶Abstract, ¶0064. The approach of Coste-Maniere uses a combined probabilistic and gradient descent approach where configurations of the joints are randomly drawn in a robot articular space. This set of randomly drawn configurations represent a plurality of starting states, ¶0065, Claims 59-60. Once the configuration is selected (i.e., the cost function), those active joints are moved over all the targets to very that there is no collision, ¶0065. Hence, this verification steps acts as a simulation of the expected movement for that specific configuration, ¶0065, ¶0097. Note; the primary concern of Coste-Maniere addressed for surgical planning is to avoid “collision of two or more robotic arms during a robotic procedure”, ¶0011.
-Coste-Maniere teaches that the simulation and validation steps use patient-specific data, such as the imaging data, which is processed to create the model of the surgical site, ¶0015, ¶0049-0050, ¶0081. Note this model is a “3D model for at least a portion of the patient’s body”, ¶0081. These steps use the predictive model of expected surgeon operational instrument movements for the procedure, and the collision prediction algorithm is applied during this simulation to determine if collisions will occur, ¶0097.
-Coste-Maniere teaches that in the step of validation is specifically performed to very that the identified location are feasible, ¶0066. Specifically, during validation, the movement of the robot as expected during the operation is carried out to look for the possible collisions between the robotic arms, ¶0066. If a collision is detected during validation, the system may return to the planning stage to select other entry locations and/or robotic arm positions, ¶0066, ¶0098-0099. Specifically, during validation step (130) the system performs “movement of the robot as will be done during the operation” to check for uses, ¶0066. ‘the trajectory between two target areas is a straight line, and this is the way a surgeon is expected to navigate.’, ¶0066, This describes the trajectory. Step 15 involves performing interactive surgery rehearsal by the surgeon, including surgeon inputs for simulated robotic manipulations, applying collision prediction algorithms, and/or inputting surgeon subjective assessment of effectiveness, ¶0099. This simulation involves the movement and therefore the mapping of trajectories of the robotic arms. See also associated paragraphs, ¶0049-0050, ¶0097, Claims 59-60, Claim 68, ¶0112-0113. In sum, Coste-Maniere teaches pre-surgical planning simulation processing and in real-time during the actual surgical procedure.
select a first starting state of the plurality of starting states that has a least number of collisions among the plurality of starting states based on the least number collisions identified for each of the respective starting state of the plurality of starting states, wherein the first starting state identifies includes respective starting poses for each arm of corresponding to the plurality of robotic arms;
-Coste-Maniere as mentioned above teaches the process of selecting an advantageous starting state (i.e., configuration) for the robotic system based on a minimizing criteria, based on the core of Coste-Maniere that teaches collision avoidance, from a plurality of simulated starting states. The process claimed is part of Coste-Maniere determining of an advantageous robotic system pre-surgical set up configuration or planning the robot position, ¶0018, ¶0092. Coste-Maniere confirms that determining the advantageous robot position uses the combined probabilistic and gradient decent approach. The configurations of the passive joints are randomly drawn. To each configuration, a cost function is associated and the cost function is defined bas on constraints, including separation between the arms (i.e., collision avoidance), ¶0064-65, ¶0092. Coste-Maniere further discusses minizing the collisions/cost wherein the low cost function gives its corresponding robot configuration a high selection probability. That is, a “low cost function” indicates a more desirable configuration, and such configurations are given “a high selection probability”, ¶0065. The process continues until a configuration arrives at a cost function that is less than a given threshold. In other words, the process aims to reach a configuration where the cost function is “less than a given threshold”, implying a state where collisions are minimized or eliminated, ¶0065. In other words, the selection of Coste-Maniere system is iterative to select the starting state, the state that has zero collisions during simulation, which are based on the collisions identified and for each respective starting state of the plurality of starting states.
-Coste-Maniere teaches that the selected configuration (i.e., first starting state) identifies the position of the robotic system base and values for the set-up joints, ¶0092. The step-up position includes determining the respective starting poses for each arm of the plurality of robotic arms, ¶0092. The subsequent steps further confirm that this configuration allows the active joints to move over all targets without collision, ¶0065, ¶0066, ¶0097.
provide pose information for the first starting state to an operator of the robotic medical system, the pose information identifying the respective starting poses corresponding to the plurality of robotic arms; and
-Coste-Maniere teaches the process of collision avoidance includes determining the advantageous robotic system pre-surgical set up configuration which includes the set-up joint positions, ¶0092. This chosen configuration (i.e., first starting state) is transferred to the actual surgical system for use, ¶0105. The results of the planning (i.e., robotic position) are registered (150), which involves transferring entry port and robot place from the simulation or an actual defined volume (i.e. operating room), ¶0072. This transfer includes reproducing the planned initial position and alignment of the robotic arms, ¶0105. Positioning of the ports is achieved moving the robotic arms according to the precomputed articulated values, ¶0072-0073. The system explicitly uses an Application Program Interface to read the corresponding articular values and then moves the joints so that the articular values match the computed ones, ¶0072-0073. Hence, this information effectively provides the pose information (i.e., starting poses) needed for the system setup to the control system and, implicitly, to the operator or assistance for setup, ¶0073.
perform the medical procedure with the plurality of robotic arms initially in the respective starting poses of the first starting state.
-Coste-Maniere entire planning process is deigned for this claimed recitation. Specifically, the entire planning process of Coste-Maniere is deisgned to prepare the robot and entry ports for the actual surgical procedure, ensuring feasibility and avoidance of collisions, ¶Abstract, ¶0014-0015, ¶0049-0050, ¶0092, ¶0105. The surgical system of Coste-Maniere includes a robot having at least two robotic arms and computer coupled with the robot for controlling movements of the robotic arms, ¶0023. The determined position is the advantageous pre-surgical set-up configuration, ¶0092. The system is used for performing a robotic operation, ¶0022-0023. During the actual surgical procedure, the aforementioned collision prediction/detection algorithms are applied to real-time robotic arm and instrument position and orientation to predict, warn of, and/or avoid collisions, ¶0112-0113. This confirms that the procedure is performed using the set-up based on the determined starting state.
Coste-Maniere fails to disclose:
identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
record the respective number of collisions resulting from the respective starting state;
and that the selection is based on the respective number of collisions identified and recorded (i.e., that the selection of a first starting state of the plurality of starting states that has the least number of collision is “based on” the respective number of collisions identified for each respective starting state of the plurality of starting states).
However, Andre in the context of collision avoidance of robotic arms, discloses, identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
-Andre teaches generating paths (i.e., trajectories) for two robotic arms using a motion planner and inspects these paths at every point in time to detect the possible collisions: [3.3 System Architecture / pg. 14-16]: “Our proposed algorithm functions in real-time. The sampling based planner RRT* is used to calculate optimal and collisionless paths around static obstacles [61][62]. In the planning state, the two paths are inspected interactively at every point in time to check for possible collisions. Following collision inspection, there are three possible outcomes: 1. No collision detected 2. A possible collision detected along the path 3. A possible collision detected at a common goal location. In the case that no collisions are detected, arms proceed to their respective goal without any change in velocity or trajectory. Once the first arm arrives at its destination, we compute a new path and once again check for possible collisions with the remaining path left of the second arm. A possible collision detected along the path in the second case implies that there exists a list of points in both paths which are under an arbitrary threshold at the same timestamp. The algorithm adjusts the velocity non-linearly to avoid the collision with minimal disturbance to the initial trajectory and velocity. Once the fast arm reaches the velocity reset point, set as the first collision point in the path, we reset the velocity and check for collisions again.”
-The system of Andre computes these trajectories based on the initial configuration (i.e., respective starting state. During the evaluations, the system test across multiple trials where the arms begin from generated starting states, either as fixed starting positions near each other or randomized, [3.3 Motion Planning / pg. 17-18], see also Experimental Setup on pages 37 & 42.
record the respective number of collisions resulting from the respective starting state;
-The collision is identified by calculating the Euclidean distance between the positions on the two paths and detecting when the distance falls below a specific threshold, [3.4 Collision Detection / pg. 19]: “The arm paths at default velocity are represented as a set of equidistant points forming a polyline. As the probability of two polylines intersecting at exactly one point in a 3D space is very low, we define a possible collision as a list of points in the two paths, at which the euclidean distance is under a certain threshold. To detect potential collisions on two paths, we use a method of collision prediction using the known initial velocity and distance traveled for every 𝑣 time steps.”
-Andre teaches the number of collisions as a performance metric defined as a measure of the collisions between the two arms or between an arm and the environment, [4.1.1 Performance Metrics / pg 26-27]. It also record the number of collisions avoided which counts the total number of times the system had to adjust arm to prevent an identified collision, [4.1.1 Performance Metrics / pg 26-27]. The result of the trials are recorded and compared in data tables. For example, the study records the execution resulting in an average of 3.2 collisions per while the proposed coordinate framework recorded 0 collisions (i.e., the ideal), [Table 4.4.1 / pg. 45].
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the one or more processors collision determination for each respective starting state of the plurality of starting states of Coste-Maniere to identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and record the respective number of collisions resulting from the respective starting state as taught by Andre. The motivation to do this yield predictable results such as “to eliminate all collisions while offering a more flexible system by operating in a decentralized manner.”, as suggested by Andre, pg. 44.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the selection of the first starting state of the plurality of starting states of modified Coste-Maniere to incorporate the teachings of Andre, such that the selection of the first starting state of the plurality of starting states in modified Coste-Maniere is performed based on the respective number of collisions identified and recorded, as taught by Andre. The motivation to do this yield predictable results such as “to eliminate all collisions while offering a more flexible system by operating in a decentralized manner.”, as suggested by Andre, pg. 44. The modified combination would disclose select a first starting state of the plurality of starting states that has a least number of collisions among the plurality of starting states based on the respective number of collisions identified and recorded for each respective starting state of the plurality of starting states.
Coste-Maniere fails to explicitly recite that the degrees of freedom is: seven or more degrees of freedom. (i.e., each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints;)
However, Ho in the context of robotic system with boundaries for robotic arm, discloses, each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints; (¶0048, ‘Each of the arms 12 have seven joints, and thus provide seven degrees of freedom. A multitude of joints result in a multitude of degrees of freedom, allowing for “redundant” degrees of freedom. Redundant degrees of freedom allow the robotic arms 12 to position their respective end effectors 22 at a specific position, orientation, and trajectory in space using different linkage positions and joint angles. This allows for the system to position and direct a medical instrument from a desired point in space while allowing the physician to move the arm joints into a clinically advantageous position away from the patient to create greater access, while avoiding arm collisions.”)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify robotics arms of modified Coste-Maniere such that each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints as taught by Ho. The motivation to do this yield predictable results such as “allows for the system to position and direct a medical instrument from a desired point in space while allowing the physician to move the arm joints into a clinically advantageous position away from the patient to create greater access, while avoiding arm collisions” as suggested by Ho, ¶0048.
Claim 2: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to identify an anatomic target for the medical procedure, wherein the simulating further uses the identifed anatomic target.
- Coste-Maniere teaches: ¶0052, ‘Data acquisition 112 generally involves acquiring any data regarding a volume which is to be operated upon, such as a portion of a patient's body, as well as, in some embodiments, data regarding a robot, surgical tools, and the like, to be used in performing the operation. Data may include, for example, CT scan data, with or without contrast, MRI data, coronary artery angiograms, conventional radiographs, digital representations of conventional radiographs, and/or the like. In a totally endoscopic coronary artery bypass graft (TECAB) operation, for example, CT scan data is typically used. This generally involves acquiring helical CT scans of a patient, with 3 mm spacing, from approximately the neck region to the hip region of the patient. Slice size is often decreased to 1 cm in the area of the heart, to acquire more image information, and often a dye is injected to better visualize the heart and aorta. Additionally, such CT data acquisition will often be synchronized with electrocardiogram (ECG) data acquisition. Coronary angiograms may also be acquired, to enable an accurate diagnosis of the state of heart vessels. Data from multiple types of imaging studies, such as CT scans and angiograms, may be used together in various embodiments to enhance planning of port placement.’. This imaging data is used to model the patient”, ¶0081 by acquiring patient-specific data, segmenting it, and reconstructing a 3D model. The ‘simulating the surgical procedure by manipulating the plurality of robotic arms so as to perform the surgical procedure while viewing a virtual image of the target area generated from the received information of the target area’, Claim 56.
Claim 3: Modified Coste-Maniere teaches all the elements above in claim 2, Coste-Maniere discloses, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to identify the anatomical target using the imaging data of the patient.
-Coste-Maniere teaches: ¶0052, ‘Data acquisition 112 generally involves acquiring any data regarding a volume which is to be operated upon, such as a portion of a patient's body, as well as, in some embodiments, data regarding a robot, surgical tools, and the like, to be used in performing the operation. Data may include, for example, CT scan data, with or without contrast, MRI data, coronary artery angiograms, conventional radiographs, digital representations of conventional radiographs, and/or the like. In a totally endoscopic coronary artery bypass graft (TECAB) operation, for example, CT scan data is typically used. This generally involves acquiring helical CT scans of a patient, with 3 mm spacing, from approximately the neck region to the hip region of the patient. Slice size is often decreased to 1 cm in the area of the heart, to acquire more image information, and often a dye is injected to better visualize the heart and aorta. Additionally, such CT data acquisition will often be synchronized with electrocardiogram (ECG) data acquisition. Coronary angiograms may also be acquired, to enable an accurate diagnosis of the state of heart vessels. Data from multiple types of imaging studies, such as CT scans and angiograms, may be used together in various embodiments to enhance planning of port placement.’. This imaging data is used to model the patient”, ¶0081 by acquiring patient-specific data, segmenting it, and reconstructing a 3D model. The ‘simulating the surgical procedure by manipulating the plurality of robotic arms so as to perform the surgical procedure while viewing a virtual image of the target area generated from the received information of the target area’, Claim 56.
Claim 5: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to select a recommended port location based on the simulating.
-Coste-Maniere teaches simulation to allow surgeons to “perform a practice operation” and “confirm that the selected combination of robot position and entry portion locations is feasible”, ¶0067-0068. If the simulation is “unsatisfactory”, the port placement and or robot position may be rejected, ¶0100. If rejected due to unsatisfactory simulation, “port placements and/or robot positioning may be rejected and steps 11 through 15 may be repeated to select and validate new placements and/or positions.”, ¶0100. “Step 11: Determining optimized multiple-port combination’, ¶0088.
Claim 6: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the first starting state includes, for each robotic arm of the plurality of robotic arms, positioning information for each joint of the plurality of joints.
- Coste-Maniere refers to “links and joints” as part of the robotic system model, ¶0116. It mentions “degrees of freedom (dofs)” for each arm, which are associated with the joints, ¶0056, ¶0064, ¶0092. “Joint position sensors” are used to monitor arm motion and direct/confirm step, ¶0111. These sensors are used to “direct and/or confirm setup arm positioning according to the optimized procedure plan”, ¶0111.
Claim 7: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to obtain imaging data of environs of the robotic medical system, wherein the simulating further uses the imaging data of the environs.
-Coste-Maniere discloses, Step 1: Robotic system modeling. Step 1 typically includes defining a model of the insertable surgical tool portion, including structure, range of motion (ROM) limits, and optionally tool-type specific properties. Step 1 also includes defining a model of the external portion or robotic tool and manipulator arm structure and ROM limits. Finally, step 1 includes defining a multiple-arm robotic system model. Optionally, the model may include adjacent OR equipment such as operating table and accessories, ¶0075. Simulation using imaging data of the environs, ¶0079-0080, ¶0098-0099.
Claim 8: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to obtain positional information for one or more objects in a vicinity of the robotic medical system, wherein the simulating further uses the positional information.
-Coste-Maniere discloses, Step 1: Robotic system modeling. Step 1 typically includes defining a model of the insertable surgical tool portion, including structure, range of motion (ROM) limits, and optionally tool-type specific properties. Step 1 also includes defining a model of the external portion or robotic tool and manipulator arm structure and ROM limits. Finally, step 1 includes defining a multiple-arm robotic system model. Optionally, the model may include adjacent OR equipment such as operating table and accessories, ¶0075. Simulation using imaging data of the positional information, ¶0079-0080, ¶0098-0099.
Claim 9: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the medical procedure requires a medical instrument, and wherein the pose information includes information regarding to which arm of the plurality of robotic arms the medical instrument should be mounted.
- Coste-Maniere teaches, Step 1 typically includes defining a model of the insertable surgical tool portion, including structure, range of motion (ROM) limits, and optionally “tool-type specific properties” and “manipulator arm structure”, ¶0075. The planning phases identifies “advantageous entry port locations for three tools, such as two robot arms and an endoscope”, ¶0057. The process involves choose an entry port for an endoscope first, then ranking other tool ports, ¶0062, ¶0088.
Claim 12: A method for determining a starting state of a robotic system that includes a plurality of robotic arms, (¶Abstract, ¶0092) each robotic arm of the plurality of robotic arms having degrees of freedom provided by a plurality of links and a plurality of joints, the method comprising: (¶0045, ‘Cart 300 here includes three robotic manipulator arm assemblies 302, each manipulator supporting an instrument 100.’; Claim 53, “a plurality of robotic arms individually manipulatable in a plurality of degrees of freedom movement”; ¶0064, ‘number of robot arms and number of degrees of freedom for each arm, potential collisions between the robot arms, potential collisions between an arm and the patient, other potential collisions (e.g. with anesthesia equipment or operating room table), and/or miscellaneous constraints (e.g. endoscope orientation for assistant surgeon).’; ¶0116, ‘active manipulator links and joints (servo-operated and passively responding joints which move during tissue treatment operation).’)
obtaining imaging data of a patient; (¶0052, ‘Data acquisition 112 generally involves acquiring any data regarding a volume which is to be operated upon, such as a portion of a patient's body, as well as, in some embodiments, data regarding a robot, surgical tools, and the like, to be used in performing the operation. Data may include, for example, CT scan data, with or without contrast, MRI data, coronary artery angiograms, conventional radiographs, digital representations of conventional radiographs, and/or the like. In a totally endoscopic coronary artery bypass graft (TECAB) operation, for example, CT scan data is typically used. This generally involves acquiring helical CT scans of a patient, with 3 mm spacing, from approximately the neck region to the hip region of the patient. Slice size is often decreased to 1 cm in the area of the heart, to acquire more image information, and often a dye is injected to better visualize the heart and aorta. Additionally, such CT data acquisition will often be synchronized with electrocardiogram (ECG) data acquisition. Coronary angiograms may also be acquired, to enable an accurate diagnosis of the state of heart vessels. Data from multiple types of imaging studies, such as CT scans and angiograms, may be used together in various embodiments to enhance planning of port placement.’; ¶0081, ‘Modeling the patient. Various exemplary embodiments include the use of patient-specific data to characterize the body portion being treated. In some embodiments and surgical procedures, port optimization planning on a representative sample of patients will have sufficient generality to be useful as a generic port placement plan. Modeling a patient may involve several sub-steps, such as: 1. acquiring patient-specific data for at least a portion of patent's body via such modalities as CT, MRI, and or arterial angiograms; 2. segmenting acquired data to distinguish organ, bone, vessel and other tissue structures (may be automated, manual or a combination of these); 3. reconstructing segmented, acquired data to construct a 3D model for at least a portion of patent's body. Optionally, such a model may include additional overlaid patient data, a body cavity insufflation space model, and/or the like.’)
obtaining information on a medical procedure for patient; (¶Abstract, ‘Methods and apparatus for enhancing surgical planning’; The determination of the port locations and robot positioning considers “factors such as patient anatomy, surgeon performance, the surgical procedure to be performed”, ¶0011. “Surgeon preferences” incorporate including specifying “target points within the patient” and “preferred ‘attack directions’”, ¶0058. Step 6, “Step 6: Defining operative motion prediction algorithms. This step involves defining a predictive model of expected range of surgeon-commanded operational tool movements during surgical task (generic task, specific procedural and/or tool-types).”, ¶0080. Simulations, as well as validation, involves, ‘applying the predictive model of expected surgeon-commanded operational instrument movements for a surgical procedure during manipulations at a surgical target site within the body.’, ¶0097.)
for each respective starting state of a plurality of starting states: simulating the medical procedure from the respective starting state using the imaging data and the information on the medical procedure, the simulating comprising mapping trajectories for each robotic arm of the plurality of robotic arms during the medical procedure,
-Coste-Maniere simulating the medical procedure from various starting states to identify resulting collisions, particularly when determining the advantageous position for the robotic system. The core of Coste-Maniere is to configure a planning process for robotic surgery involving the determining of an advantageous position for one or more arms of the plurality of robotic arms relative to the patient based on entry port placement, ¶Abstract, ¶0064. The approach of Coste-Maniere uses a combined probabilistic and gradient descent approach where configurations of the joints are randomly drawn in a robot articular space. This set of randomly drawn configurations represent a plurality of starting states, ¶0065, Claims 59-60. Once the configuration is selected (i.e., the cost function), those active joints are moved over all the targets to very that there is no collision, ¶0065. Hence, this verification steps acts as a simulation of the expected movement for that specific configuration, ¶0065, ¶0097. Note; the primary concern of Coste-Maniere addressed for surgical planning is to avoid “collision of two or more robotic arms during a robotic procedure”, ¶0011.
-Coste-Maniere teaches that the simulation and validation steps use patient-specific data, such as the imaging data, which is processed to create the model of the surgical site, ¶0015, ¶0049-0050, ¶0081. Note this model is a “3D model for at least a portion of the patient’s body”, ¶0081. These steps use the predictive model of expected surgeon operational instrument movements for the procedure, and the collision prediction algorithm is applied during this simulation to determine if collisions will occur, ¶0097.
-Coste-Maniere teaches that in the step of validation is specifically performed to very that the identified location are feasible, ¶0066. Specifically, during validation, the movement of the robot as expected during the operation is carried out to look for the possible collisions between the robotic arms, ¶0066. If a collision is detected during validation, the system may return to the planning stage to select other entry locations and/or robotic arm positions, ¶0066, ¶0098-0099. Specifically, during validation step (130) the system performs “movement of the robot as will be done during the operation” to check for uses, ¶0066. ‘the trajectory between two target areas is a straight line, and this is the way a surgeon is expected to navigate.’, ¶0066, This describes the trajectory. Step 15 involves performing interactive surgery rehearsal by the surgeon, including surgeon inputs for simulated robotic manipulations, applying collision prediction algorithms, and/or inputting surgeon subjective assessment of effectiveness, ¶0099. This simulation involves the movement and therefore the mapping of trajectories of the robotic arms. See also associated paragraphs, ¶0049-0050, ¶0097, Claims 59-60, Claim 68, ¶0112-0113. In sum, Coste-Maniere teaches pre-surgical planning simulation processing and in real-time during the actual surgical procedure.
selecting a first starting state of the plurality of starting states that has a least number of the collisions among the plurality of starting states based on the least number of collisions identified for each respective starting state of the plurality of starting states, wherein the first starting state includes respective starting poses corresponding to the plurality of robotic arms;
-Coste-Maniere as mentioned above teaches the process of selecting an advantageous starting state (i.e., configuration) for the robotic system based on a minimizing criteria, based on the core of Coste-Maniere that teaches collision avoidance, from a plurality of simulated starting states. The process claimed is part of Coste-Maniere determining of an advantageous robotic system pre-surgical set up configuration or planning the robot position, ¶0018, ¶0092. Coste-Maniere confirms that determining the advantageous robot position uses the combined probabilistic and gradient decent approach. The configurations of the passive joints are randomly drawn. To each configuration, a cost function is associated and the cost function is defined bas on constraints, including separation between the arms (i.e., collision avoidance), ¶0064-65, ¶0092. Coste-Maniere further discusses minizing the collisions/cost wherein the low cost function gives its corresponding robot configuration a high selection probability. That is, a “low cost function” indicates a more desirable configuration, and such configurations are given “a high selection probability”, ¶0065. The process continues until a configuration arrives at a cost function that is less than a given threshold. In other words, the process aims to reach a configuration where the cost function is “less than a given threshold”, implying a state where collisions are minimized or eliminated, ¶0065. In other words, the selection of Coste-Maniere system is iterative to select the starting state, the state that has zero collisions during simulation, which are based on the collisions identified and for each respective starting state of the plurality of starting states.
-Coste-Maniere teaches that the selected configuration (i.e., first starting state) identifies the position of the robotic system base and values for the set-up joints, ¶0092. The step-up position includes determining the respective starting poses for each arm of the plurality of robotic arms, ¶0092. The subsequent steps further confirm that this configuration allows the active joints to move over all targets without collision, ¶0065, ¶0066, ¶0097.
providing pose information for the first starting state to an operator of the robotic system, the pose information identifying the respective starting poses corresponding to the plurality of robotic arms; and
-Coste-Maniere teaches the process of collision avoidance includes determining the advantageous robotic system pre-surgical set up configuration which includes the set-up joint positions, ¶0092. This chosen configuration (i.e., first starting state) is transferred to the actual surgical system for use, ¶0105. The results of the planning (i.e., robotic position) are registered (150), which involves transferring entry port and robot place from the simulation or an actual defined volume (i.e. operating room), ¶0072. This transfer includes reproducing the planned initial position and alignment of the robotic arms, ¶0105. Positioning of the ports is achieved moving the robotic arms according to the precomputed articulated values, ¶0072-0073. The system explicitly uses an Application Program Interface to read the corresponding articular values and then moves the joints so that the articular values match the computed ones, ¶0072-0073. Hence, this information effectively provides the pose information (i.e., starting poses) needed for the system setup to the control system and, implicitly, to the operator or assistance for setup, ¶0073.
performing the medical procedure with the plurality of robotic arms initially in the respective starting poses of the first starting state.
-Coste-Maniere entire planning process is deigned for this claimed recitation. Specifically, the entire planning process of Coste-Maniere is deisgned to prepare the robot and entry ports for the actual surgical procedure, ensuring feasibility and avoidance of collisions, ¶Abstract, ¶0014-0015, ¶0049-0050, ¶0092, ¶0105. The surgical system of Coste-Maniere includes a robot having at least two robotic arms and computer coupled with the robot for controlling movements of the robotic arms, ¶0023. The determined position is the advantageous pre-surgical set-up configuration, ¶0092. The system is used for performing a robotic operation, ¶0022-0023. During the actual surgical procedure, the aforementioned collision prediction/detection algorithms are applied to real-time robotic arm and instrument position and orientation to predict, warn of, and/or avoid collisions, ¶0112-0113. This confirms that the procedure is performed using the set-up based on the determined starting state.
Coste-Maniere fails to disclose:
identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
record the respective number of the collisions resulting from the respective starting state;
and that the selection is based on the respective number of collisions identified and recorded (i.e., that the selection of a first starting state of the plurality of starting states that has the least number of collision is “based on” the respective number of collisions identified for each respective starting state of the plurality of starting states).
However, Andre in the context of collision avoidance of robotic arms, discloses, identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
-Andre teaches generating paths (i.e., trajectories) for two robotic arms using a motion planner and inspects these paths at every point in time to detect the possible collisions: [3.3 System Architecture / pg. 14-16]: “Our proposed algorithm functions in real-time. The sampling based planner RRT* is used to calculate optimal and collisionless paths around static obstacles [61][62]. In the planning state, the two paths are inspected interactively at every point in time to check for possible collisions. Following collision inspection, there are three possible outcomes: 1. No collision detected 2. A possible collision detected along the path 3. A possible collision detected at a common goal location. In the case that no collisions are detected, arms proceed to their respective goal without any change in velocity or trajectory. Once the first arm arrives at its destination, we compute a new path and once again check for possible collisions with the remaining path left of the second arm. A possible collision detected along the path in the second case implies that there exists a list of points in both paths which are under an arbitrary threshold at the same timestamp. The algorithm adjusts the velocity non-linearly to avoid the collision with minimal disturbance to the initial trajectory and velocity. Once the fast arm reaches the velocity reset point, set as the first collision point in the path, we reset the velocity and check for collisions again.”
-The system of Andre computes these trajectories based on the initial configuration (i.e., respective starting state. During the evaluations, the system test across multiple trials where the arms begin from generated starting states, either as fixed starting positions near each other or randomized, [3.3 Motion Planning / pg. 17-18], see also Experimental Setup on pages 37 & 42.
record the respective number of collisions resulting from the respective starting state;
-The collision is identified by calculating the Euclidean distance between the positions on the two paths and detecting when the distance falls below a specific threshold, [3.4 Collision Detection / pg. 19]: “The arm paths at default velocity are represented as a set of equidistant points forming a polyline. As the probability of two polylines intersecting at exactly one point in a 3D space is very low, we define a possible collision as a list of points in the two paths, at which the euclidean distance is under a certain threshold. To detect potential collisions on two paths, we use a method of collision prediction using the known initial velocity and distance traveled for every 𝑣 time steps.”
-Andre teaches the number of collisions as a performance metric defined as a measure of the collisions between the two arms or between an arm and the environment, [4.1.1 Performance Metrics / pg 26-27]. It also record the number of collisions avoided which counts the total number of times the system had to adjust arm to prevent an identified collision, [4.1.1 Performance Metrics / pg 26-27]. The result of the trials are recorded and compared in data tables. For example, the study records the execution resulting in an average of 3.2 collisions per while the proposed coordinate framework recorded 0 collisions (i.e., the ideal), [Table 4.4.1 / pg. 45].
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the one or more processors collision determination for each respective starting state of the plurality of starting states of Coste-Maniere to identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and record the respective number of collisions resulting from the respective starting state as taught by Andre. The motivation to do this yield predictable results such as “to eliminate all collisions while offering a more flexible system by operating in a decentralized manner.”, as suggested by Andre, pg. 44.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the selection of the first starting state of the plurality of starting states of modified Coste-Maniere to incorporate the teachings of Andre, such that the selection of the first starting state of the plurality of starting states in modified Coste-Maniere is performed based on the respective number of collisions identified and recorded, as taught by Andre. The motivation to do this yield predictable results such as “to eliminate all collisions while offering a more flexible system by operating in a decentralized manner.”, as suggested by Andre, pg. 44. The modified combination would disclose select a first starting state of the plurality of starting states that has a least number of collisions among the plurality of starting states based on the respective number of collisions identified and recorded for each respective starting state of the plurality of starting states.
Coste-Maniere fails to explicitly recite that the degrees of freedom is: seven or more degrees of freedom. (i.e., each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints;)
However, Ho in the context of robotic system with boundaries for robotic arm, discloses, each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints; (¶0048, ‘Each of the arms 12 have seven joints, and thus provide seven degrees of freedom. A multitude of joints result in a multitude of degrees of freedom, allowing for “redundant” degrees of freedom. Redundant degrees of freedom allow the robotic arms 12 to position their respective end effectors 22 at a specific position, orientation, and trajectory in space using different linkage positions and joint angles. This allows for the system to position and direct a medical instrument from a desired point in space while allowing the physician to move the arm joints into a clinically advantageous position away from the patient to create greater access, while avoiding arm collisions.”)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify robotics arms of modified Coste-Maniere such that each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints as taught by Ho. The motivation to do this yield predictable results such as “allows for the system to position and direct a medical instrument from a desired point in space while allowing the physician to move the arm joints into a clinically advantageous position away from the patient to create greater access, while avoiding arm collisions” as suggested by Ho, ¶0048.
Claim 13: Modified Coste-Maniere teaches all the elements above in claim 12, Coste-Maniere discloses, further comprising identifying an anatomic target for the medical procedure, wherein the simulating further uses the identified anatomic target.
- Coste-Maniere teaches: ¶0052, ‘Data acquisition 112 generally involves acquiring any data regarding a volume which is to be operated upon, such as a portion of a patient's body, as well as, in some embodiments, data regarding a robot, surgical tools, and the like, to be used in performing the operation. Data may include, for example, CT scan data, with or without contrast, MRI data, coronary artery angiograms, conventional radiographs, digital representations of conventional radiographs, and/or the like. In a totally endoscopic coronary artery bypass graft (TECAB) operation, for example, CT scan data is typically used. This generally involves acquiring helical CT scans of a patient, with 3 mm spacing, from approximately the neck region to the hip region of the patient. Slice size is often decreased to 1 cm in the area of the heart, to acquire more image information, and often a dye is injected to better visualize the heart and aorta. Additionally, such CT data acquisition will often be synchronized with electrocardiogram (ECG) data acquisition. Coronary angiograms may also be acquired, to enable an accurate diagnosis of the state of heart vessels. Data from multiple types of imaging studies, such as CT scans and angiograms, may be used together in various embodiments to enhance planning of port placement.’. This imaging data is used to model the patient”, ¶0081 by acquiring patient-specific data, segmenting it, and reconstructing a 3D model. The ‘simulating the surgical procedure by manipulating the plurality of robotic arms so as to perform the surgical procedure while viewing a virtual image of the target area generated from the received information of the target area’, Claim 56.
Claim 15: Modified Coste-Maniere teaches all the elements above in claim 12, Coste-Maniere discloses, further comprising selecting a recommended port location based on the simulating.
-Coste-Maniere teaches simulation to allow surgeons to “perform a practice operation” and “confirm that the selected combination of robot position and entry portion locations is feasible”, ¶0067-0068. If the simulation is “unsatisfactory”, the port placement and or robot position may be rejected, ¶0100. If rejected due to unsatisfactory simulation, “port placements and/or robot positioning may be rejected and steps 11 through 15 may be repeated to select and validate new placements and/or positions.”, ¶0100. “Step 11: Determining optimized multiple-port combination’, ¶0088.
Claim 16: Modified Coste-Maniere teaches all the elements above in claim 12, Coste-Maniere discloses, further comprising obtaining imaging data of environs of the robotic system, wherein the simulating further uses the imaging data of the environs.
-Coste-Maniere discloses, Step 1: Robotic system modeling. Step 1 typically includes defining a model of the insertable surgical tool portion, including structure, range of motion (ROM) limits, and optionally tool-type specific properties. Step 1 also includes defining a model of the external portion or robotic tool and manipulator arm structure and ROM limits. Finally, step 1 includes defining a multiple-arm robotic system model. Optionally, the model may include adjacent OR equipment such as operating table and accessories, ¶0075. Simulation using imaging data of the environs, ¶0079-0080, ¶0098-0099.
Claim 17: Modified Coste-Maniere teaches all the elements above in claim 12, Coste-Maniere discloses, further comprising obtaining positional information for one or more objects in a vicinity of the robotic system, wherein the simulating further uses the positional information.
-Coste-Maniere discloses, Step 1: Robotic system modeling. Step 1 typically includes defining a model of the insertable surgical tool portion, including structure, range of motion (ROM) limits, and optionally tool-type specific properties. Step 1 also includes defining a model of the external portion or robotic tool and manipulator arm structure and ROM limits. Finally, step 1 includes defining a multiple-arm robotic system model. Optionally, the model may include adjacent OR equipment such as operating table and accessories, ¶0075. Simulation using imaging data of the positional information, ¶0079-0080, ¶0098-0099.
Claim 21: Coste-Maniere discloses, A robotic medical system comprising: (¶Abstract)
a plurality of robotic arms (¶0045, ‘Cart 300 here includes three robotic manipulator arm assemblies 302, each manipulator supporting an instrument 100.’) each robotic arm of the plurality of robotic arms having degrees of freedom provided by a plurality of links and a plurality of joints; (¶0045, ‘Cart 300 here includes three robotic manipulator arm assemblies 302, each manipulator supporting an instrument 100.’; Claim 53, “a plurality of robotic arms individually manipulatable in a plurality of degrees of freedom movement”; ¶0064, ‘number of robot arms and number of degrees of freedom for each arm, potential collisions between the robot arms, potential collisions between an arm and the patient, other potential collisions (e.g. with anesthesia equipment or operating room table), and/or miscellaneous constraints (e.g. endoscope orientation for assistant surgeon).’; ¶0116, ‘active manipulator links and joints (servo-operated and passively responding joints which move during tissue treatment operation).’)
one or more processors; and (Claim 61: “a processor configured to determine whether any two or more of the plurality of robotic arms may collide during the surgical procedure using information of a target area in which the surgical procedure is to be performed within a patient, and information mechanically characterizing the plurality of robotic arms.’)
memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: (¶0046, ‘processor 400 coupled with master control station 200 and cart 300 and a tangible medium 410 embodying machine readable code, or software. The software typically includes instructions which enable various embodiments of the methods of the present invention. The tangible medium 410 may be coupled with the processor 400 for use. Generally, the software may be used with any suitable hardware, such as a personal computer work station with graphics capabilities, such as but not limited to a PENTIUM Ill: or equivalent processor with a GEFORCE2® graphics card. Other hardware which may be used with software of the present invention includes a display monitor, such as a 17″ monitor, a processor with 256 Mbytes of RAM and a 10 Gigabytes hard disk. Input devices will typically include a mouse and may also include a 3D mouse or a PHANTOM® arm.’; see also ¶0047-0048; ¶0113, ‘Recorded procedure history and computer data, including surgeon inputs, tool motions and robotic arm movements, may be used to refine models, optimization criteria, feasibility criteria and/or cost function terms.’)
obtain three-dimensional (3-D) scan data that includes a view of a patient and one or more objects surrounding the patient; (¶0052, ‘Data acquisition 112 generally involves acquiring any data regarding a volume which is to be operated upon, such as a portion of a patient's body, as well as, in some embodiments, data regarding a robot, surgical tools, and the like, to be used in performing the operation. Data may include, for example, CT scan data, with or without contrast, MRI data, coronary artery angiograms, conventional radiographs, digital representations of conventional radiographs, and/or the like. In a totally endoscopic coronary artery bypass graft (TECAB) operation, for example, CT scan data is typically used. This generally involves acquiring helical CT scans of a patient, with 3 mm spacing, from approximately the neck region to the hip region of the patient. Slice size is often decreased to 1 cm in the area of the heart, to acquire more image information, and often a dye is injected to better visualize the heart and aorta. Additionally, such CT data acquisition will often be synchronized with electrocardiogram (ECG) data acquisition. Coronary angiograms may also be acquired, to enable an accurate diagnosis of the state of heart vessels. Data from multiple types of imaging studies, such as CT scans and angiograms, may be used together in various embodiments to enhance planning of port placement.’; ¶0075, ‘the model may include adjacent OR equipment such as operating table and accessories.’ ¶0081, ‘Modeling the patient. Various exemplary embodiments include the use of patient-specific data to characterize the body portion being treated. In some embodiments and surgical procedures, port optimization planning on a representative sample of patients will have sufficient generality to be useful as a generic port placement plan. Modeling a patient may involve several sub-steps, such as: 1. acquiring patient-specific data for at least a portion of patent's body via such modalities as CT, MRI, and or arterial angiograms; 2. segmenting acquired data to distinguish organ, bone, vessel and other tissue structures (may be automated, manual or a combination of these); 3. reconstructing segmented, acquired data to construct a 3D model for at least a portion of patent's body. Optionally, such a model may include additional overlaid patient data, a body cavity insufflation space model, and/or the like.’)
obtain information on a medical procedure for the patient; (¶Abstract, ‘Methods and apparatus for enhancing surgical planning’; The determination of the port locations and robot positioning considers “factors such as patient anatomy, surgeon performance, the surgical procedure to be performed”, ¶0011. “Surgeon preferences” incorporate including specifying “target points within the patient” and “preferred ‘attack directions’”, ¶0058. Step 6, “Step 6: Defining operative motion prediction algorithms. This step involves defining a predictive model of expected range of surgeon-commanded operational tool movements during surgical task (generic task, specific procedural and/or tool-types).”, ¶0080. Simulations, as well as validation, involves, ‘applying the predictive model of expected surgeon-commanded operational instrument movements for a surgical procedure during manipulations at a surgical target site within the body.’, ¶0097.)
for each respective starting state of a plurality of starting states: simulate the medical procedure from the respective starting state using the 3-D scan data and the information on the medical procedure, the simulating comprising mapping trajectories for each robotic arm of the plurality of robotic arms during the medical procedure,
-Coste-Maniere simulating the medical procedure from various starting states to identify resulting collisions, particularly when determining the advantageous position for the robotic system. The core of Coste-Maniere is to configure a planning process for robotic surgery involving the determining of an advantageous position for one or more arms of the plurality of robotic arms relative to the patient based on entry port placement, ¶Abstract, ¶0064. The approach of Coste-Maniere uses a combined probabilistic and gradient descent approach where configurations of the joints are randomly drawn in a robot articular space. This set of randomly drawn configurations represent a plurality of starting states, ¶0065, Claims 59-60. Once the configuration is selected (i.e., the cost function), those active joints are moved over all the targets to very that there is no collision, ¶0065. Hence, this verification steps acts as a simulation of the expected movement for that specific configuration, ¶0065, ¶0097. Note; the primary concern of Coste-Maniere addressed for surgical planning is to avoid “collision of two or more robotic arms during a robotic procedure”, ¶0011.
-Coste-Maniere teaches that the simulation and validation steps use patient-specific data, such as the imaging data, which is processed to create the model of the surgical site, ¶0015, ¶0049-0050, ¶0081. Note this model is a “3D model for at least a portion of the patient’s body”, ¶0081. These steps use the predictive model of expected surgeon operational instrument movements for the procedure, and the collision prediction algorithm is applied during this simulation to determine if collisions will occur, ¶0097.
-Coste-Maniere teaches that in the step of validation is specifically performed to very that the identified location are feasible, ¶0066. Specifically, during validation, the movement of the robot as expected during the operation is carried out to look for the possible collisions between the robotic arms, ¶0066. If a collision is detected during validation, the system may return to the planning stage to select other entry locations and/or robotic arm positions, ¶0066, ¶0098-0099. Specifically, during validation step (130) the system performs “movement of the robot as will be done during the operation” to check for uses, ¶0066. ‘the trajectory between two target areas is a straight line, and this is the way a surgeon is expected to navigate.’, ¶0066, This describes the trajectory. Step 15 involves performing interactive surgery rehearsal by the surgeon, including surgeon inputs for simulated robotic manipulations, applying collision prediction algorithms, and/or inputting surgeon subjective assessment of effectiveness, ¶0099. This simulation involves the movement and therefore the mapping of trajectories of the robotic arms. See also associated paragraphs, ¶0049-0050, ¶0097, Claims 59-60, Claim 68, ¶0112-0113. In sum, Coste-Maniere teaches pre-surgical planning simulation processing and in real-time during the actual surgical procedure.
select a first starting state of the plurality of starting states that has a least number of the collisions among the plurality of starting states based on the least number of the collisions identified for each respective starting state of the plurality of starting states, wherein the first starting state includes respective starting poses corresponding to the plurality of robotic arms;
-Coste-Maniere as mentioned above teaches the process of selecting an advantageous starting state (i.e., configuration) for the robotic system based on a minimizing criteria, based on the core of Coste-Maniere that teaches collision avoidance, from a plurality of simulated starting states. The process claimed is part of Coste-Maniere determining of an advantageous robotic system pre-surgical set up configuration or planning the robot position, ¶0018, ¶0092. Coste-Maniere confirms that determining the advantageous robot position uses the combined probabilistic and gradient decent approach. The configurations of the passive joints are randomly drawn. To each configuration, a cost function is associated and the cost function is defined bas on constraints, including separation between the arms (i.e., collision avoidance), ¶0064-65, ¶0092. Coste-Maniere further discusses minizing the collisions/cost wherein the low cost function gives its corresponding robot configuration a high selection probability. That is, a “low cost function” indicates a more desirable configuration, and such configurations are given “a high selection probability”, ¶0065. The process continues until a configuration arrives at a cost function that is less than a given threshold. In other words, the process aims to reach a configuration where the cost function is “less than a given threshold”, implying a state where collisions are minimized or eliminated, ¶0065. In other words, the selection of Coste-Maniere system is iterative to select the starting state, the state that has zero collisions during simulation, which are based on the collisions identified and for each respective starting state of the plurality of starting states.
-Coste-Maniere teaches that the selected configuration (i.e., first starting state) identifies the position of the robotic system base and values for the set-up joints, ¶0092. The step-up position includes determining the respective starting poses for each arm of the plurality of robotic arms, ¶0092. The subsequent steps further confirm that this configuration allows the active joints to move over all targets without collision, ¶0065, ¶0066, ¶0097.
provide pose information for the first starting state to an operator of the robotic medical system, the pose information identifying the respective starting poses corresponding to the plurality of robotic arms; and
-Coste-Maniere teaches the process of collision avoidance includes determining the advantageous robotic system pre-surgical set up configuration which includes the set-up joint positions, ¶0092. This chosen configuration (i.e., first starting state) is transferred to the actual surgical system for use, ¶0105. The results of the planning (i.e., robotic position) are registered (150), which involves transferring entry port and robot place from the simulation or an actual defined volume (i.e. operating room), ¶0072. This transfer includes reproducing the planned initial position and alignment of the robotic arms, ¶0105. Positioning of the ports is achieved moving the robotic arms according to the precomputed articulated values, ¶0072-0073. The system explicitly uses an Application Program Interface to read the corresponding articular values and then moves the joints so that the articular values match the computed ones, ¶0072-0073. Hence, this information effectively provides the pose information (i.e., starting poses) needed for the system setup to the control system and, implicitly, to the operator or assistance for setup, ¶0073.
perform the medical procedure with the plurality of robotic arms initially in the respective starting poses of the first starting state.
-Coste-Maniere entire planning process is deigned for this claimed recitation. Specifically, the entire planning process of Coste-Maniere is deisgned to prepare the robot and entry ports for the actual surgical procedure, ensuring feasibility and avoidance of collisions, ¶Abstract, ¶0014-0015, ¶0049-0050, ¶0092, ¶0105. The surgical system of Coste-Maniere includes a robot having at least two robotic arms and computer coupled with the robot for controlling movements of the robotic arms, ¶0023. The determined position is the advantageous pre-surgical set-up configuration, ¶0092. The system is used for performing a robotic operation, ¶0022-0023. During the actual surgical procedure, the aforementioned collision prediction/detection algorithms are applied to real-time robotic arm and instrument position and orientation to predict, warn of, and/or avoid collisions, ¶0112-0113. This confirms that the procedure is performed using the set-up based on the determined starting state.
Coste-Maniere fails to disclose:
identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
record the respective number of collisions resulting from the respective starting state;
and that the selection is based on the respective number of collisions identified and recorded (i.e., that the selection of a first starting state of the plurality of starting states that has the least number of collision is “based on” the respective number of collisions identified for each respective starting state of the plurality of starting states).
However, Andre in the context of collision avoidance of robotic arms, discloses, identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and
-Andre teaches generating paths (i.e., trajectories) for two robotic arms using a motion planner and inspects these paths at every point in time to detect the possible collisions: [3.3 System Architecture / pg. 14-16]: “Our proposed algorithm functions in real-time. The sampling based planner RRT* is used to calculate optimal and collisionless paths around static obstacles [61][62]. In the planning state, the two paths are inspected interactively at every point in time to check for possible collisions. Following collision inspection, there are three possible outcomes: 1. No collision detected 2. A possible collision detected along the path 3. A possible collision detected at a common goal location. In the case that no collisions are detected, arms proceed to their respective goal without any change in velocity or trajectory. Once the first arm arrives at its destination, we compute a new path and once again check for possible collisions with the remaining path left of the second arm. A possible collision detected along the path in the second case implies that there exists a list of points in both paths which are under an arbitrary threshold at the same timestamp. The algorithm adjusts the velocity non-linearly to avoid the collision with minimal disturbance to the initial trajectory and velocity. Once the fast arm reaches the velocity reset point, set as the first collision point in the path, we reset the velocity and check for collisions again.”
-The system of Andre computes these trajectories based on the initial configuration (i.e., respective starting state. During the evaluations, the system test across multiple trials where the arms begin from generated starting states, either as fixed starting positions near each other or randomized, [3.3 Motion Planning / pg. 17-18], see also Experimental Setup on pages 37 & 42.
record the respective number of collisions resulting from the respective starting state;
-The collision is identified by calculating the Euclidean distance between the positions on the two paths and detecting when the distance falls below a specific threshold, [3.4 Collision Detection / pg. 19]: “The arm paths at default velocity are represented as a set of equidistant points forming a polyline. As the probability of two polylines intersecting at exactly one point in a 3D space is very low, we define a possible collision as a list of points in the two paths, at which the euclidean distance is under a certain threshold. To detect potential collisions on two paths, we use a method of collision prediction using the known initial velocity and distance traveled for every 𝑣 time steps.”
-Andre teaches the number of collisions as a performance metric defined as a measure of the collisions between the two arms or between an arm and the environment, [4.1.1 Performance Metrics / pg 26-27]. It also record the number of collisions avoided which counts the total number of times the system had to adjust arm to prevent an identified collision, [4.1.1 Performance Metrics / pg 26-27]. The result of the trials are recorded and compared in data tables. For example, the study records the execution resulting in an average of 3.2 collisions per while the proposed coordinate framework recorded 0 collisions (i.e., the ideal), [Table 4.4.1 / pg. 45].
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the one or more processors collision determination for each respective starting state of the plurality of starting states of Coste-Maniere to identify a respective number of collisions resulting from the respective starting state occurring with the plurality of robotic arms based on the trajectories, and record the respective number of collisions resulting from the respective starting state as taught by Andre. The motivation to do this yield predictable results such as “to eliminate all collisions while offering a more flexible system by operating in a decentralized manner.”, as suggested by Andre, pg. 44.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the selection of the first starting state of the plurality of starting states of modified Coste-Maniere to incorporate the teachings of Andre, such that the selection of the first starting state of the plurality of starting states in modified Coste-Maniere is performed based on the respective number of collisions identified and recorded, as taught by Andre. The motivation to do this yield predictable results such as “to eliminate all collisions while offering a more flexible system by operating in a decentralized manner.”, as suggested by Andre, pg. 44. The modified combination would disclose select a first starting state of the plurality of starting states that has a least number of collisions among the plurality of starting states based on the respective number of collisions identified and recorded for each respective starting state of the plurality of starting states.
Coste-Maniere fails to explicitly recite that the degrees of freedom is: seven or more degrees of freedom. (i.e., each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints;)
However, Ho in the context of robotic system with boundaries for robotic arm, discloses, each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints; (¶0048, ‘Each of the arms 12 have seven joints, and thus provide seven degrees of freedom. A multitude of joints result in a multitude of degrees of freedom, allowing for “redundant” degrees of freedom. Redundant degrees of freedom allow the robotic arms 12 to position their respective end effectors 22 at a specific position, orientation, and trajectory in space using different linkage positions and joint angles. This allows for the system to position and direct a medical instrument from a desired point in space while allowing the physician to move the arm joints into a clinically advantageous position away from the patient to create greater access, while avoiding arm collisions.”)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify robotics arms of modified Coste-Maniere such that each robotic arm of the plurality of robotic arms having seven or more degrees of freedom provided by a plurality of links and a plurality of joints as taught by Ho. The motivation to do this yield predictable results such as “allows for the system to position and direct a medical instrument from a desired point in space while allowing the physician to move the arm joints into a clinically advantageous position away from the patient to create greater access, while avoiding arm collisions” as suggested by Ho, ¶0048.
Claim 22: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the medical procedure involves a plurality of sequential steps in which at least a subset of the sequential steps have corresponding required robotic arm positions, and wherein the mapping maps the trajectories as the plurality of robotic arms move from respective starting poses to the required robotic arm positions for at least the subset of the sequential steps (¶0065-0066, ¶0079-0080, ¶0084-0085, ¶0097, ¶0101-0105).
Claims 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Coste-Maniere et al (US 2007/0293734 A1), in view of Andre ("A Motion Planning and Velocity Collision Avoidance Framework for Bilateral Manipulation")(December 30, 2021) (Year: 2021), in view of Ho et al (US 20190105776 A1), as applied to claim 1 and 12, in further view of Felder et al (US 10136949 B2).
Claim 10: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere discloses, wherein the memory further includes instructions that, when executed by the one or more processors, cause the one or more processors to move the plurality of robotic arms into the respective starting poses prior to docking the plurality of robotic arms to the plurality of respective ports.
-Coste-Maniere teaches Determining an advantageous robotic system pre-surgical set-up configuration which includes “set-up joint position(s) for the robotic system, ¶0092. Positioning the ports may simply be achieved by moving the robot arms according to the precomputed articular values that correspond to having the remote center on the port, ¶0073. Joint position sensors are used to direct and/or confirm setup arm positioning according to the optimized procedure plan, and to direct and/or confirm instrument orientation and tip location to touch the body surface at a modeled port location and orientation, ¶0111. ¶0073, ‘Once the robot is registered to the simulation skeleton, positioning the ports may simply be achieved by moving the robot arms according to the precomputed articular values that correspond to having the remote center on the port. On the other hand, the results of the planning can also be expressed as a quantitative description of the positions of the port, for example endoscope arm at third intercostal space at the limit of the cartilage. This is a relatively accurate description since the ports are planned to be located in the intercostal spacing. When entry port locations are identified and the robot is positioned, a surgeon or other operator may begin the procedure.’; ¶0105, ‘Transfer and registration may include the marking of port locations, and reproducing the planned initial positions and alignment of the instruments and/or robotic arms. [...] making incisions at determined port locations for instrument insertion.’;
Coste-Maniere fails to explicitly disclose: automatically move the plurality robotic arms into the respective starting poses.
However, Felder in the context of gathering and analyzing data for robotic surgical systems discloses, automatically move the plurality robotic arms into the respective starting poses. (Claim 1, ‘thereby move one or more of the plurality of movable arms such that the one or more of the plurality of movable arms are automatically positioned in accordance with the recommendation of the initial position of each of the arms’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the movement of the plurality of robotic arms into the respective starting poses of modified Coste-Maniere such that the movement is automatic as taught by Felder. The motivation to do this yield predictable results such as to help save the user's time since the user need not manually move the arms, as suggested by Felder, [Col 26 l.14-15].
Claim 19: Modified Coste-Maniere teaches all the elements above in claim 12, Coste-Maniere discloses, further comprising moving the plurality of robotic arms into the respective starting poses prior to docking the plurality of robotic arms to a plurality of respective ports.
-Coste-Maniere teaches Determining an advantageous robotic system pre-surgical set-up configuration which includes “set-up joint position(s) for the robotic system, ¶0092. Positioning the ports may simply be achieved by moving the robot arms according to the precomputed articular values that correspond to having the remote center on the port, ¶0073. Joint position sensors are used to direct and/or confirm setup arm positioning according to the optimized procedure plan, and to direct and/or confirm instrument orientation and tip location to touch the body surface at a modeled port location and orientation, ¶0111. ¶0073, ‘Once the robot is registered to the simulation skeleton, positioning the ports may simply be achieved by moving the robot arms according to the precomputed articular values that correspond to having the remote center on the port. On the other hand, the results of the planning can also be expressed as a quantitative description of the positions of the port, for example endoscope arm at third intercostal space at the limit of the cartilage. This is a relatively accurate description since the ports are planned to be located in the intercostal spacing. When entry port locations are identified and the robot is positioned, a surgeon or other operator may begin the procedure.’; ¶0105, ‘Transfer and registration may include the marking of port locations, and reproducing the planned initial positions and alignment of the instruments and/or robotic arms. [...] making incisions at determined port locations for instrument insertion.’;
Coste-Maniere fails to explicitly disclose: automatically moving the plurality robotic arms into the respective starting poses.
However, Felder in the context of gathering and analyzing data for robotic surgical systems discloses, automatically moving the plurality robotic arms into the respective starting poses. (Claim 1, ‘thereby move one or more of the plurality of movable arms such that the one or more of the plurality of movable arms are automatically positioned in accordance with the recommendation of the initial position of each of the arms’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the movement of the plurality of robotic arms into the respective starting poses of modified Coste-Maniere such that the movement is automatic as taught by Felder. The motivation to do this yield predictable results such as to help save the user's time since the user need not manually move the arms, as suggested by Felder, [Col 26 l.14-15].
Claims 11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Coste-Maniere et al (US 2007/0293734 A1), in view of Andre ("A Motion Planning and Velocity Collision Avoidance Framework for Bilateral Manipulation")(December 30, 2021) (Year: 2021), in view of Ho et al (US 20190105776 A1), as applied to claim 1 and 12, in further view of Wascher (US 7,311,714 B1)
Claim 11: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere fails to disclose: wherein the imaging data for the patient includes images of the patient after the patient is prepared for the medical procedure.
However, Wascher in the context of medical device placement for surgical procedures discloses, wherein the imaging data for the patient includes images of the patient after the patient is prepared for the medical procedure. ([Col 7. l.59-65], ‘Note that pre-operative imaging 32 typically may take place in a separate room from the surgical procedure and before the patient is prepared and positioned for surgery. However, it is also possible that the pre-operative imaging 32 may take place in the operating room after the patient has been prepared and positioned for surgery therein.’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the imaging data of Coste-Maniere to include images of the patient after the patient is prepared for the medical procedure as taught by Wascher. The motivation to do this yields predictable results such as improving the surgical results by providing the most recent image of the patient, [Col 6] & [Col 8] of Wascher.
Claim 20: Modified Coste-Maniere teaches all the elements above in claim 12, Coste-Maniere fails to disclose: wherein the imaging data for the patient includes images of the patient after the patient is prepared for the medical procedure.
However, Wascher in the context of medical device placement for surgical procedures discloses, wherein the imaging data for the patient includes images of the patient after the patient is prepared for the medical procedure. ([Col 7. l.59-65], ‘Note that pre-operative imaging 32 typically may take place in a separate room from the surgical procedure and before the patient is prepared and positioned for surgery. However, it is also possible that the pre-operative imaging 32 may take place in the operating room after the patient has been prepared and positioned for surgery therein.’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the imaging data of modified Coste-Maniere to include images of the patient after the patient is prepared for the medical procedure as taught by Wascher. The motivation to do this yields predictable results such as improving the surgical results by providing the most recent image of the patient, [Col 6] & [Col 8] of Wascher.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Coste-Maniere et al (US 2007/0293734 A1), in view of Andre ("A Motion Planning and Velocity Collision Avoidance Framework for Bilateral Manipulation")(December 30, 2021) (Year: 2021), in view of Ho et al (US 20190105776 A1), as applied to claim 1, in further view of Blumenkranz et al (US20010013764A1).
Claim 23: Modified Coste-Maniere teaches all the elements above in claim 1, Coste-Maniere fails to disclose, wherein, for each robotic arm of the plurality of robotic arms, the plurality of joints includes a subset of joints that lock in place during the medical procedure, and wherein the respective starting poses include locking positions for the subset of joints.
However, Blumenkranz in the context of positioning linkage for robotic surgery discloses, wherein, for each robotic arm of the plurality of robotic arms, the plurality of joints includes a subset of joints that lock in place during the medical procedure, and wherein the respective starting poses include locking positions for the subset of joints (¶0016, ¶0038-0039, ¶0053, ¶0069, ¶0071, Claims 9-14).
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the plurality of arms of modified Coste-Maniere such that the plurality of joints includes a subset of joints that lock in place during the medical procedure, and wherein the respective starting poses include locking positions for the subset of joints as taught by Blumenkranz. The motivation to do this yield predictable results such as preventing inadvertent movement during the surgical procedure, as suggested by Blumenkranz ¶0053.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nicholas Robinson whose telephone number is (571)272-9019. The examiner can normally be reached M-F 9:00AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.A.R./Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798