Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9, 11-18, and 20-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Roh (US 20190262084 A1), and further in view of Fuerst (US 20210121232 A1).
Regarding claim 1, Roh teaches a computer-implemented method for clinical workplace simulation, the method comprising:
capturing a surgical parameter from one or more prior robotic surgical operations, based on a sensor (par. 0115: “A method of capturing the data from drill mounted “point of view” camera in which the data is uploaded into a historical database as it is collected and refined to improve the operation of the system in future operations.”); and
determining an optimized surgical parameter using a machine learning network trained on the captured surgical parameter, the optimized surgical parameter being determined based on the received at least one of the dimensions or the geometric configuration of the simulated clinical workspace (par. 0144: “Improvements in “planning and mapping” are made such as using a Machine Learning Workflow Optimization for Surgical Robot, using a method of workflow optimization by collecting data throughout a surgical process and then using the collected data for the following workflow. This data can be incorporated into an AI system to provide surgical robot machine learning capabilities on the surgical procedures.”),
wherein the surgical parameter includes at least one of a patient habitus of a previous patient (par. 0040: “The procedure database can include medical records data, images (e.g., pre-and post-surgical images), physician input, sensor data, or the like. The images can include MRI or CAT scans, fluoroscopic images, or other types of images. The sensor data can be collected during procedures, etc. related to all procedures of this type. This database is queried by the surgical control for all medical imaging from the current patient and by the progression module for data for all similar patients who had the same procedure.”), a surgical port location in the previous patient (par. 0064: “the system is applicable to one or more steps in the spinal surgery process, from initial incision, port placement, retractor docking, lamina removal, disc removal and/or hardware insertion.”), or a robotic arm placement relative to the previous patient (par. 0112: “A method of using a hand-held ball-tip probe with sensors located in the robotic arm/surgical tool to determine the position of the ball tip probes location for creating a 3D map of a patient's spine to assist the surgeon during the operation.”), and
wherein the optimized surgical parameter includes at least one of an optimized surgical port placement location in a simulated patient (par. 0064, as above) or an optimized robotic arm placement location relative to the simulated patient (par. 0112, as above).
Roh fails to teach receiving at least one of dimensions or a geometric configuration of a simulated clinical workspace, including importing a computer-aided design (CAD) file representing an operating room.
Roh does teach a method of collecting sensor data from an operating room for use as context data to inform the artificial intelligence system of the robot to enhance the learning capabilities of the robot, which is considered by the examiner to be analogous to “receiving at least one of dimensions or a geometric configuration of a simulated clinical workspace” (par. 0138: “Improvements in “Imaging” are made such as using Motion and Sensor Context Data to Inform AI using a method of collecting sensor data from an operating room (i.e. visual, audio, or environmental data) and collecting motion data from the tool being used to perform the surgery. The sensor data and motion data is used as context data to inform the Artificial Intelligence system of the robot to enhance the learning of the robot”). Based on this, it is the opinion of the examiner that, while Roh fails to explicitly teach a CAD file, it would be obvious to one familiar in the art that a CAD file could serve a similar function of providing context data as required by the invention of Roh. Indeed, environmental data such as that collected by Roh is often stored as a CAD file, as would be appreciated by one familiar in the art.
Fuerst teaches a computer-implemented method for clinical workspace simulation, the method comprising:
capturing a surgical parameter from one or more prior robotic surgical operations, based on a sensor (par. 0044: “A geometry and anatomy of a virtual patient can be determined based on sensor information or surgical robotic movement data, captured from one or more surgical procedures. The surgical procedures can be real surgical procedures or simulated procedures with real patients or dummies. Generating or modifying the virtual patient can include estimating internal volume and/or anatomy of the virtual patient based on sensors such as a) camera images taken internal of a patient, and/or b) surgical robotic tool motions, captured during the one or more surgical procedures.”);
receiving at least one of dimensions or a geometric configuration of a simulated clinical workspace (par. 0030: “Referring to FIG. 3, a system 20 for generating and optimizing a workflow setup is shown. The system can generate a virtual reality (VR) simulation 100 of an operating room and surgical equipment, also described herein as a virtual surgical environment. Such a simulation can include a virtual patient 114, and virtual surgical equipment (including one or more virtual robotic arms 103) representative of the surgical robotic system described in FIG. 1. The virtual surgical robotic equipment and the virtual patient can be arranged in a virtual operating room, to help plan for a surgical robotic procedure (e.g., laparoscopic surgery).”), including importing a computer-aided design (CAD) file representing an operating room (par. 0032: “In one embodiment, the kinetic parameters are defined in robot models 102 (e.g., computer aided drawing (CAD) models or unified robot description format (URDF) models). The kinematics processor can extract these parameters from the models to be used in determining the setup of the virtual surgical robotic arm or the virtual tool in the VR simulation.” NOTE: it is the opinion of the examiner that, if the same system is generating a VR simulation of both an operating room and surgical equipment, it can be assumed that the CAD models used to define the latter could also logically be used to define the former.); and
wherein the surgical parameter includes at least one of a patient habitus of a previous patient, a surgical port location in the previous patient, or a robotic arm placement relative to the previous patient (par. 0026: “Workflow optimization can include determining workspace, tool placement, port locations, and setup of surgical robotic arms, to prepare for a surgical procedure.”), and
wherein the optimized surgical parameter includes at least one of an optimized surgical port placement location in a simulated patient or an optimized robotic arm placement location relative to the simulated patient (par. 0026, as above).
Fuerst fails to teach determining an optimized surgical parameter using a machine learning network trained on the captured surgical parameter, the optimized surgical parameter being determined based on the received at least one of the dimensions or the geometric configuration of the simulated clinical workspace, as Fuerst is not directed to a machine learning network.
Fuerst does teach determining an optimized surgical parameter based on the received at least one of the dimensions or the geometric configuration of the simulated clinical workspace (par. 0030, as above.). Based on this, it is the opinion of the examiner that, while Fuerst fails to explicitly teach a machine learning network, it would be obvious to one familiar in the art that a machine learning network could be used as an alternate means to optimizing the workflow of Fuerst.
It would have been obvious to one familiar in the art prior to the effective filing date of the claimed invention to combine the surgical workflow simulation of Fuerst with the AI guidance system of Roh, as both are in the same fields of endeavor of computer-aided planning, simulation or modelling of surgical operations and ICT specially adapted for mechanical surgeries. AI is commonly used in the medical profession, and would be obviously beneficial in implementing the workflow optimization of Fuerst.
Regarding claim 2, Roh and Fuerst teach the computer-implemented method of claim 1. Fuerst further teaches generating a simulated patient including a simulated patient habitus based on at least one of a first user input or a patient medical record (par. 0037: “In one embodiment, an optimization procedure 110 is performed to arrange the virtual objects (e.g., the patient, the robotic arm and tool, the platform, the user console, the control tower, and other surgical equipment) based on one or more inputs and changes to inputs. For example, inputs can [include] workspace, patient geometry, operating room, procedure type, platform height, etc.”).
Regarding claim 3, Roh and Fuerst teach the computer-implemented method of claim 2. Fuerst further teaches receiving a second user input for modifying the simulated patient habitus (par. 0037: “Changes to inputs can include a modification to a virtual patient, a change to a procedure, a change to the operating room, a change to platform height, and/or any change that would modify a location or size of a workspace in a virtual patient.”); and
modifying the simulated patient habitus based on the user input (as above).
Regarding Claim 4, Roh and Fuerst teach the computer-implemented method of claim 2. Fuerst further teaches generating a setup guide based on the generated simulated patient habitus and the determined optimized surgical parameter (par. 0047: “the user can tailor the workflow and layout optimization to suit a particular patient and procedure type. This can streamline setup of robotic arms for different patients, different procedures, and help selection of operating rooms to facilitate such procedures.”); and
displaying the setup guide on a display (par. 0047: “In one embodiment, the VR simulation 100 can be communicated or output through a display, for example, on a user console 120.”),
wherein the setup guide includes at least one of the optimized surgical port placement location or the optimized robotic arm placement location (par. 0047, as above).
Regarding Claim 5, Roh and Fuerst teach the computer-implemented method of claim 4. Fuerst further teaches wherein generating the setup guide includes:
generating a revised surgical port placement location in a trunk segment of the simulated patient (par. 0027: “A port 312 can be determined based on where the trocar passes through the virtual patient's abdomen 308.”).
Regarding Claim 6, Roh and Fuerst teach the computer-implemented method of claim 5. Fuerst further teaches wherein the revised surgical port placement location is further based on the simulated patient habitus (par. 0027: “A position of a virtual surgical robotic arm 302 can be determined based on the port location, the trocar position, and/or the workspace.”).
Regarding Claim 7, Roh and Fuerst teach the computer-implemented method of claim 4. Fuerst further teaches wherein generating the setup guide includes:
generating a revised optimized robotic arm placement location relative to the simulated patient (par. 0027, as above in claim 6 rejection).
Regarding Claim 8, Roh and Fuerst teach the computer-implemented method of claim 7. Fuerst further teaches wherein the optimized robotic arm placement location is further based on the simulated patient habitus (par. 0027, as above in claim 6 rejection).
Regarding Claim 9, Roh and Fuerst teach the computer-implemented method of claim 7. Fuerst further teaches receiving an indicated anatomy from a user (par. 0047: “The view of the simulation can then be re-generated for the user, thus providing adjustments and optimization of the workflow based on user generated inputs. In this manner, the user can tailor the workflow and layout optimization to suit a particular patient and procedure type. This can streamline setup of robotic arms for different patients, different procedures, and help selection of operating rooms to facilitate such procedures.”),
wherein the optimized robotic arm placement location is further based on targeting the indicated anatomy (par. 0047, as above).
Regarding claim 11, Roh teaches a system for clinical workplace simulation, the system comprising:
a sensor configured to sense a surgical parameter (par. 0118: “As a drill or knife is robotically controlled, the drill or knife would have highly sensitive sensors for (1) RPMs, (2) armature current, (3) angle and direction, (4) sound of motor, etc. These parameters provide real-time robot set of data.”);
a processor (par. 0157: “The computer-readable instructions can be executed by a processor of a mobile unit, a network element, and/or any other computing device.”); and
a memory, including instructions stored thereon, which, when executed by the processor, cause the system to perform the elements of the method of claim 1 taught by Roh (par. 0159: “Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, a memory such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices, such as a touch pad or screen, and/or control systems including feedback loops and control motors (e.g., feedback for sensing position and/or velocity; control motors for moving and/or adjusting components and/or quantities).”).
Fuerst teaches receiving at least one of dimensions or a geometric configuration of a simulated clinical workspace, including importing a computer-aided design (CAD) file representing an operating room (pars. 0030 & 0032, as above in claim 1 rejection).
It would have been obvious to one familiar in the art prior to the effective filing date of the claimed invention to combine the surgical workflow simulation of Fuerst with the AI guidance system of Roh, as both are in the same fields of endeavor of computer-aided planning, simulation or modelling of surgical operations and ICT specially adapted for mechanical surgeries. AI is commonly used in the medical profession, and would be obviously beneficial in implementing the workflow optimization of Fuerst.
Claim 12 is functionally identical to Claim 2, save that it relies on Claim 11 as opposed to Claim 1. As such, it is rejected on the same basis as Claim 2.
Claim 13 is functionally identical to Claim 3, save that it relies on Claim 12 as opposed to Claim 2. As such, it is rejected on the same basis as Claim 3.
Claim 14 is functionally identical to Claim 4, save that it relies on Claim 12 as opposed to Claim 2. As such, it is rejected on the same basis as Claim 4.
Claim 15 is functionally identical to Claim 5, save that it relies on Claim 14 as opposed to Claim 4. As such, it is rejected on the same basis as Claim 5.
Claim 16 is functionally identical to Claim 6, save that it relies on Claim 15 as opposed to Claim 5. As such, it is rejected on the same basis as Claim 6.
Claim 17 is functionally identical to Claim 7, save that it relies on Claim 14 as opposed to Claim 4. As such, it is rejected on the same basis as Claim 7.
Claim 18 is functionally identical to Claim 8, save that it relies on Claim 17 as opposed to Claim 7. As such, it is rejected on the same basis as Claim 8.
Regarding claim 20, Roh teaches a non-transitory computer-readable medium storing instructions which, when executed by a processor, cause the processor to perform a method comprising:
accessing a surgical parameter from one or more prior robotic surgical operations, based on a sensor (par. 0029, as above in claim 1 rejection), wherein the surgical parameter includes at least one of a patient habitus of a previous patient (par. 0040, as above in claim 1 rejection), a surgical port location in the previous patient (par. 0064, as above in claim 1 rejection), or a robotic arm placement relative to the previous patient (par. 0112, as above in claim 1 rejection); and
determining an optimized surgical parameter using a machine learning network trained on the captured surgical parameter (par. 0144, as above in claim 1 rejection), wherein the optimized surgical parameter includes an optimized surgical port placement location in at least one of a simulated patient, or an optimized robotic arm placement location relative to the simulated patient (par. 0029, as above in claim 1 rejection).
Roh fails to teach receiving at least one of dimensions or a geometric configuration of a simulated clinical workspace, including importing a computer-aided design (CAD) file representing an operating room;
generating the simulated patient including a simulated patient habitus based on at least one of a first user input or a patient medical record;
generating a setup guide based on the generated simulated patient habitus and the determined optimized surgical parameter, wherein the setup guide includes at least one of the optimized surgical port placement location or the optimized robotic arm placement location; and
displaying the setup guide on a display.
Fuerst teaches receiving at least one of dimensions or a geometric configuration of a simulated clinical workspace, including importing a computer-aided design (CAD) file representing an operating room (pars. 0030 & 0032, as above in claim 1 rejection);
generating the simulated patient including a simulated patient habitus based on at least one of a first user input or a patient medical record (par. 0037, as above in claim 2 rejection);
generating a setup guide based on the generated simulated patient habitus and the determined optimized surgical parameter, wherein the setup guide includes at least one of the optimized surgical port placement location or the optimized robotic arm placement location (par. 0047, as above in claim 4 rejection); and
displaying the setup guide on a display (par. 0047, as above in claim 4 rejection).
It would have been obvious to one familiar in the art prior to the effective filing date of the claimed invention to combine the surgical workflow simulation of Fuerst with the AI guidance system of Roh, as both are in the same fields of endeavor of computer-aided planning, simulation or modelling of surgical operations and ICT specially adapted for mechanical surgeries. AI is commonly used in the medical profession, and would be obviously beneficial in implementing the workflow optimization of Fuerst.
Regarding claim 21, Roh and Fuerst teach the computer-implemented method of claim 4. Fuerst further teaches wherein displaying the setup guide includes:
displaying an indication of a surgical port placement location as a first distance and a second distance relative to an anatomical feature of the simulated patient (par. 0026: “Workflow optimization can include determining workspace, tool placement, port locations, and setup of surgical robotic arms, to prepare for a surgical procedure.”; par. 0007: “The optimized layout of the virtual surgical equipment can be rendered on a display, which can be used as reference to rearrange the physical surgical equipment in the physical OR.”; par. 0027: “A port 312 can be determined based on where the trocar passes through the virtual patient's abdomen 308. A position of a virtual surgical robotic arm 302 can be determined based on the port location, the trocar position, and/or the workspace.” NOTE: By combining these references, we can see that the optimized workflow, and therefore, the surgical port placement location, is rendered on a display, and is determined relative to a number of points on a virtual patient’s abdomen.), the first distance and the second distance being based on the simulated patient habitus (par. 0055: “The position of the surgical tool can be based on the surgical workspace. For example, as shown in FIG. 2, a position (e.g., location and orientation) of one or more tools 304 can be determined based on the reach 306 of each tool and workspace location 310. One or more port locations 312 can be determined based on the tool position, e.g., where the tool passes through the virtual patient's abdomen 308. Tool position can be determined to provide sufficient movement and reach of a tool in the workspace.”).
Claim(s) 10 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Roh (US 20190262084 A1), Kimura (US 20180036882 A1), and Feurst (US 20210121232 A1) as applied to claim 7 above, and further in view of Meglan (US 11548140 B2).
Regarding Claim 10, Roh and Fuerst teach the computer-implemented method of claim 7. Fuerst further teaches wherein the optimized robotic arm placement location includes a location for a first robotic arm and a second location for a second robotic arm (par. 0029: “In some cases, workflow optimization can determine positions of the trocars, ports and surgical robotic arms so that reach of the two or more robots and corresponding tools overlap in the workspace.”),
wherein the method further comprises:
determining a risk of collision between the first robotic arm and a second robotic arm, based on at least one of the first location, the second location, the optimized surgical port placement location, or the simulated patient habitus (par. 0029: “Optimization, as used herein, can refer to software and hardware supported algorithms that arrange virtual surgical equipment (e.g., workflow and/or layout) based on one or more criteria and inputs described herein such as, but not limited to, location of a workspace, reach and movements in a workspace, convenience, and risk of collisions.”).
Fuerst fails to teach displaying a warning of the risk of collision on the display.
Meglan teaches displaying a warning of the risk of collision on the display (col. 2, lines 43-48: “In yet another aspect of the present disclosure, the system may further include a display. The instructions, when executed, may further cause the system to predict a possible collision with a second robotic arm based on the cross-reference and display an alert, on the display, indicating the possibility of a collision.”).
It would have been obvious to one familiar in the art prior to the effective filing date of the claimed invention to include Meglan’s warning system in Fuerst’s display, as both are in the same field of endeavor of robotic arms to be used in surgery. Such a warning system would alert the user to a potential collision while in the midst of a delicate operation, which would be obviously helpful in avoiding such collisions. Avoiding collisions is stated in Fuerst to be one goal of optimization.
Claim 19 is functionally identical to Claims 9 and 10, save that it is a single claim which relies on Claim 17 as opposed to two claims which each rely on Claim 7. As such, it is rejected on the same basis as Claims 9 and 10.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 11, and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments with respect to claim(s) 21 have been fully considered but they are not persuasive. New claim 21 is dependent on claim 4, and would be objected to even if it were not rejected.
Regarding Applicant’s arguments with respect to claim 21, Examiner asserts that Fuerst teaches “displaying an indication of a surgical port placement location as a first distance and a second distance relative to an anatomical feature of the simulated patient,” as explained above.
To explain in greater detail, the port location of Fuerst is determined based on “where the trocar passes through the virtual patient’s abdomen” implies the referencing of at least two points on the patient’s anatomy; to “pass through” a virtual three-dimensional object implies both an entry point and an exit point.
Examiner concedes applicant’s argument that Fuerst does not explicitly describe expressing a port placement location as distances relative to an anatomical feature of a patient; however, as referenced above in the rejection of claim 21, Fuerst does describe displaying the optimized workflow, and that workflow includes a port placement location. Thus, it is logical to conclude that a port placement location would implicitly be displayed as part of Fuerst’s invention, and displaying such a location in relation to the trocar passing through the patient’s abdomen is an obvious means of displaying this information.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN A BARHAM whose telephone number is (571)272-4338. The examiner can normally be reached Mon-Fri, 8:30am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN ALLEN BARHAM/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
