Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-13 are rejected under 35 U.S.C. 103 as being unpatentable over Shelton, IV et al, U.S. PG Pub 2022/0370138, published May 27, 2021 in view of Walsh et al, U.S. PG Pub 2023/0237370, published February 8, 2022.
With regard to independent claim 1,
Shelton, IV teaches, “A computer-implemented method for training a neural network for actuating a technical device,” (Paragraph 0878-0882, 0894-0897; EN: This denotes the use of a computer-implemented method that trains a neural network, which can give actions to a robot). “comprising: establishing a first data link between the neural network and a first virtual simulation of the technical device” (Paragraph 0890; EN: This denotes that the simulation can help train a neural network by providing additional information in regards to riskier surgical choices). “via a first data interface of the first simulation for reading out status data from the first simulation and transferring control data to the first simulation;” (Paragraph 0116; EN: This denotes the use of an application creation device, which includes a GUI (graphical user interface). This device can communicate with a simulation device to retrieve, modify, and load various application modules, which allows a user to select simulation activities, to input various simulation parameters, to set simulation objectives, and to confirm simulation execution through the GUI). “setting a first training goal for an actuation of the first simulation;” (Paragraph 0460-0461; EN: This denotes that the simulation device can receive and output various surgical tasks results, while the ML model can predict the outcomes of a dataset). “training the neural network based on the first simulation being actuated by the neural network, and checking the training progress of the neural network against the first training goal;” (Paragraph 0460-0461; EN: This denotes that the simulation device, which houses various tasks and activities, can train the ML model and that the training progress of the ML can be checked based on an algorithm that predicts when the model will reach a minimal outcome). “wherein the second simulation is configured to be more realistic than the first simulation and requires more mathematical operations than the first simulation for a respective simulation cycle owing to its higher degree of realism,” (Paragraph 0294; EN: This denotes updating simulations based on live surgery procedures and patient anatomy). “… via a second data interface of the second simulation for reading out status data from the second simulation and transferring control data to the second simulation; and” (Paragraph 0116; EN: This denotes the use of an application creation device, which includes a GUI (graphical user interface). This device can communicate with a simulation device to retrieve, modify, and load various application modules, which allows a user to select simulation activities, to input various simulation parameters, to set simulation objectives, and to confirm simulation execution through the GUI. Because the application creation device can modify a simulation, the first simulation and first interface can be considered the second simulation and second interface due to no longer containing the same original elements). “training the neural network based on the second simulation being actuated by the neural network” (Paragraph 0460; EN: This denotes that the simulation device, which can receive various simulated task, surgical activities, setup information, and activity information, can train the ML model on the results or datasets gained from the simulations).
However, Shelton, IV fails to explicitly disclose “breaking the first data link based on the first training goal having been achieved, and then establishing a second data link between the neural network and a second virtual simulation of the technical device,” and “and wherein the second data link is established…”
Walsh teaches “breaking the first data link based on the first training goal having been achieved, and then establishing a second data link between the neural network and a second virtual simulation of the technical device,” (Paragraph 0059; EN: This denotes recording the state before the occurrence of a specific event and starting a new scenario to teach the agent how to avoid the event). “and wherein the second data link is established…” (Paragraph 0056; EN: This denotes the use of deep reinforcement learning architecture and various representations, which can be deep neural networks. It also links specific scenarios to tables, which are linked to replay buffers and deep neural networks).
Shelton, IV and Walsh are considered to be analogous art to the claimed invention due to the fact that they both disclose the use of simulations, where the simulations learn through the use of reinforcement learning and neural network. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the interactive surgical simulation system of Shelton, IV with agent training through the use of various scenarios of Walsh. One would be motivated to do so to improve simulations, which can be used to train various robots.
With regard to dependent claim 2,
Shelton IV teaches “The method according to claim 1, wherein the first data interface and the second data interface are configured identically” (Paragraph 0116; EN: This denotes the use of an application creation device, which includes a GUI (graphical user interface). This device can communicate with a simulation device to retrieve, modify, and load various application modules, which allows a user to select simulation activities, to input various simulation parameters, to set simulation objectives, and to confirm simulation execution through the GUI. Because the application creation device can modify a simulation, the first interface can be considered the second interface due to no longer containing the same original elements).
With regard to dependent claim 3,
Shelton, IV teaches “The method according to claim 1, further comprising: generating simulated sensor data, which contain information on a virtual environment of the technical device;” (Paragraph 0340-0342; EN: This denotes a simulation system using tracking devices that may include various types of sensors that can measure data related to different biomarkers and may enhance user information for incorporation into simulation operation, training, performance scoring, and the like). “transferring the simulated sensor data to the neural network; and” (Paragraph 0460; EN: This denotes the simulation device, which houses various complex systems and setup information, being used to train a ML model).
“training the neural network to evaluate the simulated sensor data and to take account of the simulated sensor data when actuating the first simulation and/or the second simulation” (Paragraph 0460; EN: This denotes the simulation device, which houses various complex systems and setup information, being used to train a ML model).
With regard to dependent claim 4,
Shelton, IV teaches “The method according to claim 1, wherein the technical device is a robot belonging to at least one of the following categories: a vehicle; a robot arm; a robot for positioning or attaching material and/or objects; a robot for cleaning surfaces; a robot for examining spaces or surfaces; a robot for applying a chemical or a robot for carrying out a medical intervention” (Paragraph 0472-0474; EN: This denotes that a simulation device used to train a ML model may provide a robotic guidance path for real-world medical events).
With regard to dependent claim 5,
Shelton, IV teaches “The method according to claim 4, wherein the first training goal belongs to at least one of the following categories of training goals: a stretch traveled on a virtual training course or a virtual test route; a number of movement patterns carried out without any collisions or within a predefined movement range; a time period within which the neural network actuates the first simulation properly and without any undesired events occurring; or a threshold value being reached for a reward function” (Paragraph 0201-0209; EN: This denotes that the simulator can use both the surgical data system and surgical interaction criteria to take note of triggering events and its corresponding consequence and use that information to modify the simulation based on whether the triggering event has happened or if the simulation has taken a certain amount of time).
With regard to dependent claim 6,
Shelton teaches “The method according to claim 1,wherein the second simulation: takes account of more mechanical and/or electrical components of the technical device than the first simulation; takes account of more degrees of mechanical freedom than the first simulation; simulates physical phenomena and/or laws in a more realistic manner and/or to a higher degree of accuracy than the first simulation; takes account of a larger number of physical forces and/or interactions than the first simulation; and/or has a smaller simulation step size than the first simulation” (Paragraph 0294; EN: This denotes updating simulations based on live surgery procedures and patient anatomy. This can provide new outcomes, choices, and impacts for a surgeon or surgical robot, which aligns with live procedures).
With regard to dependent claim 7,
Shelton teaches “The method according to claim 1, further comprising: setting a second training goal for an actuation of the second simulation;” (Paragraph 0294; EN: This denotes updating simulations based on live surgery procedures and patient anatomy. This can provide new outcomes, choices, and impacts for a surgeon or surgical robot, which aligns with live procedures). “during the actuation of the second simulation, checking the training progress of the neural network against the second training goal;” (Paragraph 0461; EN: This denotes that the training progress of the ML can be checked based on an algorithm that predicts when the model will reach a minimal outcome).
However, Shelton, IV fails to explicitly teach “establishing a physical data link between the neural network and the technical device based on the second training goal having been achieved; and” and “the neural network actuating the technical device”.
Walsh teaches “establishing a physical data link between the neural network and the technical device based on the second training goal having been achieved; and” (Paragraph 0046; EN: This denotes that the actions of a physical agent include being able to control physical or simulated servos or actuators). “the neural network actuating the technical device” (Paragraph 0056; EN: This denotes the use of trainers, which refine the various models).
With regard to independent claim 8,
Shelton teaches “… a virtual training environment for training a neural network for actuating a technical device,” (Paragraph 0472; EN: This denotes the use of a simulation device to train a ML model, which may provide a robotic guidance path). “the virtual training environment comprising: a first simulation of the technical device having a first data interface for reading out status data from the first simulation and transferring control data to the first simulation;” (Paragraph 0116; EN: This denotes the use of an application creation device, which includes a GUI (graphical user interface). This device can communicate with a simulation device to retrieve, modify, and load various application modules, which allows a user to select simulation activities, to input various simulation parameters, to set simulation objectives, and to confirm simulation execution through the GUI). “a programming interface” (Paragraph 0116; EN: This denotes the use of an application creation device, which includes a GUI (graphical user interface)). “for establishing a first data link between the neural network and the first simulation;” (Paragraph 0890; EN: This denotes that the simulation can help train a neural network by providing additional information in regards to riskier surgical choices). “a training functionality configured to set a training goal for the first simulation to be actuated by the neural network and to check the training progress of the neural network against the training goal; and” (Paragraph 0460-0461; EN: This denotes that the simulation device, which houses various tasks and activities, can train the ML model and that the training progress of the ML can be checked based on an algorithm that predicts when the model will reach a minimal outcome). “a second simulation of the technical device, wherein the second simulation is configured to be more realistic than the first simulation and requires more mathematical operations than the first simulation for a respective simulation cycle owing to its higher degree of realism,” (Paragraph 0294; EN: This denotes updating simulations based on surgery procedures and patient anatomy). “and wherein the second simulation comprises a second data interface for reading out status data from the second simulation and transferring control data to the second simulation;” (Paragraph 0116; EN: This denotes the use of an application creation device, which includes a GUI (graphical user interface). This device can communicate with a simulation device to retrieve, modify, and load various application modules, which allows a user to select simulation activities, to input various simulation parameters, to set simulation objectives, and to confirm simulation execution through the GUI. Because the application creation device can modify a simulation, the first interface can be considered the second interface due to no longer containing the same original elements). “and train the neural network based on the second simulation being actuated by the neural network” (Paragraph 0460; EN: This denotes that the simulation device, which can receive various simulated task, surgical activities, setup information, and activity information, can train the ML model on the results or datasets gained from the simulations).
However, Shelton, IV fails to explicitly disclose “A system, comprising: at least one processor;”, “and at least one memory having instructions stored thereon; ”, “wherein the at least one processor is configured to execute the instructions stored on the at least one memory to provide…”, “wherein the training functionality is configured to: break the first data link based on the training goal having been achieved;”, and “replace the first data link with a second data link between the neural network and the second simulation”.
Walsh teaches “A system, comprising: at least one processor;” (Paragraph 0037; EN: This denotes the use of a computer or computing device, which utilizes a processor). “and at least one memory having instructions stored thereon;” (Paragraph 0037; EN: This denotes the use of memory, which holds instructions). “wherein the at least one processor is configured to execute the instructions stored on the at least one memory to provide…” (Paragraph 0037; EN: This denotes a processor executing instructions). “wherein the training functionality is configured to: break the first data link based on the training goal having been achieved;” (Paragraph 0059; EN: This denotes recording the state before the occurrence of a specific event and starting a new scenario to teach the agent how to avoid the event). “replace the first data link with a second data link between the neural network and the second simulation;” (Paragraph 0059; EN: This denotes recording the state before the occurrence of a specific event and starting a new scenario to teach the agent how to avoid the event).
Shelton, IV and Walsh are considered to be analogous art to the claimed invention due to the fact that they both disclose the use of simulations, where the simulations learn through the use of reinforcement learning and neural network. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the interactive surgical simulation system of Shelton, IV with agent training through the use of various scenarios of Walsh. One would be motivated to do so to improve simulations, which can be used to train various robots.
With regard to dependent claim 9,
This claim is similar in scope to claim 2 and is rejected under a similar rationale.
With regard to dependent claim 10,
This claim is similar in scope to claim 3 and is rejected under a similar rationale.
With regard to dependent claim 11,
This claim is similar in scope to claim 4 and is rejected under a similar rationale.
With regard to dependent claim 12,
This claim is similar in scope to claim 5 and is rejected under a similar rationale.
With regard to dependent claim 13,
This claim is similar in scope to claim 6 and is rejected under a similar rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SETH CAPRIANO-UMARI SHELTON whose telephone number is (571)270-0213. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SETH CAPRIANO-UMARI SHELTON/Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights