DETAILED ACTION
This action is responsive to the amendments filed 5/21/2025.
Claims 1-23 are pending.
The prior rejections under 35 U.S.C. § 101 are withdrawn.
The prior rejections under 35 U.S.C. § 103 are maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-7, 10-14 and 16-23 are rejected under 35 U.S.C. 103 as being unpatentable over Fernald, et al., U.S. PGPUB No. 2019/0365252 (“Fernald”), in view of Schroeder, U.S. PGPUB No. 2018/0349519 (“Schroeder”).
With regard to Claim 1, Fernald teaches a computer assisted surgery system comprising
an image capture apparatus, a display, a user interface and circuitry (Fig. 3), wherein the circuitry is configured to:
receive information indicating a predetermined predetermined surgical scenario that may occur during a future predetermined surgical stage and a predetermined predetermined surgical process associated with the predetermined predetermined surgical scenario ([0087]-[0088] describe that the system can identify an event which occurs during a medical procedure, and a response procedure for the event can be initiated. An event can be of a type that is common and may occur during a future predetermined surgical stage, such as a bleed);
obtain an artificial image of the predetermined predetermined surgical scenario; and output the artificial image for display on the display ([0049]-[0050] describe that stored visual information can be used to identify information associated with an event, such as a pool of blood resulting from a bleed. The system can generate a virtual representation of the feedback data, such as by overlaying a representation of the bleed point on a current visual representation);
Schroeder teaches that the artificial image represents what a predetermined surgical scene would look like if the predetermined predetermined surgical scenario occurs during the future predetermined surgical stage; and that the circuitry is configured to receive permission information via the user interface indicating if there is permission for the predetermined predetermined surgical process to be performed automatically without further approval if the predetermined predetermined surgical scenario is determined to occur.
Schroeder teaches at [0009] that a surgeon can view a position and function of a predetermined surgical device being designed, as well as carry out a simulation of surgery on a virtual patient. [0010] describes that while operating, a computer-assisted surgery using robotics can be carried out, where the predetermined surgical robot is able to perform tasks in accordance with the previously authorized design of the device, such as creating a contoured resection as specified by the approval of the specific device shown previously in images, where the robot is programmed to carry out the tasks prior to surgery. [0163]-[0164] further describe prior surgeon authorization of both the device design and predetermined surgical procedures after carrying out a virtual simulation of the implant surgery.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine Fernald with Schroeder. One of skill in the art would have sought the combination, to improve system functioning by giving a surgeon additional control over when a computer-controlled predetermined surgical robot carries out procedures in response to predetermined surgical events.
Claims 20 and 21 recite an apparatus and method, respectively, which are commensurate in scope with the system of Claim 1, and are similarly rejected. Claims 22 and 23 recite a program and medium storing a program to perform the method of Claim 21, and are likewise rejected.
With regard to Claim 2, Fernald teaches that the circuitry is configured to: receive a real image captured by the image capture apparatus; and determine if the real image indicates occurrence of the predetermined predetermined surgical scenario. [0049] describes that the system analyzes current visual information during surgery, in order to identify if an event such as a bleed is taking place.
Fernald, in view of Schroeder teaches that if the real image indicates occurrence of the predetermined predetermined surgical scenario, determine if there is permission for the predetermined surgical process to be performed; and if there is permission for the predetermined predetermined surgical process to be performed, control the predetermined process to be performed. Fernald teaches at [0088] that a predetermined surgical process can be performed in response, such as the system changing camera views and settings on predetermined surgical equipment. Schroeder teaches at [0010] that a robot aiding a surgeon in a computer assisted surgery can start and stop work in response to the surgeon’s control.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine Fernald with Schroeder. One of skill in the art would have sought the combination, to improve system functioning by giving a surgeon additional control over when a computer-controlled predetermined surgical robot carries out procedures in response to predetermined surgical events.
With regard to Claim 3, Fernald teaches the artificial image is obtained using feature visualization of an artificial neural network configured to output information indicating the predetermined predetermined surgical scenario when a real image of the predetermined surgical scenario captured by the image capture apparatus is input to the artificial neural network; and it is determined the real image indicates occurrence of the predetermined surgical scenario when the artificial neural network outputs information indicating the predetermined surgical scenario when the real image is input to the artificial neural network.
Fernald describes at [0046]-[0048] that machine learning algorithms can be used for machine vision to identify features of interest, such as a pool of blood indicating a bleed. [0049]-[0050] describe that feedback information which includes the virtual representation is obtained using machine learning algorithms taught using training sets through an artificial intelligence component, and that the feedback information is generated when the specific scenario such as the pool of blood indicating a bleed has been detected using the machine vision. As artificial neural networks are well-known to those of skill in the art in the field of machine learning and artificial intelligence, Fernald would suggest the use of an artificial network to one of ordinary skill in the art to carry out machine vision and generation of the feedback data.
With regard to Claim 4, Fernald teaches that the predetermined surgical process comprises controlling a predetermined surgical apparatus to perform a predetermined surgical action. [0112] describes that the system is able to remove a clamp or tissue retraction.
With regard to Claim 5, Fernald teaches that the predetermined surgical process comprises adjusting a field of view of the image capture apparatus. [0088] describes that the system can change the field of view or magnification of the camera.
With regard to Claim 6, Fernald teaches that the predetermined surgical scenario is one in which a bodily fluid may collide with the image capture apparatus; and the predetermined surgical process comprises adjusting a position of the image capture apparatus to reduce the risk of the collision. [0088] describes that a field of view for a camera can be changed in response to a bleed; one of skill in the art would understand that a camera field of view adjustment would be made so as not to collide with the blood in order to preserve the view.
With regard to Claim 7, Fernald teaches that the predetermined surgical scenario is one in which there is a different field of view of the image capture apparatus is beneficial; and the predetermined surgical process comprises adjusting the field of view of the image capture apparatus to the different field of view. [0088] describes that a field of view for a camera can be changed in response to a bleed, in order to provide focus on the bleed point.
With regard to Claim 10, Fernald teaches that the predetermined surgical scenario is one in which an item is dropped; and the predetermined surgical process comprises adjusting the field of view of the image capture apparatus to keep the dropped item within the field of view. [0085] describes that an item of interest can be a medical instrument. [0056] describes that the system is used to keep features of interest in the field of view when not in the surgeon’s current area of interest, such as happens if an instrument is dropped.
With regard to Claim 11, Fernald teaches that the predetermined surgical scenario is one in which there is evidence within the field of view of the image capture apparatus of an event not within the field of view; and the predetermined surgical process comprises adjusting the field of view of the image capture apparatus so that the event is within the field of view. [0049] describes that when a bleed event is detected such as by observing a pool of blood, previous frames can be used to determine the source, where [0088] describes adjusting a field of view to capture a bleed. Therefore, in cases where a FOV did not include a source, the source can be located and focused on when a bleed is recognized.
With regard to Claim 12, Fernald teaches that the event is a bleed. [0049] describes that when a bleed event is detected such as by observing a pool of blood, previous frames can be used to determine the source, where [0088] describes adjusting a field of view to capture a bleed. Therefore, in cases where a FOV did not include a source, the source can be located and focused on when a bleed is recognized.
With regard to Claim 13, Fernald teaches that the predetermined surgical scenario is one in which an object occludes the field of view of the image capture apparatus; and the predetermined surgical process comprises adjusting the field of view of the image capture apparatus to avoid the occluding object. [0049] describes that output is generated that highlights a portion of visual information which may be obscured, such as when a source of a bleed is obscured by the pool of blood. The view displayed is adjusted to highlight the hidden source.
With regard to Claim 14, Fernald teaches that the predetermined surgical scenario is one in which a work area approaches a boundary of the field of view of the image capture apparatus; and the predetermined surgical process comprises adjusting the field of view of the image capture apparatus so that the work area remains within the field of view. [0088] describes that the system can respond to an identified event by adjusting a camera field of view to focus on the event. Therefore, in instances where the event occurs near the boundary of the frame, the camera will adjust the view to place the work area within the field of view.
With regard to Claim 16, Fernald teaches that the circuitry is configured to: compare the real image to the artificial image; and perform the predetermined surgical process if a similarity between the real image and artificial image exceeds a predetermined threshold. [0093] describes that possible locations of blood vessels identified by the system can be overlaid during a surgery. Therefore, a surgeon can perform surgery if the virtual image correctly matches the real image.
With regard to Claim 17, Fernald teaches that the predetermined surgical process is one of a plurality of predetermined surgical processes performable if the predetermined surgical scenario is determined to occur; each of the plurality of predetermined surgical processes is associated with a respective level of invasiveness; and each predetermined surgical process other than the predetermined surgical process is given permission to be performed if a level of invasiveness of that other predetermined surgical process is less than or equal to the level of invasiveness of the predetermined surgical process given permission to be performed. [0112] describes that a system can have multiple possible procedures in response to a detected condition. A process can therefore be given permission and performed if determined to be less invasive than another permitted process.
With regard to Claim 18, Fernald teaches that the image capture apparatus is a predetermined surgical camera or medical vision scope. [0035] describes a camera configured to capture video of the predetermined surgical procedure.
With regard to Claim 19, Fernald teaches that the computer assisted surgery system is a computer assisted medical vision scope system, a master-slave system or an open surgery system. [0035] describes that the system includes an optical sensor, which can be an external scope.
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Fernald, in view of Schroeder, and in view of Kornblau, et al., U.S. PGPUB No. 2011/0105895 (“Kornblau”).
With regard to Claim 8, Kornblau teaches that the predetermined surgical scenario is one in which a planned incision is performed; and the different field of view provides an improved view of the performance of the incision. [0074] describes that a tracking system with a movable camera can move to a new position or angle to get a better view of an incision made by a surgeon.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine Kornblau with Fernald and Schroeder. One of skill in the art would have sought the combination, to improve user experience by ensuring an optimal view of every step of a predetermined surgical procedure.
With regard to Claim 9, Kornblau teaches that the predetermined surgical scenario comprises the incision deviating from the planned incision; and the different field of view provides an improved view of the deviation. [0074] describes that a imaging device can move to get a better angle or view of an incision. Therefore, a camera can provide a different field of view wherever an incision is made.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine Kornblau with Fernald and Schroeder. One of skill in the art would have sought the combination, to improve user experience by ensuring an optimal view of every step of a predetermined surgical procedure.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Fernald, in view of Schroeder, and in view of Azizian, et al., U.S. PGPUB No. 2016/0008078 (“Azizian”).
With regard to Claim 15, Azizian teaches that the predetermined surgical scenario is one in which the image capture apparatus may collide with another object; and the predetermined surgical process comprises adjusting a position of the image capture apparatus to reduce the risk of the collision. [0104] describes the movement of an image capture robot arm and another robot arm to ensure that they do not collide with one another.
It would have been obvious to one of ordinary skill in the art at the time this application was filed to combine Azizian with Fernald and Schroeder. One of skill in the art would have sought the combination, to improve user experience by ensuring that various mechanical arms do not collide during surgery, which could cause a safety issue for patients and medical staff.
Response to Arguments
Applicant’s arguments with regard to the rejection under § 101 are moot, as the rejections have been withdrawn. Applicant's arguments regarding the rejections under § 103 have been fully considered but they are not persuasive. Applicant argues with regard to the amended claims that Schroeder does not disclose generating artificial images of specific predetermined scenarios.
However, as described above, Schroeder describes a surgeon designing a device, performing virtual surgery, and approving both the device and surgical procedure. Schroeder describes that a surgical robot can perform a function specific to the procedure and specific approved device, such as creating a contoured resection based on the design. Therefore, a particular approved scenario is carried out using the approval provided by the surgeon after images of the surgery have been created and viewed by the surgeon. The claims remain properly rejected under § 103 because the references continue to teach or suggest each element of the amended claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEITH D BLOOMQUIST whose telephone number is (571)270-7718. The examiner can normally be reached M-F, 8:30-5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEITH D BLOOMQUIST/Primary Examiner, Art Unit 2171
7/15/2025
Prosecution Insights