DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 8 and 12 are objected to because of the following informalities: the claims should be amended to read in part “generate display information including the surgical instrument area”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 8, 12, and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Overmyer (US 2020/0188046) in view of Zhao (US 8,792,963).
As per claims 1, 8, and 12, Overmyer teaches a force sense display device comprising at least one processor configured to at least:
obtain image information from an image-capturing device configured to capture an image of an affected part of a patient (see at least para. 59 for image capture device 310 arranged to capture images of the surgical site 618);
obtain position information on at least one surgical instrument, the position information being received from a surgical assist robot (see at least para. 60 for determining position of end effector using sensors 622; see at least para. 43 for surgical tool 400 includes end effector 404);
obtain force sense information comprising information on a magnitude of and a direction of a force acting on the at least one surgical instrument (see at least para. 62 for determining loading data such as force and direction of force on end effector 404 from sensors 622);
generate a force sense representation based on the force sense information, the force sense representation comprising a force amount representation that represents the magnitude of the force and a direction representation that represents the direction of the force (see at least para. 64 for augmenting visual display with force indicator that provides magnitude and direction of loading);
detect
generate display information including the surgical instrument area with the force sense representation superimposed thereon and output the display information for display on the display (see at least para. 65-66 for force indicator 704 graphically tied to the known reference point).
Overmyer is not explicit regarding detecting a surgical instrument area in an image based on image information and position information.
Zhao teaches detecting a surgical instrument area based on image information and position information (see at least col. 33, ln. 35-42, for using sequence of images and kinematics information, equivalent to positional information, to determine tool tip location within the image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the system of Overmyer with the features of Zhao to provide a tool tracking system that is accurate and reliable by fusing kinematics and visual information compared to using only optical or magnetic devices which may only provide a single clue of the position of a robotic tool.
As per claim 3, Overmyer further teaches wherein the force amount representation comprises a numerical value (see at least Fig. 7-8 and 13 for numerical representation of force).
As per claims, 16-18, Overmyer further teaches wherein the display information including the surgical instrument area with the force sense representation superimposed thereon is displayed on the display.
Claim(s) 2, 9, and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Overmyer (US 2020/0188046) in view of Zhao (US 8,792,963) and further in view of Reiley (“Effects of visual force feedback on robot-assisted surgical task performance”).
As per claim 2, 9, and 13, modified Overmyer is silent regarding, but Reiley teaches wherein the force amount representation comprises at least one of a color type, a color brightness/darkness, or a transparency (see at least Fig. 3 and section Real-Time Visual Force Feedback Graphic Overlays for different colored visual indicators overlayed on the surgical images based on the measured force range at each instrument tip).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the device of modified Overmyer with the features of Reiley because providing visual force feedback using overlays that change color across three discrete force magnitude intervals provides an quick and easy to understand visual indication to avoid applying excessive forces.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Overmyer (US 2020/0188046) in view of Zhao (US 8,792,963) and further in view of Jun (KR 20160079223).
As per claim 4, modified Overmyer is silent regarding, but Jun teaches wherein the force amount representation comprises a character size of the numerical value to be displayed (see at least para. 43 for displaying the measured force numerically but also varying the size of the numbers based on whether the measured force is less than, equal to, or greater than reference forces).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the device of modified Overmyer with the features of Jun because the varying font sizes provide clear and easy to distinguish visual indicators of when the measured forces are less than, equal to, or greater than reference forces.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Overmyer (US 2020/0188046) in view of Zhao (US 8,792,963) and further in view of Dempe (US 2022/0163409).
As per claim 5, modified Overmyer is silent regarding, but Dempte teaches a storage configured to store at least one of a color tone or a pattern of the at least one surgical instrument, wherein the surgical instrument area is detected in the image, based on the at least one of the color tone or the pattern of the at least one surgical instrument stored in the storage (see at least para. 37-39 for registering and tracking fiducial marks).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the device of modified Overmyer with the features of Dempe because providing a fiducial mark associated with a tool allows a system to track the tool as it moves in the environment as it is a unique identifier.
Allowable Subject Matter
Claims 6-7, 10-11, and 14-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT NGUYEN whose telephone number is (571)272-4838. The examiner can normally be reached M-F 8AM - 4PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANNA MOMPER can be reached at (571) 270-5788. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT T NGUYEN/PRIMARY EXAMINER, Art Unit 3619
Prosecution Insights