Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office action is in response to the amendment filed on 03/28/2026. Claims 10-22 are currently pending with claim 22 being amended, and claims 23-24 being cancelled.
Response to Amendment
The amendments to the claims submitted on 03/28/2026 have been reviewed and accepted and they overcome the claim objections set forth in the previous Office action except for those set forth in the claim objection section.
Response to Arguments
Applicant's arguments filed 03/28/2026 have been fully considered but they are not persuasive. The Applicant asserts that the cited prior art does not teach “a restoring force component” which is “a force that actuates the actuator in an opposite direction”. The currently provided claim language does not require the restoring force to actuate the actuator in an opposite direction. The claim requires that the restoring force “counteract an actuation” which may be interpreted under the broadest reasonable interpretation to be a force feedback against actuation in a direction in order to simulate the resistive force of the tool interacting with the environment. The specification, specifically paragraph 0022 indicates that the restoring force may be a force that simulates contact with an obstacle to provide the operator haptic feedback in order to "feel" the obstacle or external force on the system. The Applicant further asserts that the concept of an untethered user input device to provide pressure feedback to a user is not known in the art. In order to demonstrate this concept, the Examiner has cited Kurtz et al. (US 20080167662 A1) as additional relevant prior art. Kurtz demonstrates that an “untethered” user input device may provide tactile feedback to a user in the form of pressure by receiving signals from the robotic control system. This suggests that “untethered” user input devices are known to be used to provide haptic feedback in the form of force against movement to an operator.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Regarding claim 20, “means for specifying” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “means”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0080) as a processor.
Regarding claim 20, “means for commanding a target pose” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “means”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0080) as a processor.
Regarding claim 20, “means for commanding a target force” will be interpreted under 112(f) because of the following three-prong analysis:
Prong 1: The claim uses the nonce term “means”.
Prong 2: The claim uses functional language to modify the nonce term.
Prong 3: Sufficient structure for performing the function is not recited within the claim.
This limitation is being interpreted according to the specification (paragraph 0080) as a processor.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 10-12, 16, and 20-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Freiin von Kapri et al. (US 20240268900 A1), hereinafter Freiin von Kapri in view of Verner et al. (US 20190015169 A1), hereinafter Verner.
Regarding claim 10, Freiin von Kapri teaches:
10. (CURRENTLY AMENDED) A method for controlling a telerobotic robot using an input device … the method comprising:
commanding movement of the telerobotic robot to a target pose with the input device (Paragraph 0005, "The surgeon may provide input commands to the surgical robotic system, and one or more processors of the surgical robotic system can control system components in response to the input commands. For example, the surgeon may hold in her hand a user input device that she manipulates to generate control signals to cause motion of the surgical robotic system components, e.g., an actuator, a robotic arm, and/or a surgical tool of the robotic system." and Paragraph 0044, "In one embodiment, the remote operator 109 holds and moves the UID 114 to provide an input command to move a robot arm actuator 117 in the robotic system 100. The UID 114 may be communicatively coupled to the rest of the robotic system 100, e.g., via a console computer system 116. Representatively, in some embodiments, UID 114 may be a portable handheld user input device or controller that is ungrounded with respect to another component of the surgical robotic system. For example, UID 114 may be ungrounded while either tethered or untethered from the user console. The term “ungrounded” is intended to refer to implementations where, for example, both UIDs are neither mechanically nor kinematically constrained with respect to the user console. For example, a user may hold a UID 114 in a hand and move freely to any possible position and orientation within a workspace, only limited by, for example, a predetermined three-dimensional surgical workspace limit or boundary recognized by the system 100. Representatively, the system may include a tracking mechanism that tracks the location of the UID 114 within, and relative to, the surgical workspace limit or boundary. The UID 114 can generate spatial state signals corresponding to movement of the UID 114, e.g., position and orientation of the handheld housing of the UID, and the spatial state signals may be input signals to control a motion of the robot arm actuator 117. The signals (e.g., tracking sensor signals, clutch signals or engage/disengage teleoperation mode signals) may be wirelessly communicated between UID 114 and the computer system 116. In addition, a power source, such as a rechargeable battery, may be stored within the housing of UID 114 so that it does not need to be mechanically connected to a power source, such as by a wire or cable. The robotic system 100 may use control signals derived from the spatial state signals, to control proportional motion of the actuator 117. In one embodiment, a console processor of the console computer system 116 receives the spatial state signals and generates the corresponding control signals. Based on these control signals, which control how the actuator 117 is energized to move a segment or link of the arm 104, the movement of a corresponding surgical tool that is attached to the arm may mimic the movement of the UID 114. Similarly, interaction between the remote operator 109 and the UID 114 can generate for example a grip control signal that causes a jaw of a grasper of the surgical tool 107 to close and grip the tissue of patient 106.") … and
…
wherein at least one virtual border is specified between a permissible region and an impermissible region for the telerobotic robot, (Paragraph 0008, "Representatively, in one aspect, a method of determining a location of a user input device of a surgical robotic system within a surgical workspace using a virtual workspace includes determining, by one or more processors communicatively coupled to a user input device, that a user is engaging with the user input device within a surgical workspace; in response to determining the user is engaging with the user input device, displaying a virtual user input device within a first virtual workspace boundary, the first virtual workspace boundary representing a first workspace limit within which the user input device is operable to control a surgical robotic instrument in a teleoperation mode, wherein at least a portion of the first virtual workspace boundary is operable to move in response to a movement of the user input device; displaying a second virtual workspace boundary that represents a second workspace limit beyond which the user input device is inoperable to control the surgical robotic instrument in the teleoperation mode; and determining, by one or more processors, a location of the user input device within the surgical workspace based on a proximity of the portion of the first virtual workspace boundary relative to the second virtual workspace boundary. In some aspects, the first virtual workspace boundary comprises a first three dimensional shape and the second virtual workspace boundary comprises a second three dimensional shape that encompasses the first three dimensional shape. In some aspects, an area between the first three dimensional shape and the second three dimensional shape defines a warning zone that indicates the user input device is nearing the second workspace limit. In still further aspects, the first virtual workspace boundary comprises a cube. The portion of the first virtual workspace boundary operable to move may be a side wall of the cube. In some aspects, determining the location may include detecting that the portion of the first virtual workspace boundary has moved closer to the second virtual workspace boundary, and the method may further include in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary. Determining the location may include detecting that the portion of the first virtual workspace boundary intersects with the second virtual workspace boundary, and the method further includes in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary or the second virtual workspace boundary. The feedback may include an audible alert or haptic feedback. The user input device may be an ungrounded user input device." as well as Paragraph 0073, "Accordingly, the user would understand this to mean that the UID in the user's right hand is in the warning zone, but is still closer to the operable workspace than the workspace limit or boundary beyond which operation of the associated surgical component is not allowed.") … starting from the virtual border; (Paragraph 0066, "In other aspects, the indicator 808 may, in addition to indicating to the user that the UID is in a warning zone, indicate to the user which UID is within the warning zone and/or which boundary of the warning zone the UID is closest to. For example, FIG. 8B illustrates an indictor 810A that indicates to the surgeon which UID is approaching a workspace boundary or limit. Representatively, in FIG. 8B, display screen 802 is shown having a left side panel 802A and a right side panel 802B. The left side panel 802A may display information relating to a left UID (e.g., a UID held in the left hand), and the right side panel 802B may display information relating to a right UID (e.g., a UID held in the right hand). Accordingly, when the right UID is detected within the warning zone, an indicator 810A will appear in the right side panel 802B, as shown in FIG. 8B. In this aspect, the indicator 810A is shown as a line along the right side panel 802B. The indicator 810A could, however, be any type of shape, image, icon or the like that can be viewed by the user in the right side panel 802B. The surgeon will therefore understand that the UID held in his or her right hand is in the warning zone and nearing the workspace limit. Similarly, although not shown, if the left UID is detected in the warning zone, a similar warning indicator may appear in the left side panel 802B. The indicator 810A may appear in the center of the left or right panels 802A-B, or in any other region of the left or right panels 802A-B that is easily viewed and understood by the surgeon to mean a left and/or right UID is within the warning zone. In addition, in some aspects, the display may also include an indicator 810B including text which also informs the surgeon that the UID is approaching the workspace limit and/or which side of the workspace the UID is near. For example, the indicator 810B may also appear in the right side panel 802B to indicate that the UID is nearing the right boundary of the workspace, and also include text stating that the UID is “Approaching the workspace limit.” Although both indicators 810A, 810B are shown in FIG. 8B, it is contemplated that in some aspects, only one of indicators 810A, 810B may be displayed to indicate that a UID is approaching a workspace boundary or limit (i.e., in the warning zone), and which UID (right or left) is approaching the workspace boundary or limit." as well as Paragraph 0064, "The feedback may be any one or more of the previously discussed feedback mechanisms, including a visual feedback, haptic feedback or audio feedback, or a combination of any one or more of a visual, haptic or audio feedback.")
…
Freiin von Kapri does not specifically discuss there being an actuator in the user input device to assist in providing haptic feedback to the user or teach using a restoring force to counteract the commanded movement when the operator attempts to control the robot to move outside of the operating envelope. However, Verner, in the same field of endeavor of robotics, teaches:
… which comprises a movable actuator, (Paragraph 0018, “The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”) … based on a detected position of the actuator; … commanding a target force of the actuator; (Paragraph 0030, “FIGS. 3A and 3B shows block diagram of a surgical system 300 incorporating haptic feedback at an input device 330 and means for providing a consistent haptic experience for the user as described above with respect to FIGS. 1, 2A, and 2B. Surgical system 300 includes an instrument 310 for performing a surgical task (e.g., forceps, cutter, retractor, vessel sealer, needle driver, catheter, etc.), an input device 330 (e.g., a lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input) for receiving inputs from a user (e.g., surgeon), and a controller 320 for receiving input instructions from input device 330, controlling the actions of instrument 310 accordingly via a manipulation structure 313, and providing instructions to a haptic feedback actuation mechanism 340 to provide haptic feedback to input device 330 according to a desired haptic feedback profile. In various embodiments, manipulation structure 313 can include any number of systems and structures for maneuvering, positioning, actuating, or otherwise controlling the behavior of instrument 310, including a robotic arm(s)/manipulator(s), set up structure(s), and/or positioning element(s) such as a boom(s) or cart(s), among others. Controller 320 can include any combination of hardware, software, firmware, and other modalities for generating, managing, controlling, and effecting the actions described herein. In various embodiments, controller 320 can be integrated with instrument 310, input device 330, and/or discrete control hardware (e.g., a standalone processing unit or computing platform).”) … and the target force comprises a restoring force component … the restoring force component counteracting an actuation of the actuator for commanding a movement of the telerobotic robot away from the border in the direction of the impermissible region. (Paragraphs 0017-0018, “FIG. 1 shows an exemplary method for providing directionally consistent haptic feedback when actuator output limits are exceeded. In a PROVIDE HAPTIC FEEDBACK step 110, a surgical system that allows a user (e.g., surgeon) to control a surgical instrument (and/or other elements of the surgical system, such as a robotic arm, set up structure, or positioning element such as a boom or cart) via an input device(s) (e.g., lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input), and then provides force feedback to that input device based on a desired haptic feedback profile (a set of one or more haptic feedback effects that at least partially reproduce or represent the physical experience of a real or virtual/modeled interaction). The haptic feedback profile can be based on any haptic model input, such as sensed forces at the instrument (e.g., tissue or other instrument interactions) or robotic arm (e.g., arm collisions with structures or staff), user guidance (e.g., haptic detents, fences, or other profiles to provide guidance for the user to move the input device(s) along a desired path or trajectory), and user interface (UI) elements (e.g., presenting a virtual handle or steering wheel to the user). This haptic feedback can be anything from direct replication of the haptic feedback profile, to scaling of the haptic feedback profile, to applying a non-linear modification of the haptic feedback profile, or any other transformation (e.g., force scaling that varies depending on one or more other factors such as instrument state/speed, viewing magnification, etc.).
The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods using haptic feedback as taught by Freiin von Kapri with the ability to use directional force feedback as a communication method between the operator and the system as taught by Verner. This would ensure that the operator does not move outside of the determined workspace and allows them to learn the workspace safely and efficiently without causing damage by leaving the desired workspace region.
Regarding claim 11, where all the limitations of claim 10 are discussed above, Freiin von Kapri further teaches:
11. (PREVIOUSLY PRESENTED) The method of claim 10, wherein the steps of the method are repeated at least once, in sequence. (Paragraph 0064, "The process 700 therefore continues to monitor the UID location within the workspace and can detect if the UID is in a warning zone of the virtual workspace at operation 706. If the system detects that the UID is in a warning zone, and therefore nearing the workspace limit, the system may provide feedback to the user about a location of the UID at operation 708. Representatively, the feedback may be that a UID is in the warning zone, a distance of a UID to the workspace limit when within the warning zone, whether the UID nearing the workspace limit is a left or right UID, which edge or side of the workspace limit a UID is closest to, or which tool associated with the UID location is affected. This information about the UID relative to the workspace may be determined using the proximity information previously discussed in reference to FIGS. 4A-5B. The feedback may be any one or more of the previously discussed feedback mechanisms, including a visual feedback, haptic feedback or audio feedback, or a combination of any one or more of a visual, haptic or audio feedback.")
Regarding claim 12, where all the limitations of claim 10 are discussed above, Freiin von Kapri does not specifically teach simulating the contact of the robot with an obstacle. However, Verner, in the same field of endeavor of robotics, teaches:
12. (PREVIOUSLY PRESENTED) The method of claim 10, wherein the restoring force component simulates contact of the telerobotic robot with an obstacle. (Paragraph 0017, " FIG. 1 shows an exemplary method for providing directionally consistent haptic feedback when actuator output limits are exceeded. In a PROVIDE HAPTIC FEEDBACK step 110, a surgical system that allows a user (e.g., surgeon) to control a surgical instrument (and/or other elements of the surgical system, such as a robotic arm, set up structure, or positioning element such as a boom or cart) via an input device(s) (e.g., lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input), and then provides force feedback to that input device based on a desired haptic feedback profile (a set of one or more haptic feedback effects that at least partially reproduce or represent the physical experience of a real or virtual/modeled interaction). The haptic feedback profile can be based on any haptic model input, such as sensed forces at the instrument (e.g., tissue or other instrument interactions) or robotic arm (e.g., arm collisions with structures or staff), user guidance (e.g., haptic detents, fences, or other profiles to provide guidance for the user to move the input device(s) along a desired path or trajectory), and user interface (UI) elements (e.g., presenting a virtual handle or steering wheel to the user). This haptic feedback can be anything from direct replication of the haptic feedback profile, to scaling of the haptic feedback profile, to applying a non-linear modification of the haptic feedback profile, or any other transformation (e.g., force scaling that varies depending on one or more other factors such as instrument state/speed, viewing magnification, etc.).")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Freiin von Kapri with the ability to use force feedback as a communication method between the operator and the system as taught by Verner. This would ensure that the operator has an accurate grasp on the environment and does not move outside of the determined workspace and allows them to learn the workspace safely and efficiently without causing damage by leaving the desired workspace region.
Regarding claim 16, where all the limitations of claim 10 are discussed above, Freiin von Kapri further teaches:
16. (PREVIOUSLY PRESENTED) The method of claim 10, wherein the at least one virtual border (Paragraph 0008, "Representatively, in one aspect, a method of determining a location of a user input device of a surgical robotic system within a surgical workspace using a virtual workspace includes determining, by one or more processors communicatively coupled to a user input device, that a user is engaging with the user input device within a surgical workspace; in response to determining the user is engaging with the user input device, displaying a virtual user input device within a first virtual workspace boundary, the first virtual workspace boundary representing a first workspace limit within which the user input device is operable to control a surgical robotic instrument in a teleoperation mode, wherein at least a portion of the first virtual workspace boundary is operable to move in response to a movement of the user input device; displaying a second virtual workspace boundary that represents a second workspace limit beyond which the user input device is inoperable to control the surgical robotic instrument in the teleoperation mode; and determining, by one or more processors, a location of the user input device within the surgical workspace based on a proximity of the portion of the first virtual workspace boundary relative to the second virtual workspace boundary. In some aspects, the first virtual workspace boundary comprises a first three dimensional shape and the second virtual workspace boundary comprises a second three dimensional shape that encompasses the first three dimensional shape. In some aspects, an area between the first three dimensional shape and the second three dimensional shape defines a warning zone that indicates the user input device is nearing the second workspace limit. In still further aspects, the first virtual workspace boundary comprises a cube. The portion of the first virtual workspace boundary operable to move may be a side wall of the cube. In some aspects, determining the location may include detecting that the portion of the first virtual workspace boundary has moved closer to the second virtual workspace boundary, and the method may further include in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary. Determining the location may include detecting that the portion of the first virtual workspace boundary intersects with the second virtual workspace boundary, and the method further includes in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary or the second virtual workspace boundary. The feedback may include an audible alert or haptic feedback. The user input device may be an ungrounded user input device." as well as Paragraph 0073, "Accordingly, the user would understand this to mean that the UID in the user's right hand is in the warning zone, but is still closer to the operable workspace than the workspace limit or boundary beyond which operation of the associated surgical component is not allowed.") …
or as a virtual stop of at least one joint of the telerobotic robot. (Paragraph 0054, “To indicate to the user the workspace limit, boundary or region beyond which control of an associated surgical component by the UID is no longer allowed (e.g., the UID is too far outside of the workspace), the training operation also provides the user with a second virtual workspace boundary at operation 308. The second virtual workspace boundary may be displayed to the user and provide a visual representation of an area within which the UID must be manipulated in order to operate an associated surgical component (e.g., a surgical tool). If the movement of the UID exceeds the second virtual workspace boundary, the UID is considered to have exceeded the workspace limit or boundary and is prevented from controlling an associated surgical component by the system. In some cases, the teleoperation mode is disengaged in response to detecting that the workspace boundary or limit has been met or exceeded by the UID. In this aspect, the second virtual workspace boundary may be displayed to the user as a boundary or limit that is outside of the first virtual workspace boundary. For example, in some aspects, the first virtual workspace boundary and the second virtual workspace boundary may be displayed to the user as three-dimensional shapes. The three-dimensional shape of the second virtual workspace boundary may be larger (e.g., define a larger area) than that of the first virtual workspace boundary. In this aspect, the three-dimensional shapes may be displayed to the user as one inside of the other. For example, the three-dimensional shape of the first virtual workspace boundary may displayed to the user as located within, or otherwise encompassed by, the three-dimensional shape of the second virtual workspace boundary. In this aspect, the area within (or at) the boundary of the inner three-dimensional shape (e.g., the first virtual workspace) may be understood by the user as the desired area of operation of the UID. The area or zone between the inner three-dimensional boundary (e.g., the first virtual workspace boundary) and the outer three-dimensional boundary (e.g., the second virtual workspace boundary) may be understood by the user as the warning zone or area indicating the user is approaching the operation limit of the UID. The area beyond (or at) the outer three-dimensional boundary (e.g., the second virtual workspace boundary) may be understood by the user as an area in which the UID has exceeded its operational limit and is therefore no longer able to control an associated surgical component. Accordingly, when the user moves the real UID, and this movement is shown as the virtual UID exceeding the inner three-dimensional boundary, approaching the outer three-dimensional boundary, or exceeding the outer three-dimensional boundary, the user begins to recognize and learn the operational boundaries of the real workspace.”)
Freiin von Kapri does not specifically teach the boundary being a virtual wall or virtual stop. However, Verner, in the same field of endeavor of robotic control, teaches:
… is specified as a virtual wall in a working space of the telerobotic robot (Paragraph 0017, " FIG. 1 shows an exemplary method for providing directionally consistent haptic feedback when actuator output limits are exceeded. In a PROVIDE HAPTIC FEEDBACK step 110, a surgical system that allows a user (e.g., surgeon) to control a surgical instrument (and/or other elements of the surgical system, such as a robotic arm, set up structure, or positioning element such as a boom or cart) via an input device(s) (e.g., lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input), and then provides force feedback to that input device based on a desired haptic feedback profile (a set of one or more haptic feedback effects that at least partially reproduce or represent the physical experience of a real or virtual/modeled interaction). The haptic feedback profile can be based on any haptic model input, such as sensed forces at the instrument (e.g., tissue or other instrument interactions) or robotic arm (e.g., arm collisions with structures or staff), user guidance (e.g., haptic detents, fences, or other profiles to provide guidance for the user to move the input device(s) along a desired path or trajectory), and user interface (UI) elements (e.g., presenting a virtual handle or steering wheel to the user). This haptic feedback can be anything from direct replication of the haptic feedback profile, to scaling of the haptic feedback profile, to applying a non-linear modification of the haptic feedback profile, or any other transformation (e.g., force scaling that varies depending on one or more other factors such as instrument state/speed, viewing magnification, etc.).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Freiin von Kapri with the ability to use force feedback as a communication method between the operator and the system as taught by Verner. This would ensure that the operator does not move outside of the determined workspace and allows them to learn the workspace safely and efficiently without causing damage by leaving the desired workspace region.
Regarding claim 20, Freiin von Kapri further teaches:
20. (PREVIOUSLY PRESENTED) A system for controlling a telerobotic robot using an input device … the system comprising an input device controller that comprises:
means for specifying at least one virtual border between a permissible and an impermissible region for the telerobotic robot; (Paragraph 0008, "Representatively, in one aspect, a method of determining a location of a user input device of a surgical robotic system within a surgical workspace using a virtual workspace includes determining, by one or more processors communicatively coupled to a user input device, that a user is engaging with the user input device within a surgical workspace; in response to determining the user is engaging with the user input device, displaying a virtual user input device within a first virtual workspace boundary, the first virtual workspace boundary representing a first workspace limit within which the user input device is operable to control a surgical robotic instrument in a teleoperation mode, wherein at least a portion of the first virtual workspace boundary is operable to move in response to a movement of the user input device; displaying a second virtual workspace boundary that represents a second workspace limit beyond which the user input device is inoperable to control the surgical robotic instrument in the teleoperation mode; and determining, by one or more processors, a location of the user input device within the surgical workspace based on a proximity of the portion of the first virtual workspace boundary relative to the second virtual workspace boundary. In some aspects, the first virtual workspace boundary comprises a first three dimensional shape and the second virtual workspace boundary comprises a second three dimensional shape that encompasses the first three dimensional shape. In some aspects, an area between the first three dimensional shape and the second three dimensional shape defines a warning zone that indicates the user input device is nearing the second workspace limit. In still further aspects, the first virtual workspace boundary comprises a cube. The portion of the first virtual workspace boundary operable to move may be a side wall of the cube. In some aspects, determining the location may include detecting that the portion of the first virtual workspace boundary has moved closer to the second virtual workspace boundary, and the method may further include in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary. Determining the location may include detecting that the portion of the first virtual workspace boundary intersects with the second virtual workspace boundary, and the method further includes in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary or the second virtual workspace boundary. The feedback may include an audible alert or haptic feedback. The user input device may be an ungrounded user input device." as well as Paragraph 0073, "Accordingly, the user would understand this to mean that the UID in the user's right hand is in the warning zone, but is still closer to the operable workspace than the workspace limit or boundary beyond which operation of the associated surgical component is not allowed.")
means for commanding a target pose of the telerobotic robot (Paragraph 0005, "The surgeon may provide input commands to the surgical robotic system, and one or more processors of the surgical robotic system can control system components in response to the input commands. For example, the surgeon may hold in her hand a user input device that she manipulates to generate control signals to cause motion of the surgical robotic system components, e.g., an actuator, a robotic arm, and/or a surgical tool of the robotic system." as well as Paragraph 0044, "In one embodiment, the remote operator 109 holds and moves the UID 114 to provide an input command to move a robot arm actuator 117 in the robotic system 100. The UID 114 may be communicatively coupled to the rest of the robotic system 100, e.g., via a console computer system 116. Representatively, in some embodiments, UID 114 may be a portable handheld user input device or controller that is ungrounded with respect to another component of the surgical robotic system. For example, UID 114 may be ungrounded while either tethered or untethered from the user console. The term “ungrounded” is intended to refer to implementations where, for example, both UIDs are neither mechanically nor kinematically constrained with respect to the user console. For example, a user may hold a UID 114 in a hand and move freely to any possible position and orientation within a workspace, only limited by, for example, a predetermined three-dimensional surgical workspace limit or boundary recognized by the system 100. Representatively, the system may include a tracking mechanism that tracks the location of the UID 114 within, and relative to, the surgical workspace limit or boundary. The UID 114 can generate spatial state signals corresponding to movement of the UID 114, e.g., position and orientation of the handheld housing of the UID, and the spatial state signals may be input signals to control a motion of the robot arm actuator 117. The signals (e.g., tracking sensor signals, clutch signals or engage/disengage teleoperation mode signals) may be wirelessly communicated between UID 114 and the computer system 116. In addition, a power source, such as a rechargeable battery, may be stored within the housing of UID 114 so that it does not need to be mechanically connected to a power source, such as by a wire or cable. The robotic system 100 may use control signals derived from the spatial state signals, to control proportional motion of the actuator 117. In one embodiment, a console processor of the console computer system 116 receives the spatial state signals and generates the corresponding control signals. Based on these control signals, which control how the actuator 117 is energized to move a segment or link of the arm 104, the movement of a corresponding surgical tool that is attached to the arm may mimic the movement of the UID 114. Similarly, interaction between the remote operator 109 and the UID 114 can generate for example a grip control signal that causes a jaw of a grasper of the surgical tool 107 to close and grip the tissue of patient 106.") … and
…
wherein … starting from the virtual border, (Paragraph 0066, "In other aspects, the indicator 808 may, in addition to indicating to the user that the UID is in a warning zone, indicate to the user which UID is within the warning zone and/or which boundary of the warning zone the UID is closest to. For example, FIG. 8B illustrates an indictor 810A that indicates to the surgeon which UID is approaching a workspace boundary or limit. Representatively, in FIG. 8B, display screen 802 is shown having a left side panel 802A and a right side panel 802B. The left side panel 802A may display information relating to a left UID (e.g., a UID held in the left hand), and the right side panel 802B may display information relating to a right UID (e.g., a UID held in the right hand). Accordingly, when the right UID is detected within the warning zone, an indicator 810A will appear in the right side panel 802B, as shown in FIG. 8B. In this aspect, the indicator 810A is shown as a line along the right side panel 802B. The indicator 810A could, however, be any type of shape, image, icon or the like that can be viewed by the user in the right side panel 802B. The surgeon will therefore understand that the UID held in his or her right hand is in the warning zone and nearing the workspace limit. Similarly, although not shown, if the left UID is detected in the warning zone, a similar warning indicator may appear in the left side panel 802B. The indicator 810A may appear in the center of the left or right panels 802A-B, or in any other region of the left or right panels 802A-B that is easily viewed and understood by the surgeon to mean a left and/or right UID is within the warning zone. In addition, in some aspects, the display may also include an indicator 810B including text which also informs the surgeon that the UID is approaching the workspace limit and/or which side of the workspace the UID is near. For example, the indicator 810B may also appear in the right side panel 802B to indicate that the UID is nearing the right boundary of the workspace, and also include text stating that the UID is “Approaching the workspace limit.” Although both indicators 810A, 810B are shown in FIG. 8B, it is contemplated that in some aspects, only one of indicators 810A, 810B may be displayed to indicate that a UID is approaching a workspace boundary or limit (i.e., in the warning zone), and which UID (right or left) is approaching the workspace boundary or limit." as well as Paragraph 0064, "The feedback may be any one or more of the previously discussed feedback mechanisms, including a visual feedback, haptic feedback or audio feedback, or a combination of any one or more of a visual, haptic or audio feedback.") …
Freiin von Kapri does not specifically discuss there being an actuator in the user input device to assist in providing haptic feedback to the user or teach using a restoring force to counteract the commanded movement when the operator attempts to control the robot to move outside of the operating envelope. However, Verner, in the same field of endeavor of robotics, teaches:
… which includes a movable actuator, (Paragraph 0018, “The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”) … on the basis of a detected position of the actuator; … means for commanding a target force of the actuator, (Paragraph 0030, “FIGS. 3A and 3B shows block diagram of a surgical system 300 incorporating haptic feedback at an input device 330 and means for providing a consistent haptic experience for the user as described above with respect to FIGS. 1, 2A, and 2B. Surgical system 300 includes an instrument 310 for performing a surgical task (e.g., forceps, cutter, retractor, vessel sealer, needle driver, catheter, etc.), an input device 330 (e.g., a lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input) for receiving inputs from a user (e.g., surgeon), and a controller 320 for receiving input instructions from input device 330, controlling the actions of instrument 310 accordingly via a manipulation structure 313, and providing instructions to a haptic feedback actuation mechanism 340 to provide haptic feedback to input device 330 according to a desired haptic feedback profile. In various embodiments, manipulation structure 313 can include any number of systems and structures for maneuvering, positioning, actuating, or otherwise controlling the behavior of instrument 310, including a robotic arm(s)/manipulator(s), set up structure(s), and/or positioning element(s) such as a boom(s) or cart(s), among others. Controller 320 can include any combination of hardware, software, firmware, and other modalities for generating, managing, controlling, and effecting the actions described herein. In various embodiments, controller 320 can be integrated with instrument 310, input device 330, and/or discrete control hardware (e.g., a standalone processing unit or computing platform).”) … the target force comprises a restoring force component … said restoring force component counteracting an actuation of the actuator for commanding a movement of the telerobotic robot away from the border in the direction of the impermissible region. (Paragraphs 0017-0018, “FIG. 1 shows an exemplary method for providing directionally consistent haptic feedback when actuator output limits are exceeded. In a PROVIDE HAPTIC FEEDBACK step 110, a surgical system that allows a user (e.g., surgeon) to control a surgical instrument (and/or other elements of the surgical system, such as a robotic arm, set up structure, or positioning element such as a boom or cart) via an input device(s) (e.g., lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input), and then provides force feedback to that input device based on a desired haptic feedback profile (a set of one or more haptic feedback effects that at least partially reproduce or represent the physical experience of a real or virtual/modeled interaction). The haptic feedback profile can be based on any haptic model input, such as sensed forces at the instrument (e.g., tissue or other instrument interactions) or robotic arm (e.g., arm collisions with structures or staff), user guidance (e.g., haptic detents, fences, or other profiles to provide guidance for the user to move the input device(s) along a desired path or trajectory), and user interface (UI) elements (e.g., presenting a virtual handle or steering wheel to the user). This haptic feedback can be anything from direct replication of the haptic feedback profile, to scaling of the haptic feedback profile, to applying a non-linear modification of the haptic feedback profile, or any other transformation (e.g., force scaling that varies depending on one or more other factors such as instrument state/speed, viewing magnification, etc.).
The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods using haptic feedback as taught by Freiin von Kapri with the ability to use directional force feedback as a communication method between the operator and the system as taught by Verner. This would ensure that the operator does not move outside of the determined workspace and allows them to learn the workspace safely and efficiently without causing damage by leaving the desired workspace region.
Regarding claim 21, Freiin von Kapri further teaches:
21. (PREVIOUSLY PRESENTED) A computer program product for controlling a telerobotic robot using an input device … the computer program product comprising program code stored on a non-transitory, computer-readable medium, the program code, when executed on a computer, causing the computer to:
command a target pose of the telerobotic robot .(Paragraph 0005, "The surgeon may provide input commands to the surgical robotic system, and one or more processors of the surgical robotic system can control system components in response to the input commands. For example, the surgeon may hold in her hand a user input device that she manipulates to generate control signals to cause motion of the surgical robotic system components, e.g., an actuator, a robotic arm, and/or a surgical tool of the robotic system." and also Paragraph 0044, "In one embodiment, the remote operator 109 holds and moves the UID 114 to provide an input command to move a robot arm actuator 117 in the robotic system 100. The UID 114 may be communicatively coupled to the rest of the robotic system 100, e.g., via a console computer system 116. Representatively, in some embodiments, UID 114 may be a portable handheld user input device or controller that is ungrounded with respect to another component of the surgical robotic system. For example, UID 114 may be ungrounded while either tethered or untethered from the user console. The term “ungrounded” is intended to refer to implementations where, for example, both UIDs are neither mechanically nor kinematically constrained with respect to the user console. For example, a user may hold a UID 114 in a hand and move freely to any possible position and orientation within a workspace, only limited by, for example, a predetermined three-dimensional surgical workspace limit or boundary recognized by the system 100. Representatively, the system may include a tracking mechanism that tracks the location of the UID 114 within, and relative to, the surgical workspace limit or boundary. The UID 114 can generate spatial state signals corresponding to movement of the UID 114, e.g., position and orientation of the handheld housing of the UID, and the spatial state signals may be input signals to control a motion of the robot arm actuator 117. The signals (e.g., tracking sensor signals, clutch signals or engage/disengage teleoperation mode signals) may be wirelessly communicated between UID 114 and the computer system 116. In addition, a power source, such as a rechargeable battery, may be stored within the housing of UID 114 so that it does not need to be mechanically connected to a power source, such as by a wire or cable. The robotic system 100 may use control signals derived from the spatial state signals, to control proportional motion of the actuator 117. In one embodiment, a console processor of the console computer system 116 receives the spatial state signals and generates the corresponding control signals. Based on these control signals, which control how the actuator 117 is energized to move a segment or link of the arm 104, the movement of a corresponding surgical tool that is attached to the arm may mimic the movement of the UID 114. Similarly, interaction between the remote operator 109 and the UID 114 can generate for example a grip control signal that causes a jaw of a grasper of the surgical tool 107 to close and grip the tissue of patient 106.") … and
…
wherein at least one virtual border is specified between a permissible region and an impermissible region for the telerobotic robot, (Paragraph 0008, "Representatively, in one aspect, a method of determining a location of a user input device of a surgical robotic system within a surgical workspace using a virtual workspace includes determining, by one or more processors communicatively coupled to a user input device, that a user is engaging with the user input device within a surgical workspace; in response to determining the user is engaging with the user input device, displaying a virtual user input device within a first virtual workspace boundary, the first virtual workspace boundary representing a first workspace limit within which the user input device is operable to control a surgical robotic instrument in a teleoperation mode, wherein at least a portion of the first virtual workspace boundary is operable to move in response to a movement of the user input device; displaying a second virtual workspace boundary that represents a second workspace limit beyond which the user input device is inoperable to control the surgical robotic instrument in the teleoperation mode; and determining, by one or more processors, a location of the user input device within the surgical workspace based on a proximity of the portion of the first virtual workspace boundary relative to the second virtual workspace boundary. In some aspects, the first virtual workspace boundary comprises a first three dimensional shape and the second virtual workspace boundary comprises a second three dimensional shape that encompasses the first three dimensional shape. In some aspects, an area between the first three dimensional shape and the second three dimensional shape defines a warning zone that indicates the user input device is nearing the second workspace limit. In still further aspects, the first virtual workspace boundary comprises a cube. The portion of the first virtual workspace boundary operable to move may be a side wall of the cube. In some aspects, determining the location may include detecting that the portion of the first virtual workspace boundary has moved closer to the second virtual workspace boundary, and the method may further include in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary. Determining the location may include detecting that the portion of the first virtual workspace boundary intersects with the second virtual workspace boundary, and the method further includes in response to the detecting, providing user feedback. The feedback may include changing a visual characteristic of the first virtual workspace boundary or the second virtual workspace boundary. The feedback may include an audible alert or haptic feedback. The user input device may be an ungrounded user input device." as well as Paragraph 0073, "Accordingly, the user would understand this to mean that the UID in the user's right hand is in the warning zone, but is still closer to the operable workspace than the workspace limit or boundary beyond which operation of the associated surgical component is not allowed.") and … starting from the virtual border; (Paragraph 0066, "In other aspects, the indicator 808 may, in addition to indicating to the user that the UID is in a warning zone, indicate to the user which UID is within the warning zone and/or which boundary of the warning zone the UID is closest to. For example, FIG. 8B illustrates an indictor 810A that indicates to the surgeon which UID is approaching a workspace boundary or limit. Representatively, in FIG. 8B, display screen 802 is shown having a left side panel 802A and a right side panel 802B. The left side panel 802A may display information relating to a left UID (e.g., a UID held in the left hand), and the right side panel 802B may display information relating to a right UID (e.g., a UID held in the right hand). Accordingly, when the right UID is detected within the warning zone, an indicator 810A will appear in the right side panel 802B, as shown in FIG. 8B. In this aspect, the indicator 810A is shown as a line along the right side panel 802B. The indicator 810A could, however, be any type of shape, image, icon or the like that can be viewed by the user in the right side panel 802B. The surgeon will therefore understand that the UID held in his or her right hand is in the warning zone and nearing the workspace limit. Similarly, although not shown, if the left UID is detected in the warning zone, a similar warning indicator may appear in the left side panel 802B. The indicator 810A may appear in the center of the left or right panels 802A-B, or in any other region of the left or right panels 802A-B that is easily viewed and understood by the surgeon to mean a left and/or right UID is within the warning zone. In addition, in some aspects, the display may also include an indicator 810B including text which also informs the surgeon that the UID is approaching the workspace limit and/or which side of the workspace the UID is near. For example, the indicator 810B may also appear in the right side panel 802B to indicate that the UID is nearing the right boundary of the workspace, and also include text stating that the UID is “Approaching the workspace limit.” Although both indicators 810A, 810B are shown in FIG. 8B, it is contemplated that in some aspects, only one of indicators 810A, 810B may be displayed to indicate that a UID is approaching a workspace boundary or limit (i.e., in the warning zone), and which UID (right or left) is approaching the workspace boundary or limit." as well as Paragraph 0064, "The feedback may be any one or more of the previously discussed feedback mechanisms, including a visual feedback, haptic feedback or audio feedback, or a combination of any one or more of a visual, haptic or audio feedback.")
…
Freiin von Kapri does not specifically discuss there being an actuator in the user input device to assist in providing haptic feedback to the user or teach using a restoring force to counteract the commanded movement when the operator attempts to control the robot to move outside of the operating envelope. However, Verner, in the same field of endeavor of robotics, teaches:
… that includes a movable actuator, (Paragraph 0018, “The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”) … based on of a detected position of the actuator; … command a target force of the actuator; (Paragraph 0030, “FIGS. 3A and 3B shows block diagram of a surgical system 300 incorporating haptic feedback at an input device 330 and means for providing a consistent haptic experience for the user as described above with respect to FIGS. 1, 2A, and 2B. Surgical system 300 includes an instrument 310 for performing a surgical task (e.g., forceps, cutter, retractor, vessel sealer, needle driver, catheter, etc.), an input device 330 (e.g., a lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input) for receiving inputs from a user (e.g., surgeon), and a controller 320 for receiving input instructions from input device 330, controlling the actions of instrument 310 accordingly via a manipulation structure 313, and providing instructions to a haptic feedback actuation mechanism 340 to provide haptic feedback to input device 330 according to a desired haptic feedback profile. In various embodiments, manipulation structure 313 can include any number of systems and structures for maneuvering, positioning, actuating, or otherwise controlling the behavior of instrument 310, including a robotic arm(s)/manipulator(s), set up structure(s), and/or positioning element(s) such as a boom(s) or cart(s), among others. Controller 320 can include any combination of hardware, software, firmware, and other modalities for generating, managing, controlling, and effecting the actions described herein. In various embodiments, controller 320 can be integrated with instrument 310, input device 330, and/or discrete control hardware (e.g., a standalone processing unit or computing platform).”) … the target force comprises a restoring force component … the restoring force component counteracting an actuation of the actuator for commanding a movement of the telerobotic robot away from the border in the direction of the impermissible region. (Paragraphs 0017-0018, “FIG. 1 shows an exemplary method for providing directionally consistent haptic feedback when actuator output limits are exceeded. In a PROVIDE HAPTIC FEEDBACK step 110, a surgical system that allows a user (e.g., surgeon) to control a surgical instrument (and/or other elements of the surgical system, such as a robotic arm, set up structure, or positioning element such as a boom or cart) via an input device(s) (e.g., lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input), and then provides force feedback to that input device based on a desired haptic feedback profile (a set of one or more haptic feedback effects that at least partially reproduce or represent the physical experience of a real or virtual/modeled interaction). The haptic feedback profile can be based on any haptic model input, such as sensed forces at the instrument (e.g., tissue or other instrument interactions) or robotic arm (e.g., arm collisions with structures or staff), user guidance (e.g., haptic detents, fences, or other profiles to provide guidance for the user to move the input device(s) along a desired path or trajectory), and user interface (UI) elements (e.g., presenting a virtual handle or steering wheel to the user). This haptic feedback can be anything from direct replication of the haptic feedback profile, to scaling of the haptic feedback profile, to applying a non-linear modification of the haptic feedback profile, or any other transformation (e.g., force scaling that varies depending on one or more other factors such as instrument state/speed, viewing magnification, etc.).
The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods using haptic feedback as taught by Freiin von Kapri with the ability to use directional force feedback as a communication method between the operator and the system as taught by Verner. This would ensure that the operator does not move outside of the determined workspace and allows them to learn the workspace safely and efficiently without causing damage by leaving the desired workspace region.
Claim(s) 13-15 and 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Feiin von Kapri in view of Verner and in further view of Kang et al. (US 20210128253 A1), hereinafter Kang, Dozeman et al. (US 20210298846 A1), hereinafter Dozeman and Bowling et al. (US 20200030046 A1), hereinafter Bowling.
Regarding claim 13, where all the limitations of claim 10 are discussed above, Freiin von Kapri does not specifically teach using a virtual spring for haptic feedback or a restoring force to counteract an operators commands when they attempt to control the robot to leave a desired region. However, Kang, in the same field of endeavor of robotics, further teaches:
13. (PREVIOUSLY PRESENTED) The method of claim 10, wherein … is a force of a virtual compression spring. (Paragraph 0065, "In the preferred embodiment, the surgical system 10 employs point-based haptic interaction where only a virtual point, or haptic interaction point (HIP), interacts with virtual objects in the virtual environment. The HIP corresponds to a physical point on the haptic device 30, such as, for example, a tip of the tool 50. The HIP is coupled to the physical point on the haptic device 30 by a virtual spring/damper model. The virtual object with which the HIP interacts may be, for example, a haptic object 705 (shown in FIG. 11) having a surface 707 and a haptic force normal vector F.sub.n. A penetration depth d.sub.i is a distance between the HIP and the nearest point on the surface 707. The penetration depth d.sub.i represents the depth of penetration of the HIP into the haptic object 705.")
However, Verner, in the same field of endeavor of robotic control, teaches:
… the restoring force component (Paragraphs 0017-0018, “FIG. 1 shows an exemplary method for providing directionally consistent haptic feedback when actuator output limits are exceeded. In a PROVIDE HAPTIC FEEDBACK step 110, a surgical system that allows a user (e.g., surgeon) to control a surgical instrument (and/or other elements of the surgical system, such as a robotic arm, set up structure, or positioning element such as a boom or cart) via an input device(s) (e.g., lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input), and then provides force feedback to that input device based on a desired haptic feedback profile (a set of one or more haptic feedback effects that at least partially reproduce or represent the physical experience of a real or virtual/modeled interaction). The haptic feedback profile can be based on any haptic model input, such as sensed forces at the instrument (e.g., tissue or other instrument interactions) or robotic arm (e.g., arm collisions with structures or staff), user guidance (e.g., haptic detents, fences, or other profiles to provide guidance for the user to move the input device(s) along a desired path or trajectory), and user interface (UI) elements (e.g., presenting a virtual handle or steering wheel to the user). This haptic feedback can be anything from direct replication of the haptic feedback profile, to scaling of the haptic feedback profile, to applying a non-linear modification of the haptic feedback profile, or any other transformation (e.g., force scaling that varies depending on one or more other factors such as instrument state/speed, viewing magnification, etc.).
The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”) …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and control methods as taught by Freiin von Kapri with the virtual spring as taught by Kang and the ability to use force feedback as a communication method between the operator and the system as taught by Verner. This would ensure that the operator does not move outside of the determined workspace and allows them to learn the workspace safely and efficiently without causing damage by leaving the desired workspace region.
Regarding claim 14, where all the limitations of claim 13 are discussed above, Freiin von Kapri does not specifically teach using a virtual spring for haptic feedback or the virtual spring being used to resist movement out of the desired workspace or varying the spring force based on the position of the robot/the position input by the operator, a desired stiffness, or a specified scale. However, Kang, in the same field of endeavor of robotics, further teaches:
14. (CURRENTLY AMENDED) The method of claim 13, wherein at least one of:
a) the virtual compression spring is only tensioned by at least one of:
… or b) a spring force of the virtual compression spring depends on at least one of:
… (Paragraph 0065, "In the preferred embodiment, the surgical system 10 employs point-based haptic interaction where only a virtual point, or haptic interaction point (HIP), interacts with virtual objects in the virtual environment. The HIP corresponds to a physical point on the haptic device 30, such as, for example, a tip of the tool 50. The HIP is coupled to the physical point on the haptic device 30 by a virtual spring/damper model. The virtual object with which the HIP interacts may be, for example, a haptic object 705 (shown in FIG. 11) having a surface 707 and a haptic force normal vector F.sub.n. A penetration depth d.sub.i is a distance between the HIP and the nearest point on the surface 707. The penetration depth d.sub.i represents the depth of penetration of the HIP into the haptic object 705.")
However, Verner, in the same field of endeavor of robotics, teaches:
… a movement of the telerobotic robot away from the border in the direction of the impermissible region, or an actuation of the actuator for commanding a movement of the telerobotic robot away from the border in the direction of the impermissible region; (Paragraphs 0017-0018, “FIG. 1 shows an exemplary method for providing directionally consistent haptic feedback when actuator output limits are exceeded. In a PROVIDE HAPTIC FEEDBACK step 110, a surgical system that allows a user (e.g., surgeon) to control a surgical instrument (and/or other elements of the surgical system, such as a robotic arm, set up structure, or positioning element such as a boom or cart) via an input device(s) (e.g., lever(s), gripper(s), joystick(s), or any other structure capable of receiving user input), and then provides force feedback to that input device based on a desired haptic feedback profile (a set of one or more haptic feedback effects that at least partially reproduce or represent the physical experience of a real or virtual/modeled interaction). The haptic feedback profile can be based on any haptic model input, such as sensed forces at the instrument (e.g., tissue or other instrument interactions) or robotic arm (e.g., arm collisions with structures or staff), user guidance (e.g., haptic detents, fences, or other profiles to provide guidance for the user to move the input device(s) along a desired path or trajectory), and user interface (UI) elements (e.g., presenting a virtual handle or steering wheel to the user). This haptic feedback can be anything from direct replication of the haptic feedback profile, to scaling of the haptic feedback profile, to applying a non-linear modification of the haptic feedback profile, or any other transformation (e.g., force scaling that varies depending on one or more other factors such as instrument state/speed, viewing magnification, etc.).
The actual force feedback provided at the input device is generated by two or more actuators (e.g., motors, drives, or any other motive elements) that work cooperatively to provide feedback of varying force and direction. For example, an input device having pitch and yaw capabilities may be coupled to a first pair of actuators that apply forces in opposing directions about the pitch axis, and a second pair of actuators that apply forces in opposing directions about the yaw axis. Two or more of the pitch and yaw actuators can then be used simultaneously to provide force feedback that is offset from the pitch and yaw axes.”) …
However, Dozeman, in the same field of endeavor of robotics, teaches:
… a current and/or previous position of the actuator, a current and/or previous pose of the telerobotic robot, (Paragraph 0322, "Coefficient KWS E is a spring coefficient. This coefficient may be variable. This is because as the energy applicator 184 moves outwardly from the workspace boundary towards the workspace limit, there is a need to appreciably increase the force that prevents the continued movement of the energy applicator 184 towards the workspace limit. Consequently, there is often greater than first order relationship between the magnitude of this force and the workspace boundary exceeded distance.") … or a specified scaling between adjustments of the actuator and movements of the telerobotic robot. (Paragraph 0149, "A second variable used to selectively scale the defined feed rate is force and torque to which the energy applicator 184 is exposed (SNSD F/T). The energy applicator 184 is rigidly attached to the instrument 160 and the instrument is rigidly attached to the end effector 110. Accordingly, the signals output by the end effector force/torque sensor 108 are the signals representative of the forces and torques to which energy applicator 184 is exposed. Feed rate calculator 284 sets the instrument rate based on the principle that there is relationship between the amount of force/torque that the manipulator applies to the instrument and energy applicator 184 and the rate of instrument advancement. Generally, it is a goal of modern medical practice to minimize the heating of tissue that is not being removed. One reason for this goal is to minimize the attendant damage this needless heating can cause to the tissue. Accordingly, manipulator 50 of this invention is configured to, when it is determined that an appreciable amount of force and/or torque is applied to the instrument or energy applicator 184, slow the advancement of the instrument along the path segment.")
However, Bowling, in the same field of endeavor of robotics, teaches:
… a specified spring stiffness of the virtual compression spring, (Paragraph 0115, "The virtual constraints are not infinitely rigid, but instead each of the virtual constraints has tuning parameters to adjust the stiffness of the virtual constraints, e.g., by incorporating spring and damping parameters into the constraints. Such parameters may include a constraint force mixing parameter (C) and an error reduction parameter (E). The spring and damping parameters may be adjusted during operation. In some versions, values for the tuning parameters may change based on certain relationships, e.g., a curvature of the tool path TP (for path constraints), a relationship between the virtual boundary 71 and the TCP (for boundary constraints), a relationship between the current state and the target state (for guide constraints), etc. The tuning parameters may be different for different virtual constraints. For example, the boundary constraints may be stiffer than the other constraints. The virtual constraints may comprise a first virtual constraint that has a first value for a tuning parameter and a second virtual constraint that has a second value for the tuning parameter, the first value being greater than the second value so that the resulting virtual forces and/or torques embodied in the constraint force F.sub.e are adapted to effect movement of the tool 20 more strongly as a result of the first virtual constraint as compared to the second virtual constraint. The values of the tuning parameters may be greater (e.g., stiffer) for position constraints than for orientation constraints, or vice versa.") …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine haptic feedback and robotic control system/methods as taught by Freiin von Kapri with the ability to make adjustments to a spring configuration in order to resist movement toward undesired regions of the workspace as taught by Kang, Verner, Dozeman, and Bowling. This would allow for a highly configurable system where an operator may select different configurations that suit their needs/style of operation. This would provide better feedback and communication with the operator making the system easier to use and more effective.
Examiner Note: For the purpose of compact prosecution, multiple references have been shown which demonstrate these limitations. However, only one is positively recited in the current claim language.
Regarding claim 15, where all the limitations of claim 10 are discussed above, Freiin von Kapri further teaches:
15. (PREVIOUSLY PRESENTED) The method of claim 10, further comprising:
… in response to an actuation of the actuator that would result in movement of the telerobotic robot from a position starting from the border in the direction of the impermissible region. (Paragraph 0054, "To indicate to the user the workspace limit, boundary or region beyond which control of an associated surgical component by the UID is no longer allowed (e.g., the UID is too far outside of the workspace), the training operation also provides the user with a second virtual workspace boundary at operation 308. The second virtual workspace boundary may be displayed to the user and provide a visual representation of an area within which the UID must be manipulated in order to operate an associated surgical component (e.g., a surgical tool). If the movement of the UID exceeds the second virtual workspace boundary, the UID is considered to have exceeded the workspace limit or boundary and is prevented from controlling an associated surgical component by the system. In some cases, the teleoperation mode is disengaged in response to detecting that the workspace boundary or limit has been met or exceeded by the UID. In this aspect, the second virtual workspace boundary may be displayed to the user as a boundary or limit that is outside of the first virtual workspace boundary. For example, in some aspects, the first virtual workspace boundary and the second virtual workspace boundary may be displayed to the user as three-dimensional shapes. The three-dimensional shape of the second virtual workspace boundary may be larger (e.g., define a larger area) than that of the first virtual workspace boundary. In this aspect, the three-dimensional shapes may be displayed to the user as one inside of the other. For example, the three-dimensional shape of the first virtual workspace boundary may displayed to the user as located within, or otherwise encompassed by, the three-dimensional shape of the second virtual workspace boundary. In this aspect, the area within (or at) the boundary of the inner three-dimensional shape (e.g., the first virtual workspace) may be understood by the user as the desired area of operation of the UID. The area or zone between the inner three-dimensional boundary (e.g., the first virtual workspace boundary) and the outer three-dimensional boundary (e.g., the second virtual workspace boundary) may be understood by the user as the warning zone or area indicating the user is approaching the operation limit of the UID. The area beyond (or at) the outer three-dimensional boundary (e.g., the second virtual workspace boundary) may be understood by the user as an area in which the UID has exceeded its operational limit and is therefore no longer able to control an associated surgical component. Accordingly, when the user moves the real UID, and this movement is shown as the virtual UID exceeding the inner three-dimensional boundary, approaching the outer three-dimensional boundary, or exceeding the outer three-dimensional boundary, the user begins to recognize and learn the operational boundaries of the real workspace.")
Freiin von Kapri does not specifically teach limiting the commanded movement of the robot. However, Dozeman, in the same field of endeavor of robotics, teaches:
… suppressing the commanding of a movement of the telerobotic robot (Paragraph 0003, "During operation, the surgical system may limit movement of the tool to avoid violating a virtual boundary established to protect portions of the patient's anatomy from the tool." As well as Paragraph 0138, "The boundary handler 90 performs various collision checks depending on the mode of operation, user input state, etc. A first type of collision check involves checking whether/how a current state (e.g., current pose) of the tool 20 or a proposed state (e.g., proposed pose) of the tool 20 generated in the virtual simulation by the virtual simulator 86 violates the virtual boundary 71. This collision check is performed to determine the boundary constraints that need to be generated by the boundary handler 90 and applied by the constraint solver 84 so that the current state/proposed state is altered in a way to prevent, or at least to limit, the violation of the virtual boundary 71 by the tool 20 during normal operation in the manual mode or the semi-autonomous mode. In some versions, this type of collision check is performed in each frame during operation in the manual mode or the semi-autonomous mode and occurs before the generation of a new commanded pose by the virtual simulator 86 so that the commanded pose that is ultimately generated and carried out by the motion control 76 limits violations of the virtual boundary 71 by the tool 20. In some versions, this type of collision check could be performed by the boundary handler 90 based on the commanded pose computed in the prior iteration (e.g., the commanded pose of the prior time frame is set as the current pose). In that case, the boundary handler 90 determines boundary constraints that need to be generated to at least limit violation of the virtual boundary 71. For example, the commanded pose from the prior frame may be one that results in the tool 20 being moved slightly across the virtual boundary 71, but the boundary handler 90 generates boundary constraints in the current frame to bring the tool 20 back.") …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system and operating methods as taught by Freiin von Kapri with the ability to suppress the input command as taught by Dozeman. This would ensure that the boundary is not violated and the operator does not cause damage by leaving the desired region of the workspace.
Regarding claim 17, where all the limitations of claim 10 are discussed above, Freiin von Kapri further teaches:
17. (PREVIOUSLY PRESENTED) The method of claim 10, wherein the target force (Paragraph 0044, "In one embodiment, the remote operator 109 holds and moves the UID 114 to provide an input command to move a robot arm actuator 117 in the robotic system 100. The UID 114 may be communicatively coupled to the rest of the robotic system 100, e.g., via a console computer system 116. Representatively, in some embodiments, UID 114 may be a portable handheld user input device or controller that is ungrounded with respect to another component of the surgical robotic system. For example, UID 114 may be ungrounded while either tethered or untethered from the user console. The term “ungrounded” is intended to refer to implementations where, for example, both UIDs are neither mechanically nor kinematically constrained with respect to the user console. For example, a user may hold a UID 114 in a hand and move freely to any possible position and orientation within a workspace, only limited by, for example, a predetermined three-dimensional surgical workspace limit or boundary recognized by the system 100. Representatively, the system may include a tracking mechanism that tracks the location of the UID 114 within, and relative to, the surgical workspace limit or boundary. The UID 114 can generate spatial state signals corresponding to movement of the UID 114, e.g., position and orientation of the handheld housing of the UID, and the spatial state signals may be input signals to control a motion of the robot arm actuator 117. The signals (e.g., tracking sensor signals, clutch signals or engage/disengage teleoperation mode signals) may be wirelessly communicated between UID 114 and the computer system 116. In addition, a power source, such as a rechargeable battery, may be stored within the housing of UID 114 so that it does not need to be mechanically connected to a power source, such as by a wire or cable. The robotic system 100 may use control signals derived from the spatial state signals, to control proportional motion of the actuator 117. In one embodiment, a console processor of the console computer system 116 receives the spatial state signals and generates the corresponding control signals. Based on these control signals, which control how the actuator 117 is energized to move a segment or link of the arm 104, the movement of a corresponding surgical tool that is attached to the arm may mimic the movement of the UID 114. Similarly, interaction between the remote operator 109 and the UID 114 can generate for example a grip control signal that causes a jaw of a grasper of the surgical tool 107 to close and grip the tissue of patient 106.") …
Freiin von Kapri does not specifically teach the target force comprising a force feedback component based on an external force on the robot. However, Dozeman, in the same field of endeavor of robotics, teaches:
… further comprises a force feedback component that depends on an external force on the telerobotic robot. (Paragraph 0156, "In step 146, the constraint force Fe is summed with the external force Fext transformed to the virtual mass coordinate system VM (Fegext), the damping force Fdamping, and the inertial force Finertial to yield the total force FT. In step 148, the total force FT is applied to the virtual rigid body in the virtual simulation conducted by the virtual simulator 86 to determine a proposed state (e.g., pose and velocity) of the virtual rigid body, and ultimately to transform the initial state and the proposed state to the TCP in step 150.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system as taught by Freiin von Kapri with the ability to provide force feedback based on external forces as taught by Dozeman. This would give the operator a more realistic experience and allow them to adjust their input based on an accurate environment.
Regarding claim 18, where all the limitations of claim 17 are discussed above, Freiin von Kapri does not specifically teach force feedback emulating an external force on the robot or a scaled value of an external force. However, Dozeman, in the same field of endeavor of robotics, teaches:
18. (PREVIOUSLY PRESENTED) The method of claim 17, wherein the force feedback component emulates one of: the external force on the telerobotic robot; (Paragraph 0156, "In step 146, the constraint force Fe is summed with the external force Fext transformed to the virtual mass coordinate system VM (Fegext), the damping force Fdamping, and the inertial force Finertial to yield the total force FT. In step 148, the total force FT is applied to the virtual rigid body in the virtual simulation conducted by the virtual simulator 86 to determine a proposed state (e.g., pose and velocity) of the virtual rigid body, and ultimately to transform the initial state and the proposed state to the TCP in step 150.") or a scaled magnitude of the external force on the telerobotic robot. (Paragraph 0192, "The tuning parameters for the SIFs may also be set to: remain constant; rise/fall exponentially with constraint distance; vary linearly with constraint distance; vary with constraint direction; take gravitational effects into account; and the like. The tuning parameters can also be scaled depending on the constraint force Fe that is ultimately computed based on the virtual constraints, such as by increasing/decreasing the stiffness depending on the magnitude of the constraint force Fe, or any components thereof. The tuning parameters of the SIFs and their values, their correlation to a particular relationship, and the manner in which they may be scaled, may be stored in one or more look-up tables in any suitable memory in the control system 60 for later retrieval.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system as taught by Freiin von Kapri with the ability to provide force feedback based on external forces as taught by Dozeman. This would give the operator a more realistic experience and allow them to adjust their input based on an accurate environment.
Regarding claim 19, where all the limitations of claim 10 are discussed above, Freiin von Kapri further teaches:
19. (PREVIOUSLY PRESENTED) The method of claim 10, wherein the target force comprises a damping component (Paragraph 0044, "In one embodiment, the remote operator 109 holds and moves the UID 114 to provide an input command to move a robot arm actuator 117 in the robotic system 100. The UID 114 may be communicatively coupled to the rest of the robotic system 100, e.g., via a console computer system 116. Representatively, in some embodiments, UID 114 may be a portable handheld user input device or controller that is ungrounded with respect to another component of the surgical robotic system. For example, UID 114 may be ungrounded while either tethered or untethered from the user console. The term “ungrounded” is intended to refer to implementations where, for example, both UIDs are neither mechanically nor kinematically constrained with respect to the user console. For example, a user may hold a UID 114 in a hand and move freely to any possible position and orientation within a workspace, only limited by, for example, a predetermined three-dimensional surgical workspace limit or boundary recognized by the system 100. Representatively, the system may include a tracking mechanism that tracks the location of the UID 114 within, and relative to, the surgical workspace limit or boundary. The UID 114 can generate spatial state signals corresponding to movement of the UID 114, e.g., position and orientation of the handheld housing of the UID, and the spatial state signals may be input signals to control a motion of the robot arm actuator 117. The signals (e.g., tracking sensor signals, clutch signals or engage/disengage teleoperation mode signals) may be wirelessly communicated between UID 114 and the computer system 116. In addition, a power source, such as a rechargeable battery, may be stored within the housing of UID 114 so that it does not need to be mechanically connected to a power source, such as by a wire or cable. The robotic system 100 may use control signals derived from the spatial state signals, to control proportional motion of the actuator 117. In one embodiment, a console processor of the console computer system 116 receives the spatial state signals and generates the corresponding control signals. Based on these control signals, which control how the actuator 117 is energized to move a segment or link of the arm 104, the movement of a corresponding surgical tool that is attached to the arm may mimic the movement of the UID 114. Similarly, interaction between the remote operator 109 and the UID 114 can generate for example a grip control signal that causes a jaw of a grasper of the surgical tool 107 to close and grip the tissue of patient 106.") …
Freiin von Kapri does not specifically teach adjusting the target force based on the velocity of the actuator. However, Dozeman, in the same field of endeavor of robotics, teaches:
… that depends on an adjustment speed of the actuator. (Paragraph 0121, "The virtual mass matrix M combines 3×3 mass and inertia matrices. The damping and inertial forces F.sub.damping and F.sub.inertial are calculated/known by the virtual simulator 86 and are based on the virtual mass velocity V.sub.eg1 (e.g., the velocity of the virtual mass coordinate system VM) output by the virtual simulator 86 in a prior time step. The virtual mass velocity V.sub.eg1 is a 6-DOF velocity vector comprising linear and angular velocity components. The damping force F.sub.damping is a 6-DOF force/torque vector computed as a function of the virtual mass velocity V.sub.eg1 and a damping coefficient matrix (linear and rotational coefficients may not be equal). Damping is applied to the virtual mass to improve its stability. The inertial force F.sub.inertial is also a 6-DOF force/torque vector computed as a function of the virtual mass velocity V.sub.eg1 and the virtual mass matrix M. The damping and inertial forces, F.sub.damping and F.sub.inertial, can be determined in the manner described in U.S. Pat. No. 9,566,122 to Bowling et al., hereby incorporated herein by reference.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic system as taught by Freiin von Kapri with the ability to adjust a damping force based on the velocity as taught by Dozeman. This would improve the stability of the movement of the robot.
Allowable Subject Matter
Claim 22 is allowed.
Conclusion
The Examiner has cited particular paragraphs or columns and line numbers in the referencesapplied to the claims above for the convenience of the Applicant. Although the specified citations arerepresentative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the Applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP 2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed Invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
Additional prior art relevant to the application but not relied upon:
US 20080167662 A1
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
/H.J.K./Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights