DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 35-36, 39, 45-46, 49 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 35 recites “in response to the user initiating the following mode, cause the out-of-view indication to be presented; and after a predetermined time period after initiating the following mode, cause the presentation of the out-of-view indication to be ceased” in lines 3-4, which renders the claim indefinite because it is unclear what functional meaning applicant meant to impart by reciting “in response” as it is unclear what measure or measurements are performed then executed which would constitute “in response”.
Claims 36, 45-46 are also rejected for reciting the same and/or limitations outlined above.
All dependent claims are also rejected by the nature of their dependency.
Claim 39 recites “receive an acknowledgement of an out-of-view instrument status for the instrument” in line 3, which renders the claim indefinite because it is unclear who or what structure “receives” an “acknowledgement” of the out-of-view instrument status, and if this limitation is meant to encompass a human user, general purpose computer, or a processor that has specifically been configured to perform the reception of an “acknowledgement”. Similarly, it is unclear what structure “sends” a message seeking an “acknowledgement”. To avoid being rejected under 35 USC 101, applicant is advised to amend the claim to recite the structural components/devices involved in this limitation.
Claim 49 is also rejected for reciting the same and/or limitations outlined above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to co click form Wizard click form Wizard click add page move mouse to display number three insider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 32-34, 38-39, 41, 42-44, 48-49, and 51 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. (US20090088634, hereafter “Zhao”), in view of Moll et al. (US20070197896, hereafter “Moll”).
Regarding claims 32, 42, and 51; Zhao discloses a method for controlling a teleoperational (FIG. 6A, claim 13) and a teleoperational system (FIG. 1, claim 1), a non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a processing unit of a teleoperational system ([0049] computer having a hard drive stores the instructions to be executed), comprising:
a teleoperational assembly configured to support an instrument (endoscope FIG. 2) and an imaging device ([0052] stereo viewer, three dimensional maps (a depth map with respect to a camera coordinate system derived from alternative imaging modalities (e.g. CT scan, XRAY, or MRI), the instrument having an instrument tip (endoscope in FIG. 2); and
a processing unit including one or more processors ([0049] the computer having one or more microprocessors), wherein the processing unit is configured to:
determine an instrument position of the instrument ([0204] determine an estimation error (EET) between the actual tip location 1520 and the estimated tip location 1518 using tool tracking methods);
determine an instrument position error ([0061], FIG. 5A, tool tracking is determined by determinig a pose of a robotic instrument 101 including its position or location (Xt,Yt,Zt) and its orientation in the camera coordinate system as it moves in, around, and out of the surgical site, a full pose description may not only include the location and orientation of a robotic instrument in a three-dimensional space but may further include the pose of an end-effector, if any [a lack of indicator of a pose of an end-effector corresponds to the device with an end-effector being outside of the field of view], then positional information or pose as used herein may be used to refer to one or both the location and orientation [being within or outside of the field of view] of a robotic instrument); and
determine, based on the instrument position and the instrument position error, that at least a portion of the instrument is outside a field of view of the imaging device ([0204], FIGS. 5A, 9A-9B, image sequences 1514 along with kinematics information may be adaptively fused together to translate a model of the tool tip (MET) into the endoscopic camera frame of reference (ECM)…The operational error EOP between the actual location of the tumor and the actual location of tool tips [position error] is a sum of errors); and
in response to determining that at least a portion of the instrument is outside the field of view of the imaging device and based on a mode of operation of the teleoperational system ([0090], [0166], FIG. 8, tool tracking system adaptively fuses visual information [based on a determination] and robot kinematics in order to achieve robust, accurate and efficient tool tracking, the unknown full pose of a robotic instrument, at a time instant t, is represented as a state st in a Bayesian state-space model, the state-space model uses a plurality of posed states to perform tool tracking in the surgical site. The state-space model generates the corrected kinematics information of the robotic instrument, and the robotic kinematics information [including bounding volume of the instrument tip] indicates that the one or both of the robotic surgical tools are outside the field of view), cause an out-of-view indication for the instrument to be presented (see FIG. 5A for determining the tip pose is outside the field of view, [0216] when the robotic surgical tool is out of the field of view, then one of a plurality of compass icons may be displayed [indicate when the instrument tip is outside of the field of view] in the field of view to show a direction of tool reentrance), but the determined instrument tip position error disclosed by Zhao does not explicitly disclosed as being determined relative to the imaging device.
However, in the same field of endeavor, Moll teaches determining an instrument tip position error relative to the imaging device ([0277], FIG. 151, the instrument operated by an instrument driver and a closed-loop control system incorporating a localization technology to measure actual instrument position is depicted so that when the instrument tip is driven through a range of motion, such as +pitch to -pitch, then back to neutral and +yaw to -yaw, at some cyclic interval, loads encountered by tissue structure contact, as opposed to free cavity space in blood, for example, will tend to increase the error detected between the measured tip position determined by the localization system, and the predicted tip location, determined via the inverse kinematics of the instrument structure, and other cyclic patterns of motion may also be utilized, e.g., repeated spiral motion, circular motion, etc. A threshold may be utilized, depending upon the experimentally determined systematic error between the predicted and measured tip locations in free space given a particular instrument structure, the threshold is beyond which error is considered an indicator of tissue contact. And, depending upon the cyclic motion pattern selected, the direction of contact between the instrument and another object may also be detected by observing the directionality of error between predicted and measured instrument position).
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the system and method disclosed by Zhao with the step to determine an instrument tip position error relative to the imaging device as taught by Moll in order to provide a real-time representation of where the computerized system expects the instrument to be relative to the surrounding anatomy to facilitate the user's comprehension of relative positioning of the various structures ([0306] of Moll).
Regarding claims 33 and 43, Zhao substantially discloses all the limitations of the claimed invention, specifically, Zhao discloses wherein the processing unit is configured to cause the out-of-view indication for the instrument to be presented based on the mode of operation of the teleoperational system being in a camera control mode in which a user of the teleoperational system controls the imaging device ([0047]-[0048] the operator's head sets off the viewing sensor to enable the control of the robotic instruments when the system operator aligns his or her eyes with the binocular eye pieces of the display to view a stereoscopic image [part of imaging system/device] of the surgical worksite and the operator O can use their feet to control the foot-pedals to change the configuration of the surgical system and generate additional control signals to control the robotic instruments as well as the endoscopic camera [imaging device]).
Regarding claims 34 and 44, Zhao substantially discloses all the limitations of the claimed invention, specifically, Zhao discloses wherein the processing unit is configured to cause the out-of-view indication for the instrument to be presented based on the mode of operation of the teleoperational system being in a following mode* in which a user of the teleoperational system controls the instrument ([0047]-[0048] the operator's head sets off the viewing sensor to enable the control of the robotic instruments when the system operator aligns his or her eyes with the binocular eye pieces of the display to view a stereoscopic image [part of imaging system/device] of the surgical worksite and the operator O can use their feet to control the foot-pedals to change the configuration of the surgical system and generate additional control signals to control the robotic instruments).
*The term has been interpreted as defined by the applicant in [0048] of the PG Pub to mean “the out-of-view indicator may be displayed for a few seconds after initiating a mode of the system in which the operator controls movement of the instruments, a mode which may be known as a following mode”, where the term “instruments” has been interpreted under the broadest reasonable interpretation to mean any device used by an operator during a surgical operation as known in the surgical arts.
Regarding claims 38 and 48, Zhao, in view of Moll, substantially discloses all the limitations of the claimed invention, specifically, Zhao discloses wherein the processing unit is further configured to, in response to determining that at least a portion of the instrument is outside the field of view of the imaging device and based on a mode of operation of the teleoperational system ([0047]-[0048] the operator's head sets off the viewing sensor to enable the control of the robotic instruments when the system operator aligns his or her eyes with the binocular eye pieces of the display to view a stereoscopic image [part of imaging system/device] of the surgical worksite and the operator O can use their feet to control the foot-pedals to change the configuration of the surgical system and generate additional control signals to control the robotic instruments as well as the endoscopic camera [imaging device]), and specifically, Moll discloses disable a function of the instrument (FIG. 2.1, [0131], a device disabling switch (7) configured to temporarily disable the activity of the instrument).
Regarding claims 39 and 49, Zhao substantially discloses all the limitations of the claimed invention, specifically, Zhao discloses wherein the processing unit is further configured to:
receive an acknowledgement of an out-of-view instrument status for the instrument; and
in response to receiving the acknowledgment of the out-of-view instrument status, enable the function of the instrument ([0047] the operator, via the operator's head, sets off the viewing sensor 320 to enable the control of the robotic instruments 101).
Regarding claim 41, Zhao substantially discloses all the limitations of the claimed invention, specifically, Zhao discloses wherein the out-of-view indication for the instrument is based on an instrument type* of the instrument ([0195], [0211] one type of tracked tool (e.g., an ultrasound tool) is used to draw marks to indicate regions of interest and then use a different type of tracked tool (e.g., a cautery tool) to operate or perform a surgical procedure, prior knowledge used in the tracking information includes the knowledge of the type of instrument).
*The limitation has been interpreted under the broadest reasonable interpretation to mean any type of instrument known in the surgical arts.
Claim(s) 35-37, 40, 45-47 and 50 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao, in view of Moll, as applied to claims 32, 34 and 42, 44 above, further in view of Bendall (US20170337705).
Regarding claims 35 and 45, Zhao substantially discloses all the limitations of the claimed invention, but does not explicitly disclose wherein the processing unit is further configured to: in response to the user initiating the following mode, cause the out-of-view indication to be presented; and after a predetermined time period after initiating the following mode, cause the presentation of the out-of-view indication to be ceased.
However, in solving the same problem, Bendall teaches wherein the processing unit is further configured to:
in response to the user initiating the following mode, cause the out-of-view indication to be presented ([0159] the graphic overlay may be updated in real time as the cursors are moved by the user); and
after a predetermined time period after initiating the following mode, cause the presentation of the out-of-view indication to be ceased ([0160] the graphic overlay is only displayed temporarily when a cursor is moved and may be removed a few seconds [predetermined period] after cursor movement stops).
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the method and system disclosed by Zhao with the processing unit is further configured to: in response to the user initiating the following mode, cause the out-of-view indication to be presented; and after a predetermined time period after initiating the following mode, cause the presentation of the out-of-view indication to be ceased as taught by Bendall in order to provide a visual indication of the orientation of the tip of the probe of the video inspection device with respect to the object ([0166] of Bendall).
Regarding claims 36 and 46, Zhao substantially discloses all the limitations of the claimed invention, but does not explicitly disclose wherein the processing unit is further configured to: receive a user input to view information relating to the instrument's location; and cause the out-of-view indication for the instrument to be presented based further on receiving the user input.
However, in solving the same problem, Bendall wherein the processing unit is further configured to:
receive a user input to view information relating to the instrument's location ([0077] the video inspection device monitors the movement of the cursor 234 and detect when the cursor 234 has stopped moving, the video inspection device 100 (e.g., the CPU 150) can determine the deepest surface point proximate to the cursor 234); and
cause the out-of-view indication for the instrument to be presented based further on receiving the user input ([0160] the graphic overlay may be shown temporarily and is removed a few seconds after cursor movement stops).
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the method and system disclosed by Zhao with receive a user input to view information relating to the instrument's location; and cause the out-of-view indication for the instrument to be presented based further on receiving the user input as taught by Bendall in order to provide a visual indication of the orientation of the tip of the probe of the video inspection device with respect to the object ([0166] of Bendall).
Regarding claims 37 and 47, Zhao substantially discloses all the limitations of the claimed invention, but does not explicitly disclose wherein the processing unit is further configured to: receive a user input to disable the out-of-view indication; and cause the presentation of the out-of-view indication to be ceased in response to receiving the user input.
However, in solving the same problem, Bendall teaches wherein the processing unit is further configured to:
receive a user input to disable the out-of-view indication ([0160] the graphic overlay may be shown temporarily and is removed a few seconds after cursor movement stops as input by a user); and
cause the presentation of the out-of-view indication to be ceased in response to receiving the user input ([0160] the graphic overlay may be hidden whenever a reference surface cursor is inactive).
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the method and system disclosed by Zhao with the processing unit is further configured to: receive a user input to disable the out-of-view indication; and cause the presentation of the out-of-view indication to be ceased in response to receiving the user input as taught by Bendall in order to provide a visual indication of the orientation of the tip of the probe of the video inspection device with respect to the object ([0166] of Bendall).
Regarding claims 40 and 50, Zhao substantially discloses all the limitations of the claimed invention, but does not explicitly disclose wherein the processing unit is further configured to present, based on an instrument type of the instrument, a warning message or an acknowledgment relating to the at least the portion of the instrument being outside of the field of view of the imaging device.
However, in solving the same problem, Bendall teaches wherein the processing unit is further configured to present, based on an instrument type of the instrument, a warning message or an acknowledgment relating to the at least the portion of the instrument being outside of the field of view of the imaging device ([0175] the video inspection device outputs a warning system to a user via a visual or audible warning when there is an undesirable (e.g., far from perpendicular) viewing perspective at the location where a measurement cursor is being placed on an edge of the viewing frame).
It would have been obvious to one ordinarily skilled in the art before the effective filing date of the claimed invention to modify the method and system disclosed by Zhao with the processing unit is further configured to present, based on an instrument type of the instrument, a warning message or an acknowledgment relating to the at least the portion of the instrument being outside of the field of view of the imaging device the user input as taught by Bendall in order to provide a visual indication of the orientation of the tip of the probe of the video inspection device with respect to the object ([0166] of Bendall).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY SHAFQAT whose telephone number is (571)272-4054. The examiner can normally be reached Monday-Friday 9:30AM-5:30PM MST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Keith Raymond can be reached at (571) 270-1790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S./Examiner, Art Unit 3798
/KEITH M RAYMOND/Supervisory Patent Examiner, Art Unit 3798