DETAILED ACTION
Receipt is acknowledged of applicant’s argument/remarks filed on December 18, 2025, claims 1, 4-8, and 13-26 are pending and an action on the merits is as follows.
Applicant's arguments with respect to amended claims have been fully considered but are moot in view of the following ground(s) of rejection. Applicant has amended claims 1 and 17 and added claim 27.
Previously, claims 2-3 and 9-12 has been canceled.
Per amendment of the claims, the rejection of the claims under nonstatutory double patenting rejection had been withdrawn.
Response to Argument
Regarding applicant’s arguments with respect to the amendment of the claims, applicant is kindly invited to consider the Office Action below to view the new ground of rejection, cited section of the references and motivation.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), first paragraph:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 and 17 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention; this constitutes new matter. The invention fails to disclose “ … establishing a persistent tracking state …” (added remarks), and (ii) its critical roles with respect to the scope of the invention.
Applicant did not provide specification support for the above limitation. The specification provides a general description for tracking the movement of surgical tool (par. 138, 140, 285 and other section); however, it is completely devoid of the feature " establishing a persistent tracking state.” Applicants may consider using terms clearly disclose within the specification.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the “establishing a persistent tracking state …” or other new matter cited on the claim limitations must be shown or the feature(s) canceled from the amended claims 1 and 17. No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-8, 14-16, 17, 20-22, and 24-27 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Pat. No.: US 6,063,095) in view of Devam et al. (Pub. No.: US 2016/0249989 A1) and Wang et al. (Pat. No.: US 6,463,361 B1).
Regarding claim 1, Wang et al. disclose a method for performing minimally invasive surgical procedures using voice instruction, the method comprising:
introducing a robotically controlled surgical endoscope and at least one robotically controlled surgical tool into a surgical environment in a body cavity (e.g., endoscope 28 and surgical instruments 22 inserted into a chest cavity of a patient to perform surgical procedure (col. 5, lines 41-46, col. 17, lines 34-36 and lines 62-65 and Figures 9 and 10A-10C));
detecting said at least one vocal command generated by at least one surgeon in a surgical setting (e.g., a surgeon inputs vocabulary / voice instruction(s) via a microphone to move an endoscope – see col. 6, lines 45-52 and Figure 1);
in response to said vocal command (e.g., a controller 46 moves the endoscope 28 and robotic arm assembly 26 in accordance with the input vocabulary / voice instruction(s) of the surgeon – col. 6, line 66 – col. 7, line 2 and col. 6, lines 34-35 and Figure 1).
Wang et al. disclose the controller 46 configured to move the endoscope 28 and robotic arm assembly 26 in accordance with the input vocabulary / voice instruction(s) of the surgeon – col. 6, line 66 – col. 7, line 2 and col. 6, lines 34-35 and Figure 1).
However, Wang et al. failed to specifically disclose tagging at least one of said at least one robotically controlled surgical tools, said tagging establishing tracking state that cause the robotically controlled surgical endoscope to move within the body cavity to maintain visualization of said at least one tagged robotically controlled surgical tools within the body cavity.
However, Devam et al. teach the technique of identifying a surgical instrument based on user request (limitation: tagging at least one of said at least one robotically controlled surgical tools) and tracking the identified instrument via a laparoscopic camera based on user vocal command during surgical procedure (par. 43, 26 and 37) to display the surgical instrument during a surgical procedure(par. 26, 43, 50), which requires the camera to move, for instance, within the patient body.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the controller for operating surgical equipment (e.g., endoscope) taught by Wang et al., such that the controller identifies a surgical instrument based on user request (e.g., vocal command) and tracking the identified instrument via a laparoscopic camara to display the surgical instrument during a surgical procedure based on user vocal command during surgical procedure, in view of Devam et al., with reasonable expectation of success, since doing so would have achieved the benefit of allowing a surgeon to view and accurately locate patient organ and other features prior to and during surgery via laparoscopic camara thereby reducing morbidity and mortality due to surgical error associated with variance in patient feature location (par. 25).
However, Wang et al., as modified by Devam et al., failed to specifically disclose to autonomously move the surgical endoscope within the body cavity as said at least one robotically controlled surgical tool moves within the body cavity.
However, Wang et al (‘361) teach an automated endoscope system configured to move an endoscope to automatically track a movement of an instrument that is inserted into a patient based on voice command (col. 4, lines 9-15 and col. 3, lines 36-41).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to further modify the controller for operating surgical equipment (e.g., endoscope) as taught by the combination of Wang et al. in view of Devam et al., such that the controller moves the endoscope to automatically track a movement of an instrument that is inserted into a patient based on voice command, in view of Wang et al (‘361), with reasonable expectation of success, since doing so would have achieved the benefit of moving a surgical instrument in response to voice command from a user (col. 1, lines 6-8) while avoiding holding an endoscope (e.g., by an assistance) during a surgical procedure (col. 1, lines 21-27).
Regarding claim 6, Wang et al. disclose a method for performing minimally invasive surgical procedures using voice instruction wherein said at least one vocal command can be unqualified or can comprise at least one qualifier (e.g., voice instruction(s): “up,” “down,” “back,” and “in”), said at least one qualifier configured to modify at least one component of a response (col. 6, lines 45-48), and wherein at least one of the following:
said qualifier is selected from a group consisting of an amount, a surgical tool (e.g., endoscope), an apparatus, and any combination thereof (col. 6, lines 45-52 and Figure 1); and
said amount is either a fixed-term value or a fractional value.
Regarding claim 7, the claim limitations recited features on alternative form of rejected claim 6; therefore, Wang et al. in view of Devam et al. and Wang et al (‘361) still read on the claimed combination alternative form.
Regarding claims 8 and 14, Wang et al., as modified by Devam et al., teach a vocabulary / voice instructions to be input by a surgeon to control the movement of the endoscope to perform different task(s) (Wang et al.’s col. 6, lines 40-52), wherein the surgeon defines specific voice command for the endoscope to perform particular surgical procedure (Wang et al.’s col. 14, 8-11), which covers voice command to stop tracking / tagging the tool and/or tracking / tagging the tool for a predetermined time.
Regarding claims 15-16, Wang et al. failed to specifically disclose wherein tagging at least one of said at least one robotically controlled surgical tools and causing said at least one robotically controlled surgical endoscope to track movement of said at least one tagged robotically controlled surgical tools comprises (i) tagging only one of said at least one robotically controlled surgical tools, and causing said robotically controlled surgical endoscope to track movement of said only one tagged robotically controlled surgical tool (claim 15) and (ii) tagging a set of robotically controlled surgical tools, and causing said robotically controlled surgical endoscope to track movement of said set of tagged robotically controlled surgical tool (claim 16).
However, Devam et al. teach the technique of identifying a surgical instrument(s) based on user request and tracking the identified instrument(s) via a laparoscopic camera based on user vocal command during surgical procedure (par. 43, 26 and 37) – which covers only one or a set of robotically controlled surgical tool.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the controller for operating surgical equipment (e.g., endoscope) taught by Wang et al., such that the controller identifies a surgical instrument(s) based on user request (e.g., vocal command) and tracking the identified instrument(s) via a laparoscopic camara based on user vocal command during surgical procedure, in view of Devam et al., with reasonable expectation of success, since doing so would have achieved the benefit of allowing a surgeon to view and accurately locate patient organ and other features prior to and during surgery via laparoscopic camara thereby reducing morbidity and mortality due to surgical error associated with variance in patient feature location (par. 25).
Regarding claim 17, Wang et al. disclose a method for performing minimally invasive surgical procedures using voice instruction, the method comprising:
introducing a robotically controlled surgical endoscope and at least one surgical tool into a surgical environment in a body cavity (e.g., endoscope 28 and surgical instruments 22 inserted into a chest cavity of a patient to perform surgical procedure (col. 5, lines 41-46, col. 17, lines 34-36 and lines 62-65 and Figures 9 and 10A-10C));
detecting said at least one vocal command generated by at least one surgeon in a surgical setting (e.g., a surgeon inputs vocabulary / voice instruction(s) via a microphone to move an endoscope – see col. 6, lines 45-52 and Figure 1);
in response to said vocal command (e.g., a controller 46 moves the endoscope 28 and robotic arm assembly 26 in accordance with the input vocabulary / voice instruction(s) of the surgeon – col. 6, line 66 – col. 7, line 2 and col. 6, lines 34-35 and Figure 1).
Wang et al. disclose the controller 46 configured to move the endoscope 28 and robotic arm assembly 26 in accordance with the input vocabulary / voice instruction(s) of the surgeon – col. 6, line 66 – col. 7, line 2 and col. 6, lines 34-35 and Figure 1).
However, Wang et al. failed to specifically disclose establishing a persistent tracking state that causes said robotically controlled surgical endoscope to move within the body cavity to maintain visualization of said at least one surgical tool.
However, Devam et al. teach the technique of identifying a surgical instrument based on user request and tracking the identified instrument via a laparoscopic camera based on user vocal command during surgical procedure (par. 43, 26 and 37) to display the surgical instrument during a surgical procedure(par. 26, 43, 50), which requires the camera to move, for instance, within the patient body.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the controller for operating surgical equipment (e.g., endoscope) taught by Wang et al., such that the controller identifies a surgical instrument based on user request (e.g., vocal command) and tracking the identified instrument via a laparoscopic camara to display the surgical instrument during a surgical procedure based on user vocal command during surgical procedure, in view of Devam et al., with reasonable expectation of success, since doing so would have achieved the benefit of allowing a surgeon to view and accurately locate patient organ and other features prior to and during surgery via laparoscopic camara thereby reducing morbidity and mortality due to surgical error associated with variance in patient feature location (par. 25).
However, Wang et al., as modified by Devam et al., failed to specifically disclose to autonomously move the surgical endoscope within the body cavity as said at least one surgical tool moves within the body cavity.
However, Wang et al (‘361) teach an automated endoscope system configured to move an endoscope to automatically track a movement of an instrument that is inserted into a patient based on voice command (col. 4, lines 9-15 and col. 3, lines 36-41).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to further modify the controller for operating surgical equipment (e.g., endoscope) as taught by the combination of Wang et al. in view of Devam et al., such that the controller moves the endoscope to automatically track a movement of an instrument that is inserted into a patient based on voice command, in view of Wang et al (‘361), with reasonable expectation of success, since doing so would have achieved the benefit of moving a surgical instrument in response to voice command from a user (col. 1, lines 6-8) while avoiding holding an endoscope (e.g., by an assistance) during a surgical procedure (col. 1, lines 21-27).
Regarding claim 20, Regarding claim 6, Wang et al. disclose a method for performing minimally invasive surgical procedures using voice instruction wherein said at least one vocal command can be unqualified or can comprise at least one qualifier (e.g., voice instruction(s): “up,” “down,” “back,” and “in”), said at least one qualifier configured to modify at least one component of a response (col. 6, lines 45-48), and wherein at least one of the following:
said qualifier is selected from a group consisting of an amount, a surgical tool (e.g., endoscope), an apparatus, and any combination thereof (col. 6, lines 45-52 and Figure 1); and
said amount is either a fixed-term value or a fractional value.
Regarding claim 21, the claim limitations recited features on alternative form of rejected claim 20; therefore, Wang et al. in view of Devam et al. and Wang et al (‘361) still read on the claimed combination alternative form.
Regarding claims 22 and 24, Wang et al., as modified by Devam et al., teach a vocabulary / voice instructions to be input by a surgeon to control the movement of the endoscope to perform different task(s) (Wang et al.’s col. 6, lines 40-52), wherein the surgeon defines specific voice command for the endoscope to perform particular surgical procedure (Wang et al.’s col. 14, 8-11), which covers voice command to stop tracking / tagging the tool and/or tracking / tagging the tool for a predetermined time.
Regarding claim 25-26, Wang et al. failed to specifically disclose wherein tagging at least one of said at least one robotically controlled surgical tools and causing said at least one robotically controlled surgical endoscope to track movement of said at least one tagged robotically controlled surgical tools comprises (i) tagging only one of said at least one robotically controlled surgical tools, and causing said robotically controlled surgical endoscope to track movement of said only one tagged robotically controlled surgical tool (claim 15) and (ii) tagging a set of robotically controlled surgical tools, and causing said robotically controlled surgical endoscope to track movement of said set of tagged robotically controlled surgical tool (claim 16).
However, Devam et al. teach the technique of identifying a surgical instrument(s) based on user request and tracking the identified instrument(s) via a laparoscopic camera based on user vocal command during surgical procedure (par. 43, 26 and 37) – which covers only one or a set of robotically controlled surgical tool.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the controller for operating surgical equipment (e.g., endoscope) taught by Wang et al., such that the controller identifies a surgical instrument(s) based on user request (e.g., vocal command) and tracking the identified instrument(s) via a laparoscopic camara based on user vocal command during surgical procedure, in view of Devam et al., with reasonable expectation of success, since doing so would have achieved the benefit of allowing a surgeon to view and accurately locate patient organ and other features prior to and during surgery via laparoscopic camara thereby reducing morbidity and mortality due to surgical error associated with variance in patient feature location (par. 25).
Regarding claim 27, Wang et al. disclose a method for performing minimally invasive surgical procedures using voice instruction, wherein said at least one surgical tool is a robotically controlled surgical tool (e.g., endoscope 28 and surgical instruments 22 inserted into a chest cavity of a patient to perform surgical procedure (col. 5, lines 41-46, col. 17, lines 34-36 and lines 62-65 and Figures 9 and 10A-10C));
Claims 4 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Pat. No.: US 6,063,095) in view of Devam et al. (Pub. No.: US 2016/0249989 A1, Wang et al. (Pat. No.: US 6,463,361 B1) and Scheib et al. (Pub. No.: US 2020/0015901 A1).
Regarding claims 4 and 18, further modified Wang et al. failed to specifically disclose wherein at least one response to at least one said at least one vocal command is a smart response, said smart response being configured to take into account at least one feature of the an environment surrounding said robotically controlled endoscope, said feature selected from a group consisting of a hazard or obstacle, an interference with a second surgical tool, an interference between two parts of an apparatus, interference between two apparatus, possibility of damage to a portion of a patient, and any combination thereof.
However, Scheib et al. teach a process for applying control motion to a surgical tool (e.g., gasper) to automatically navigate around a tissue structure during a surgical procedure (par. 188).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to furthermore modify the controller for operating surgical equipment (e.g., the endoscope and robotic arm assembly) via a voice instruction(s) as taught by the combination of Wang et al. in view of Devam et al. and Wang et al. ( ‘361), such that the controller applies control motion to surgical tool (e.g., endoscope) to automatically navigate around a tissue structure during a surgical procedure, in view of Scheib et al., with reasonable expectation of success, since doing so would have achieved the benefit of avoiding certain areas in order prevent inadvertent dissection of a critical structure of patient’s organ during a surgical procedure (par. 70).
Claims 5 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Pat. No.: US 6,063,095) in view of Devam et al. (Pub. No.: US 2016/0249989 A1), Wang et al. (Pat. No.: US 6,463,361 B1) and Jung (Pub. No.: US 2009/0299751 A1).
Regarding claims 5 and 19, Wang et al. disclose a method for performing minimally invasive surgical procedures using voice instruction, wherein a surgeon inputs command (e.g., voice instruction: “up,” “down,” “back,” and “in”) via a microphone to control an endoscope (col. 6, lines 40-48).
However, further modified Wang et al. failed to specifically disclose wherein said at least one vocal command comprises at least one complex sentence, and wherein the method further includes recognizing said at least one vocal command from said complex sentence.
However, Jung teaches a robot configured to received voice command from a user for controlling the robot to perform a predetermined task; wherein the voice command comprise a complex sentence – e.g., “get me a glass of water from a dining room” (par. 38 and 7).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to furthermore modify the controller for operating surgical equipment (e.g., the endoscope and robotic arm assembly) as taught by the combination of Wang et al. in view of Devam et al. and Wang et al. ( ‘361), such that the controller received and processes user complex voice command, in view of Jung, with reasonable expectation of success, since doing so would have achieved the benefit of controlling a robot using complex voice command (par. 43).
Claims 13 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Pat. No.: US 6,063,095) in view of Devam et al. (Pub. No.: US 2016/0249989 A1), Wang et al. (Pat. No.: US 6,463,361 B1) and Lee et al. (Pub. No.: US 2018/0249973 A1).
Regarding claims 13 and 23, further modified Wang et al. failed to specifically disclose estimating a 3D spatial position of said at least one surgical tool based on data from said real-time image.
However, Lee et al. teach estimating a three-dimensional location of a surgical tool based on image during a surgical operation (par. 43, 26).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to furthermore modify the controller for operating surgical equipment (e.g., the endoscope and robotic arm assembly) as taught by the combination of Wang et al. in view of Devam et al. and Wang et al. (‘361), such that the controller estimating a three-dimensional location of a surgical tool based on image, in view of Lee et al., with reasonable expectation of success, since doing so would have achieved the benefit of estimating the posture of a surgical instrument by using tracking technique and markers while reducing large incision on patient during surgical operation (par. 7 and 5).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Pandya et al. (US 2013/0331644) is directed to an intelligent autonomous camera configured to follow a tool and keep the tool in the center of view.
Wang et al. (US 6,965,812 B2) is directed to an endoscope system configured to automatically track the movement of a second instrument that is inserted into the patient based on voice command.
This application is a continuation application of U.S. application no. 18/084,477 filed on December 19, 2022, now U.S. Patent 11,886,772 (“Parent Application”) which is a continuation application of U.S. application no. 17/064,549 filed on October 6, 2020, now U.S. Patent 11,561,762 (“Parent Application”). See MPEP §201.07. In accordance with MPEP §609.02 A. 2 and MPEP §2001.06(b) (last paragraph), the Examiner has reviewed and considered the prior art cited in the Parent Application. Also, in accordance with MPEP §2001.06(b) (last paragraph), all documents cited or considered ‘of record’ in the Parent Application are now considered cited or ‘of record’ in this application. Additionally, Applicant(s) are reminded that a listing of the information cited or ‘of record’ in the Parent Application need not be resubmitted in this application unless Applicant(s) desire the information to be printed on a patent issuing from this application. See MPEP §609.02 A. 2. Finally, Applicant(s) are reminded that the prosecution history of the Parent Application is relevant in this application.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jorge O. Peche whose telephone number is (571)270-1339. The examiner can normally be reached Monday-Friday 8:30 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi H. Tran can be reached on 571 272 6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.O.P/Examiner, Art Unit 3656 /KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656