Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 2, 2026 has been entered.
Response to Amendment
Applicant's response to the last office action, filed April 6, 2026 has been entered and made of record. Claims 49, 51-52, 55-56, 58-59, 62-63, 65-66 have been amended; claims 50, 57, 64 have been cancelled; and claims 1-48 were previously cancelled. By this amendment, claims 49, 51-56, 58-63, and 65-68 are currently pending in this application.
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required:
a. The specification is objected to because of lacking support for the limitations “the portion of the user interface in the frame indicative of a surgical phase”; and ”determining a derived data value identifying a phase boundary of the surgical phase”, cited in claims 49, 56, and 63.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 49, 51-56, 58-63, and 65-68 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claims 49, 56, and 63 recite the following limitations: “the portion of the user interface in the frame indicative of a surgical phase”; and ”determining a derived data value identifying a phase boundary of the surgical phase”. These limitations do not have any support from the specification.
For the purpose of prior art consideration, the claims in question will be construed as best understood.
Allowable Subject Matter
Claims 49, 51-56, 58-63, and 65-68 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, set forth in this Office action.
The following is a statement of reasons for the indication of allowable subject matter:
-- Claims 49, 56, and 63 are allowable over the prior art of record.
-- Claims 51-55 are allowable in view of their dependency from claim 49
-- Claims 58-62 are allowable in view of their dependency from claim 56
-- Claims 65-68 are allowable in view of their dependency from claim 63.
With respect to claim 49, the prior art of record, alone or in reasonable combination, does not teach or suggest, the following limitation(s), (in consideration of the claim as a whole):
“determining a derived data value identifying a phase boundary of the surgical phase by consolidating data derived from the detected portion of the user interface, the recognized text, the detected surgical tool, and the optical flow into consolidated derived data”.
The relevant prior art of record, Donhowe et al, (US-Patent 11,443,501) discloses a computer-implemented method for determining derived data from surgical video data, the method comprising:
detecting one or more user interface elements in a portion of a user interface in a frame of a plurality of frames of surgical video data, (see at least: col. 5, lines 43-36, the robotic surgical device 114 may be equipped with one or more cameras 128, such as an endoscope camera, configured to provide a view of the operating site to guide the surgeon 102 during the surgery, and from col. 6, lines 8-13, surgical images 132 of a robotic surgical procedure captured by the camera 128 can also be displayed on a video monitor 108 in real time so that the surgeon 102 can view the procedure while the surgical tools 126 are being operated on the patient 130, “i.e., displaying surgical images 132 captured by the camera 128, to view the procedure while the surgical tools 126 are being operated on the patient 130, by the surgeon 102”. Further, from col. 7, line 65 through col. 8, line 18, safety module 120 can analyze the surgical images 132 captured during the surgical procedure ….; where the analysis can include identifying surgical tools 126 used in the surgical procedure from the surgical images 132, for example, through object recognition or other image/video processing techniques, [i.e., detecting one or more user interface elements, “identifying surgical tools 126 from the surgical images 132 through object recognition”, in a portion of a user interface, “displaying the surgical tools 126 based on displaying the surgical images 132 on video monitor 108”, in a frame of a plurality of frames of surgical video data, “in at least one image frame of a plurality of image frames of the surgical images, captured by one or more cameras 128”]); and
detecting the surgical tool in the frame and tracking the surgical tool across a plurality of frames, ((see at least: col. 5, lines 43-36, col. 6, lines 8-13, and col. 7, line 65 through col. 8, line 18, where the surgical tool corresponds to the tool, “see the rejection of claim 49 above, for more details”. Further from col. 6, lines 22-26, surgeon 102 can then move the camera 128 to a position where a surgical tool 126 to be used is located to bring the surgical tool 126 inside the field of view of the camera 128, [i.e., tracking the tool, “surgical tool”, using at least one tracker, “camera”]);
calculating an optical flow of movement of the surgical tool between at least the frame and a subsequent frame of the plurality of frames of the surgical video data, (see at least: col. 4, lines 29-33, a kinematic model of the robotic arm holding the surgical tool implicitly determines a motion of surgical tool between the frame and a subsequent frame);
determining the derived data value based upon the calculated optical flow, (see at least: col. 6, lines 46-50, robotic surgical device 114 can include a position predictor 118 to predict the position of the surgical tool 126, through a kinematic chain model of the robotic arm of the robotic surgical device 114 holding the surgical tool 126, [i.e., determining the derived data value, “position of the surgical tool 126”, based upon the calculated optical flow, “based motion of the surgical tool through the kinematic model of the robotic arm holding the surgical tool”]). Further, from col. 7, line 65 through col. 8, line 24, identifying the presence of surgical tools based on the position of the surgical tools in surgical images, “see the Final office action-rejection of claim 49 for more details”).
However, while disclosing determining a derived data value based upon the calculated optical flow; Donhowe fails to teach or suggest, either alone or in combination with the other cited references, determining a derived data value identifying a phase boundary of the surgical phase by consolidating data derived from the detected portion of the user interface, the recognized text, the detected surgical tool, and the optical flow into consolidated derived data.
A further prior art of record, Wolf et al, (US-PGPUB 20200273581) discloses detecting one or more user interface elements in a portion of a user interface in a frame of a plurality of frames of surgical video data, the portion of the user interface in the frame indicative of a surgical phase, (see at least: Par. 0104, an interface may allow a surgeon to review surgical video (of their own surgeries, other's surgeries, or compilations) with a surgical timeline simultaneously displayed. Further, from Par. 0109, 0111, the surgical timeline may include markers identifying at least one of a surgical phase within a surgical procedure, [i.e., the portion of the user interface in the frame, “implicitly portion of surgical timeline”, indicative of a surgical phase, “surgical timeline provides surgical phases within a surgical procedure”]).
However, Wolf et al fails to teach or suggest, either alone or in combination with the other cited references, determining a derived data value identifying a phase boundary of the surgical phase by consolidating data derived from the detected portion of the user interface, the recognized text, the detected surgical tool, and the optical flow into consolidated derived data.
Regarding claim 56, claim 56 recites substantially similar limitations as set forth in claim 49. As such, claim 56 is in condition for allowance, for at least similar reasons, as stated above.
Regarding claim 63, claim 63 recites substantially similar limitations as set forth in claim 49. As such, claim 63 is in condition for allowance, for at least similar reasons, as stated above.
Other prior art listed on the attached form PTO-892 show various aspects of the
invention but none, either alone or in combination, teach or suggest all the claimed limitations.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMARA ABDI whose telephone number is (571)272-0273. The examiner can normally be reached 9:00am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMARA ABDI/Primary Examiner, Art Unit 2668 04/17/2026
Prosecution Insights