DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 7/17/2026 have been fully considered but they are not persuasive.
Applicant auges there’s no teaching in the prior arts for “receiving an input of auxiliary information reported by at least one medica device used during the surgical procedure;…analyzing the plurality of frames of the surgical footage and auxiliary information…” The Examiner disagrees. Figure 1 and paragraph 42 of Barral et al teach “The computing device 107 receives video signals from the camera, including any associated metadata generated at or by the camera”. Paragraph 43 of Barral et al teach: “computing device 107 then identifies segments in the surgical video by analyzing at least one of the surgical video or the associated metadata” and paragraph 39 specify camera 101 is part of the surgical robot 121. So These teaching corresponds to the claimed “receiving an input of auxiliary information reported by at least one medica device used during the surgical procedure;…analyzing the plurality of frames of the surgical footage and auxiliary information…”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21-27, 29-37 and 39-40 are rejected under 35 U.S.C. 103 as being unpatentable over Farjo et al (US 2015/0227501) and further in view of Barral et al (US 2019/0110856).
For claim 21, Farjo et al teach a non-transitory computer readable medium containing instructions that, when executed by at least one processor, cause the at least one processor to execute operations enabling automatically populating a post-operative report of a surgical procedure (e.g. abstract: processor), the operations comprising:
receiving an input of an identifier of a patient (e.g. paragraphs 50-52: data is loaded in the memory system 14, paragraph 52: patient information may included, for example, patient name…);
receiving an input of an identifier of a health care provider (e.g. paragraph 51: For example, information about the name and address of the hospital or surgery center is save in a database in the memory system 14);
receiving an input of surgical footage of a surgical procedure performed on the patient by the health care provider (e.g. paragraph 70: user is prompted to add operative photographs 48 showing the problem for inclusion in the report…);
analyzing a plurality of frames of the surgical footage to derive image-based information for populating a post-operative report of the surgical procedure (e.g. figure 10, paragraph 70: the user may select and drag one or more photographs to locations on a graphic background indicating the anatomic structure at issue); and
causing the derived image-based information to populate the post-operative report of the surgical procedure (e.g. figures 3-9, paragraph 61: the operative report overview screen includes an image 24 of the procedure to be performed for quick visual confirmation by the user).
Farjo et al do not further disclose analyzing the surgical footage to identify a surgical event within the surgical footage; derived image-based information being based on the identified surgical event; receiving an input of auxiliary information reported by at least one medical device used during the surgical procedure; analyzing the auxiliary information.
Barral et al teach analyzing the surgical footage to identify a surgical event within the surgical footage; derived image-based information being based on the identified surgical event (e.g. abstract, paragraph 116: “the surgical video may be provided to one or more ML techniques to recognize the type of the surgical procedure as well as steps, sub-steps, or events within the surgical video. As each ML technique analyzes the surgical video, it may update existing bookmark information, or it may output information that the computing device 107 can incorporate into existing bookmarks (or create new bookmarks, as appropriate if no corresponding bookmark exists)”, figure 4, bookmarks);
receiving an input of auxiliary information reported by at least one medical device used during the surgical procedure; analyzing the auxiliary information (e.g. Figure 1 and paragraph 42 of Barral et al teach “The computing device 107 receives video signals from the camera, including any associated metadata generated at or by the camera”. Paragraph 43 of Barral et al teach: “computing device 107 then identifies segments in the surgical video by analyzing at least one of the surgical video or the associated metadata” and paragraph 39 specify camera 101 is part of the surgical robot 121.).
It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
Claims 39 and 40 are rejected for the same reasons as discussed in claim 21 above.
For claim 26, Farjo et la teach transmitting data to the health care provider, the transmitted data including the patient identifier and the derived image-based information (e.g. paragraph 44, figure 26).
For claim 27, Farjo et al teach analyzing the surgical footage to identify at least one recommendation for post-operative treatment; and providing the identified at least one recommendation (e.g. figures 12-14, paragraph 73: “The system may display default treatment techniques 54 selected based on, the most common treatment techniques for the selected procedure”).
For claim 29, Farjo et al teach the caused populating of the post-operative report of the surgical procedure is configured to cause at least part of the derived image-based information to be identified in the post- operative report as automatically generated data (e.g. figures 12-14, paragraph 73: “The system may display default treatment techniques 54 selected based on, the most common treatment techniques for the selected procedure”).
For claim 33, Farjo et al teach the operations further comprise providing the derived image-based information in a form enabling updating of an electronic medical record (e.g. figure 5, paragraph 62: “edit different portions of the operative note”).
For claim 34, Farjo et al teach the derived image-based information is based in part on user input (e.g. figure 10, paragraph 70: the user may select and drag one or more photographs to locations on a graphic background indicating the anatomic structure at issue). Farjo et al do not further disclose the surgical footage is being captured to indicate a particular point in time within the surgical footage. Barral et al teach the surgical footage is being capture to indicate a particular point in time within the surgical footage (e.g. Figure 5, “Creation of Pouch” start at 3:02, Gastrojejunal Anastomosis starts at 22:12). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 36, Farjo et al teach analyzing the surgical footage to select at least part of at least one frame of the surgical footage; and causing the selected at least part of at least one frame of the surgical footage to be included in the post-operative report of the surgical procedure (e.g. figure 10, paragraph 70: the user may select and drag one or more photographs to locations on a graphic background indicating the anatomic structure at issue).
For claim 37, Farjo et al teach receiving a preliminary post-operative report; and analyzing the preliminary post-operative report and the surgical footage to select the at least part of at least one frame of the surgical footage (e.g. figure 10, paragraph 70: the user may select and drag one or more photographs to locations on a graphic background indicating the anatomic structure at issue).
For claim 22, Farjo et al do not further disclose analyzing the surgical footage to identify one or more phases of the surgical procedure and to identify a property of at least one phase of the identified phases; and wherein the derived image-based information is based on the identified at least one phase and the identified property of the at least one phase. Barral et la teach analyzing the surgical footage to identify one or more phases of the surgical procedure and to identify a property of at least one phase of the identified phases; and wherein the derived image-based information is based on the identified at least one phase and the identified property of the at least one phase (e.g. figure 5: marker 422a: “Creation of Pouch”, marker 422b: “Gastrojejunal Anastomosis”, marker 422c: “Measurement of Alimentary Limb”). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 23, Farjo et al do not further disclose analyzing the surgical footage to associate a name with the at least one phase; and wherein the derived image-based information includes the name associated with the at least one phase. Barral et al teach analyzing the surgical footage to associate a name with the at least one phase; and wherein the derived image-based information includes the name associated with the at least one phase. (e.g. figure 5: marker 422a: “Creation of Pouch”, marker 422b: “Gastrojejunal Anastomosis”, marker 422c: “Measurement of Alimentary Limb”). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 24, Farjo et al do not further disclose determining at least a beginning of the at least one phase; and wherein the derived image-based information is based on the determined beginning. Barral et al teach determining at least a beginning of the at least one phase; and wherein the derived image-based information is based on the determined beginning (e.g. Figure 5, “Creation of Pouch” start at 3:02, Gastrojejunal Anastomosis starts at 22:12). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 25, Farjo et al do not further disclose associating a time marker with the at least one phase; and wherein the derived image-based information includes the time marker indicate a time range associated with the at least one phase. Barral et al teach associating a time marker with the at least one phase; and wherein the derived image-based information includes the time marker indicate a time rage associated with the at least one phase (e.g. Figure 5, “Creation of Pouch” start at 3:02 and ends at 22:12, Gastrojejunal Anastomosis starts at 22:12 and ends at 38:04). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 30, Farjo et al do not further disclose analyzing the surgical footage to identify a property of the identified surgical event; and wherein the derived image-based information is further based on the identified property. Barral et al teach analyzing the surgical footage to identify a property of the identified surgical event; and wherein the derived image-based information is further based on the identified property. (e.g. figure 5: marker 422a: “Creation of Pouch”, marker 422b: “Gastrojejunal Anastomosis”, marker 422c: “Measurement of Alimentary Limb”). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 32, Farjo et al do not further disclose associating a time marker indicate of a time range with the identified surgical event; and wherein the derived image-based information includes the time marker indicate of time range.
Barral et al teach associating a time marker indicated of time range (e.g. Figure 5, “Creation of Pouch” start at 3:02 and ends at 22:12, Gastrojejunal Anastomosis starts at 22:12 and ends at 38:04) with the identified surgical event; and wherein the derived image-based information includes the time marker indicate of time range (e.g. abstract, paragraph 116: “the surgical video may be provided to one or more ML techniques to recognize the type of the surgical procedure as well as steps, sub-steps, or events within the surgical video. As each ML technique analyzes the surgical video, it may update existing bookmark information, or it may output information that the computing device 107 can incorporate into existing bookmarks (or create new bookmarks, as appropriate if no corresponding bookmark exists)”, figure 4, bookmarks). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 31, Farjo et al do not further disclose analyzing the surgical footage to determine an event name of the identified surgical event; and wherein the derived image-based information includes the determined event name. Barral et al teach analyzing the surgical footage to determine an event name of the identified surgical event; and wherein the derived image-based information includes the determined event name (e.g. paragraph 143: “As discussed above, each of these lists may include lists of standardized procedure, step, and event names. For example, gastric bypass procedures may be represented by the standard procedure name ‘Gastric Bypass.’ ” or Figure 5: “Creation of Pouch”, “Gastrojejunal Anastomosis” “Measurement of Alimentary Limb”). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
For claim 35, Farjo et al do not disclose the derived image-based information comprises a first part associated with a first portion of the surgical procedure and a second part associated with a second portion of the surgical procedure, and wherein the operations further comprise: receiving a preliminary post-operative report; analyzing the preliminary post-operative report to select a first position and a second position within the preliminary post-operative report, the first position is associated with the first portion of the surgical procedure and the second position is associated with the second portion of the surgical procedure; and causing the first part of the derived image-based information to be inserted at the selected first position and the second part of the derived image-based information to be inserted at the selected second position. Barral et al teach the derived image-based information comprises a first part associated with a first portion of the surgical procedure and a second part associated with a second portion of the surgical procedure, and wherein the operations further comprise: receiving a preliminary post-operative report; analyzing the preliminary post-operative report to select a first position and a second position within the preliminary post-operative report, the first position is associated with the first portion of the surgical procedure and the second position is associated with the second portion of the surgical procedure; and causing the first part of the derived image-based information to be inserted at the selected first position and the second part of the derived image-based information to be inserted at the selected second position (e.g. figure 8: Books 422a, 422b, 810, 820, 422c…are inserted on the timeline 412 of the video). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Barral et al into the teaching of Farjo et al to help surgeon to view surgical footage to improve the surgeon’s proficient with a particular type of surgery (e.g. paragraph 5, Barral et al).
Allowable Subject Matter
Claims 28, 38 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and if the Double Patenting rejecting was overcome.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAQUAN ZHAO whose telephone number is (571)270-1119. The examiner can normally be reached M-Thur: 7:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Tran can be reached on 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Email: daquan.zhao1@uspto.gov.
Phone: (571)270-1119
/DAQUAN ZHAO/Primary Examiner, Art Unit 2484
Prosecution Insights