Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 03/24/2026, 12/05/2025, 05/15/2025, 10/13/2023 are being considered by the examiner.
Priority
Acknowledgement is made of applicant’s claim for priority based on DE10 2021 109 118.7 dated 04/13/2021.
Drawings
The applicant’s drawings submitted are acceptable for examination purposes.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5 and 7-14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Leiderman (WO2020163845A2), as cited in the IDS. All references to Leiderman (WO2020163845A2) will be made in reference to English language equivalence Leiderman (US20220104884A1).
Regarding claim 1, Leiderman teaches in Fig. 8: a system for an ophthalmic microscope system (“ IGSS 800”; [0074], Fig. 8, “image-guided surgical system (IGSS)”; [0051]), the system (800) comprising one or more processors (“a computer processor 806”; [0074]) and one or more storage devices (“a memory device 808”; [0074]), wherein the system (800) is configured to:
obtain intraoperative sensor data of an eye from at least one imaging device (“image data received from the imaging system 802”; [0074], “real-time cross-sectional images”; [0084]) of the ophthalmic microscope system (800);
process the intraoperative sensor data using a machine-learning model, the machine-learning model being trained to output information on one or more anatomical features of the eye based on the intraoperative sensor data (“An analysis loop can be configured to distinguish anatomical tissue information from the data and to train the feedback loop via machine learning to discriminate between anatomical layers”; [0012]); and
generate a display signal for a display device (“display 810”; [0074]) of the ophthalmic microscope system (800) based on the information on the one or more anatomical features of the eye (“It is contemplated that all IGSS systems would include the display 810, though the display 810 may, in various embodiments, provide more or less feedback to the surgeon, and may provide that feedback in a variety of forms”; [0074]), the display signal comprising a visual guidance overlay for guiding a user of the ophthalmic microscope system with respect to the one or more anatomical features of the eye (“the display 810 may also include a depiction, in real-time, within the surgical field, of a surgical instrument 816 wielded by the surgeon, and may identify on the display a tip or working end of the surgical instrument 816”; [0075], “the IGSS 800 may, using the determined distance, provide real-time visual, auditory, and/or haptic feedback to the surgeon, via the display 810, the speaker 812, and/or the haptic feedback system 814”; [0076]).
Regarding claim 2, Leiderman teaches the system according to claim 1. Leiderman further teaches: the visual guidance overlay comprises an annotation of the one or more anatomical features of the eye, suitable for guiding a user of the ophthalmic microscope system during a surgical procedure (“When the surgical instrument approaches the retinal surface with a system constructed in accordance with the principles herein, two changes in the system occur happen: first, the color of the dot changes from green to red (Fig. 6), indicating that the surgical instrument is close or proximal to the boundary of biologic tissue that has been defined as an avoidance or exclusion zone, in this case the retina”; [0050], “the IGSS provides a computer-augmented image and/or other feedback to a surgeon to allow the surgeon to reliably recognize tissue boundaries in the surgical field and understand the relationship between those tissue boundaries and a surgical instrument”; [0061]).
Regarding claim 3, Leiderman teaches the system according to claim 1. Leiderman further teaches: the system (800) is configured to obtain the intraoperative sensor data as a continuously updated stream of intraoperative sensor data, and wherein the system is configured to update the visual guidance overlay based on the continuously updated stream of intraoperative sensor data (“the IGSS facilitates the provision of real-time actionable image data and feedback to a surgeon during a surgical procedure”; [0051]).
Regarding claim 4, Leiderman teaches the system according to claim 1. Leiderman further teaches: the system is configured to overlay the visual guidance overlay over a visual representation of the intraoperative sensor data within the display signal (“When the surgical instrument approaches the retinal surface with a system constructed in accordance with the principles herein, two changes in the system occur happen: first, the color of the dot changes from green to red (Fig. 6), indicating that the surgical instrument is close or proximal to the boundary of biologic tissue that has been defined as an avoidance or exclusion zone, in this case the retina”; [0061], “the IGSS provides a computer-augmented image and/or other feedback to a surgeon to allow the surgeon to reliably recognize tissue boundaries in the surgical field and understand the relationship between those tissue boundaries and a surgical instrument”; [0051]).
Regarding claim 5, Leiderman teaches the system according to claim 1. Leiderman further teaches: the system (800) is configured to generate the visual guidance overlay with one or more of a plurality of visual indicators (“the IGSS provides a computer-augmented image and/or other feedback to a surgeon to allow the surgeon to reliably recognize tissue boundaries in the surgical field and understand the relationship between those tissue boundaries and a surgical instrument”; [0051]),
the plurality of visual indicators comprising one or more of a textual annotation of at least a subset of the one or more anatomical features (“text warnings or status may be added to the images displayed on the display 810 to provide a state of the parameter”; [0144]),
an overlay for highlighting one or more surfaces of the one or more anatomical features (“The identified tissue boundaries, may be added to the image data to provide augmented images for display to the surgeon on the display 810 and/or the identified tissue boundaries may be used to identify and highlight tissue layers (as opposed to merely boundaries) to provide augmented images for display to the surgeon on the display 810”; [0079]),
an overlay for highlighting one or more edges of the one or more anatomical features (“The identified tissue boundaries, may be added to the image data to provide augmented images for display to the surgeon on the display 810”; [0079]), and
one or more directional indicators, and
one or more indicators related to one or more anomalies regarding the one or more anatomical features (“the surgeon may receive feedback regarding aspects of the anatomy and/or the surgical procedure including, but not limited to: location of an identified tissue boundary; proximity of a surgical instrument tip to an identified tissue boundary; tissue deformation (e.g., resulting from manipulation of the tissue with a surgical instrument or movement of the surgical instrument within the surgical field); tissue volume; tissue attachments (e.g., retinal detachment); shear stress on the tissue; tissue movement; tissue position; exposed area of the tissue; and occluded area of the tissue, i.e., occlusion of any instrument with tissue”; [0140]).
Regarding claim 7, Leiderman teaches the system according to claim 1. Leiderman further teaches: the intraoperative sensor data comprises intraoperative optical coherence tomography sensor data (“OCT data are processed and reconstructed to display three-dimensional anatomic representations that are offset to accommodate human stereoscopic visual perception”; [0065]), wherein
the machine-learning model is trained to output information on one or more layers of the eye based on the intraoperative optical coherence tomography sensor data (“image data from the iOCT system (the imaging system 802) may be input into a trained AI model”; [0086], “an analysis loop configured to distinguish ocular tissue data from OCT data, and further configured to train the feedback loop via machine learning to discriminate among anatomical layers”; [0225]), wherein
the system (800) is configured to generate the visual guidance overlay with a visual indicator highlighting or annotating at least a subset of the one or more layers of the eye (“The identified tissue boundaries, may be added to the image data to provide augmented images for display to the surgeon on the display 810 and/or the identified tissue boundaries may be used to identify and highlight tissue layers (as opposed to merely boundaries) to provide augmented images for display to the surgeon on the display 810”; [0079]).
Regarding claim 8, Leiderman teaches the system according to claim 1. Leiderman further teaches: the machine-learning model is trained to output information on a classification of the one or more anatomical features within the intraoperative sensor data (“a trained artificial intelligence (AI) model configured to identify tissue boundaries and/or tissue types based on the image data”; [0162], “an autonomous guidance system requires the ability to discriminate between the different tissue types segmented in the image”; [0118]),
wherein the system is configured to generate the visual guidance overlay with a visual indicator related to the classification of the one or more anatomical features (“The identified tissue boundaries, may be added to the image data to provide augmented images for display to the surgeon on the display 810 and/or the identified tissue boundaries may be used to identify and highlight tissue layers (as opposed to merely boundaries) to provide augmented images for display to the surgeon on the display 810”; [0079]).
Regarding claim 9, Leiderman teaches the system according to claim 1. Leiderman further teaches: the machine-learning model is trained to output information on one or more anomalies regarding the one or more anatomical features of the eye (“The Al model receives as an input the image data obtained from the iOCT imaging system and provides a segmentation probability map of the location of the tissue in question (e.g., the retina, the lens, etc.). The segmentation probability map may also, in embodiments, provide utility measures such as the relative area change and/or volume change of the tissue (e.g., the retina, the lens, etc.) between different images (useful for the surgeon to estimate how much the tissue’s area is changing, therefore providing information about the amount of stress the tissue is undergoing at a particular instant in the procedure), the relative change in height of the tissue between images (provides a similar, but different, type of stress indication), the position and/or motion of the tissue relative to adjacent ocular tissues (e.g., to identify retinal detachment), or identify occlusion of an instrument by a tissue, etc.”; [0087]), wherein
the system is configured to generate the visual guidance overlay with a visual indicator related to the one or more anomalies (“the surgeon may receive feedback regarding aspects of the anatomy and/or the surgical procedure including, but not limited to: location of an identified tissue boundary; proximity of a surgical instrument tip to an identified tissue boundary; tissue deformation (e.g., resulting from manipulation of the tissue with a surgical instrument or movement of the surgical instrument within the surgical field); tissue volume; tissue attachments (e.g., retinal detachment); shear stress on the tissue; tissue movement; tissue position; exposed area of the tissue; and occluded area of the tissue, i.e., occlusion of any instrument with tissue”; [0140]).
Regarding claim 10, Leiderman teaches the system according to claim 9. Leiderman further teaches: the intraoperative sensor data comprises first intraoperative sensor data from a first imaging device and second intraoperative sensor data from a second imaging device (“a stream of data comprising consecutive images from each of two (or more) cameras”; [0124], see Fig. 18),
wherein the system is configured to generate the display signal with a first visual representation of the first intraoperative sensor data and with a second visual representation of the second intraoperative sensor data (“a stream of data comprising consecutive images from each of two (or more) cameras”; [0124], ,
wherein the system is configured to overlay a visual indicator of an anomaly detected by the machine-learning model based on the first intraoperative sensor data (“The output of this step is a disparity map of the entire scene where landmarks are in the same coordinates system of the left frame”; [0130], “an average disparity value for the retina”; [0130]) over a corresponding position of the second visual representation of the second intraoperative sensor data within the display signal (“the output of the example pipeline 900 acting on stereoscopic images received from the imaging system. In each, the pipeline 900 successfully identifies the tip of a surgical instrument in the image data of the surgical field, highlighting the tip of the surgical instrument in each frame with a box”; [0137], “binocular optical microscopes have been supplanted by digital stereoscopic viewing systems that allow for three-dimensional perception of the surgical field using sterocameras and digital displays”; [0121]).
Regarding claim 11, Leiderman teaches the system according to claim 1. Leiderman further teaches: the system is configured to detect a presence of one or more surgical instruments in the intraoperative sensor data (“a trained AI model may implement tip localization module 822. The trained AI model may be trained according to principles similar to those used to train the image segmentation module 820 when it is implemented as an AI model. That is, the model may be trained using a series of images of instrument tips present in the surgical field, in which images the tip of the surgical instrument has been manually marked/identified”; [0106]), to determine a distance between the detected one or more surgical instruments and the one or more anatomical features (“analysis module 824 may compare the identified coordinates of the tip of the surgical instrument to the coordinates of the tissue boundary”; [0119]), and to generate the visual guidance overlay with a visual indicator representing the distance between the detected one or more surgical instruments and the one or more anatomical features (“and an indication 948 of a tip of the surgical instrument 940. The indication 948 may be displayed in a particular color (e.g., green) and/or style (e.g., square with 1 pt line, square with a first brightness) to indicate that a determined distance between the tip of the surgical instrument 940 and the tissue boundary is sufficiently large that there is no danger to the tissue”; [0145]).
Regarding claim 12, Leiderman teaches the system according to claim 1. Leiderman further teaches: the intraoperative sensor data comprises one or more of intraoperative optical coherence tomography sensor data of an intraoperative optical coherence tomography device of the ophthalmic microscope system (“image data from the iOCT system (the imaging system 802) may be input into a trained AI model”; [0086]),
intraoperative imaging sensor data of an imaging sensor of a microscope of the ophthalmic microscope system (“binocular optical microscopes have been supplanted by digital stereoscopic viewing systems that allow for three-dimensional perception of the surgical field using sterocameras and digital displays”; [0121]), and
intraoperative endoscope sensor data of an endoscope of the ophthalmic microscope system.
Regarding claim 13, Leiderman teaches the system according to claim 1. Leiderman further teaches: an ophthalmic microscope system (“intraoperative Optical Coherence Tomography (iOCT) image-guided microsurgery of the eye”; [0068]) comprising at least one imaging device (“imaging system 802”; [0105]), a display device (“display 810”; [0074]), and the system (800) according to claim 1.
Regarding claim 14, Leiderman teaches in Fig. 8: a method for an ophthalmic microscope system (“IGSS 800”; [0074], Fig. 8, “image-guided surgical system (IGSS)”; [0051]), the method comprising:
obtaining intraoperative sensor data of an eye from at least one imaging device (“image data received from the imaging system 802”; [0074], “real-time cross-sectional images”; [0084]) of the ophthalmic microscope system (800);
processing the intraoperative sensor data using a machine-learning model, the machine-learning model being trained to output information on one or more anatomical features of the eye based on the intraoperative sensor data (“An analysis loop can be configured to distinguish anatomical tissue information from the data and to train the feedback loop via machine learning to discriminate between anatomical layers”; [0012]); and
generating a display signal (“display 810”; [0074]) based on the information on the one or more anatomical features of the eye (“It is contemplated that all IGSS systems would include the display 810, though the display 810 may, in various embodiments, provide more or less feedback to the surgeon, and may provide that feedback in a variety of forms”; [0074]), the display signal comprising a visual guidance overlay for guiding a user of the ophthalmic microscope system with respect to the one or more anatomical features of the eye (“the display 810 may also include a depiction, in real-time, within the surgical field, of a surgical instrument 816 wielded by the surgeon, and may identify on the display a tip or working end of the surgical instrument 816”; [0075], “the IGSS 800 may, using the determined distance, provide real-time visual, auditory, and/or haptic feedback to the surgeon, via the display 810, the speaker 812, and/or the haptic feedback system 814”; [0076]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Leiderman (WO2020163845A2) as applied to claim 1 above, and further in view of Buch (WO 2020023740 A1), as cited in the IDS. All references to Leiderman (WO2020163845A2) will be made in reference to English language equivalence Leiderman (US20220104884A1). All references to of Buch (WO 2020023740 A1) will be made in reference to English language equivalence Buch (US20250176798A1).
Regarding claim 6, Leiderman teaches the system according to claim 5. Leiderman teaches an ophthalmic microscope system. Leiderman fails to teach: the system is configured to generate the visual guidance overlay based on a selection of a subset of the plurality of visual indicators, wherein the selection is based on an input of a user of the ophthalmic microscope system, or wherein the system is configured to determine the selection based on a progress of an ophthalmic surgical procedure being performed with the help of the ophthalmic microscope system.
However, in a related invention in the field of AI assisted surgical guidance, Buch teaches in Fig. xxx: the system is configured to generate the visual guidance overlay (“surgical guidance generator 104”; [0030]) based on a selection of a subset of the plurality of visual indicators (“a surgical guidance generator 104, which post-processes the neural network data through novel hierarchical algorithms to generate and output surgical guidance to a surgeon in real time during surgery”; [0027], “FIG. 1B is a diagram illustrating a hierarchy of functions performed by the surgical guidance generator illustrated in FIG. 1A”; [0028]), wherein
the selection is based on an input of a user of the system (“A surgeon would be able to interact with the AOA using voice activation to ask questions of the AOA (i.e. “How certain are you that this is the disc material?”, “How safe is it to retract this part of the thecal sac?”) or ask the AOA to label objects as the user sees fit (i.e. “Label this object as the carotid artery.”). Intraoperative feedback from the surgeon can also be fed back into the network to improve its output”; [0024]),
or wherein the system is configured to determine the selection based on a progress of a surgical procedure being performed with the help of the system (“At the lowest level of the hierarchy, anatomical and surgical objects are identified. At the next level, movement of the anatomical and surgical objects is tracked. At this level critical structure proximity warnings along with procedural next-step suggestions may be generated. At the next level in the hierarchy, tissue manipulation and contour monitoring are performed to warn the surgeon about complications that may result from manipulating tissue beyond safe mechanical ranges. At the next level in the hierarchy, intraoperative outcome metrics may be generated and utilized to create feedback to the surgeon”; [0028]).
Furthermore, Buch teaches this configuration such that “The AOA provides real-time automated feedback on the surgical field to augment a surgeon's decision making and has the potential to become essential to any surgical procedure” (Buch, [0016]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Leiderman to incorporate the teachings of Buch to provide a device in which “the system is configured to generate the visual guidance overlay based on a selection of a subset of the plurality of visual indicators, wherein the selection is based on an input of a user of the ophthalmic microscope system, or wherein the system is configured to determine the selection based on a progress of an ophthalmic surgical procedure being performed with the help of the ophthalmic microscope system,” for the purpose of providing real-time automated feedback on the surgical field to augment a surgeon's decision making (Buch, [0016]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 11798676 B2: AI guided surgical planning and support.
US 20220079675 A1: systems, devices and methods for performing a surgical step or surgical procedure with visual guidance using a head mounted display.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RUBY L KAUFFMAN whose telephone number is (571)272-1738. The examiner can normally be reached Mon-Fri 7:30am - 5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pinping Sun can be reached at (571) 270-1284. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RUBY L KAUFFMAN/ Examiner, Art Unit 2872
/WILLIAM R ALEXANDER/ Primary Examiner, Art Unit 2872
Prosecution Insights