Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 19 November, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of pre-AIA 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a) the invention was known or used by others in this country, or patented or described in a printed publication in this or a foreign country, before the invention thereof by the applicant for a patent.
(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on sale in this country, more than one year prior to the date of application for patent in the United States.
Claims 1, 2, 5 – 8, 11, 12, 14 – 16, and 19 are rejected under pre-AIA 35 U.S.C. 102(a)(2) as being anticipated by Moretti et al (U.S. Patent Publication No. 2022/0280028 A1, hereinafter “Moretti”).
Regarding claim 1, Moretti teaches a device for imaging an anatomy, the device comprising:
a bracket for attaching the device to an instrument head of an optical viewing device, the instrument head including an eyepiece for examining the anatomy (Figure 3; ¶ 0037 : FIG. 3 illustrates a diagnostic device 305 (sometimes referred to as a base unit or base system) and a plurality of interchangeable imaging modules 350A, 350B, and 350C.);
a camera that captures images through the eyepiece of the instrument head (Figure 3; ¶ 0037 : FIG. 3 illustrates a diagnostic device 305 (sometimes referred to as a base unit or base system) and a plurality of interchangeable imaging modules 350A, 350B, and 350C.);
a display screen that displays the images captured by the camera (¶ 0017: Easy and comfortable visualization of the patient's retina can be facilitated using such retina camera, which can be placed over the patient's eye, display the retina image on a high resolution display…; ¶ 0038: The base system 305 can include a handle 310 (which can be similar to the handle 110) and a user interface 330 (which can be similar to the display 130).);
at least one processing device (¶ 0035: Image or video analysis, including the application of one or more machine learning models to one or more images or video, can be performed by execution of program instructions by a processor); and
at least one computer readable data storage device storing software instructions that, when executed by the at least one processing device (¶ 0035: Image or video analysis, including the application of one or more machine learning models to one or more images or video, can be performed by execution of program instructions by a processor), cause the device to:
determine a type of the optical viewing device (¶ 0040: The base system 305 can detect the attachment of an imaging module (such as, via one or more controllers).); and
present a workflow on the display screen based on the type of the optical viewing device (¶ 0042: For example, when the imaging module 350B for the ear is attached to the base system 305, the base system can detect that the imaging module 350B is an ear imaging module and alter the image processing workflow such that images captured are processed through the ear-specific machine learning model. In another example, when the imaging module 350B for the ear is swapped for the imaging module 350A for the eye, the base system 305 can detect the change and alter the image processing workflow such that images captured are processed through the eye-specific AI workflow.), the workflow enabling capture of a video of the anatomy viewed through the eyepiece of the instrument head (¶ 0029: One or more preset modes can facilitate easy and efficient capture of multiple images or video. Such one or more preset modes can automatically focus, capture, verify image quality, and store the video or image(s).), and the workflow including one or more tools for annotating one or more frames selected from the video (¶ 0030: The user can select a region of an image for analysis, for instance, by outlining the region on the touch screen display 130, zooming in on region of interest on the display 130, or the like. In some cases, by default the entire image may be analyzed.).
Regarding claim 2, Moretti teaches the device of claim 1.
Additionally, Moretti teaches wherein the type of the optical viewing device is from a group consisting of an ophthalmoscope, an otoscope, and a dermatoscope (¶ 0038: When the imaging module 350B is attached to the base system 305, the diagnostic device can function as an otoscope.)
Regarding claim 5, Moretti teaches the device of claim 1.
Additionally, Moretti teaches wherein the instructions, when executed by the at least one processing device, further cause the device to:
determine an environment of the optical viewing device (¶ 0040: The base system 305 can determine the type of the imaging module attached to the base system… In some cases, the determination can be made based on decoding the image data of a body part captured after the imaging module has been installed (such as, by detecting one or more of the anatomical features of the eye, ear, or skin in the image data).); and
modify the workflow based on the environment (¶ 0042: For example, when the imaging module 350B for the ear is attached to the base system 305, the base system can detect that the imaging module 350B is an ear imaging module and alter the image processing workflow such that images captured are processed through the ear-specific machine learning model. In another example, when the imaging module 350B for the ear is swapped for the imaging module 350A for the eye, the base system 305 can detect the change and alter the image processing workflow such that images captured are processed through the eye-specific AI workflow.; Examiner’s note: The examiner is interpreting “an environment” to include the portions of a patient being imaged. As the device determines the specific camera type of the device based on the anatomical features of the patient, then determining the AI workflow based on this determination, this is understood to read on “determine an environment of the optical viewing device; and modify the workflow based on the environment.).
Regarding claim 6, Moretti teaches the device of claim 1.
Additionally, Moretti teaches wherein the one or more tools for annotating the one or more frames selected from the video include at least one of a predefined annotation and a filter (¶ 0030: The user can select a region of an image for analysis, for instance, by outlining the region on the touch screen display 130, zooming in on region of interest on the display 130, or the like. In some cases, by default the entire image may be analyzed.; Examiner’s note: Annotating an image to include an outline, or zooming in on the image to identify a region of interest are understood to be “predefined annotations” as they are the tools readily provided to the user of the tool to specifically identify areas within an image.).
Regarding claim 7, Moretti teaches the device of claim 1.
Additionally, Moretti teaches wherein the instructions, when executed by the at least one processing device, further cause the device to:
automatically detect the type of the optical viewing device based on imaging performed by the camera of the imaging device, machine-readable data provided by the instrument head, or a connection established between the imaging device and the instrument head (¶ 0040: The base system 305 can determine the type of the imaging module attached to the base system. The determination can be made by one or more of: 1) reading a value ( such as, an identifier) stored in a memory of the imaging module (such as, 0 for a retinal imaging module, 1 for an ear imaging module, or 2 for a skin imaging module); 2) receiving the value (such as, the identifier) via a communication protocol, which can be wired or wireless (such as, radio frequency identification (RFID) or near field communication (NFC)); or 3) decoding a visual marker positioned on the imaging module (such as, a quick response or QR code, colored dot, or another image pattern, which may be positioned on the exterior surface of the imaging module). In some cases, the determination can be made based on decoding the image data of a body part captured after the imaging module has been installed (such as, by detecting one or more of the anatomical features of the eye, ear, or skin in the image data).; ¶ 0042: The base system 305 can make such workflow changes automatically responsive to a particular imaging module being attached and without the need for user input.).
The rejection of device claim 1 above applies mutatis mutandis to the corresponding limitations of method claim 8 while noting that the rejection above cites to both device and method disclosures.
The rejection of device claim 5 above applies mutatis mutandis to the corresponding limitations of method claim 11 while noting that the rejection above cites to both device and method disclosures.
Regarding claim 12, Moretti teaches the method of claim 8.
Additionally, Moretti teaches wherein the one or more tools for annotating the one or more frames selected from the video include one or more predefined annotations based on the anatomy,
the one or more predefined annotations being selectable. (¶ 0030: The user can select a region of an image for analysis, for instance, by outlining the region on the touch screen display 130, zooming in on region of interest on the display 130, or the like. In some cases, by default the entire image may be analyzed.; Examiner’s note: Annotating an image to include an outline, or zooming in on the image to identify a region of interest are understood to be “predefined annotations” as they are the tools readily provided to the user of the tool to specifically identify areas within an image. Additionally, the chosen method is understood to be selected by the user ).
Regarding claim 14, Moretti teaches the method of claim 8.
Additionally, Moretti teaches further comprising:
capturing the video of the anatomy viewed through the eyepiece of the instrument head (¶ 0045: Light gathered by the optical components of an attached imaging module can be projected on the image sensor(s) to generate video or images for processing by one or more machine learning models.).
The rejection of device claim 1 above applies mutatis mutandis to the corresponding limitations of device claim 15.
For the device limitations of claim 15 see Moretti’s teaching on:
A system for imaging an anatomy (¶ 0037: FIG. 3 illustrates a diagnostic device 305 (sometimes referred to as a base unit or base system) and a plurality of interchangeable imaging modules 350A, 350B, and 350C.), the system comprising:
an optical viewing device including an instrument head having an eyepiece for examining the anatomy (Figure 3, Ref. No. 305); and
an imaging device configured for attachment to the optical viewing device (Figure 3, Ref. No. 350A-C; ¶ 0037: FIG. 3 illustrates a diagnostic device 305 (sometimes referred to as a base unit or base system) and a plurality of interchangeable imaging modules 350A, 350B, and 350C.)), the imaging device including:
a camera for capturing images through the eyepiece of the instrument head (¶ 0037: imaging module 350A-C…);
a display screen for displaying the images captured by the camera (¶ 0038: The base system 305 can include a handle 310 (which can be similar to the handle 110) and a user interface 330 (which can be similar to the display 130). The base system 305 can include electronic components);
at least one processing device communicatively connected to the camera and the display screen (¶ 0035: Image or video analysis, including the application of one or more machine learning models to one or more images or video, can be performed by execution of program instructions by a processor and/or by a specialized integrated circuit that implements the machine learning model in hardware.); and
at least one computer readable data storage device storing software instructions that, when executed by the at least one processing device (¶ 0035: Image or video analysis, including the application of one or more machine learning models to one or more images or video, can be performed by execution of program instructions by a processor and/or by a specialized integrated circuit that implements the machine learning model in hardware.), cause the imaging device to…
The rejection of device claim 2 above applies mutatis mutandis to the corresponding limitations of device claim 16.
The rejection of device claim 5 above applies mutatis mutandis to the corresponding limitations of device claim 19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 9, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Moretti et al (U.S. Patent Publication No. 2022/0280028 A1, hereinafter “Moretti”) in view of Abulnaga et al (U.S. Patent Publication No. 2022/0246298 A1, hereinafter “Abulnaga”).
Regarding claim 3, Moretti teaches the device of claim 1.
Moretti does not explicitly teach wherein the instructions, when executed by the at least one processing device, further cause the device to: determine a user of the imaging device; and modify the workflow based on the user.
However, Abulnaga does teach wherein the instructions, when executed by the at least one processing device, further cause the device to:
determine a user of the imaging device (¶ 0017: The device may be portable or handheld by a user (which may be a patient or healthcare provider.; ¶ 0034: If an electronic medical ( or electronic health) record module is present (as described below), the user (or system) can choose for the module to provide patient demographic or other relevant patient information to one or more machine learning models.) and
modify the workflow based on the user (¶ 0034: One or more tags can be stored along with one or more pertinent images in the storage 224. One or more tags can be used during analysis by one or more machine learning models during analysis and evaluation. One or more images along with one or more tags can be used as training data.; ¶ 0035: In some cases, the diagnostic report may alternatively or additionally provide information indicating increased risk of disease or condition for a physician's (such as, ophthalmologist's) consideration or indicating the presence (or absence) of disease of condition. Physician can use this information during subsequent evaluation of the patient.).
Abulnaga is considered to be analogous art as it pertains to modular optical imaging devices. Therefore, it would have been obvious to one of ordinary skill in the art to combine the interchangeable imaging modules (as taught by Moretti) and the modular imaging modules (as taught by Abulnaga) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Abulnaga allows a user to enter medical information regarding the patient which is then used by machine learning models. This additional information is understood to improve the diagnostic methods of the system (See ¶ 0034).
This motivation for the combination of Moretti and Abulnaga is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
The rejection of device claim 3 above applies mutatis mutandis to the corresponding limitations of method claim 9 while noting that the rejection above cites to both device and method disclosures.
The rejection of device claim 3 above applies mutatis mutandis to the corresponding limitations of device claim 17.
Claims 4, 10, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Moretti et al (U.S. Patent Publication No. 2022/0280028 A1, hereinafter “Moretti”) in view of Anderson et al (U.S. Patent Publication No. 202/0196863 A1, hereinafter “Anderson”).
Regarding claim 4, Moretti teaches the device of claim 1.
Moretti does not explicitly teach wherein the instructions, when executed by the at least one processing device, further cause the device to: determine a patient undergoing an examination by the optical viewing device; and modify the workflow based on the patient
However, Anderson does teach wherein the instructions, when executed by the at least one processing device, further cause the device to:
determine a patient undergoing an examination by the optical viewing device (¶ 0077: Patient data 360 of the currently selected patient can be displayed on the top of the acquisition screen 345.); and
modify the workflow based on the patient (¶ 0077: FIG. 3b shows an example of an acquisition window/screen 345 used for acquiring (e.g., capturing) patient images. In this window, various display elements and icons are displayed to the instrument operator (the user) to select the types of images to be acquired and to ensure the patient is properly aligned to the instrument. Patient data 360 of the currently selected patient can be displayed on the top of the acquisition screen 345.).
Anderson is considered to be analogous art as it pertains to ophthalmic imaging of patients. Therefore, it would have been obvious to one of ordinary skill in the art to combine the interchangeable imaging modules (as taught by Moretti) and the systems for improved ophthalmic imaging (as taught by Anderson) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Anderson provides an infrared reflectance image, as the eye is not sensitive to infrared wavelengths, this permits a user to continuously take images without disturbing the eye of the patient. (See ¶ 0070).
This motivation for the combination of Moretti and Anderson is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
The rejection of device claim 4 above applies mutatis mutandis to the corresponding limitations of method claim 10 while noting that the rejection above cites to both device and method disclosures.
The rejection of device claim 4 above applies mutatis mutandis to the corresponding limitations of device claim 18.
Claims 13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Moretti et al (U.S. Patent Publication No. 2022/0280028 A1, hereinafter “Moretti”) in view of Tran et al (U.S. Patent Publication No. 2022/0248955 A1, hereinafter “Tran”).
Regarding claim 13, Moretti teaches the method of claim 8.
Additionally, Moretti teaches wherein the one or more tools for annotating the one or more frames selected from the video include one or more filters based on the anatomy (¶ 0016: The base system can load the appropriate anatomy or disease specific AI (or machine learning) model(s) that perform the image analysis and disease detection.; ¶ 0030: The user can select a region of an image for analysis, for instance, by outlining the region on the touch screen display 130, zooming in on region of interest on the display 130, or the like. In some cases, by default the entire image may be analyzed.; Examiner’s note: Annotating an image to include an outline, or zooming in on the image to identify a region of interest are understood to be “predefined annotations” as they are the tools readily provided to the user of the tool to specifically identify areas within an image.),
Moretti does not explicitly teach the one or more filters being selectable for each frame of the one or more frames.
However, Tran does teach wherein the one or more tools for annotating the one or more frames selected from the video include one or more filters (¶ 0073: In any of the methods described herein, receiving the image may include receiving a video frame ( e.g., an image taken from a video)… For example, a raw image may be processed to provide a different scale by filtering (e.g., blurring, sharpening, etc.) and the resulting processed image may provide an additional scale of the image.).
Tran is considered to be analogous art as it pertains to ophthalmic imaging of patients. Therefore, it would have been obvious to one of ordinary skill in the art to combine the interchangeable imaging modules (as taught by Moretti) and the metaverse system (as taught by Tran) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Tran is usable with phones with multiple cameras, allowing stereoscopic images which display depth information and are better in visualizing the transparent medium, like the cornea. (See ¶ 0059).
This motivation for the combination of Moretti and Tran is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 20, Moretti teaches the system of claim 15.
Additionally, Moretti teaches wherein the one or more tools for annotating the one or more frames selected from the video include:
one or more predefined annotations based on the anatomy, the one or more predefined annotations being selectable for each frame of the one or more frames (¶ 0030: The user can select a region of an image for analysis, for instance, by outlining the region on the touch screen display 130, zooming in on region of interest on the display 130, or the like. In some cases, by default the entire image may be analyzed.; Examiner’s note: Annotating an image to include an outline, or zooming in on the image to identify a region of interest are understood to be “predefined annotations” as they are the tools readily provided to the user of the tool to specifically identify areas within an image. Additionally, the chosen method is understood to be selected by the user); and
one or more filters based on the anatomy (¶ 0016: The base system can load the appropriate anatomy or disease specific AI (or machine learning) model(s) that perform the image analysis and disease detection.),
Additionally, Tran teaches the one or more filters being selectable for each frame of the one or more frames (¶ 0073: In any of the methods described herein, receiving the image may include receiving a video frame ( e.g., an image taken from a video)… For example, a raw image may be processed to provide a different scale by filtering (e.g., blurring, sharpening, etc.) and the resulting processed image may provide an additional scale of the image.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Small et al (U.S. Patent Publication No. 2023/0240523 A1) teaches a system for tele-otoscopy wherein the otoscope apparatus may capture a picture or video of the outer ear, the middle ear, and/or the ear canal of a patient utilizing the camera of the smart device. Such a picture or video of the outer ear, the middle ear, and/or the ear canal may be sent to a remote healthcare professional for diagnosis of an ear infection.
Perkins et al (U.S. Patent Publication No. 2017/0303857 A1) teaches an examination system having separate enabled interchangeable operating modes which includes an adapter that is configured for aligning a plurality of disparate smart devices with the optical system of the medical device when the adapter is attached to the medical device, thereby enabling multiple operating modes without modification to the device.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW JONES whose telephone number is (703)756-4573. The examiner can normally be reached Monday - Friday 8:00-5:00 EST, off Every Other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW B. JONES/Examiner, Art Unit 2667
Prosecution Insights