Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/16/2026 has been considered by the examiner.
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Section 33(a) of the America Invents Act reads as follows:
Notwithstanding any other provision of law, no patent may issue on a claim directed to or encompassing a human organism.
Claims 11-20 are rejected under 35 U.S.C. 101 and section 33(a) of the America Invents Act as being directed to or encompassing a human organism. See also Animals - Patentability, 1077 Off. Gaz. Pat. Office 24 (April 21, 1987) (indicating that human organisms are excluded from the scope of patentable subject matter under 35 U.S.C. 101).
Claim 11 is considered to be directed towards a human organism because the limitation positively recites the instrument in contact with a human organism (“receive image data captured by a camera disposed on a distal end of an instrument inserted within an anatomy”) as a structural requirement of the claim. To overcome these rejections, examiner suggests amending claim 11 to recite the instrument is configured for inserting within an anatomy.
Claims 12-20 are dependent of claim 11, and therefore rejected under this 101 rejection as well.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
For claim 1, and similarly for claim 11, the limitation “inferring enhanced image data” is indefinite. It is unclear what is meant by “inferring” data. For the purpose of advancing prosecution, the examiner assumes “inferring” should be “generating” for clarity.
Claims 2-10 are dependent of claim 1, and claims 12-20 are dependent of claim 11, and therefore rejected under this 112(b) rejection as well.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 4, 6, 8, 10-12, 14, 16, 18, and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Berg et al. (US 20240156330 A1, published may 16, 2024), hereinafter referred to as Berg.
Regarding claim 1, and similarly for claim 11, Berg teaches a method for controlling a medical system, comprising:
receiving image data captured by a camera disposed on a distal end of an instrument inserted within an anatomy (Fig. 11C, “FrameSource” as image data; see para. 0068 – “The endoscope may be a “chip on a tip” scope, having an illumination source and camera 410.” see para. 0110 – “Referring again to FIG. 2A, the endoscope may have a handle 112, 114, 120, and a shaft 110 for insertion into a body. At or near distal tip 116 of the shaft 110 may be a lens, electronic image sensor, filter, or other optical component 410.”);
inferring (see 112(b) rejection above, “generating”) enhanced image data from the received image data based on a neural network model trained to filter visual artifacts or obstructions from image data (Fig. 11C, “Super Resolution” as neural network model; see para. 0273 – “The “Super Resolution” box may in turn have a block diagram as shown in FIG. 11D. A machine learning model may be used to combine noise reduction, lens resolution correction, edge enhancement, local contrast enhancement, and upscaling [filtering visual artifacts or obstructions from image data to enhance image data] as an integrated module.”);
extracting information from the enhanced image data based on one or more image processing operations (Fig. 11C, “Mask” (image processing operation) to extract information (lesion, polyps, etc) from output of “Super Resolution” (enhanced image data); see para. 0293 – “In some cases, the image processing pipeline of FIG. 11B may include processing to detect various lesions. For example, during colonoscopy, the image processing pipeline may have a processor to detect polyps. During esophageoscopy, the image processing pipeline may have a processor to detect Barrett's esophagus.”); and
generating a graphical user interface (GUI) for navigating the instrument within the anatomy based at least in part on the information extracted from the enhanced image data (Fig. 11C, “Frame Writer” as displaying image data from output of “Mask” (information extracted from enhanced image data); see para. 0247 – “Frame Writer is the last stage, putting the video into the video system's frame buffer for display or to storage. The fully-processed video stream [enhanced image data] may be displayed on a video monitor, or may be sent to a storage device or network interface.”; see para. 0294 – “The scope [instrument] may have several controls, including…a graphical user interface….”; see para. 0007 – “The processor is programmed to receive video image data from an image sensor at the distal end of an endoscope and to display the image data to a surgeon in real time [navigating the instrument within the anatomy based on information extracted from the enhanced image data].”).
Furthermore, regarding claims 2 and 12, Berg further teaches wherein the visual artifacts include blur, lighting variations, specular reflections, camera saturation, over-exposure, or under-exposure (see para. 0290 – “In some cases, several machine learning systems may be chained together, for example, one to enhance dynamic range, one to reduce blur and for edge sharpening, one to recognize frame-to-frame motion, one to improve contrast, and one to upsample for super resolution.”).
Furthermore, regarding claims 4 and 14, Berg further teaches wherein the anatomy comprises a lung (see para. 0003 – “An endoscope may be an…tracheoscope (trachea and bronchi)…”).
Furthermore, regarding claims 6 and 16, Berg further teaches wherein the neural network model comprises a generative image inpainting model, an artificial intelligence (AI)-based super resolution model, a generative style transfer model, or a Neural Radiance Field (NeRF) or Gaussian splatting model (Fig. 11C, “Super Resolution”; see para. 0273 – “The “Super Resolution” box may in turn have a block diagram as shown in FIG. 11D. A machine learning model may be used to combine noise reduction, lens resolution correction, edge enhancement, local contrast enhancement, and upscaling as an integrated module.”).
Furthermore, regarding claims 8 and 18, Berg further teaches wherein the extracted information includes a shape, boundary, eccentricity, texture, or position of a feature of the anatomy (Fig. 11C, “Mask” (image processing operation) to extract information (lesion, polyps, etc) from output of “Super Resolution”; see para. 0293 – “In some cases, the image processing pipeline of FIG. 11B may include processing to detect various lesions. For example, during colonoscopy, the image processing pipeline may have a processor to detect polyps. During esophageoscopy, the image processing pipeline may have a processor to detect Barrett's esophagus.”).
Furthermore, regarding claims 10 and 20, Berg further teaches wherein the enhanced image data is displayed as a live camera view in the GUI (see para. 0004 – “The processor is programmed to receive video image data from an image sensor at the distal end of an endoscope and to display the image data to a surgeon in real time.”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3, 5, 7, 9, 13, 15, 17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Berg in view of Lin et al. (US 20210369384 A1, published December 2, 2021), hereinafter referred to as Lin.
Regarding claims 3 and 13, Berg teaches all of the elements disclosed in claim 1 and 11 above.
Berg teaches generating an enhanced image based on filtering visual artifacts or obstructions from image data, but does not explicitly teach where the visual artifacts or obstructions includes mucus, blood, stone dust or fragments, bubbles, or other medical instruments.
Whereas, Lin, in an analogous field of endeavor, teaches wherein the obstructions include mucus, blood, stone dust or fragments, bubbles, or other medical instruments (see para. 0127 – “…ureteroscopic images of internal renal anatomy, wherein the images include certain instrument components and/or kidney stones or other objects.”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified generating an enhanced image based on filtering visual artifacts or obstructions from image data, as disclosed in Berg, by having the visual artifacts or obstructions includes mucus, blood, stone dust or fragments, bubbles, or other medical instruments, as disclosed in Lin. One of ordinary skill in the art would have been motivated to make this modification in order to remove kidney stones, as taught in Lin (see para. 0077).
Furthermore, regarding claims 5 and 15, Lin further teaches wherein the anatomy comprises a kidney (see para. 0127 – “…ureteroscopic images of internal renal anatomy, wherein the images include certain instrument components and/or kidney stones or other objects.”).
Furthermore, regarding claims 7 and 17, Lin further teaches wherein the extracted information includes a position or orientation of the medical instrument (see para. 0077 – “With further reference to FIG. 1, the medical system 100 can provide a variety of benefits, such as providing guidance to assist a physician in performing a procedure (e.g., instrument tracking, instrument alignment information, etc.)…”; see para. 0119 – “In the image 701, the basket 35 and stone 80 are relatively large in the field-of-view due to the proximity of the basket 35 to the scope camera 48.”).
Furthermore, regarding claims 9 and 19, Lin further teaches wherein the GUI includes an anatomical map indicating a spatial relationship between the instrument and a target within the anatomy (Fig. 6A; see para. 0119 – “In the image 701 [anatomical map indicating spatial relationship via display], the basket 35 [part of instrument] and stone 80 [target within anatomy (renal pelvis 71)] are relatively large in the field-of-view due to the proximity of the basket 35 to the scope camera 48.”; see para. 0053 – “The control system 50 can provide information via the display(s) 56 that is associated with the medical instrument 40, such as real-time endoscopic images captured therewith, and/or other instruments of the system 100, to assist the physician 5 in navigating/controlling such instrumentation.”; see para. 0070 – “Positioned at the upper end of column 14, the console 13 can provide both a user interface for receiving user input and a display screen 16 (or a dual-purpose device such as, for example, a touchscreen) to provide the physician/user with both pre-operative and intra-operative data.”).
The motivation for claims 5, 7, 9, 15, 17, and 19 was shown previously in claims 3 and 13.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Lent et al. (US 20220280245 A1, published September 8, 2022) discloses determining anatomical distance and/or position measurements during an endoscopic procedure.
Ye et al. (US 20210059765 A1, published March 4, 2021) discloses providing both robotic controls as well as preoperative and real time information of the procedure, such as navigational and localization information of the endoscope.
Nekhendzy et al. (US 20220008140 A1, published January 13, 2022) discloses robotic movement and control of an intubation tube introducer or guide, and may include utilizing image data from one or more image sensors.
Herda et al. (US20250095839 A1, published March 20, 2025 with a priority date of September 14, 2023) discloses the system may provide live high-definition video to be displayed on a video monitor, and to be captured as stored video and still images; illumination of the surgical cavity, irrigation and/or inflation (insufflation) of the surgical site, and image refinement such as zoom, rotation, removal or reduction of hotspots and other artifacts, etc.
Makihira et al. (US 20210304363 A1, published September 30, 2021) discloses analyzing a medical image subjected to various kinds of artifact removal processing.
Choi et al. (US 20250089985 A1, published March 20, 2025 with a priority date of July 28, 2021) discloses training datasets may also comprise a parametrization by which parameters can be varied to broaden the scope of the training. These parameters may include camera exposure data, lighting data, airway surface characteristics, image artifacts, and other confounding variables.
Jorgensen et al. (US 20230172428 A1, published June 8, 2023) discloses the processor receives image data from the medical device interfaces and outputs video signals incorporating the GUI and image data.
Yeung et al. (US 20180296281 A1, published October 18, 2018) discloses automated steering control of a robotic endoscope.
Ummalaneni (US 20190110843 A1, published April 18, 2019) discloses the instrument may be equipped with a camera to provide vision data and process the vision data to enable one or more vision-based location tracking modules or features.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nyrobi Celestine whose telephone number is 571-272-0129. The examiner can normally be reached on Monday - Thursday, 7:00AM - 5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached on 571-272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.C./Examiner, Art Unit 3798
Prosecution Insights