Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the:
“data processing unit”
must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 12-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the recitation of “computer program product” is not considered one of the four categories of patent eligible subject matter. The proper recitation of such subject matter in the art is “a non-transitory computer readable medium with a program”. The recited “program product” is indefinite also and therefore could be interpreted as a computer program per se, which does not fall within the four categories of patent eligible subject matter. It will be interpreted as “a non-transitory computer readable medium with a program”. Appropriate correction is required.
Claim Rejections - 35 USC § 112a
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 1 and dependent claims are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “a data processing unit”. This element is not sufficiently described in the details of the specification, which also does not provide a reference character or a proper drawing. The term “data processing unit” does not exist in the details of the specification and therefore deemed to not be sufficiently described.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 4, 5, 8, 12, 15 is/are rejected under 35 U.S.C. 102a1 as being anticipated by Sarvestani US20240315778.
Sarvestani discloses for claim 1, “A device for augmenting image information of an endoscope, wherein the endoscope comprises a longitudinal axis, a distal tip and a viewing direction away from the distal tip, and wherein the endoscope is configured to provide video images of at least a part of an outer surface of an anatomy of interest in front of the distal tip (fig 1; 0071),
the device comprising a data processing unit (control unit 12; fig 1; 0071) configured to
determine a 3D position and orientation of the endoscope relative to the anatomy of interest (0071: Thus, the pose of the recording head 6 of the real endoscope 2 relative to the body of the patient P is determined. In other words, the (real) intracorporeal image IA is initially correlated, i.e. registered, with the (virtual or digital) 3D image data 3DA via at least one characteristic landmark 15, 15′, which occurs in both images, in order to register or determine the pose of the (real) endoscope 2, in particular of the recording head 6, relative to the patient P),
to receive a model of the anatomy of interest (0049: reading in digital 3D image data, in particular pre-operative 3D image data, of a patient), and
to augment a video image provided by the endoscope with information of the model of the anatomy of interest (0049: creating an intracorporeal image through an endoscope; detecting at least one landmark and/or orientation in the intracorporeal image; determining a corresponding landmark and/or orientation in the 3D image data; registering the 3D image data to the intracorporeal image via the at least one detected landmark and/or orientation; generating at least one correlation display with a display of the intracorporeal image and a display of a view of the 3D image data and preferably outputting the correlation display by the display device)”.
Sarvestani discloses for claim 4, “The device of claim 1, wherein the model of the anatomy of interest is based on an anatomy scan (0069: preoperative 3D image data 3DA of the patient P to be treated are stored in digital/computer-readable form. 3D image data 3DA can be (segmented) MRI image data or CT image data, for example)”.
Sarvestani discloses for claim 5, “The device of claim 1, wherein determining the 3D position and orientation of the endoscope relative to the anatomy of interest comprises determining the viewing direction of the endoscope and a distance between the distal tip of the endoscope and the outer surface of the anatomy of interest visible in the video image provided by the endoscope (0071-0072: the pose of the recording head 6 of the real endoscope 2 relative to the body of the patient P is determined. In other words, the (real) intracorporeal image IA is initially correlated, i.e. registered, with the (virtual or digital) 3D image data 3DA via at least one characteristic landmark 15, 15′, which occurs in both images, in order to register or determine the pose of the (real) endoscope 2, in particular of the recording head 6, relative to the patient P)”.
Sarvestani discloses for claim 8, “The device of claim 1, wherein determining the 3D position and orientation of the endoscope relative to the anatomy of interest is based on a real time tracking system providing localization information of at least one of the endoscope and of the anatomy of interest (0018, 0041 describes real time/repeated re-registration)”.
Sarvestani discloses for claim 12, “A computer program product executable on a data processing unit of a device for augmenting image information of an endoscope, the computer program product comprising sets of instructions which when executed by the data processing unit causing the device (fig 1; 0071)
to determine a 3D position and orientation of the endoscope relative to the anatomy of interest (0071: Thus, the pose of the recording head 6 of the real endoscope 2 relative to the body of the patient P is determined. In other words, the (real) intracorporeal image IA is initially correlated, i.e. registered, with the (virtual or digital) 3D image data 3DA via at least one characteristic landmark 15, 15′, which occurs in both images, in order to register or determine the pose of the (real) endoscope 2, in particular of the recording head 6, relative to the patient P),
to receive a model of the anatomy of interest (0049: reading in digital 3D image data, in particular pre-operative 3D image data, of a patient), and
to augment at least one video image provided by the endoscope with information of the model of the anatomy of interest (0049: creating an intracorporeal image through an endoscope; detecting at least one landmark and/or orientation in the intracorporeal image; determining a corresponding landmark and/or orientation in the 3D image data; registering the 3D image data to the intracorporeal image via the at least one detected landmark and/or orientation; generating at least one correlation display with a display of the intracorporeal image and a display of a view of the 3D image data and preferably outputting the correlation display by the display device)”.
Sarvestani discloses for claim 15, “The computer program product of claim 12, comprising a non-transitory computer readable medium having stored the sets of instructions of the computer program product (0080)”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sarvestani as applied to claim 1 above, and further in view of Chow US10758309.
Sarvestani does not disclose for claim 2, “The device of claim 1, wherein a deep neural net is applied to classify anatomical structures visible in the video image provided by the endoscope”.
Chow teaches in the same field of endeavor, using deep neural net/deep residual network to classify anatomy from images (16:64-17:21). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Chow into the invention of Sarvestani in order to configure the device e.g. as claimed because it provides the advantage afforded from grouping multiple similar elements into one single common label for simpler identification.
Sarvestani does not disclose for claim 13, “The computer program product of claim 12, comprising sets of instructions which when executed by the data processing unit causing the device to apply a deep neural net to classify anatomical structures visible in the video images provided by the endoscope, and/or to augment the video images provided by the endoscope by outlining anatomical structures visible in the video images”.
Chow teaches in the same field of endeavor, using deep neural net/deep residual network to classify anatomy from images (16:64-17:21). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Chow into the invention of Sarvestani in order to configure the device e.g. as claimed because it provides the advantage afforded from grouping multiple similar elements into one single common label for simpler identification.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sarvestani as applied to claim 1 above, and further in view of Chen US20240169533.
Sarvestani does not disclose for claim 3, “The device of claim 1, wherein a deep neural net is applied to augment the video image provided by the endoscope by outlining anatomical structures visible in the video image”.
Chen teaches in the same field of endeavor, using deep neural net/deep learning to help identify structures by applying an outline (0006). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Chen into the invention of Sarvestani in order to configure the device e.g. as claimed because outlining helps to identify relevant items.
Claim(s) 6-7, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sarvestani as applied to claim 1 above, and further in view of Blau US20210248779.
Sarvestani does not disclose for claim 6, “The device of claim 1, wherein determining the 3D position and orientation of the endoscope relative to the anatomy of interest is based on an X-ray image of at least a part of the endoscope including the distal tip and of the anatomy of interest in front of the distal tip”.
Blau teaches in the same field of endeavor,
determining relative 3D positions and relative 3D orientations between multiple objects at least partially visible in an X-ray projection image 0019, and
If multiple X-ray images are acquired, the information from them may be fused (registered) to increase the accuracy of 3D reconstruction and/or determination of spatial positions or orientations. The matching of a virtual projection to an actual projection in an X-ray image (e.g., using Deep Morphing) may be performed with higher accuracy if the imaging direction is known or can be determined, or in case of registering multiple X-ray images, the 3D angles (which may, for instance, be represented by Euler angles) between imaging directions are known or can be determined 0053.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Blau into the invention of Sarvestani in order to configure the device e.g. as claimed because it provides increased accuracy (0053).
Modified Sarvestani discloses for claim 7, “The device of claim 6, wherein the data processing unit is configured to receive the X-ray image of the endoscope and of the anatomy of interest, to determine a longitudinal axis of the endoscope, and to determine the 3D position and orientation of the endoscope relative to the anatomy of interest based on the X-ray image, on the model of the anatomy of interest, and on the determined longitudinal axis of the endoscope (0071-0072: the pose of the recording head 6 of the real endoscope 2 relative to the body of the patient P is determined. In other words, the (real) intracorporeal image IA is initially correlated, i.e. registered, with the (virtual or digital) 3D image data 3DA via at least one characteristic landmark 15, 15′, which occurs in both images, in order to register or determine the pose of the (real) endoscope 2, in particular of the recording head 6, relative to the patient P)”.
Sarvestani does not disclose for claim 14, “The computer program product of claim 13, comprising sets of instructions which when executed by the data processing unit causing the device to receive an X-ray image of the endoscope and of the anatomy of interest, to determine a longitudinal axis of the endoscope, and to determine the 3D position and orientation of the endoscope relative to the anatomy of interest based on the X-ray image, on the model of the anatomy of interest, and on the determined longitudinal axis of the endoscope”.
To this end though, Sarvestani 0018, 0041 describes real time/repeated re-registration.
Additionally, Blau teaches in the same field of endeavor,
determining relative 3D positions and relative 3D orientations between multiple objects at least partially visible in an X-ray projection image 0019, and
If multiple X-ray images are acquired, the information from them may be fused (registered) to increase the accuracy of 3D reconstruction and/or determination of spatial positions or orientations. The matching of a virtual projection to an actual projection in an X-ray image (e.g., using Deep Morphing) may be performed with higher accuracy if the imaging direction is known or can be determined, or in case of registering multiple X-ray images, the 3D angles (which may, for instance, be represented by Euler angles) between imaging directions are known or can be determined 0053.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Blau into the invention of Sarvestani in order to configure the device e.g. as claimed because it provides increased accuracy (0053).
Claim(s) 9-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sarvestani as applied to claim 1 above, and further in view of Mosnier US 20030225422 A1.
Sarvestani does not disclose for claim 9, “The device of claim 1, wherein a trocar holder is utilized to arrange the endoscope in a fixed relation to the anatomy of interest, while the anatomy is moving”. Mosnier teaches in the same field of endeavor, providing a robot positioning trocar (0008). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Mosnier into the invention of Sarvestani in order to configure the device e.g. as claimed because it allows instruments to be fixed and held in any desired position (0008).
Modified Sarvestani discloses for claim 10, “The device of claim 9, wherein the trocar holder is configured to be fixedly arranged at the moving anatomy so that the trocar holder moves together with the moving anatomy (Mosnier: 0008)”.
Modified Sarvestani discloses for claim 11, “The device of claim 10, wherein the trocar holder is a robotic trocar holder configured to move the trocar holder together with the moving anatomy so as to keep the trocar holder in the fixed relation to the anatomy of interest (Mosnier: 0008)”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAE K WOO whose telephone number is (571)272-0837. The examiner can normally be reached M-F 8:30-2:30p, 6p-9p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jae Woo/Examiner, Art Unit 3795
/ANHTUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795 10/9/26