DETAILED ACTION
Claims 1-22 and 24 are pending. Claims 23 and 25-91 are cancelled.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 15, 2025 has been considered by the examiner.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “460” has been used to designate both client terminal(s) and training dataset in Figure 4. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
Page 47 line 13-14 recite “Schematic 614 depicts markers 606 and 6068 overlapping...” (emphasis added). This appears to be a typographical error and should be “markers 606 and 608…” (emphasis added).
Appropriate correction is required.
Claim Objections
Claims objected to because of the following informalities:
Claim 16 lines 3-4 state “estimate a roll angle of the medical implement based on the appearance of the mark in the first or enhanced image” (emphasis added). This appears to be a typographical error and should be “the marker…” (emphasis added).
Claim 20 lines 1-2 state “The system of claim 16, the instructions…” This appears to be a typographical error and should be “The system of claim 16, wherein the instructions…” (emphasis added).
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 16-18, 20 and 24 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 16 recites the limitation "the appearance of the mark" in line 3 (emphasis added). Although “an enhanced image showing at least said marker” is mentioned in claim 12 line 8-9, there is no mention of “appearance” prior to claim 16. It is unclear what “the appearance” is referring to. Therefore, there is insufficient antecedent basis for this limitation in the claim.
Claims 17, 18, and 20 are rejected by the virtue of their dependency upon rejected claim 16.
Claim 17 recites the limitation "the length of the marker" in line 3 (emphasis added). Although “an enhanced image showing at least said marker” is mentioned in claim 12 line 8-9, there is no mention of “length” prior to claim 17. It is unclear what “the length” is referring to. Therefore, there is insufficient antecedent basis for this limitation in the claim.
Claim 24 recites the limitation "the imaging device" in line 3 (emphasis added). There is no mention of “imaging device” prior to claim 24. It is unclear what “the imaging device” is referring to. Therefore, there is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 2 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sejor et al. (International Publication No. WO 2021/185811 A1) ("Sejor").
Regarding claim 2, Sejor discloses a computer implemented method (Fig. 4; [0001] and [0018]) for generating a presentation of at least one image for assisting an operator in bringing a medical implement having a marker to a target location within a body of a patient ([Fig. 4; [0001], [0018], and [0075], wherein the surgical instrument (i.e. medical implement) has a marker on a tip end and assists a practitioner/surgical robot (i.e. operator) in verifying the right positioning (i.e. target location)), the method comprising:
receiving a first image of the medical implement in the body of the patient, the implement being on its way to the target location (Fig. 4 step 71; [0018], [0038], and [0075], wherein the one frame (i.e. first image) including the surgical instrument (i.e. medical implement) is received and the right positioning corresponds to the target location);
processing the first image to obtain an enhanced image showing at least said marker (Fig. 4 steps 72-82; [0038] and [0041-0042], wherein an enlarged image (i.e. enhanced image) of an area of interest, including the tip end of the surgical instrument (i.e. marker), of the one frame (i.e. first image) is obtained); and
providing the enhanced image for display (Fig. 4 step 82; [0003] and [0040-0042], wherein the enlarged area of interest (i.e. enhanced image) is displayed),
wherein the processing comprises inputting the first image to a machine learning model trained to identify the region of interest encompassing at least said marker ([0064] and [0075], wherein an artificial intelligence algorithm (i.e. machine learning model) to identify markers of the surgical instrument within a captured image (i.e. first image); [0031-0032] and [0038], wherein an area of interest (i.e. region of interest) is the part of the frame that includes the tip ends of the surgical instrument, including markers).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-4, 8, 12-15, 19, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Sejor et al. (International Publication No. WO 2021/185811 A1) ("Sejor") in view of Kono et al. (U.S. Publication No. US 2024/0221152 A1) (“Kono”).
Regarding claim 1, Sejor discloses a computer implemented method (Fig. 4; [0001] and [0018]) for generating a presentation of at least one image for assisting an operator in bringing a medical implement having a marker to a target location within a body of a patient ([Fig. 4; [0001], [0018], and [0075], wherein the surgical instrument (i.e. medical implement) has a marker on a tip end and assists a practitioner/surgical robot (i.e. operator) in verifying the right positioning (i.e. target location)), the method comprising:
receiving a first image of the medical implement in the body of the patient, the implement being on its way to the target location (Fig. 4 step 71; [0018], [0038], and [0075], wherein the one frame (i.e. first image) including the surgical instrument (i.e. medical implement) is received and the right positioning corresponds to the target location);
processing the first image to obtain an enhanced image showing at least said marker (Fig. 4 steps 72-82; [0038] and [0041-0042], wherein an enlarged image (i.e. enhanced image) of an area of interest, including the tip end of the surgical instrument (i.e. marker), of the one frame (i.e. first image) is obtained); and
providing the enhanced image for display, wherein the enhanced image is provided for display as an inset on a display (Fig. 4 step 82; [0003] and [0040-0042], wherein the enlarged area of interest is displayed on only a part of a screen as a magnifying glass (i.e. inset on a display)).
However, Sejor fails to teach providing the first image and the enhanced image for display, wherein the enhanced image is provided for display as an inset on a display of the first image, specifically. Kono, on the other hand, teaches displaying the X-ray images on a display device. More specifically and as it relates to the applicant’s claims, Kono discloses providing the first image and enhanced image for display (Fig. 1 element 2; [0054], wherein x-ray images including devices (i.e. first image) are displayed on a display device; Fig. 14; [0095] and [0152], wherein an enlarged region of interest (i.e. enhanced image) is superimposed onto the real-time display screen (i.e. first image)), wherein the enhanced image is provided for display as an inset on a display of the first image (Fig. 14; [0095] and [0152], wherein an enlarged window of a region of interest (i.e. enhanced image) is superimposed (i.e. inset) onto the real-time display screen (i.e. first image)).
Kono is combinable with Sejor because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to improve visualization of the details of the movement of the guide wire tip (Kono, [0151]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate providing the first image and the enhanced image for display, wherein the enhanced image is provided for display as an inset on a display of the first image, as taught by Kono, into the computer implemented method, as taught by Sejor, to obtain the invention as specified in claim 1.
Regarding claim 3, Sejor and Kono disclose the computer implemented method of claim 1.
Sejor additionally discloses wherein said processing comprises identifying a portion of the first image as a region of interest comprising the marker ([0031-0032] and [0038], wherein an area of interest (i.e. region of interest) is the part of the frame that includes the tip ends of the surgical instrument, including markers);
However, Sejor fails to teach cropping the identified portion from the first image; and enhancing the cropped portion of the first image, specifically (emphasis added). Kono, on the other hand, teaches cropping and enlarging the region of interest. More specifically and as it relates to the applicant’s claims, Kono discloses cropping the identified portion from the first image; and enhancing the cropped portion of the first image (Fig. 13; [0152], wherein a region of interest is cropped from the acquired image, and the cropped image becomes an enlarged window of a region of interest (i.e. enhanced image)).
Kono is combinable with Sejor because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to improve visualization of the details of the movement of the guide wire tip (Kono, [0151]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate cropping the identified portion from the first image; and enhancing the cropped portion of the first image, as taught by Kono, into the computer implemented method, as taught by Sejor and Kono, to obtain the invention as specified in claim 3.
Regarding claim 4, Sejor and Kono disclose the computer implemented method of claim 1.
Sejor additionally discloses wherein the enhanced image shows a portion of the medical implement enlarged in comparison to its size in the first image (Fig. 4 step 82; [0040-0042], wherein an enlarged image of the area of interest (i.e. enhanced image) includes at least one tip end (i.e. a portion) of the surgical instrument (i.e. medical implement) and this enlarged image of the surgical instrument is obtained by zooming into the area of interest of the frame (i.e. enlarged in comparison to its size in the first image).
Regarding claim 8, Sejor and Kono disclose the computer implemented method of claim 3.
Sejor additionally discloses wherein the processing comprises inputting the first image to a machine learning model trained to identify the region of interest encompassing at least said marker ([0064] and [0075], wherein an artificial intelligence algorithm (i.e. machine learning model) to identify markers of the surgical instrument within a captured image (i.e. first image); [0031-0032] and [0038], wherein an area of interest (i.e. region of interest) is the part of the frame that includes the tip ends of the surgical instrument, including markers).
Regarding claim 12, Sejor discloses a system ([0009], [0066], and [0109]) for generating a presentation of at least one image for assisting an operator in bringing a medical implement having a marker to a target location within a body of a patient ([Fig. 4; [0001], [0018], and [0075], wherein the surgical instrument (i.e. medical implement) has a marker on a tip end and assists a practitioner/surgical robot (i.e. operator) in verifying the right positioning (i.e. target location)), the system comprising:
receive a first image of the medical implement (Fig. 4 step 71; [0018], [0038], and [0075], wherein the one frame (i.e. first image) including the surgical instrument (i.e. medical implement) is received));
process the first image to obtain an enhanced image showing at least said marker (Fig. 4 steps 72-82; [0038] and [0041-0042], wherein an enlarged image (i.e. enhanced image) of an area of interest, including the tip end of the surgical instrument (i.e. marker), of the one frame (i.e. first image) is obtained); and
provide the enhanced image for display, wherein the enhanced image is provided for display as an inset on a display (Fig. 4 step 82; [0003] and [0040-0042], wherein the enlarged area of interest is displayed on only a part of a screen as a magnifying glass (i.e. inset on a display)).
However, Sejor fails to teach a memory, storing instructions; and a processor, configured to execute the instructions. Kono, on the other hand, discloses a memory, storing instructions; (Fig. 2 element 10; [0058], wherein a ROM corresponds to memory and includes an application program with instructions) and a processor, configured to execute the instructions (Fig. 2 element 11; [0059], wherein a CPU/GPU corresponds to a processor and executes the program with instructions).
Kono is combinable with Sejor because they are from the same art of image processing. It is well-known in the art to use a computer to execute instructions regarding a method, within a system, and using a processor is required to execute the instructions stored within the memory of the computer.
Additionally, Sejor fails to teach providing the first image and the enhanced image for display, wherein the enhanced image is provided for display as an inset on a display of the first image, specifically. Kono, on the other hand, teaches displaying the X-ray images on a display device. More specifically and as it relates to the applicant’s claims, Kono discloses a processor to provide the first image and enhanced image for display (Fig. 1 element 2; [0054], wherein x-ray images including devices (i.e. first image) are displayed on a display device; Fig. 14; [0095] and [0152], wherein an enlarged region of interest (i.e. enhanced image) is superimposed onto the real-time display screen (i.e. first image)), wherein the enhanced image is provided for display as an inset on a display of the first image (Fig. 14; [0095] and [0152], wherein an enlarged window of a region of interest (i.e. enhanced image) is superimposed (i.e. inset) onto the real-time display screen (i.e. first image)).
Kono is combinable with Sejor because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to improve visualization of the details of the movement of the guide wire tip (Kono, [0151]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a memory and a processor to provide the first image and the enhanced image for display, wherein the enhanced image is provided for display as an inset on a display of the first image, as taught by Kono, into the system, as taught by Sejor, to obtain the invention as specified in claim 12.
Regarding claim 13, Sejor discloses a system ([0009], [0066], and [0109]) for generating a presentation of at least one image for assisting an operator in bringing a medical implement having a marker to a target location within a body of a patient ([Fig. 4; [0001], [0018], and [0075], wherein the surgical instrument (i.e. medical implement) has a marker on a tip end and assists a practitioner/surgical robot (i.e. operator) in verifying the right positioning (i.e. target location)), the system comprising:
receive a first image of the medical implement in the body of the patient, the implement being on its way to the target location (Fig. 4 step 71; [0018], [0038], and [0075], wherein the one frame (i.e. first image) including the surgical instrument (i.e. medical implement) is received and the right positioning corresponds to the target location);
apply to the first image a machine-learning model trained to identify a region of interest encompassing at least said marker ([0064] and [0075], wherein an artificial intelligence algorithm (i.e. machine learning model) to identify markers of the surgical instrument within a captured image (i.e. first image); [0031-0032] and [0038], wherein an area of interest (i.e. region of interest) is the part of the frame that includes the tip ends of the surgical instrument, including markers);
process the first image to obtain an enhanced image showing at least said marker (Fig. 4 steps 72-82; [0038] and [0041-0042], wherein an enlarged image (i.e. enhanced image) of an area of interest, including the tip end of the surgical instrument (i.e. marker), of the one frame (i.e. first image) is obtained); and
provide the enhanced image for display (Fig. 4 step 82; [0003] and [0040-0042], wherein the enlarged area of interest (i.e. enhanced image) is displayed).
However, Sejor fails to teach a memory, storing instructions; and a processor, configured to execute the instructions. Kono, on the other hand, discloses a memory, storing instructions; (Fig. 2 element 10; [0058], wherein a ROM corresponds to memory and includes an application program with instructions) and a processor, configured to execute the instructions (Fig. 2 element 11; [0059], wherein a CPU/GPU corresponds to a processor and executes the program with instructions).
Kono is combinable with Sejor because they are from the same art of image processing. It is well-known in the art to use a computer to execute instructions regarding a method, within a system, and using a processor is required to execute the instructions stored within the memory of the computer.
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a memory, storing instructions, and a processor, configured to execute the instructions, as taught by Kono, into the system, as taught by Sejor, to obtain the invention as specified in claim 13.
Regarding claim 14, Sejor and Kono disclose the system of claim 12.
Sejor additionally discloses identifying a portion of the first image as a region of interest comprising the marker ([0031-0032] and [0038], wherein an area of interest (i.e. region of interest) is the part of the frame that includes the tip ends of the surgical instrument, including markers);
However, Sejor fails to teach cropping the identified portion from the first image; and enhance at least the the cropped portion of the first image, specifically (emphasis added). Kono, on the other hand, teaches cropping and enlarging the region of interest. More specifically and as it relates to the applicant’s claims, Kono discloses a processor to crop the identified portion from the first image; and enhance the cropped portion of the first image (Fig. 13; [0152], wherein a region of interest is cropped from the acquired image, and the cropped image becomes an enlarged window of a region of interest (i.e. enhanced image)).
Kono is combinable with Sejor because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to improve visualization of the details of the movement of the guide wire tip (Kono, [0151]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a processor to crop the identified portion from the first image; and enhance the cropped portion of the first image, as taught by Kono, into the system, as taught by Sejor and Kono, to obtain the invention as specified in claim 14.
Regarding claim 15, Sejor and Kono disclose the system of claim 12.
Sejor additionally discloses wherein the enhanced image shows a portion of the medical implement enlarged in comparison to its size in the first image (Fig. 4 step 82; [0040-0042], wherein an enlarged image of the area of interest (i.e. enhanced image) includes at least one tip end (i.e. a portion) of the surgical instrument (i.e. medical implement) and this enlarged image of the surgical instrument is obtained by zooming into the area of interest of the frame (i.e. enlarged in comparison to its size in the first image).
Regarding claim 19, Sejor and Kono disclose the system of claim 14.
Sejor additionally discloses wherein the instructions cause the processor to apply, to the first image or to the enhanced image, a machine-learning model trained to identify the region of interest encompassing at least said marker ([0064] and [0075], wherein an artificial intelligence algorithm (i.e. machine learning model) to identify markers of the surgical instrument within a captured image (i.e. first image); [0031-0032] and [0038], wherein an area of interest (i.e. region of interest) is the part of the frame that includes the tip ends of the surgical instrument, including markers).
Regarding claim 24, Sejor and Kono disclose the system of claim 12.
Although Sejor teaches providing the enhanced image for (Fig. 4 step 82; [0003] and [0040-0042], wherein the enlarged area of interest (i.e. enhanced image) is displayed), Sejor fails to teach wherein the instructions cause the processor to provide the enhanced image for display to an output
connected to a display of the imaging device, specifically (emphasis added). Kono, on the other hand, teaches displaying an enlarged image on the display screen connected to an X-ray imaging apparatus. More specifically and as it relates to the applicant’s claims, Kono discloses wherein the instructions cause the processor to provide the enhanced image for display to an output connected to a display of the imaging device (Fig. 1 element 2 and 3; [0052] and [0152], wherein the enlarged image of the area of interest (i.e. enhanced image) is displayed on the display device 2, which is connected to the X-ray imaging apparatus 3 (i.e. imaging device)).
Kono is combinable with Sejor because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to improve visualization of the details of the movement of the guide wire tip (Kono, [0151]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the instructions cause the processor to provide the enhanced image for display to an output connected to a display of the imaging device, as taught by Kono, into the system, as taught by Sejor and Kono, to obtain the invention as specified in claim 24.
Claims 5, 6, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Sejor et al. (International Publication No. WO 2021/185811 A1) ("Sejor") in view of Kono et al. (U.S. Publication No. US 2024/0221152 A1) (“Kono”) and further in view of Dumoulin et al. (U.S. Patent No. US 5318025 A) (“Dumoulin”).
Regarding claim 5, Sejor and Kono disclose the computer implemented method of claim 1.
Although Sejor teaches estimating a roll angle of the medical implement based on appearance of the marker in the first or enhanced image ([0051] and [0055-0056], wherein the orientation (i.e. roll angle) of the surgical instrument is estimated from the size (i.e. appearance) of the marker in the captured frame (i.e. first image)), Sejor and Kono fail to teach providing for display an indication to the estimated roll angle. Dumoulin, on the other hand, teaches providing the orientation of the device on the display. More specifically and as it relates to the applicant’s claims, Dumoulin discloses providing for display an indication to the estimated roll angle (column 4 lines 8-11, wherein the orientation (i.e. roll angle) of the device is displayed).
Dumoulin is combinable with Sejor and Kono because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to obtain information regarding the depth of an object within the field-of-view of the procedure (Dumoulin, column 1 lines 45-48).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate providing for display an indication to the estimated roll angle, as taught by Dumoulin, into the computer implemented method, as taught by Sejor and Kono, to obtain the invention as specified in claim 5.
Regarding claim 6, Sejor, Kono, and Dumoulin disclose the computer implemented method of claim 5. Sejor additionally discloses wherein the marker is shaped to display on the first image a portion of its length depending on the roll angle, and the processing comprises estimating the roll angle based on the length of the marker shown in the first or enhanced image ([0055-0056], wherein determining the orientation (i.e. roll angle) of the surgical instrument on the captured frame (i.e. first image) is based on the deformation of the marker consisting of a modification of its length (i.e. length of the marker)).
Regarding claim 16, Sejor and Kono disclose the system of claim 12.
Although Sejor teaches wherein the instructions further cause the processor to estimate a roll angle of the medical implement based on appearance of the marker in the first or enhanced image ([0051] and [0055-0056], wherein the orientation (i.e. roll angle) of the surgical instrument is estimated from the size (i.e. appearance) of the marker in the captured frame (i.e. first image)), Sejor and Kono fail to teach providing for display an indication to the estimated roll angle. Dumoulin, on the other hand, teaches providing the orientation of the device on the display. More specifically and as it relates to the applicant’s claims, Dumoulin discloses to estimate for display an indication to the estimated roll angle (column 4 lines 8-11, wherein the orientation (i.e. roll angle) of the device is displayed).
Dumoulin is combinable with Sejor and Kono because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to inform the user of the orientation for tracking/monitoring of the device (Dumoulin, Abstract; column 2 lines 39-43).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate providing for display an indication to the estimated roll angle, as taught by Dumoulin, into the system, as taught by Sejor and Kono, to obtain the invention as specified in claim 16.
Regarding claim 17, Sejor, Kono, and Dumoulin disclose the system of claim 16. Sejor additionally discloses wherein the instructions cause the processor to estimate the roll angle based on the length of the marker shown in the first or enhanced image ([0055-0056], wherein determining the orientation (i.e. roll angle) of the surgical instrument on the captured frame (i.e. first image) is based on the deformation of the marker consisting of a modification of its length (i.e. length of the marker)).
Claims 7 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sejor et al. (International Publication No. WO 2021/185811 A1) ("Sejor") in view of Kono et al. (U.S. Publication No. US 2024/0221152 A1) (“Kono”) and Dumoulin et al. (U.S. Patent No. US 5318025 A) (“Dumoulin”) and further in view of Wu et al. (U.S. Publication No. US 2017/0209225 A1) (“Wu”).
Regarding claim 7, Sejor, Kono, and Dumoulin disclose the computer implemented method of claim 5.
However, Sejor, Kono, and Dumoulin fail to teach wherein estimating the roll angle comprises identifying spatial relationships between marking elements of the marker shown in the first or enhanced image. Wu, on the other hand, teaches identifying the orientation of a surgical tool by determining the spatial relationship between two sets of markers. More specifically and as it relates to the applicant’s claims, Wu discloses wherein estimating the roll angle comprises identifying spatial relationships between marking elements of the marker shown in the first or enhanced image ([0047], wherein the orientation (i.e. roll angle) is estimated by identifying the spatial relationship between two sets of markers (i.e. between marking elements) in the image (i.e. first image)).
Wu is combinable with Sejor, Kono, and Dumoulin because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to allow the navigation system to track the tool in relation to the anatomy (Wu, [0049]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein estimating the roll angle comprises identifying spatial relationships between marking elements of the marker shown in the first or enhanced image, as taught by Wu, into the computer implemented method, as taught by Sejor, Kono, and Dumoulin, to obtain the invention as specified in claim 7.
Regarding claim 18, Sejor, Kono, and Dumoulin disclose the system of claim 16.
However, Sejor, Kono, and Dumoulin fail to teach wherein the instructions cause the processor to estimate the roll angle by identifying spatial relationships between marking elements of the marker shown in the first or enhanced image. Wu, on the other hand, teaches identifying the orientation of a surgical tool by determining the spatial relationship between two sets of markers. More specifically and as it relates to the applicant’s claims, Wu discloses wherein the instructions cause the processor to estimate the roll angle by identifying spatial relationships between marking elements of the marker shown in the first or enhanced image ([0047], wherein the orientation (i.e. roll angle) is estimated by identifying the spatial relationship between two sets of markers (i.e. between marking elements) in the image (i.e. first image)).
Wu is combinable with Sejor, Kono, and Dumoulin because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to allow the navigation system to track the tool in relation to the anatomy (Wu, [0049]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the instructions cause the processor to estimate the roll angle by identifying spatial relationships between marking elements of the marker shown in the first or enhanced image, as taught by Wu, into the system, as taught by Sejor, Kono, and Dumoulin, to obtain the invention as specified in claim 18.
Claims 9 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sejor et al. (International Publication No. WO 2021/185811 A1) ("Sejor") in view of Kono et al. (U.S. Publication No. US 2024/0221152 A1) (“Kono”) and Dumoulin et al. (U.S. Patent No. US 5318025 A) (“Dumoulin”) and further in view of Lin et al. (U.S. Publication No. US 2021/0267440 A1) (“Lin”).
Regarding claim 9, Sejor, Kono, and Dumoulin disclose the computer implemented method of claim 5.
However, Sejor, Kono, and Dumoulin fail to teach wherein the processing comprises inputting the first image to a machine-learning model trained to estimate the roll angle. Lin, on the other hand, teaches using a neural network to determine the orientation of the marking relative to the catheter based on the captured image. More specifically and as it relates to the applicant’s claims, Lin discloses wherein the processing comprises inputting the first image to a machine-learning model trained to estimate the roll angle ([0088-0089], wherein an artificial neural network (i.e. machine-leaning model) is trained to determine the orientation (i.e. roll angle) of the marking relative to the catheter based on the captured image (i.e. first image)).
Lin is combinable with Sejor, Kono, and Dumoulin because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to more efficiently and accurately produce the correct results (Lin, [0093]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processing comprises inputting the first image to a machine-learning model trained to estimate the roll angle, as taught by Lin, into the computer implemented method, as taught by Sejor, Kono, and Dumoulin, to obtain the invention as specified in claim 9.
Regarding claim 20, Sejor, Kono, and Dumoulin disclose the system of claim 16.
However, Sejor, Kono, and Dumoulin fail to teach the instructions cause the processor to apply, to the first image or to the enhanced image, a machine-learning model trained to estimate the roll angle. Lin, on the other hand, teaches using a neural network to determine the orientation of the marking relative to the catheter based on the captured image. More specifically and as it relates to the applicant’s claims, Lin discloses the instructions cause the processor to apply, to the first image or to the enhanced image, a machine-learning model trained to estimate the roll angle. ([0088-0089], wherein an artificial neural network (i.e. machine-leaning model) is trained to determine the orientation (i.e. roll angle) of the marking relative to the catheter based on the captured image (i.e. first image)).
Lin is combinable with Sejor, Kono, and Dumoulin because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to more efficiently and accurately produce the correct results (Lin, [0093]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the instructions cause the processor to apply, to the first image or to the enhanced image, a machine-learning model trained to estimate the roll angle., as taught by Lin, into the system, as taught by Sejor, Kono, and Dumoulin, to obtain the invention as specified in claim 20.
Claims 10 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Sejor et al. (International Publication No. WO 2021/185811 A1) ("Sejor") in view of Kono et al. (U.S. Publication No. US 2024/0221152 A1) (“Kono”) and further in view of Crowley (U.S. Patent No. US 5131397 A) (“Crowley”).
Regarding claim 10, Sejor and Kono disclose the computer implemented method of claim 1.
However, Sejor and Kono fail to teach repeated at least 10 times per second, to provide for display of a cine of enhanced images. Crowley, on the other hand, teaches repeating the process many thousands of times per second. More specifically and as it relates to the applicant’s claims, Crowley discloses repeated at least 10 times per second (column 4 lines 62-65, wherein the process is repeated many thousands of times per second), to provide for display of a cine of enhanced images (column 4 lines 62-65, wherein a real-time, two-dimensional image of the subject (i.e. a cine of enhanced images) being studied is displayed).
Crowley is combinable with Sejor and Kono because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to know the exact position of the catheter tip in real-time (Crowley, column 3 lines 60-61).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate repeated at least 10 times per second, to provide for display of a cine of enhanced images, as taught by Crowley, into the computer implemented method, as taught by Sejor and Kono, to obtain the invention as specified in claim 10.
Regarding claim 21, Sejor and Kono disclose the system of claim 14.
However, Sejor and Kono fail to teach wherein the instructions cause the processor to repeat the receiving, processing, and providing for display at least 10 times per second, to provide for display of a cine of enhanced images. Crowley, on the other hand, teaches repeating the process many thousands of times per second. More specifically and as it relates to the applicant’s claims, Crowley discloses wherein the instructions cause the processor to repeat the receiving, processing, and providing for display at least 10 times per second (column 4 lines 62-65, wherein the process is repeated many thousands of times per second), to provide for display of a cine of enhanced images (column 4 lines 62-65, wherein a real-time, two-dimensional image of the subject (i.e. a cine of enhanced images) being studied is displayed).
Crowley is combinable with Sejor and Kono because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to know the exact position of the catheter tip in real-time (Crowley, column 3 lines 60-61).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the instructions cause the processor to repeat the receiving, processing, and providing for display at least 10 times per second, to provide for display of a cine of enhanced images, as taught by Crowley, into the system, as taught by Sejor and Kono, to obtain the invention as specified in claim 10.
Claims 11 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Sejor et al. (International Publication No. WO 2021/185811 A1) ("Sejor") in view of Kono et al. (U.S. Publication No. US 2024/0221152 A1) (“Kono”) and further in view of Lin et al. (U.S. Publication No. US 2021/0267440 A1) (“Lin”).
Regarding claim 11, Sejor and Kono disclose the computer implemented method of claim 1.
However, Sejor and Kono fail to teach wherein the first image is a fluoroscopic image. Lin, on the other hand, teaches wherein the first image is a fluoroscopic image. More specifically and as it relates to the applicant’s claims, Lin discloses wherein the first image is a fluoroscopic image ([0046] and [0051]).
Lin is combinable with Sejor and Kono because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to allow the operator to actively visualize and manipulate the medical instrument’s movements (Lin, [0046-0047])
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the first image is a fluoroscopic image, as taught by Lin, into the computer implemented method, as taught by Sejor and Kono, to obtain the invention as specified in claim 11.
Regarding claim 22, Sejor and Kono disclose the system of claim 12.
However, Sejor and Kono fail to teach wherein the first image is a fluoroscopic image. Lin, on the other hand, teaches wherein the first image is a fluoroscopic image. More specifically and as it relates to the applicant’s claims, Lin discloses wherein the first image is a fluoroscopic image ([0046] and [0051]).
Lin is combinable with Sejor and Kono because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to allow the operator to actively visualize and manipulate the medical instrument’s movements (Lin, [0046-0047]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the first image is a fluoroscopic image, as taught by Lin, into the system, as taught by Sejor and Kono, to obtain the invention as specified in claim 22.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Richmond et al. (U.S. Publication No. US 2018/0055578 A1) teaches estimating the spatial position and pose of a medical device using the tracking markers that are placed onto the device. This allows for tracking of the device within the operating theater during surgery (Abstract, [0046], [0063], [0169]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL Y DANG whose telephone number is (571)438-9519. The examiner can normally be reached Monday - Thursday: 7am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL Y DANG/Examiner, Art Unit 2661
/XUEMEI G CHEN/Primary Examiner, Art Unit 2661
Prosecution Insights