DETAILED ACTION
This office action is in response to the communication received on 08/17/2026 concerning application no. 19/225,396 filed on 06/02/2025.
Claims 2-21 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 08/17/2026 have been fully considered but they are not persuasive.
Regarding the 101 rejection, Applicant argues that the amended language is not directed to a mental process. Applicant alleges “The human mind cannot identify and use photosensitive sites within a visible-light image to determine real-time information for each of a plurality of markers of a surgical device in a reasonable amount of time. Further, the surgical computing device being configured to "generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers" is directed to more than an abstract idea. The human mind cannot compare real-time information and stored information for each of a plurality of markers to generate an image of a surgical instrument in a reasonable amount of time.”
Examiner disagrees. MPEP 716.01(c) establishes “Arguments presented by the applicant cannot take the place of evidence in the record. In re Schulze, 346 F.2d 600, 602, 145 USPQ 716, 718 (CCPA 1965) and In re De Blauwe, 736 F.2d 699, 705, 222 USPQ 191, 196 (Fed. Cir. 1984).” Applicant’s allegation is without support. The human mind is capable of identifying sites and markers by observing the image and determining what is associated to a surgical instrument and perform a comparison to stored image data. Such a comparison requires an observation of the instant image and the stored image.
Examiner maintains the rejection.
Applicant’s arguments with respect to claim 2 and 12 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments filed 08/17/2026 have been fully considered but they are not persuasive.
Applicant requests that the rejection of the pending claims be held in abeyance with respect to the double patent rejection.
Examiner disagrees. In the interest of compact prosecution, Applicant is required to address all rejections.
Examiner maintains the rejection.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 2-21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 2 recites “identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites”. While the specification discloses the identification of the surgical instrument and the markers from an image in at least paragraphs 0046-47, the specification does not disclose the identification of photosensitive sites from the visible light image. Rather, the specification, as stated in paragraph 0043, “generates an electrical signal that is representative of the captured image”. See also Fig. 2. The specification does not disclose the imaging of the photosensitive site and the identification based on an image. Rather, the photosensitive site is integrated in the camera for the observation of the surgical instrument and its markers. Therefore, the claim contains subject matter which is not described in the specification in such a way as to reasonably convey to one with ordinary skill in the art that the inventor had possession of the claim invention at the time of filing.
Claim 2 recites “determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites”. While the specification in paragraph 0047 discloses that the markers are detected based on the photosensitive sites, the specification limits the identification to “information related to the color of the detected visible-light”. Additionally, the specification does not disclose the real-time determination via the photosensitive sites. Therefore, the claim contains subject matter which is not described in the specification in such a way as to reasonably convey to one with ordinary skill in the art that the inventor had possession of the claim invention at the time of filing.
Claim 12 recites “identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites”. While the specification discloses the identification of the surgical instrument and the markers from an image in at least paragraphs 0046-47, the specification does not disclose the identification of photosensitive sites from the visible light image. Rather, the specification, as stated in paragraph 0043, “generates an electrical signal that is representative of the captured image”. See also Fig. 2. The specification does not disclose the imaging of the photosensitive site and the identification based on an image. Rather, the photosensitive site is integrated in the camera for the observation of the surgical instrument and its markers. Therefore, the claim contains subject matter which is not described in the specification in such a way as to reasonably convey to one with ordinary skill in the art that the inventor had possession of the claim invention at the time of filing.
Claim 12 recites “determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites”. While the specification in paragraph 0047 discloses that the markers are detected based on the photosensitive sites, the specification limits the identification to “information related to the color of the detected visible-light”. Additionally, the specification does not disclose the real-time determination via the photosensitive sites. Therefore, the claim contains subject matter which is not described in the specification in such a way as to reasonably convey to one with ordinary skill in the art that the inventor had possession of the claim invention at the time of filing.
Claims that are not discussed above but are cited to be rejected under 35 U.S.C. 112(a) are also rejected because they inherit the deficiencies of the claims they respectively depend upon.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 is indefinite for the following reasons:
Recites “determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what information of the marker is determined as the claim establishes in the preceding element that the markers are identified on the visible light image.
Applicant is encouraged to provide consistent and clear language.
Recites “identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the data is referring to the data of a visible light image or of the data set. The claim is establishing that the data has stored information the plurality of markers. However, the plurality of markers where imaged and identified from the visible light image.
Applicant is encouraged to provide consistent and clear language.
Recites “generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers”. This claim element is indefinite. The preceding claim element already establishes the visible light image that has the image of the markers and the photosensitive sites of the surgical instrument. It would be unclear to one with ordinary skill in the art if this is the generation of an alternative image or referring to the original image.
Applicant is encouraged to provide consistent and clear language.
Claim 3 is indefinite for the following reasons:
Recites “wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the data is referring to the data of a visible light image or of the data set. The claim is establishing that the data has stored information the plurality of markers. However, the plurality of markers where imaged and identified from the visible light image.
Applicant is encouraged to provide consistent and clear language.
Claim 12 is indefinite for the following reasons:
Recites “determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what information of the marker is determined as the claim establishes in the preceding element that the markers are identified on the visible light image.
Applicant is encouraged to provide consistent and clear language.
Recites “identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the data is referring to the data of a visible light image or of the data set. The claim is establishing that the data has stored information the plurality of markers. However, the plurality of markers where imaged and identified from the visible light image.
Applicant is encouraged to provide consistent and clear language.
Recites “generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers”. This claim element is indefinite. The preceding claim element already establishes the visible light image that has the image of the markers and the photosensitive sites of the surgical instrument. It would be unclear to one with ordinary skill in the art if this is the generation of an alternative image or referring to the original image.
Applicant is encouraged to provide consistent and clear language.
Claim 3 is indefinite for the following reasons:
Recites “wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the data is referring to the data of a visible light image or of the data set. The claim is establishing that the data has stored information the plurality of markers. However, the plurality of markers where imaged and identified from the visible light image.
Applicant is encouraged to provide consistent and clear language.
Claims that are not discussed above but are cited to be rejected under 35 U.S.C. 112(b) are also rejected because they inherit the indefiniteness of the claims they respectively depend upon.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 2-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical computing device and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers; calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the identification of the markers and sites of an instrument on an image and the determination of associated information and performing a comparison to stored image data for generating an illustration of a surgical instrument and the calculating a temperature, and modifying an image according to the temperature. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “a processor configured to: receive a visible-light image and an infrared image”. Reception of the data and generation of an image is a data gathering step that is a form of a pre-solution insignificant activity. The use of a processor does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. For these reasons, there is no inventive concept in the claim. In light of the above, claim 2 is ineligible.
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical computing device and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to comparison of positional information or color association to stored images. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim does not contain additional elements. Therefore, the claim does not integrate the judicial exception into a practical application.
Step 2B, Inventive Concept: No - Similar to Step 2A Prong 2, the claim does not contain additional elements. For these reasons, there is no inventive concept in the claim. In light of the above, claim 3 is ineligible.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 3 rejection.
Claim 4 recites the following elements: “wherein the plurality of markers is configured to reflect a predetermined frequency of light”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 4 is ineligible.
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical computing device and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to identification according to thresholding. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “wherein the processor is further configured to”. The use of a processor does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. For these reasons, there is no inventive concept in the claim. In light of the above, claim 5 is ineligible.
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical computing device and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to modification of the image according to the thresholded temperature. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim does not contain additional elements. Therefore, the claim does not integrate the judicial exception into a practical application.
Step 2B, Inventive Concept: No - Similar to Step 2A Prong 2, the claim does not contain additional elements. For these reasons, there is no inventive concept in the claim. In light of the above, claim 6 is ineligible.
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 2 rejection.
Claim 7 recites the following elements: “wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 7 is ineligible.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical computing device and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to color coding according to the temperature. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim does not contain additional elements. Therefore, the claim does not integrate the judicial exception into a practical application.
Step 2B, Inventive Concept: No - Similar to Step 2A Prong 2, the claim does not contain additional elements. For these reasons, there is no inventive concept in the claim. In light of the above, claim 8 is ineligible.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 2 rejection.
Claim 9 recites the following elements: “wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 9 is ineligible.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 2 rejection.
Claim 10 recites the following elements: “wherein the visible-light image is received from an optical system”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 10 is ineligible.
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 10 rejection.
Claim 11 recites the following elements: “wherein the processor is remote from the optical system”. The use of a remote processor does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This extra-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 11 is ineligible.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical system and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the identification of the markers and sites of an instrument on an image and the determination of associated information and performing a comparison to stored image data for generating an illustration of a surgical instrument and the calculating a temperature, and modifying an image according to the temperature. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “an optical system configured to: generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector; and a surgical computing device comprising a processor configured to: receive, from the optical system, the visible-light image and the infrared image”. Reception of the data and generation of an image is a data gathering step that is a form of a pre-solution insignificant activity. The use of a processor does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. For these reasons, there is no inventive concept in the claim. In light of the above, claim 12 is ineligible.
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical computing device and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to comparison of positional information or color association to stored images. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim does not contain additional elements. Therefore, the claim does not integrate the judicial exception into a practical application.
Step 2B, Inventive Concept: No - Similar to Step 2A Prong 2, the claim does not contain additional elements. For these reasons, there is no inventive concept in the claim. In light of the above, claim 13 is ineligible.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 13 rejection.
Claim 14 recites the following elements: “wherein the plurality of markers is configured to reflect a predetermined frequency of light”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 14 is ineligible.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical system and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to identification according to thresholding. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “wherein the processor is further configured to”. The use of a processor does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. For these reasons, there is no inventive concept in the claim. In light of the above, claim 15 is ineligible.
Claim 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical system and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to modification of the image according to the thresholded temperature. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim does not contain additional elements. Therefore, the claim does not integrate the judicial exception into a practical application.
Step 2B, Inventive Concept: No - Similar to Step 2A Prong 2, the claim does not contain additional elements. For these reasons, there is no inventive concept in the claim. In light of the above, claim 16 is ineligible.
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 12 rejection.
Claim 17 recites the following elements: “wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 17 is ineligible.
Claim 18 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a surgical system and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to color coding according to the temperature. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim does not contain additional elements. Therefore, the claim does not integrate the judicial exception into a practical application.
Step 2B, Inventive Concept: No - Similar to Step 2A Prong 2, the claim does not contain additional elements. For these reasons, there is no inventive concept in the claim. In light of the above, claim 18 is ineligible.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 12 rejection.
Claim 19 recites the following elements: “further comprising a display device to display the modified image”. This claim element is a mere display step which amounts to a post-solution insignificant activity. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 19 is ineligible.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 12 rejection.
Claim 20 recites the following elements: “wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 20 is ineligible.
Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 12 rejection.
Claim 21 recites the following elements: “wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 21 is ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-3, 5-6, 8, and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236).
Regarding claim 2, Salahieh teaches a surgical computing device, comprising:
a processor (Col. 14, lines 26-34 teaches computation of the images. See Fig. 33. It is inherent that a computational system will utilize a processor and memory for the performance of its computational functions) configured to:
receive a visible-light image and an infrared image (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time);
identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera);
determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera);
calculate a temperature of the surgical instrument based on the infrared image (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera); and
modify the image of the surgical instrument to include visible indicia based on the calculated temperature (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera).
However, Salahieh is silent regarding a surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers;
generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches a surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability);
generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Salahieh with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
Regarding claim 3, modified Salahieh teaches the surgical computing device in claim 2, as discussed above.
However, Salahieh is silent regarding a surgical computing device, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches a surgical computing device, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Salahieh with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
Regarding claim 5, modified Salahieh teaches the surgical computing device in claim 2, as discussed above.
Salahieh further teaches a surgical computing device, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 25, lines 1-9, teach that the limit is assessed with respect to the temperature and the operator is alerted about the temperature).
Regarding claim 6, modified Salahieh teaches the surgical computing device in claim 5, as discussed above.
Salahieh further teaches a surgical computing device, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 25, lines 1-9, teach that the limit is assessed with respect to the temperature and the operator is alerted about the temperature).
Regarding claim 8, modified Salahieh teaches the surgical computing device in claim 2, as discussed above.
Salahieh further teaches a surgical computing device, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 25, lines 1-9, teach that the limit is assessed with respect to the temperature and the operator is alerted about the temperature).
Regarding claim 10, modified Salahieh teaches the surgical computing device in claim 2, as discussed above.
Salahieh further teaches a surgical computing device, wherein the visible-light image is received from an optical system (Col. 2, lines 29-36, teaches the use of cameras. Col. 14, lines 26-34 teaches computation of the images. See Fig. 33. It is inherent that a computational system will utilize a processor and memory for the performance of its computational functions).
Regarding claim 11, modified Salahieh teaches the surgical computing device in claim 10, as discussed above.
Salahieh further teaches a surgical computing device, wherein the processor is remote from the optical system (Col. 2, lines 29-36, teaches the use of cameras. Col. 14, lines 26-34 teaches computation of the images. See Fig. 33. It is inherent that a computational system will utilize a processor and memory for the performance of its computational functions).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Roger et al. (PGPUB No. US 2016/0166335).
Regarding claim 4, modified Salahieh teaches the surgical computing device in claim 3, as discussed above.
However, the combination of Salahieh and Schoepp is silent regarding a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh and Schoepp with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Kassayan (PGPUB No. US 2005/0205566).
Regarding claim 7, modified Salahieh teaches the surgical computing device in claim 2, as discussed above.
However, the combination of Salahieh and Schoepp is silent regarding a surgical computing device, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument.
In an analogous imaging field of endeavor, regarding ablating patient targets, Kassayan teaches a surgical computing device, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Paragraph 0093 teaches that the infrared sensors are able to regulate and monitor the temperature with respect to a temperature threshold with respect to the frequency-based generation heat).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh and Schoepp with Kassayan’s teaching of temperature thresholding with infrared sensors that consider the frequency that is used in the heat generation. This modified apparatus would allow the user to heat tissue quickly in a pattern based on the size and shape of a target and without causing unwanted tissue vaporization (Paragraph 0009 of Kassayan). Furthermore, the modification the consideration of temperature in association the frequency of the components of the instrument (Paragraph 0093 of Kassayan).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Muszala (PGPUB No. US 2010/0063500).
Regarding claim 9, modified Salahieh teaches the surgical computing device in claim 2, as discussed above.
However, the combination of Salahieh and Schoepp is silent regarding a surgical computing device, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue.
In an analogous imaging field of endeavor, regarding ablating patient targets, Muszala teaches a surgical computing device, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Paragraph 0021 teaches the use of an end effector that operates in RF or ultrasound).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh and Schoepp with Muszala’s teaching of an ablative tool. This modified apparatus would allow the user to avoid the issue cut through the zone of sealed or coagulated tissue (Paragraph 0005 of Muszala). Furthermore, the modification utilizes energy based sectioning to cut and/or section tissue as required by an electrosurgical procedure (Paragraph 0002 of Muszala).
Claims 12-13, 15-16, 18-19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596).
Regarding claim 12, Salahieh a surgical system, comprising: an optical system (Col. 14, lines 26-34 teaches computation of the images. See Fig. 33. It is inherent that a computational system will utilize a processor and memory for the performance of its computational functions)
a surgical computing device comprising a processor (Col. 14, lines 26-34 teaches computation of the images. See Fig. 33. It is inherent that a computational system will utilize a processor and memory for the performance of its computational functions) configured to:
receive, from the optical system, the visible-light image and the infrared image (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time);
identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera);
determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera);
calculate a temperature of the surgical instrument based on the infrared image (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera); and
modify the image of the surgical instrument to include visible indicia based on the calculated temperature (Col. 20, lines 28-34 teaches the use of a CMOS camera that is able to operate in the light and infrared range. The visible light image is superimposed with the infrared data. Col. 13, lines 58-Col. 14, lines 10 shows the visualization of the instrument via the camera in real time. Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 21, lines 50-Col. 22, lines 12 teaches that the electrodes can contain markings for registration and identification by the camera).
However, Salahieh is silent regarding a surgical system, configured to:
generate a visible-light image from an optical filter positioned before a first detector; and
generate an infrared image from an infrared optical filter positioned before a second detector; identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers;
generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches a surgical system, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability);
generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Salahieh with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
However, the combination of Salahieh and Schoepp is silent regarding a surgical system, configured to:
generate a visible-light image from an optical filter positioned before a first detector; and
generate an infrared image from an infrared optical filter positioned before a second detector.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Azizian teaches a surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector (Paragraph 0077 teaches the use of a filter that is able to suppress light to allow a particular light through. This can be applied to two CCDs that operate for both NIR and visual light. See Fig. 7).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh and Schoepp with Azizian’s teaching of filtering. This modified apparatus would allow the user to have accurate re-approximation and precise reconstruction (Paragraph 0061 of Azizian). Furthermore, the modification assists in closing the loop of control of robots with special considerations of unique issues (Paragraph 0005 of Azizian).
Regarding claim 13, modified Salahieh teaches the surgical system in claim 12, as discussed above.
However, Salahieh is silent regarding a surgical system, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches a surgical system, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Salahieh with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
Regarding claim 15, modified Salahieh teaches the surgical system in claim 12, as discussed above.
Salahieh further teaches a surgical system, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 25, lines 1-9, teach that the limit is assessed with respect to the temperature and the operator is alerted about the temperature).
Regarding claim 16, modified Salahieh teaches the surgical system in claim 15, as discussed above.
Salahieh further teaches a surgical system, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 25, lines 1-9, teach that the limit is assessed with respect to the temperature and the operator is alerted about the temperature).
Regarding claim 18, modified Salahieh teaches the surgical system in claim 12, as discussed above.
Salahieh further teaches a surgical system, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 25, lines 1-9, teach that the limit is assessed with respect to the temperature and the operator is alerted about the temperature).
Regarding claim 19, modified Salahieh teaches the surgical system in claim 12, as discussed above.
Salahieh further teaches a surgical system, further comprising a display device to display the modified image (Col. 6, lines 28-35 teach the use of a camera assembly for real-time imaging. Col. 20, lines 47-64, teach the display of superimposed information where the temperature can be provided. The indicator can show a red color that is superimposed on the electrode to indicate that it is “hot”. It can also indicate if the electrode is on/off. See Fig. 38. Col. 20, lines 28-34 teach that the camera assembly comprises cameras that operate in the infrared range and that is superimposed on the visible light image of the camera to indicate temperature. The indication of a hot electrode and the indication of the temperature in the image requires an assessment of temperature with respect to a reference point. That is, a value of temperature must be used to show if an electrode is “hot” or on/off for its visualization. Col. 25, lines 1-9, teach that the limit is assessed with respect to the temperature and the operator is alerted about the temperature).
Regarding claim 21, modified Salahieh teaches the surgical system in claim 12, as discussed above.
However, the combination of Salahieh and Schoepp is silent regarding a surgical system, wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Azizian teaches a surgical system, wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument (Paragraph 0058 teaches the use of a robotic system for controlled and guided resection and ablation of the tumor. Paragraphs 0020-24 teaches the use of a robotic system for a surfical operation. Paragraphs 0080-92 teaches the operative control of the tool via the robot).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh and Schoepp with Azizian’s teaching of robotic control. This modified apparatus would allow the user to have accurate re-approximation and precise reconstruction (Paragraph 0061 of Azizian). Furthermore, the modification assists in closing the loop of control of robots with special considerations of unique issues (Paragraph 0005 of Azizian).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596) further in view of Roger et al. (PGPUB No. US 2016/0166335).
Regarding claim 14, modified Salahieh teaches the surgical system in claim 13, as discussed above.
However, the combination of Salahieh, Schoepp, and Azizian is silent regarding a surgical system, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical system, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh, Schoepp, and Azizian with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596) further in view of Kassayan (PGPUB No. US 2005/0205566).
Regarding claim 17, modified Salahieh teaches the surgical system in claim 12, as discussed above.
However, the combination of Salahieh, Schoepp, and Azizian is silent regarding a surgical system, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument.
In an analogous imaging field of endeavor, regarding ablating patient targets, Kassayan teaches a surgical system, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Paragraph 0093 teaches that the infrared sensors are able to regulate and monitor the temperature with respect to a temperature threshold with respect to the frequency-based generation heat).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh, Schoepp, and Azizian with Kassayan’s teaching of temperature thresholding with infrared sensors that consider the frequency that is used in the heat generation. This modified apparatus would allow the user to heat tissue quickly in a pattern based on the size and shape of a target and without causing unwanted tissue vaporization (Paragraph 0009 of Kassayan). Furthermore, the modification the consideration of temperature in association the frequency of the components of the instrument (Paragraph 0093 of Kassayan).
Claims 20 are rejected under 35 U.S.C. 103 as being unpatentable over Salahieh et al. (US Patent No. 9,333,031) in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596) further in view of Muszala (PGPUB No. US 2010/0063500).
Regarding claim 20, modified Salahieh teaches the surgical system in claim 12, as discussed above.
However, the combination of Salahieh, Schoepp, and Azizian is silent regarding a surgical system, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue.
In an analogous imaging field of endeavor, regarding ablating patient targets, Muszala teaches a surgical system, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Paragraph 0021 teaches the use of an end effector that operates in RF or ultrasound).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Salahieh, Schoepp, and Azizian with Muszala’s teaching of an ablative tool. This modified apparatus would allow the user to avoid the issue cut through the zone of sealed or coagulated tissue (Paragraph 0005 of Muszala). Furthermore, the modification utilizes energy based sectioning to cut and/or section tissue as required by an electrosurgical procedure (Paragraph 0002 of Muszala).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 2-3 and 5-10 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 of U.S. Patent No. 10,537,394 in view of Schoepp (PGPUB No. US 2012/0078236).
Regarding claim 2, Claim 1 of U.S. Patent No. 10,537,394 teaches surgical computing device, comprising:
a processor configured to: receive a visible-light image and an infrared image;
identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature (Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
However, the claims of U.S. Patent No. 10,537,394 is silent regarding a surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers;generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches a surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 10,537,394 with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
Regarding claim 3, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 2, as discussed above.
Claim 4 of U.S. Patent No. 10,537,394 further teaches a surgical computing device, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Claim 3 recites: The system of claim 1, wherein the processor is configured to identify data representative of the surgical instrument based on position of the first detector with respect to the surgical instrument.
Claim 4 recites: The system of claim 3, wherein the processor is configured to determine orientation of the surgical instrument based on the identified data representative of the surgical instrument and a database of one or more surgical instruments stored in memory).
Regarding claim 5, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 2, as discussed above.
Claim 1 of U.S. Patent No. 10,537,394 further teaches a surgical computing device, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
Regarding claim 6, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 5, as discussed above.
Claim 6 of U.S. Patent No. 10,537,394 further teaches a surgical computing device, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Claim 6 recites: The system of claim 1, wherein the processor is configured to identify from the second dataset data representative of one or more regions of the surgical instrument above the predetermined threshold temperature based on the frequency of the infrared radiation produced by the one or more regions of the surgical instrument. Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
Regarding claim 7, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 2, as discussed above.
Claim 6 of U.S. Patent No. 10,537,394 further teaches a surgical computing device, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Claim 6 recites: The system of claim 1, wherein the processor is configured to identify from the second dataset data representative of one or more regions of the surgical instrument above the predetermined threshold temperature based on the frequency of the infrared radiation produced by the one or more regions of the surgical instrument. Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
Regarding claim 8, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 2, as discussed above.
Claim 11 of U.S. Patent No. 10,537,394 further teaches a surgical computing device, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Claim 11 recites: The system of claim 1, wherein the visible indicia comprises pixels of a distinct color in the modified image that represent a temperature value at or above the predetermined threshold temperature).
Regarding claim 9, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 2, as discussed above.
Claim 7 of U.S. Patent No. 10,537,394 further teaches a surgical computing device, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Claim 7 recites: The system of claim 1, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue).
Regarding claim 10, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 2, as discussed above.
Claim 1 of U.S. Patent No. 10,537,394 further teaches a surgical computing device, wherein the visible-light image is received from an optical system (Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
Claim 4 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 of U.S. Patent No. 10,537,394 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Roger et al. (PGPUB No. US 2016/0166335.
Regarding claim 4, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 3, as discussed above.
However, the claims of U.S. Patent No. 10,537,394 is silent regarding a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of U.S. Patent No. 10,537,394 and Schoepp with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Claim 11 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 of U.S. Patent No. 10,537,394 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Salahieh et al. (US Patent No. 9,333,031).
Regarding claim 11, modified U.S. Patent No. 10,537,394 teaches the surgical computing device in claim 10, as discussed above.
However, the claims of U.S. Patent No. 10,537,394 is silent regarding a surgical computing device, wherein the processor is remote from the optical system.
In an analogous imaging field of endeavor, regarding temperature observation of a surgical instrument via a camera, Salahieh teaches a surgical computing device, wherein the processor is remote from the optical system (Col. 2, lines 29-36, teaches the use of cameras. Col. 14, lines 26-34 teaches computation of the images. See Fig. 33. It is inherent that a computational system will utilize a processor and memory for the performance of its computational functions).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of U.S. Patent No. 10,537,394 and Schoepp with Salahieh’s teaching of a processor and optical system. This modified apparatus would allow the user to ensure uninterrupted lines of ablation (Col. 2, lines 4-15 of Salahieh). Furthermore, the modification allows for real-time imaging of the instrument components in relation to the anatomy during ablation (Col. 6, lines 28-35 of Salahieh).
Claims 12-13 and 15-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 of U.S. Patent No. 10,537,394 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596)).
Regarding claim 12, U.S. Patent No. 10,537,394 teaches a surgical system, comprising: an optical system configured to: a surgical computing device comprising a processor configured to: receive, from the optical system, the visible-light image and the infrared image; identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature (Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
However, the claims of U.S. Patent No. 10,537,394 is silent regarding surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector; and
identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches surgical system, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 10,537,394 with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Azizian teaches a surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector (Paragraph 0077 teaches the use of a filter that is able to suppress light to allow a particular light through. This can be applied to two CCDs that operate for both NIR and visual light. See Fig. 7).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 10,537,394 with Azizian’s teaching of filtering. This modified apparatus would allow the user to have accurate re-approximation and precise reconstruction (Paragraph 0061 of Azizian). Furthermore, the modification assists in closing the loop of control of robots with special considerations of unique issues (Paragraph 0005 of Azizian).
Regarding claim 13, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 2, as discussed above.
Claim 4 of U.S. Patent No. 10,537,394 further teaches a surgical system, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Claim 3 recites: The system of claim 1, wherein the processor is configured to identify data representative of the surgical instrument based on position of the first detector with respect to the surgical instrument.
Claim 4 recites: The system of claim 3, wherein the processor is configured to determine orientation of the surgical instrument based on the identified data representative of the surgical instrument and a database of one or more surgical instruments stored in memory).
Regarding claim 15, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 12, as discussed above.
Claim 1 of U.S. Patent No. 10,537,394 further teaches a surgical system, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
Regarding claim 16, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 15, as discussed above.
Claim 6 of U.S. Patent No. 10,537,394 further teaches a surgical system, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Claim 6 recites: The system of claim 1, wherein the processor is configured to identify from the second dataset data representative of one or more regions of the surgical instrument above the predetermined threshold temperature based on the frequency of the infrared radiation produced by the one or more regions of the surgical instrument. Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
Regarding claim 17, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 12, as discussed above.
Claim 6 of U.S. Patent No. 10,537,394 further teaches a surgical system, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Claim 6 recites: The system of claim 1, wherein the processor is configured to identify from the second dataset data representative of one or more regions of the surgical instrument above the predetermined threshold temperature based on the frequency of the infrared radiation produced by the one or more regions of the surgical instrument).
Regarding claim 18, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 2, as discussed above.
Claim 11 of U.S. Patent No. 10,537,394 further teaches a surgical system, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Claim 11 recites: The system of claim 1, wherein the visible indicia comprises pixels of a distinct color in the modified image that represent a temperature value at or above the predetermined threshold temperature).
Regarding claim 19, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 12, as discussed above.
Claim 1 of U.S. Patent No. 10,537,394 further teaches a surgical system, further comprising a display device to display the modified image (Claim 1 recites: A surgical system comprising: a first detector, comprising a first array of pixels configured to detect light reflected by a surgical instrument and generate a first signal comprising a first dataset representative of a visible image of the surgical instrument; a second detector, comprising a second array of pixels configured to detect infrared radiation produced by the surgical instrument during a procedure using the surgical instrument and generate a second signal comprising a second dataset representative of an infrared image of the surgical instrument; an optical system configured to direct at least a portion of the light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector; and a processor configured to: receive the first and second signals, identify from the first dataset data representative of the surgical instrument, identify from the second dataset data representative of one or more regions of the surgical instrument above a predetermined threshold temperature, and generate a modified image of the surgical instrument based on data identified from the first and second dataset, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature and the modified image configured to be displayed in a display screen).
Regarding claim 20, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 12, as discussed above.
Claim 7 of U.S. Patent No. 10,537,394 further teaches a surgical system, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Claim 7 recites: The system of claim 1, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue).
Regarding claim 21, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 12, as discussed above.
Claim 9 of U.S. Patent No. 10,537,394 further teaches a surgical system, wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument (Claim 9 recites: The system of claim 1, wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument).
Claim 14 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 of U.S. Patent No. 10,537,394 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596) in view of Roger et al. (PGPUB No. US 2016/0166335).
Regarding claim 14, modified U.S. Patent No. 10,537,394 teaches the surgical system in claim 13, as discussed above.
However, the claims of U.S. Patent No. 10,537,394 are silent regarding surgical system, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of U.S. Patent No. 10,537,394 and Schoepp with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Claim 2-3 and 5-10 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of U.S. Patent No. 11,213,355 in view of Schoepp (PGPUB No. US 2012/0078236).
Regarding claim 2, Claim 3 of U.S. Patent No. 11,213, teaches a surgical computing device, comprising:
a processor configured to: receive a visible-light image and an infrared image; identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature (Claim 3 recites: A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
However, the claims of U.S. Patent No. 11,213, is silent regarding a surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches a surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers ; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 11,213, with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
Regarding claim 3, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 2, as discussed above.
Claim 8 of U.S. Patent No. 11,213,355 further teaches a surgical computing device, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Claim 8 recites: The robotic system of claim 7, wherein the processor is configured to determine orientation of the at least one of the end effector and the shaft based on the identified data representative of the at least one of the end effector and the shaft and a database of one or more end effectors stored in memory. Claim 7 recites: The robotic system of claim 3, wherein the processor is configured to identify data representative of the at least one of the end effector and the shaft based on position of the first detector with respect to the at least one of the end effector and the shaft).
Regarding claim 5, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 2, as discussed above.
Claim 10 of U.S. Patent No. 11,213,355 further teaches a surgical computing device, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Claim 10 recites: The robotic system of claim 3, wherein the processor is configured to identify from the second signal data representative of the one or more regions of the end effector and the shaft above the predetermined threshold temperature based on the frequency of the infrared radiation. Claim 3 recites: A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
Regarding claim 6, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 5, as discussed above.
Claim 3 of U.S. Patent No. 11,213,355 further teaches a surgical computing device, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Claim 3 recites: A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
Regarding claim 7, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 2, as discussed above.
Claim 10 of U.S. Patent No. 11,213,355 further teaches a surgical computing device, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Claim 10 recites: The robotic system of claim 3, wherein the processor is configured to identify from the second signal data representative of the one or more regions of the end effector and the shaft above the predetermined threshold temperature based on the frequency of the infrared radiation).
Regarding claim 8, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 2, as discussed above.
Claim 12 of U.S. Patent No. 11,213,355 further teaches a surgical computing device, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Claim 12 recites: The robotic system of claim 11, wherein the one or more regions are represented by a distinct color. Claim 11 recites: The robotic system of claim 3, wherein the processor is configured to identify from the second signal data representative of the one or more regions of the end effector and the shaft above the predetermined threshold temperature based on the frequency of the infrared radiation. A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
Regarding claim 9, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 2, as discussed above.
Claim 14 of U.S. Patent No. 11,213,355 further teaches a surgical computing device, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Claim 14 recites: The robotic system of claim 3, wherein the end effector is configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue).
Regarding claim 10, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 2, as discussed above.
Claim 4 of U.S. Patent No. 11,213,355 further teaches a surgical computing device, wherein the visible-light image is received from an optical system (Claim 4 recites: The robotic system of claim 3, wherein the camera system includes an optical element configured to direct visible light to a first detector comprising a first array of pixels configured to detect visible light, and direct infrared radiation to a second detector comprising a second array of pixels configured to detect infrared radiation).
Claim 4 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of U.S. Patent No. 11,213,355 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Roger et al. (PGPUB No. US 2016/0166335).
Regarding claim 4, modified U.S. Patent No. 11,213,355, teaches the surgical computing device in claim 3, as discussed above.
However, the claims of U.S. Patent No. 11,213,355, is silent regarding a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 11,213,355, with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Claim 11 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of U.S. Patent No. 11,213,355 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Salahieh et al. (US Patent No. 9,333,031.
Regarding claim 11, modified U.S. Patent No. 11,213,355 teaches the surgical computing device in claim 10, as discussed above.
However, the claims of U.S. Patent No. 11,213,355 is silent regarding a surgical computing device, wherein the processor is remote from the optical system.
In an analogous imaging field of endeavor, regarding temperature observation of a surgical instrument via a camera, Salahieh teaches a surgical computing device, wherein the processor is remote from the optical system (Col. 2, lines 29-36, teaches the use of cameras. Col. 14, lines 26-34 teaches computation of the images. See Fig. 33. It is inherent that a computational system will utilize a processor and memory for the performance of its computational functions).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 11,213,355 and Schoepp with Salahieh’s teaching of a processor and optical system. This modified apparatus would allow the user to ensure uninterrupted lines of ablation (Col. 2, lines 4-15 of Salahieh). Furthermore, the modification allows for real-time imaging of the instrument components in relation to the anatomy during ablation (Col. 6, lines 28-35 of Salahieh).
Claims 12-13 and 15-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of U.S. Patent No. 11,213,355 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596).
Regarding claim 12, Claim 3 of U.S. Patent No. 11,213,355 further teaches a surgical system, comprising: a surgical computing device comprising a processor configured to: receive, from the optical system, the visible-light image and the infrared image; identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature(Claim 3 recites: A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
However, the claims of U.S. Patent No. 11,213,355 is silent regarding a surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector; and identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
However, the claims of U.S. Patent No. 11,213,355 is silent regarding surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector; and
identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches surgical system, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 11,213,355 with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Azizian teaches a surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector (Paragraph 0077 teaches the use of a filter that is able to suppress light to allow a particular light through. This can be applied to two CCDs that operate for both NIR and visual light. See Fig. 7).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 11,213,355 with Azizian’s teaching of filtering. This modified apparatus would allow the user to have accurate re-approximation and precise reconstruction (Paragraph 0061 of Azizian). Furthermore, the modification assists in closing the loop of control of robots with special considerations of unique issues (Paragraph 0005 of Azizian).
Regarding claim 13, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 12, as discussed above.
Claim 8 of U.S. Patent No. 11,213,355 further teaches a surgical system, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Claim 8 recites: The robotic system of claim 7, wherein the processor is configured to determine orientation of the at least one of the end effector and the shaft based on the identified data representative of the at least one of the end effector and the shaft and a database of one or more end effectors stored in memory. Claim 7 recites: The robotic system of claim 3, wherein the processor is configured to identify data representative of the at least one of the end effector and the shaft based on position of the first detector with respect to the at least one of the end effector and the shaft).
Regarding claim 15, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 12, as discussed above.
Claim 10 of U.S. Patent No. 11,213,355 further teaches a surgical system, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Claim 10 recites: The robotic system of claim 3, wherein the processor is configured to identify from the second signal data representative of the one or more regions of the end effector and the shaft above the predetermined threshold temperature based on the frequency of the infrared radiation. Claim 3 recites: A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
Regarding claim 16, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 15, as discussed above.
Claim 3 of U.S. Patent No. 11,213,355 further teaches a surgical system, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Claim 3 recites: A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
Regarding claim 17, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 12, as discussed above.
Claim 10 of U.S. Patent No. 11,213,355 further teaches a surgical system, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Claim 10 recites: The robotic system of claim 3, wherein the processor is configured to identify from the second signal data representative of the one or more regions of the end effector and the shaft above the predetermined threshold temperature based on the frequency of the infrared radiation).
Regarding claim 18, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 12, as discussed above.
Claim 12 of U.S. Patent No. 11,213,355 further teaches a surgical system, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Claim 12 recites: The robotic system of claim 11, wherein the one or more regions are represented by a distinct color. Claim 11 recites: The robotic system of claim 3, wherein the processor is configured to identify from the second signal data representative of the one or more regions of the end effector and the shaft above the predetermined threshold temperature based on the frequency of the infrared radiation. A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
Regarding claim 19, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 12, as discussed above.
Claim 13 of U.S. Patent No. 11,213,355 further teaches a surgical system, further comprising a display device to display the modified image (Claim 13 recites: The robotic system of claim 3, comprising a display device configured to display a modified image of the end effector and the shaft such that the modified image indicates the one or more regions having a temperature at or above the predetermined threshold temperature).
Regarding claim 20, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 12, as discussed above.
Claim 14 of U.S. Patent No. 11,213,355 further teaches a surgical system, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Claim 14 recites: The robotic system of claim 3, wherein the end effector is configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue).
Regarding claim 21, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 12, as discussed above.
Claim 3 of U.S. Patent No. 11,213,355 further teaches a surgical system, wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument (Claim 3 recites: A robotic surgical system, comprising: a robotic arm; a tool assembly removably coupled to the robotic arm and comprising a shaft extending distally from a housing and an end effector coupled to the shaft, the end effector configured to deliver energy to a tissue for the treatment thereof; a camera system configured to capture visible light reflected by at least one of the end effector and the shaft and to capture infrared radiation produced by the at least one of the end effector and the shaft; and a processor configured to: receive a first signal representative of the visible light and a second signal representative of the infrared radiation, identify, from the first signal, data representative of the at least one of the end effector and the shaft, identify, from the second signal, data representative of one or more regions of the end effector and the shaft above a predetermined threshold temperature; and generate a modified image of the at least one of the end effector and the shaft based on data identified from the first and the second signal, the modified image including visible indicia in the one or more regions of the surgical instrument at or above the predetermined temperature).
Claim 14 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-15 of U.S. Patent No. 11,213,355 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596) in view of Roger et al. (PGPUB No. US 2016/0166335).
Regarding claim 14, modified U.S. Patent No. 11,213,355 teaches the surgical system in claim 13, as discussed above.
However, the claims of U.S. Patent No. 11,213,355 are silent regarding surgical system, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of U.S. Patent No. 11,213,355 and Schoepp with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Claim 2-3 and 5-10 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 12,318,147 in view of Schoepp (PGPUB No. US 2012/0078236).
Regarding claim 2, modified U.S. Patent No. 12,318,147 teaches a surgical computing device, comprising: a processor configured to: receive a visible-light image and an infrared image; identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature (Claim 1 recites: A surgical system, comprising: a camera system configured to: capture an image of a surgical instrument using a first photosensitive site, generate data associated with a first signal representative of the image of the surgical instrument, capture an infrared image of the surgical instrument using a second photosensitive site, and generate data associated with a second signal representative of the infrared image of the surgical instrument; and a processor, remote to the camera system, configured to: receive, from the camera system, the data associated with the first signal representative of the image of the surgical instrument and the data associated with the second signal representative of the infrared image of the surgical instrument, calculate a temperature based on the data associated with the second signal representative of the infrared image of the surgical instrument, identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature, generate a real-time image of the surgical instrument based on the data associated with the first signal representative of the image of the surgical instrument, and modify the real-time image to include visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature).
However, the claims of U.S. Patent No. 12,318,147 is silent regarding surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches a surgical computing device, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 12,318,147 with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
Regarding claim 3, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 2, as discussed above.
Claim 5 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Claim 5 recites: The system of claim 1, wherein the processor is further configured to determine orientation of the surgical instrument based on the data associated with the first signal representative the image of the surgical instrument and a database of one or more surgical instruments stored in memory).
Regarding claim 5, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 2, as discussed above.
Claim 1 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Claim 1 recites: A surgical system, comprising: a camera system configured to: capture an image of a surgical instrument using a first photosensitive site, generate data associated with a first signal representative of the image of the surgical instrument, capture an infrared image of the surgical instrument using a second photosensitive site, and generate data associated with a second signal representative of the infrared image of the surgical instrument; and a processor, remote to the camera system, configured to: receive, from the camera system, the data associated with the first signal representative of the image of the surgical instrument and the data associated with the second signal representative of the infrared image of the surgical instrument, calculate a temperature based on the data associated with the second signal representative of the infrared image of the surgical instrument, identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature, generate a real-time image of the surgical instrument based on the data associated with the first signal representative of the image of the surgical instrument, and modify the real-time image to include visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature).
Regarding claim 6, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 5, as discussed above.
Claim 1 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Claim 1 recites: A surgical system, comprising: a camera system configured to: capture an image of a surgical instrument using a first photosensitive site, generate data associated with a first signal representative of the image of the surgical instrument, capture an infrared image of the surgical instrument using a second photosensitive site, and generate data associated with a second signal representative of the infrared image of the surgical instrument; and a processor, remote to the camera system, configured to: receive, from the camera system, the data associated with the first signal representative of the image of the surgical instrument and the data associated with the second signal representative of the infrared image of the surgical instrument, calculate a temperature based on the data associated with the second signal representative of the infrared image of the surgical instrument, identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature, generate a real-time image of the surgical instrument based on the data associated with the first signal representative of the image of the surgical instrument, and modify the real-time image to include visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature).
Regarding claim 7, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 2, as discussed above.
Claim 7 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Claim 7 recites: The system of claim 1, wherein the data associated with the second signal representative of the infrared image of the surgical instrument is a frequency of an infrared radiation produced by the one or more regions of the surgical instrument).
Regarding claim 8, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 2, as discussed above.
Claim 9 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Claim 9 recites: The system of claim 1, wherein the visible indicia comprises pixels of a distinct color in the modified image that represent a temperature value at or above the predetermined threshold temperature).
Regarding claim 9, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 2, as discussed above.
Claim 12 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Claim 12 recites: The system of claim 1, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue).
Regarding claim 10, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 2, as discussed above.
Claim 4 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the visible-light image is received from an optical system (Claim 4 recites: The system of claim 3, further comprising a lens to direct the visible light reflected by the surgical instrument and the infrared radiation produced by the surgical instrument to the optical system. Claim 3 recites: The system of claim 2, further comprising an optical system configured to direct at least a portion of the visible light reflected by the surgical instrument to the first detector and direct at least a portion of the infrared radiation produced by the surgical instrument to the second detector).
Regarding claim 11, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 10, as discussed above.
Claim 1 of U.S. Patent No. 12,318,147 further teaches a surgical computing device, wherein the processor is remote from the optical system (Claim 1 recites: A surgical system, comprising: a camera system configured to: capture an image of a surgical instrument using a first photosensitive site, generate data associated with a first signal representative of the image of the surgical instrument, capture an infrared image of the surgical instrument using a second photosensitive site, and generate data associated with a second signal representative of the infrared image of the surgical instrument; and a processor, remote to the camera system, configured to: receive, from the camera system, the data associated with the first signal representative of the image of the surgical instrument and the data associated with the second signal representative of the infrared image of the surgical instrument, calculate a temperature based on the data associated with the second signal representative of the infrared image of the surgical instrument, identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature, generate a real-time image of the surgical instrument based on the data associated with the first signal representative of the image of the surgical instrument, and modify the real-time image to include visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature).
Claim 4 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 12,318,147 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Roger et al. (PGPUB No. US 2016/0166335).
Regarding claim 4, modified U.S. Patent No. 12,318,147 teaches the surgical computing device in claim 3, as discussed above.
However, the claims of U.S. Patent No. 12,318,147 is silent regarding a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 12,318,147 with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Claims 12-13 and 15-21 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 12,318,147 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596).
Regarding claim 12, Claim 1 of U.S. Patent No. 12,318,147 further teaches a surgical system, comprising: a surgical computing device comprising a processor configured to: receive, from the optical system, the visible-light image and the infrared image; identify, from the visible-light image, a plurality of markers on a surgical instrument and a plurality of photosensitive sites; determine real-time information for each of the plurality of markers using each of the plurality of photosensitive sites; calculate a temperature of the surgical instrument based on the infrared image; and modify the image of the surgical instrument to include visible indicia based on the calculated temperature (Claim 1 recites: A surgical system, comprising: a camera system configured to: capture an image of a surgical instrument using a first photosensitive site, generate data associated with a first signal representative of the image of the surgical instrument, capture an infrared image of the surgical instrument using a second photosensitive site, and generate data associated with a second signal representative of the infrared image of the surgical instrument; and a processor, remote to the camera system, configured to: receive, from the camera system, the data associated with the first signal representative of the image of the surgical instrument and the data associated with the second signal representative of the infrared image of the surgical instrument, calculate a temperature based on the data associated with the second signal representative of the infrared image of the surgical instrument, identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature, generate a real-time image of the surgical instrument based on the data associated with the first signal representative of the image of the surgical instrument, and modify the real-time image to include visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature).
However, the claims of U.S. Patent No. 12,318,147 is silent regarding a surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector; and identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers.
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Schoepp teaches surgical system, identify data from a visible-light image dataset, wherein the data includes stored information for each of the plurality of markers; generate an image of the surgical instrument based on a comparison of the real-time information and the stored information for each of the plurality of markers (Paragraph 0021 teaches the use of an image recognition program that assess the pattern of the optical navigation markers of the instrument that is observed by the camera. The tracking is done with respect to the data saved in the database. Paragraph 0031 teaches real time operability. Paragraph 0055 teaches that the tracking elements have unique optical navigation markers that have different shapes, colors, and spacing).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 12,318,147 with Schoepp’s teaching of pattern recognition with respect to a database. This modified apparatus would allow the user to improve ease of handling during imaging and make it easier to maintain sterility during the surgical procedure (Paragraph 0028 of Schoepp). Furthermore, the modification provides for unique identification for the tool and can assess multiple tools (Paragraph 0021 of Schoepp).
In an analogous imaging field of endeavor, regarding surgical navigation and instrument tracking, Azizian teaches a surgical system, generate a visible-light image from an optical filter positioned before a first detector; and generate an infrared image from an infrared optical filter positioned before a second detector (Paragraph 0077 teaches the use of a filter that is able to suppress light to allow a particular light through. This can be applied to two CCDs that operate for both NIR and visual light. See Fig. 7).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify U.S. Patent No. 12,318,147 with Azizian’s teaching of filtering. This modified apparatus would allow the user to have accurate re-approximation and precise reconstruction (Paragraph 0061 of Azizian). Furthermore, the modification assists in closing the loop of control of robots with special considerations of unique issues (Paragraph 0005 of Azizian).
Regarding claim 13, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 12, as discussed above.
Claim 5 of U.S. Patent No. 12,318,147 further teaches a surgical system, wherein the real-time information and the stored information for each of the plurality of markers is related to at least one of positions of the plurality of markers or colors of the plurality of markers (Claim 5 recites: The system of claim 1, wherein the processor is further configured to determine orientation of the surgical instrument based on the data associated with the first signal representative the image of the surgical instrument and a database of one or more surgical instruments stored in memory).
Regarding claim 15, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 12, as discussed above.
Claim 1 of U.S. Patent No. 12,318,147 further teaches a surgical system, wherein the processor is further configured to: identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature (Claim 1 recites: A surgical system, comprising: a camera system configured to: capture an image of a surgical instrument using a first photosensitive site, generate data associated with a first signal representative of the image of the surgical instrument, capture an infrared image of the surgical instrument using a second photosensitive site, and generate data associated with a second signal representative of the infrared image of the surgical instrument; and a processor, remote to the camera system, configured to: receive, from the camera system, the data associated with the first signal representative of the image of the surgical instrument and the data associated with the second signal representative of the infrared image of the surgical instrument, calculate a temperature based on the data associated with the second signal representative of the infrared image of the surgical instrument, identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature, generate a real-time image of the surgical instrument based on the data associated with the first signal representative of the image of the surgical instrument, and modify the real-time image to include visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature).
Regarding claim 16, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 15, as discussed above.
Claim 1 of U.S. Patent No. 12,318,147 further teaches a surgical system, wherein the image of the surgical instrument is modified to include the visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature (Claim 1 recites: A surgical system, comprising: a camera system configured to: capture an image of a surgical instrument using a first photosensitive site, generate data associated with a first signal representative of the image of the surgical instrument, capture an infrared image of the surgical instrument using a second photosensitive site, and generate data associated with a second signal representative of the infrared image of the surgical instrument; and a processor, remote to the camera system, configured to: receive, from the camera system, the data associated with the first signal representative of the image of the surgical instrument and the data associated with the second signal representative of the infrared image of the surgical instrument, calculate a temperature based on the data associated with the second signal representative of the infrared image of the surgical instrument, identify one or more regions of the surgical instrument having a temperature at or above a predetermined threshold temperature based on the calculated temperature, generate a real-time image of the surgical instrument based on the data associated with the first signal representative of the image of the surgical instrument, and modify the real-time image to include visible indicia in the one or more regions of the surgical instrument at or above the predetermined threshold temperature).
Regarding claim 17, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 12, as discussed above.
Claim 7 of U.S. Patent No. 12,318,147 further teaches a surgical system, wherein the infrared image includes a frequency of an infrared radiation produced by one or more regions of the surgical instrument (Claim 7 recites: The system of claim 1, wherein the data associated with the second signal representative of the infrared image of the surgical instrument is a frequency of an infrared radiation produced by the one or more regions of the surgical instrument).
Regarding claim 18, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 12, as discussed above.
Claim 9 of U.S. Patent No. 12,318,147 further teaches a surgical system, wherein the visible indicia comprise pixels of a distinct color in the modified image that represent the calculated temperature (Claim 9 recites: The system of claim 1, wherein the visible indicia comprises pixels of a distinct color in the modified image that represent a temperature value at or above the predetermined threshold temperature).
Regarding claim 19, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 12, as discussed above.
Claim 11 of U.S. Patent No. 12,318,147 further teaches a surgical system, further comprising a display device to display the modified image (Claim 11 recites: The system of claim 1, further comprising a display device to display the modified image).
Regarding claim 20, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 12, as discussed above.
Claim 12 of U.S. Patent No. 12,318,147 further teaches a surgical system, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue (Claim 12 recites: The system of claim 1, wherein the surgical instrument comprises an energy delivering end effector configured to deliver at least one of ultrasonic energy and radio frequency energy to a tissue).
Regarding claim 21, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 12, as discussed above.
Claim 13 of U.S. Patent No. 12,318,147 further teaches a surgical system, wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument (Claim 13 recites: The system of claim 1, wherein the surgical system is a robotic surgical system and the robotic surgical system comprises at least one robotic arm configured to hold and manipulate the surgical instrument).
Claim 14 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-14 of U.S. Patent No. 12,318,147 in view of Schoepp (PGPUB No. US 2012/0078236) further in view of Azizian et al. (PGPUB No. US 2013/0274596).
Regarding claim 14, modified U.S. Patent No. 12,318,147 teaches the surgical system in claim 13, as discussed above.
However, the claims of U.S. Patent No. 12,318,147 are silent regarding surgical system, wherein the plurality of markers is configured to reflect a predetermined frequency of light.
In an analogous imaging field of endeavor, regarding infrared tracking of medical instruments, Roger teaches a surgical computing device, wherein the plurality of markers is configured to reflect a predetermined frequency of light (Paragraph 0027 teaches that the surgical markers can have reflective components that reflect light of a particular frequency and can be incorporated with a surgical drill).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of U.S. Patent No. 12,318,147 and Schoepp with Roger’s teaching of markers that reflect a particular frequency of light. This modified apparatus would allow the user to track medical instruments with respect to IR cameras and determine their position (Paragraph 0027 of Roger). Furthermore, the modification allows for navigation with respect to anatomy (Paragraphs 0026-28 of Roger).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Ren et al. (PGPUB No. US 2018/0168737): Teaches template matching for marker detection.
Nagel et al. (PGPUB No. US 2008/0221520): Teaches template matching for marker detection.
Kwak et al. (PGPUB No. US 2014/0316252): Teaches template matching for marker detection.
Razzaque et al. (PGPUB No. US 2010/0268067): Teaches template matching for marker detection.
Duret (PGPUB No. US 2016/0220105): Teaches template matching for marker detection.
Walker et al. (US Patent No. 11,504,187): Teaches template matching for marker detection.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADIL PARTAP S VIRK whose telephone number is (571)272-8569. The examiner can normally be reached Mon-Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached on 571-272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADIL PARTAP S VIRK/Primary Examiner, Art Unit 3798
Prosecution Insights