DETAILED ACTION
This office action is in response to the communication received on 12/09/2024 concerning application no. 16/304,405 filed on 11/26/2018.
Claims 21-33 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/09/2024 has been entered.
Claims 21-33 are pending.
Response to Arguments
Applicant's arguments filed 12/09/2024 have been fully considered but they are not persuasive.
Regarding the drawing objections, Applicant comments “Applicants gratefully acknowledge the approval of the previously filed Replacement Sheets of Drawings, and the withdrawal of the objections to the drawing.”
Applicant is mistaken. Applicant is reminded that the final action, filed 08/09/2024, did not accept the submitted drawings. The action, on page 4, states “The drawings were received on 05/23/2024. These drawings are not accepted.” On page 5, the action continues to maintain the objection. Similarly, the present action also maintains the objection.
Applicant's arguments filed 12/09/2024 have been fully considered but they are not persuasive.
Regarding the 102 rejection, Applicant argues that Shekhar does not teach the image transformation between the endoscopic data and the laparoscopic data.
Examiner disagrees. Applicant’s arguments are without support. MPEP 716.01(c) establishes “Arguments presented by the applicant cannot take the place of evidence in the record. In re Schulze, 346 F.2d 600, 602, 145 USPQ 716, 718 (CCPA 1965) and In re De Blauwe, 736 F.2d 699, 705, 222 USPQ 191, 196 (Fed. Cir. 1984).” Furthermore, they are contrary to the actual language of Shekhar. Paragraph 0124 teaches that the laparoscope and LUS images can be overlayed based on the transform of the LUS in relation to the laparoscope and the calculated MIPs are composited with the stereoscopic video images. Paragraph 0124 teaches that the imaging devices are spatially tracked and that the data of both are overlaid. The tracked position of the laparoscope is transformed with respect to the coordinates of the ultrasound data. Paragraph 0100 teaches the registration based on the relationship of the LUS with the stereoscopic view. Their coordinate and field of view is considered and shown in Fig. 10. Paragraphs 0080-81 teaches that the AR fusion module receives the endoscopic and the ultrasound image data from both instruments. Paragraph 01274 teaches the overlay based on the coordinate information. Fig. 11 shows that overlay of the LUS in the camera view. Nothing in the claims precludes the transformation as taught by Shekhar.
Examiner maintains the rejection.
Drawings
The drawings (Fig. 6-8 and 13-14) are objected to because, according to MPEP 608.02 and 67 CFR 1.84, "India ink, or its equivalent that secures solid black lines, must be used for drawings". Drawings should be presented as India ink drawings unless the illustration is not capable of being accurately or adequately depicted by India ink drawings.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 21-33 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 21 recites “control a fusion between the endoscopic image and the ultrasound image based on an image transformation between an endoscopic image space of the endoscope and an ultrasound image space of the laparoscopic ultrasound probe derived from a detection by the image fusion controller of the laparoscopic ultrasound probe be within the field-of-view of the endoscope of the anatomical region” (emphasis added). This is necessarily a computer-implemented functional limitation, consistent with those described in MPEP § 2161.01(I). For computer-implemented functional limitations, applicant must disclose how the result is achieved, and it is “not enough that one skilled in the art could write a program to achieve the claimed function because the specification [itself] must explain how the inventor intends to achieve the claimed function in order to satisfy the written description requirement,” as from the cited section of the manual, referencing Vasudevan Software, Inc. v. MicroStrategy, Inc., 782 F.3d 671, 681-683, 114 USPQ2d 1349, 1356, 1357 (Fed. Cir. 2015). The specification merely repeats this result without delineating the necessary “steps/procedure” (or flowcharts of sufficient specificity) as to how these results are achieved. See [0011], [0013] (paragraphs as numbered in applicant’s pre-grant publication). While the specification also describes what is targeted for transformation (2D planar US image or 3D scan US image, as in [0063]), the details for transforming either of these targets from the US probe being detected within the FOV of the endoscope are not delineated. In view of the evidence cited, applicant has not demonstrated sufficiency of disclosure for these computer-implemented functions.
Claim 21 recites “in response to a communication of the endoscopic image and the ultrasound image, control a fusion...”. This is understood to be a computer-implemented functional limitation which requires disclosure of the underlying algorithm(s) for obtaining the result in order to comply with the written description requirement. See MPEP § 2161.01(I). The specification fails to disclose what the communication is and even what possible nodes it is between. Furthermore, the specification fails to disclose operation that is responsive to this communication of the endoscopic image and the ultrasound image. The specification is silent regarding the operation and considerations that are based on the communications or what steps or procedures are prompted based on the communication. Therefore, the claim contains subject matter which is not described in the specification in such a way as to reasonably convey to one with ordinary skill in the art that the inventor had possession of the claim invention at the time of filing.
Claim 28 recites “responsive to a communication of an anatomical model, the image fusion controller is further configured to control…”. This is understood to be a computer-implemented functional limitation which requires disclosure of the underlying algorithm(s) for obtaining the result in order to comply with the written description requirement. See MPEP § 2161.01(I). The specification fails to disclose what the communication is and even what possible nodes it is between. Furthermore, the specification fails to disclose operation that is responsive to this communication of the anatomical model. The specification is silent regarding the operation and considerations that are based on the communications or what steps or procedures are prompted based on the communication. Therefore, the claim contains subject matter which is not described in the specification in such a way as to reasonably convey to one with ordinary skill in the art that the inventor had possession of the claim invention at the time of filing.
Claims that are not discussed above but are cited to be rejected under 35 U.S.C. 112(a) are also rejected because they inherit the deficiencies of the claims they respectively depend upon.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 21-33 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 21 is indefinite for the following reasons:
The claim is generally narrative and indefinite, failing to conform with current U.S. practice. They appear to be a literal translation into English from a foreign document and are replete with grammatical and idiomatic errors.
Recites “in response to a communication of the endoscopic image and the ultrasound image”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what the communication is between. The image fusion controller is responsive to the communication. However, the claim fails to establish what is conveying the communication and what is receptive of the communication.
Applicant is encouraged to provide consistent and clear language.
Recites “a fusion”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “fusion” is the same as the “image-based fusion” established in the preceding claim element or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Recites “an endoscopic image space of the endoscope”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what an endoscopic image space is. One interpretation is that it is the field of view. Another interpretation is that it is the space occupied by the endoscope in cases of tracking. A third interpretation is the spatial position of the image of the endoscope.
Applicant is encouraged to provide consistent and clear language.
Recites “an ultrasound image space of the laparoscopic ultrasound probe”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what an ultrasound image space is. One interpretation is that it is the field of view. Another interpretation is that it is the space occupied by the laparoscopic ultrasound probe in cases of tracking. A third interpretation is the spatial position of the image of the laparoscopic ultrasound probe.
Applicant is encouraged to provide consistent and clear language.
Recites “the laparoscopic ultrasound probe”. There is insufficient antecedent basis for this limitation in the claim.
Recites “the laparoscopic ultrasound probe”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “laparoscopic ultrasound probe” is the same as the “laparoscopic probe” established in the preceding claim element or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Recites “the field-of-view of the endoscope”. There is insufficient antecedent basis for this limitation in the claim.
Recites “the image fusion controller of the laparoscopic ultrasound probe”. There is insufficient antecedent basis for this limitation in the claim.
Recites “the image fusion controller of the laparoscopic ultrasound probe”. It would be unclear to one with ordinary skill in the art if the “image fusion controller of the laparoscopic ultrasound probe” is the same as the “image fusion controller” established in the preceding claim element or is a separate and distinct feature. If it is the same, the language of the claim is further unclear as the claim establishes the image fusion controller to be of the workstation.
Applicant is encouraged to provide consistent and clear language.
Recites “the endoscope of the anatomical region”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art how an endoscope, an artificial construct, can be of an anatomical region, a biological feature.
Applicant is encouraged to provide consistent and clear language.
Recites “endoscope of the anatomical region”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “endoscope of the anatomical region” is the same as the “endoscope” established in the preceding claim element or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Recites “control a fusion between the endoscopic image and the ultrasound image based on an image transformation between an endoscopic image space of the endoscope and an ultrasound image space of the laparoscopic ultrasound probe derived from a detection by the image fusion controller of the laparoscopic ultrasound probe be within the field-of-view of the endoscope of the anatomical region” (emphasis added). This claim element is indefinite. While it is clear how the transformation between each of the endoscopic image space and the ultrasound image space is derived from detection of the laparoscopic US probe within a FOV of the endoscope, it is unclear how the image fusion itself is specifically derived from detecting the ultrasound probe within the FOV of the endoscope, and the specification does not clarify this feature. For example, if the ultrasound probe is within the endoscopic FOV, those skilled would understand that the images acquired from each modality at that instant will not be of the same anatomical region (i.e., respective image space for each will be different at that time, and fusion of disparate anatomical regions at that instant would be impractical, if not impossible). If applicant intends that the image fusion is controlled between the two images based on the transformation between the two image spaces alone and it is only the transformation that is a precursor to fusion, rather than the fusion itself, which is specifically “derived from” the ultrasound probe appearing within the endoscope FOV and being thus detected, this is not effectively conveyed by the claim language and clarification of the claim is required by delineating these as separate steps/functions, for example.
Applicant is encouraged to provide consistent and clear language.
Claim 23 is indefinite for the following reasons:
Recites “a display of the ultrasound image relative to an instrument inserted into the anatomical region”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the display is the same as the image fusion display of claim 21 that is considerate of the laparoscope being present within the endoscope field of view. Such a display contains the laparoscope. Furthermore, it is unclear if the instrument is the same as the laparoscope of a separate and distinct feature given the common understanding that laparoscopic systems are medical instruments.
Applicant is encouraged to provide consistent and clear language.
Claim 24 is indefinite for the following reasons:
Recites “display the ultrasound image relative to an instrument inserted into the anatomical region”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the display is the same as the image fusion display of claim 21 that is considerate of the laparoscope being present within the endoscope field of view. Such a display contains the laparoscope. Furthermore, it is unclear if the instrument is the same as the laparoscope of a separate and distinct feature given the common understanding that laparoscopic systems are medical instruments.
Applicant is encouraged to provide consistent and clear language.
Claim 26 is indefinite for the following reasons:
The claim is generally narrative and indefinite, failing to conform with current U.S. practice. They appear to be a literal translation into English from a foreign document and are replete with grammatical and idiomatic errors.
Recites “responsive to a communication of an anatomical model”. This claim element is indefinite. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what the communication is between. The image fusion controller is responsive to the communication. However, the claim fails to establish what is conveying the communication and what is receptive of the communication.
Applicant is encouraged to provide consistent and clear language.
Recites “registration of an anatomical model of the anatomical model of the anatomical region and the fusion of the endoscope image and the ultrasound image”. This claim element is indefinite. It is unclear what the anatomical model is intended to be registered to, since “the fusion” is not an image per se, but rather an act or function. The amounts to claiming an act (registration) performed on an act (fusion), which is indefinite. Perhaps applicant intends a registration between the model and a resulting fused (or composite) image, but this is not effectively conveyed by the language recited and clarification is required.
Applicant is encouraged to provide consistent and clear language.
Recites “an anatomical model of the anatomical model of the anatomical region”. This claim element is indefinite. It would be unclear if there are multiple models and the relation between the first anatomical model recitation, the second anatomical model recitation, and the anatomical region.
Applicant is encouraged to use proper grammatical form.
Recites “an image transformation”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “image transformation” is the same as the “image transformation” established in claim 21 or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Recites “a volumetric image space of the anatomical model”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art what a volumetric image space is. One interpretation is that it is the field of view of a modality. Another interpretation is that it is the space occupied by the model.
Applicant is encouraged to provide consistent and clear language.
Claim 28 is indefinite for the following reasons:
Recites “wherein the image fusion controller and the display controller are further configured to control a display of at least one reference planar view of the anatomical model overlaid on a display of at least one of the endoscopic image, the ultrasound image, and the fusion of the endoscopic image and the ultrasound image”. This claim element is indefinite. It is unclear what the anatomical model is intended to be registered to, since “the fusion” is not an image per se, but rather an act or function. The amounts to claiming an act (registration) performed on an act (fusion), which is indefinite. Perhaps applicant intends a registration between the model and a resulting fused (or composite) image, but this is not effectively conveyed by the language recited and clarification is required.
Applicant is encouraged to provide consistent and clear language.
Recites “a display of at least one of the endoscopic image, the ultrasound image, and the fusion of the endoscopic image and the ultrasound image”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “display of at least one of the endoscopic image, the ultrasound image, and the fusion of the endoscopic image and the ultrasound image” is the same as the “display of the fusion by the image fusion controller of the endoscopic image and the ultrasound image” established in claim 21 or is a separate and distinct feature.
Applicant is encouraged to provide consistent and clear language.
Claim 29 is indefinite for the following reasons:
Recites “the registration between the anatomical model of the anatomical region and the fusion of the endoscopic image and the ultrasound image”. This claim element is indefinite. It is unclear what the anatomical model is intended to be registered to, since “the fusion” is not an image per se, but rather an act or function. The amounts to claiming an act (registration) performed on an act (fusion), which is indefinite. Perhaps applicant intends a registration between the model and a resulting fused (or composite) image, but this is not effectively conveyed by the language recited and clarification is required.
Applicant is encouraged to provide consistent and clear language.
Claim 30 is indefinite for the following reasons:
Recites “display a virtual laparoscopic probe”. This claim element is indefinite. An image is a virtual representation. Given that the claim 21 establishes that the laparoscopic probe is in the field of view of the endoscope, it would be unclear if this is establishing the probe in the field of view or some other feature. If it is a different feature, it is further unclear as the probe in the field of view is a virtual representation as captured by the endoscope.
Applicant is encouraged to provide consistent and clear language.
Recites “a display of the anatomical model based on the registered anatomical model of the anatomical region and the fusion of the endoscopic image and the ultrasound image”. This claim element is indefinite. This claim element is indefinite. It is unclear what the anatomical model is intended to be registered to, since “the fusion” is not an image per se, but rather an act or function. The amounts to claiming an act (registration) performed on an act (fusion), which is indefinite. Perhaps applicant intends a registration between the model and a resulting fused (or composite) image, but this is not effectively conveyed by the language recited and clarification is required.
Applicant is encouraged to provide consistent and clear language.
Claims that are not discussed above but are cited to be rejected under 35 U.S.C. 112(b) are also rejected because they inherit the indefiniteness of the claims they respectively depend upon.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 21-33 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite an image fusion workstation for an image-based fusion of an endoscopic image of an anatomical region generated by an endoscope and an ultrasound image of the anatomical region generated by a laparoscopic probe and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “control a fusion between the endoscopic image and the ultrasound image based on an image transformation between an endoscopic image space of the endoscope and an ultrasound image space of the laparoscopic ultrasound probe derived from a detection by the image fusion controller of the laparoscopic ultrasound probe be within the field-of-view of the endoscope of the anatomical region”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the control of image fusion based on an image transformation between image spaces that are derived upon detection of a probe in the field of view of an endoscope. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “an image fusion controller configured to: in response to a communication of the endoscopic image and the ultrasound image; and a display controller configured to control a display of the fusion by the image fusion controller of the endoscopic image and the ultrasound image”. The communication of the images is a data gather step which is a form of a pre-solution insignificant activity. The display of the fusion is a display step that merely amounts to a post-solution insignificant activity. The use of a processor does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. For these reasons, there is no inventive concept in the claim. In light of the above, claim 21 is ineligible.
Claim 22 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 21 rejection.
Claim 22 recites the following elements: “wherein the image fusion controller and the display controller are further configured to receive a user interactive selection of a depth of the ultrasound image from the laparoscopic ultrasound probe”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 22 is ineligible.
Claim 23 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 21 rejection.
Claim 23 recites the following elements: “wherein the image fusion controller and the display controller are further configured to control a display of the ultrasound image relative to an instrument inserted into the anatomical region”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 23 is ineligible.
Claim 24 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 21 rejection.
Claim 24 recites the following elements: “wherein the image fusion controller is further configured to display the ultrasound image relative to an instrument inserted into the anatomical region”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 24 is ineligible.
Claim 25 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 21 rejection.
Claim 25 recites the following elements: “wherein the image fusion controller and the display controller are further configured to display a perspective view of the ultrasound image”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 25 is ineligible.
Claim 26 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite an image fusion workstation and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein, responsive to a communication of an anatomical model, … to control a registration of an anatomical model of the anatomical model of the anatomical region and the fusion of the endoscope image and the ultrasound image based on an image transformation between a volumetric image space of the anatomical model and the endoscopic image space of the endoscope”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to registration of an anatomical model and fusion according to an image transformation. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “the image fusion controller is further configured to”. The use of a processor does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. For these reasons, there is no inventive concept in the claim. In light of the above, claim 26 is ineligible.
Claim 27 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite an image fusion workstation and therefore, is an apparatus.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “wherein the anatomical model is an anatomical atlas of an anatomical structure within the anatomical region”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to registration according to an anatomical atlas. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim does not contain additional elements. Therefore, the claim does not integrate the judicial exception into a practical application.
Step 2B, Inventive Concept: No - Similar to Step 2A Prong 2, the claim does not contain additional elements. For these reasons, there is no inventive concept in the claim. In light of the above, claim 27 is ineligible.
Claim 28 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 26 rejection.
Claim 28 recites the following elements: “wherein the image fusion controller and the display controller are further configured to control a display of at least one reference planar view of the anatomical model overlaid on a display of at least one of the endoscopic image, the ultrasound image, and the fusion of the endoscopic image and the ultrasound image”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 28 is ineligible.
Claim 29 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 26 rejection.
Claim 29 recites the following elements: “wherein the image fusion controller and the display controller are further configured to control a display a targeted ultrasound image among a plurality of ultrasound images of the anatomical region generated by the laparoscopic ultrasound probe based on the registration between the anatomical model of the anatomical region and the fusion of the endoscopic image and the ultrasound image”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 29 is ineligible.
Claim 30 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 26 rejection.
Claim 30 recites the following elements: “wherein the image fusion controller and the display controller are further configured to control a display a virtual laparoscopic probe and the ultrasound image in conjunction with a display of the anatomical model based on the registered anatomical model of the anatomical region and the fusion of the endoscopic image and the ultrasound image”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 30 is ineligible.
Claim 31 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 21 rejection.
Claim 31 recites the following elements: “wherein the image fusion controller and the display controller are further configured to control a display the ultrasound image indicative of a status of a surgical procedure performed within the anatomical region”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 31 is ineligible.
Claim 32 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 25 rejection.
Claim 32 recites the following elements: “wherein the image fusion controller and the display controller are further configured to display at least one reference planar view of the anatomical model indicative of a status of a surgical procedure performed within the anatomical region”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 32 is ineligible.
Claim 33 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 25 rejection.
Claim 33 recites the following elements: “wherein the image fusion controller and the display controller are further configured to provide a virtual display of at least one desired position of the laparoscopic ultrasound probe relative to the anatomical model based on the registered anatomical model of the anatomical region and fused image”. This claim element is a mere display step which amounts to a post-solution insignificant activity. The use of processors does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 33 is ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 21-26 and 31 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shekhar et al. (PGPUB No. US 2014/0303491).
Regarding claim 21, Shekhar teaches an image fusion workstation for an image-based fusion of an endoscopic image of an anatomical region generated by an endoscope and an ultrasound image of the anatomical region generated by a laparoscopic probe, the image fusion workstation comprising:
an image fusion controller (AR fusion module. Paragraph 0060 teaches it has major circuitry components) configured to:
in response to a communication of the endoscopic image and the ultrasound image, control a fusion between the endoscopic image and the ultrasound image based on an image transformation between an endoscopic image space of the endoscope and an ultrasound image space of the laparoscopic ultrasound probe derived from a detection by the image fusion controller of the laparoscopic ultrasound probe be within the field-of-view of the endoscope of the anatomical region (Paragraph 0061 teaches that the tracking device 120 determines the pose of the LUS and the stereoscopy device. The images are overlaid onto each other and allow for stereoscopic visualization. Fig. 11 shows that overlay of the LUS in the camera view. Paragraph 0124 teaches that the laparoscope and LUS images can be overlayed based on the transform of the LUS in relation to the laparoscope and the calculated MIPs are composited with the stereoscopic video images. Paragraph 0124 teaches that the imaging devices are spatially tracked and that the data of both are overlaid. The tracked position of the laparoscope is transformed with respect to the coordinates of the ultrasound data. Paragraph 0100 teaches the registration based on the relationship of the LUS with the stereoscopic view. Their coordinate and field of view is considered and shown in Fig. 10. Paragraphs 0080-81 teaches that the AR fusion module receives the endoscopic and the ultrasound image data from both instruments. Paragraph 01274 teaches the overlay based on the coordinate information. Fig. 11 shows that overlay of the LUS in the camera view); and
a display controller configured to control a display of the fusion by the image fusion controller of the endoscopic image and the ultrasound image (Fig. 11 shows the images of LUS and laparoscopy fused).
Regarding claim 22, Shekhar teaches the image fusion workstation in claim 21, as discussed above.
Shekhar further teaches image fusion workstation, wherein the image fusion controller and the display controller are further configured to receive a user interactive selection of a depth of the ultrasound image from the laparoscopic ultrasound probe (Paragraph 0125 teaches that the user can select the ultrasound thickness and adjust the location in the volume prior to the volume rendering. Paragraph 0124 teaches that the MIP is used in the registration of the ultrasound onto the laparoscopic video).
Regarding claim 23, Shekhar teaches the image fusion workstation in claim 21, as discussed above.
Shekhar further teaches image fusion workstation, wherein the image fusion controller and the display controller are further configured to control a display of the ultrasound image relative to an instrument inserted into the anatomical region (Fig. 20 shows a composite view where the needle is tracked in relation to the ultrasound plane and presented as a navigation aid. This is taught in paragraph 0127).
Regarding claim 24, Shekhar teaches the image fusion workstation in claim 21, as discussed above.
Shekhar further teaches image fusion workstation, wherein the image fusion controller is further configured to display the ultrasound image relative to an instrument inserted into the anatomical region (Fig. 20 shows a composite view where the needle is tracked in relation to the ultrasound plane and presented as a navigation aid. This is taught in paragraph 0127).
Regarding claim 25, Shekhar teaches the image fusion workstation in claim 21, as discussed above.
Shekhar further teaches image fusion workstation, wherein the image fusion controller and the display controller are further configured to display a perspective view of the ultrasound image (Figs. 11-12 show the LUS images and the stereoscopic AR view with the overlaid ultrasound image data. Fig. 20 shows an ultrasound plane).
Regarding claim 26, Shekhar teaches the image fusion workstation in claim 21, as discussed above.
Shekhar further teaches image fusion workstation, wherein, responsive to a communication of an anatomical model, the image fusion controller is further configured to control a registration of an anatomical model of the anatomical model of the anatomical region and the fusion of the endoscope image and the ultrasound image based on an image transformation between a volumetric image space of the anatomical model and the endoscopic image space of the endoscope (Paragraphs 0088-89 teach that the composite stereoscopic images can be created through registration form a surface model that is derived from pre-operative tomography images and a surface model that is reconstructed from the endoscope. The organ deformation model can also be used and processed by the AR fusion module. Paragraph 0082 teaches that the preoperative volumes undergo deformable image registration algorithm as well as the intra operative tomographic images).
Regarding claim 31, Shekhar teaches the image fusion workstation in claim 21, as discussed above.
Shekhar further teaches image fusion workstation, wherein the image fusion controller and the display controller are further configured to control a display the ultrasound image indicative of a status of a surgical procedure performed within the anatomical region (Paragraph 0127 teaches that the marker 2000 and 2005 can show the entry and target position of the needle. This is done with the overlaid composite image data to aid the surgeon performing the procedure. The procedures can be precision of laparoscopic ablations, biopsies, drainages, aspirations, therapeutic delivery, vascular access, and anesthesia delivery. Fig. 20 shows the tracking to the needle and the needle path between the markers).
Claims 21-29 and 31-33 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hasser et al. (US 2007/0021738).
Regarding claim 21, Hasser et al. disclose an image fusion workstation (console “C” of Fig. 1) for fusion of endoscopic and laparoscopic ultrasound images, as in [0038]-[0039] and [0050], in which “ultrasound images captured by LUS [laparoscopic ultrasound] [p]robe 150, right and left 2D camera images captured by the stereoscopic [e]ndoscope 140 [...] are calibrated and registered with each other.” See also “registered overlay” between an ultrasound view and a “camera view” in [0025], of which the camera view is understood to correspond to the images acquired by the endoscope 140. The workstation C of Fig. 1 comprises an image fusion controller in the form of a personal computer (processor 102), as in cited [0039]. The act of displaying from cited [0025] represents a communication of the endoscopic image and the ultrasound image to the workspace and display of the cited fused overlay is necessarily responsive to communication of the images at least to volatile memory associated with the console PC. The controller 102 relies on robot kinematics to report a coordinate frame for LUS probe 150 tip relative to the endoscope 140, which is also associated with a distinct coordinate frame. See “both of these coordinate frames” in [0057]. The LUS probe 150 is described as being both within the coordinate frame in [0057] and within right and left 2D camera views of the endoscope 140. See “captured images of the ultrasound probe” in the right and left 2D [endoscopic] camera views in published claim 16, from which the image space of each of the respective modalities is