DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4-6 and 16-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recites “a planned needle direction” in lines 4-5 page 2. It is not clear whether applicant intends to recite two different planned needle direction as “a planned needle direction” was recited in claim 3.
Claim 5 recites the limitation "at the end of the needle intervention" in line 9 page 2. There is insufficient antecedent basis for this limitation in the claim, since “an end of the needle intervention” was not previously recited in claims 1 and 5.
Claim 6 recites the limitation "end point of the needle" in line 14 page 2. There is insufficient antecedent basis for this limitation in the claim, since “an end point of the needle” was not previously recited in claims 1, 5, and 6. The examiner further notes that it is not clear whether it meant to refer to “end point of the head of the needle” recited in claim 5 or it refers to the other end point of the needle as claimed.
Claim 16 recites the limitation "at the end of the needle intervention" in line page 4. There is insufficient antecedent basis for this limitation in the claim, since “an end of the needle intervention” was not previously recited in claims 1 and 3.
Claim 17 is rejected due to its dependency upon rejected claim 16.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a determination unit,” “a transformation unit,” and “a generation unit” in claim 12.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
For a purpose of examination, the examiner found support for recited limitations in the specification of instant application in paragraphs [0099]-[0101], and [0113]. Thus, “a determination unit,” “a transformation unit,” and “a generation unit” are interpreted as a processor, CPU and equivalent thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Allowable Subject Matter
Claims 4, 6, and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and amened to overcome 112 rejections.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 11-12, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Maier-Hein et al.,” US 2013/0245461 (hereinafter Maier-Hein).
Regarding to claim 1, Maier-Hein teaches a computer-implemented method for generating augmented camera image data for guiding a needle intervention ([0003] and [0103]), the computer-implemented method comprising:
receiving medical image data of an examination portion of a patient (medical data image from a CT or MRI scan [0143])
receiving camera image data of the examination portion, the camera image data acquired using a camera (camera [0145], camera allows for capturing 2D image [0146])
determining information concerning a planned insertion pose of a needle based on the medical image data, the information including coordinates assigned to the planned insertion pose of the needle in a medical image data coordinate system (insertion path planned and constructed on previously acquired 3D medical data image [0142]-[0143])
determining coordinates assigned to the planned insertion pose of the needle in a camera coordinate system of the camera by transforming the coordinates assigned to the planned insertion pose of the needle in the medical image data coordinate system into the camera coordinate system (coordinates [0119]; registration of the anatomy of the patient with 3D planning data sets [0145]; registration of the planning image and the camera yielding the pose of the critical structures and other relevant planning data such as needle trajectory relative to the intra-interventionally acquired surface, the position of the planned path [0149]) and
generating augmented camera image data based on the camera image data and the coordinates assigned to the planned insertion pose of the needle in the camera coordinate system ( augmented reality visualization of anatomical data and instrument guidance [0145]; AR view, intensity image of the camera, physician may see true instrument pose as well as projected pose which represents a desired pose prior to the needle insertion with the projection [0150] Figure 3).
Regarding to claims 2-3 and 11, Maier-Hein teaches all limitations of claim 1 as set forth above.
Maier-Hein further teaches following limitations:
Of claim 2, wherein the information concerning the planned insertion pose of the needle includes a needle insertion point ( plan a predetermined path, such as an insertion path for inserting a needle into the patient, a path from a skin entry point [0064])
Of claim 3, wherein the information concerning the planned insertion pose of the needle includes a needle orientation for the needle intervention (position and orientation of the needle [0143], [0154]; instrument pose [0147]), and the determining information concerning the planned insertion pose of the needle in the medical image data includes determining at least one of (i) an end point of the needle at the start of the insertion based on a target needle tip position, (ii) a planned needle direction, or (iii) a length of the needle ( moving tip of the needle to be inserted into insertion path to a predetermined body part [0058]; initially position the tip of the needle at the planned insertion point visualized in the image [0147]).
Of claim 11, A camera-based method for guiding a needle intervention, the camera-based method comprising: generating augmented camera image data by performing the computer-implemented method according to claim 1; and displaying the augmented camera image data (computer [0003] and [0103] ; processing unit and software [0106]; virtual camera display [0108]).
Regarding to claim 12, Maier-Hein teaches an intervention control device, comprising:
a medical image data interface to receive medical image data of an examination portion of a patient (medical data image from a CT or MRI scan [0143]; image data stored [0105] and [0134]);
a camera image data interface to receive camera image data of the examination portion (camera [0145], camera allows for capturing 2D image [0146]; camera data being sent to the computer [0134]);
a determination unit to determine information concerning a planned insertion pose of a needle based on the medical image data, the information including coordinates assigned to the planned insertion pose of the needle in a medical image data coordinate system (insertion path planned and constructed on previously acquired 3D medical data image [0142]-[0143]);
a transformation unit to determine coordinates assigned to the planned insertion pose of the needle in a camera coordinate system of a camera by transforming the coordinates assigned to the planned insertion pose from the medical image data coordinate system into the camera coordinate system (coordinates [0119]; registration of the anatomy of the patient with 3D planning data sets [0145]; registration of the planning image and the camera yielding the pose of the critical structures and other relevant planning data such as needle trajectory relative to the intra-interventionally acquired surface, the position of the planned path [0149]); and
a generation unit to generate augmented camera image data based on the camera image data and the coordinates assigned to the planned insertion pose of the needle in the camera coordinate system ( augmented reality visualization of anatomical data and instrument guidance [0145]; AR view, intensity image of the camera, physician may see true instrument pose as well as projected pose which represents a desired pose prior to the needle insertion with the projection [0150] Figure 3).
Regarding to claim 13, Maier-Hein teaches all limitations of claim 12 as set forth above.
Maier-Hein further discloses an intervention system, comprising:
a medical imaging system (CT or MRI [0143]);
a camera (camera [0145]);
a needle for an intervention (needle [0043]);
an intervention control device according to claim 12, wherein the intervention control device is configured to generate augmented camera image data based on medical image data from the medical imaging system and camera image data from the camera, and wherein the intervention control device is configured to control an intervention using the needle based on the augmented camera image data (guiding the advancement of the needle in AR view [0150]-[0151]); and
a display to display the augmented camera image data generated by the intervention control device (display [0104]; virtual camera [0108]).
Regarding to 20, Maier-Hein teaches an intervention control device, comprising:
a medical image data interface to receive medical image data of an examination portion of a patient (medical data image from a CT or MRI scan [0143]; image data stored [0105] and [0134]);
a camera image data interface to receive camera image data of the examination portion (camera [0145], camera allows for capturing 2D image [0146]; camera data being sent to the computer [0134]); and
at least one processor (processing the data [0111]-[0113], computer [0103]) configured to determine information concerning a planned insertion pose of a needle based on the medical image data, the information including coordinates assigned to the planned insertion pose of the needle in a medical image data coordinate system (insertion path planned and constructed on previously acquired 3D medical data image [0142]-[0143]), determine coordinates assigned to the planned insertion pose of the needle in a camera coordinate system of the camera by transforming the coordinates assigned to the planned insertion pose from the medical image data coordinate system into the camera coordinate system (coordinates [0119]; registration of the anatomy of the patient with 3D planning data sets [0145]; registration of the planning image and the camera yielding the pose of the critical structures and other relevant planning data such as needle trajectory relative to the intra-interventionally acquired surface, the position of the planned path [0149]), and generate augmented camera image data based on the camera image data and the coordinates assigned to the planned insertion pose in the camera coordinate system ( augmented reality visualization of anatomical data and instrument guidance [0145]; AR view, intensity image of the camera, physician may see true instrument pose as well as projected pose which represents a desired pose prior to the needle insertion with the projection [0150] Figure 3).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Maier-Hein as applied to claims 1 and 3 above, and further in view of “Lavallee et al,” US 2017/0238999 (hereinafter Lavallee).
Regarding to claims 5 and 16, Maier-Hein teaches all limitations of claims 1 and 3 as set forth above.
Maier-Hein does not further disclose wherein the information concerning the planned insertion pose of the needle in the medical image data comprises a planned end point of a head of the needle at the end of the needle intervention.
However, in the analogous field of endeavor in needle guidance and navigation, Lavallee teaches image guidance of needle, including the information comprises a planned end point of a head of the needle at the end of the needle intervention (The point corresponding to the stop of the needle (claimed end point of a head of the needle) projected with the position and orientation of the needle [0087]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the planned insertion pose of the needle as taught by Maier-Hein to incorporate teaching of Lavallee, since needle intervention planning information including end point of the head of the needle was well known in the art as taught by Lavallee. One of ordinary skill in the art could have combined the elements as claimed by Maier-Hein with no change in their respective functions, configuring the position tracking of the needle to include a head of the needle, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to provide determination of the length between distal tip and the stop ([0103]), and there was reasonable expectation of success.
Claims 7-8, 10, and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Maier-Hein as applied to claim 1 above, and further in view of “Mariampillai et al,” US 2021/0153953 (hereinafter Mariampillai).
Regarding to claim 7, Maier-Hein teaches all limitations of claim 1 as set forth above.
Maier-Hein further teaches following limitations of wherein the camera image data includes 2D camera image data ( a camera allows for capturing 2D intensity image such as RGB image [0146]), but is silent with the limitation of the determining coordinates assigned to the planned insertion pose of the needle in the camera coordinate system of the camera includes transforming the coordinates assigned to the planned insertion pose from the medical image data coordinate system into a 3D camera coordinate system, and projecting the transformed coordinates into 2D camera image data to determine 2D coordinates assigned to the planned insertion pose.
However, in the analogous field of endeavor in surgical navigation, Mariampillai teaches transforming the coordinates assigned to the planned insertion pose from the medical image data coordinate system into a 3D camera coordinate system, and projecting the transformed coordinates into 2D camera image data to determine 2D coordinates assigned to the planned insertion pose (tracking system is a camera which outputs a 2D image such that a known 3D position coordinate space can projected into the 2D camera image space, using transform for mapping a 3D position from the reference frame to coordinate in the 2D camera image ([0127]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify transformation as taught by Maier-Hein to incorporate teaching of Mariampillai, since 3D transformation into 2D coordinates was well known in the art as taught by Mariampillai. One of ordinary skill in the art could have combined the elements as claimed by Maier-Hein with no change in their respective functions, employing 3D to 2D coordinate transform, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to provide a positional information projected into the 2D image space ([0127]), and there was reasonable expectation of success.
Regarding to claim 8, Maier-Hein and Mariampillai together disclose all limitations of claim 7 as set forth above.
Mariampillai further teaches, wherein the transforming the coordinates assigned to the planned insertion pose from the medical image data coordinate system into a 3D camera coordinate system includes applying a camera transform to the coordinates assigned to the planned insertion pose in the medical image data coordinate system (a perspective transform [0127], Figure 7).
Regarding to claim 10, Maier-Hein and Mariampillai together disclose all limitations of claim 7 as set forth above.
Mariampillai further teaches wherein the camera transform is determined based on an extrinsic calibration step (camera calibration includes extrinsic parameters [0126]).
Regarding to claims 14-15, Maier-Hein teaches all limitations of claim 1 as set forth above.
Maier-Hein does disclose software and using a computer to process the data, but does not explicitly disclose a non-transitory computer-readable storage medium or computer program product as set forth in claims 14-15.
However, However, in the analogous field of endeavor in surgical navigation, Mariampillai teaches computer readable media, storage medium to process and perform the software/instructions ([0083] and [0085]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the computer and memory as taught by Maier-Hein to incorporate teaching of Mariampillai , since computer readable medium and storage was well known in the art as taught by Mariampillai. One of ordinary skill in the art could have combined the elements as claimed by Maier-Hein with no change in their respective functions, configuring its method/processing to be executed in the computer readable medium, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to provide a computer readable medium for executing the instructions/method of guidance of the tool ([0083] and [0085]), and there was reasonable expectation of success.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Maier-Hein and Mariampillai as applied to claims 7-8 above, and further in view of “Chang et al,” US 2017/0100089 (hereinafter Chang).
Regarding to claims 9 and 19, Maier-Hein and Mariampillai together disclose all limitations of claims 7 and 8 as set forth above.
Mariampillai further teaches a patient support such as treatment bed or couch ([0122]).
Mariampillai does not further disclose wherein the transforming the coordinates assigned to the planned insertion pose from the medical image data coordinate system into a 3D camera coordinate system includes applying a transformation representing a new table position.
However, in the analogous field of endeavor in camera tracking of the surgical procedures, Chang teaches transformation between the coordinate of the camera and the coordinate of the patient table ([0022], [0023]-[0024]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify calibration as taught by Maier-Hein and Mariampillai to incorporate teaching of Chang, since transformation between the coordinate of the camera and the coordinate of the patient table was well known in the art as taught by Chang. One of ordinary skill in the art could have combined the elements as claimed by Mariampillai with no change in their respective functions, configuring its calibration step to include transformation between the camera and the patient table coordinates, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to provide an accurate transformation of camera and patient table ([0022]), and there was reasonable expectation of success.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wu (US2024/0074810) teaches surgery planning method including XR display which depicts virtual surgical sites, user can virtually replacing, changing a position of an insertion point and changing a moving direction of the virtual surgical instrument ([0083]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICIA J PARK whose telephone number is (571)270-1788. The examiner can normally be reached Monday-Thursday 8 am - 3 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at 571-272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICIA J PARK/Primary Examiner, Art Unit 3798
Prosecution Insights