DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-14 are pending for examination in the application filed 09/09/2024.
Priority
Acknowledgement is made of Applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been received for parent application JP2022-154021, filing date: 09/27/2022. Acknowledgement is additionally made of the present application as a national stage entry of PCT/JP2023/023631, international filing date: 06/26/2023.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/09/2024 has been considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier, as explained in MPEP §2181, subsection I (note that the list of generic placeholders below is not exhaustive, and other generic placeholders may invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph):
A. The Claim Limitation Uses the Term “Means” or “Step” or a Generic Placeholder (A Term That Is Simply A Substitute for “Means”)
With respect to the first prong of this analysis, a claim element that does not include the term “means” or “step” triggers a rebuttable presumption that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, does not apply. When the claim limitation does not use the term “means,” examiners should determine whether the presumption that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, paragraph 6 does not apply is overcome. The presumption may be overcome if the claim limitation uses a generic placeholder (a term that is simply a substitute for the term “means”). The following is a list of non-structural generic placeholders that may invoke 35 U.S.C. 112(f) or pre- AIA 35 U.S.C. 112, paragraph 6: “mechanism for,” “module for,” “device for,” “unit for,” “component for,” “element for,” “member for,” “apparatus for,” “machine for,” or “system for.” Welker Bearing Co., v. PHD, Inc., 550 F.3d 1090, 1096, 89 USPQ2d 1289, 1293-94 (Fed. Cir. 2008); Massachusetts Inst. of Tech. v. Abacus Software, 462 F.3d 1344, 1354, 80 USPQ2d 1225, 1228 (Fed. Cir. 2006); Personalized Media,161 F.3d at 704, 48 USPQ2d at 1886–87; Mas- Hamilton Group v. LaGard, Inc., 156 F.3d 1206, 1214-1215, 48 USPQ2d 1010, 1017 (Fed. Cir.1998). This list is not exhaustive, and other generic placeholders may invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, paragraph 6.
Such claim limitations are:
An irradiated position confirmation support device comprising:
an acquisition unit configured to…
(in claims 1 and 12)
a region analysis unit configured to…
(in claim 1)
a relation analysis unit configured to…
(in claims 1, 9, and 11-12)
and a generation unit configured to…
(in claims 1, 2, and 7-10)
[0018] FIG. 1 is a block diagram illustrating a functional configuration example of an irradiated position confirmation support system. FIG. 11 is a block diagram illustrating a hardware structure example of an irradiated position confirmation support system.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 11 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 11 recites the limitation “wherein the relation analysis unit calculates second change information indicating a change related to the same tissue in the second medical image and the third medical image by registering the second medical image with the second medical image”. For the sake of compact prosecution, the Examiner has mapped the claim language as “wherein the relation analysis unit calculates second change information indicating a change related to the same tissue in the second medical image and the third medical image by registering the second medical image with the third medical image” based on the context provided in the remainder of the claims and the specification.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because claim 14 is drawn to “An irradiated position confirmation support program for causing a processor to execute”.
Per the MPEP 2106.03 Eligibility Step 1: The Four Categories of Statutory Subject Matter [R-07.2022], non-limiting examples of claims that are not directed to any of the statutory categories include:
Products that do not have a physical or tangible form, such as information (often referred to as "data per se”) or a computer program per se (often referred to as "software per se") when claimed as a product without any structural recitations; and
Transitory forms of signal transmission (often referred to as "signals per se"), such as a propagating electrical or electromagnetic signal or carrier wave; and
Subject matter that the statute expressly prohibits from being patented, such as humans per se, which are excluded under The Leahy-Smith America Invents Act (AIA ), Public Law 112-29, sec. 33, 125 Stat.284 (September 16, 2011).
Therefore, since claim 14 recites a computer program per se, it does not fall within a statutory category. Claim 14 is not eligible subject matter under 35 USC § 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5, and 11-14 are rejected under 35 U.S.C. 103 as being unpatentable over He (US20250064523A1) in view of Zhang (Zhang, Xiaohui, et al. "A markerless automatic deformable registration framework for augmented reality navigation of laparoscopy partial nephrectomy." International journal of computer assisted radiology and surgery 14.8 (2019): 1285-1294).
Regarding claim 1, He teaches an irradiated position confirmation support device comprising ([0009] One of the embodiments of the present disclosure provides a medical image processing device for an interventional procedure, comprising a processor. [0048] Under real-time CT scanning, the physician master-slave controls a robot for puncture, thus the efficiency and accuracy of puncture are greatly improved, and the radiation irradiation dose on patients is reduced. However, due to the limitations of radiation dose, imaging time, etc., the range of real-time CT scanning is small. [0099] The spatial position of elements in the registration result (e.g., the target organ, the lesion, the blood vessels within the target organ, the non-interventional region, all the vital organs) provides a comprehensive and accurate reflection of the current status of the target object (e.g., the patient)):
an acquisition unit configured to acquire a first medical image acquired by imaging an inside of a body of a subject, a second medical image acquired by imaging the inside of the body of the subject after imaging the first medical image, and a third medical image acquired by imaging the inside of the body of the subject after imaging the second medical image ([0008] When executing the operating instructions, the at least one processor is directed to cause the system to perform operations including obtaining a first medical image, a second medical image, and a third medical image of a target object, respectively, at different times. [0056] For example, the processing device 140 may obtain image data from the medical scanning device 110 via the network 120. [0054] The scanning object may be biological or non-biological. Merely by way of example, the scanning object may include patients, man-made objects (e.g., man-made phantoms), or the like. As another example, the scanning object may include specific parts, organs, and/or tissues of the patient);
a region analysis unit configured to analyze a region included in the first medical image acquired by the acquisition unit ([0157] In some embodiments, segmentation of a target organ of the intraoperative scanning image by a processing device may be implemented in the following manner. [0236] For example, in FIGS. 16 and 17, the preoperative enhanced image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-left drawing) covered by the organ contour A of the target organ, and the intraoperative scanning image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-right drawing) covered by the organ contour B of the target organ);
a relation analysis unit configured to calculate first change information indicating a change related to the same tissue in the first medical image and the second medical image by registering the first medical image with the second medical image acquired by the acquisition unit, and to output a deformed region analysis result by deforming a region analysis result acquired by the region analysis unit based on the first change information ([0093] Secondly, the processing device may perform a first registration on the first medical image and the second medical image to obtain first deformation information. The first deformation information refers to information about a morphological change of an image element (e.g., a pixel or voxel) in the second medical image relative to a corresponding image element in the first medical image. [0094] Finally, the processing device may actuate the first deformation information on the interventional procedure planning information image to obtain the registration result. [0085] The registration result refers to an image obtained after registering the second medical image and the first medical image. In some embodiments, the registration result may also be referred to as a fourth medical image);
and a generation unit configured to generate an image by mapping the deformed region analysis result acquired by the relation analysis unit on the third medical image acquired by the acquisition unit ([0008] registering the first medical image and the second medical image to obtain a fourth medical image, the fourth medical image including registered interventional procedure planning information; and mapping the fourth medical image to the third medical image to guide the interventional procedure. [0117] In some embodiments, the processing device may map the fourth medical image to the third medical image to guide the interventional procedure).
He does not explicitly teach generate a superimposed image by superimposing the deformed region analysis result acquired by the relation analysis unit on the third medical image.
Zhang, in the same field of endeavor of medical image analysis, teaches generate a superimposed image by superimposing the deformed region analysis result acquired by the relation analysis unit on the third medical image ([Abstract] A coarse-to-fine deformable registration is performed to achieve a precise automatic registration between the intraoperative point cloud and the preoperative model using the iterative closest point algorithm followed by the coherent point drift algorithm. [Methods Overview] Figure 1 shows an overview of our proposed registration framework. After accomplishing the reconstruction of preoperative 3D model and the intraoperative 3D stitched point cloud, the ICP registration is carried out to obtain a coarse registration between the preoperative model point cloud and the current view point cloud patch. With the coarse registration result, the CPD algorithm is used to achieve a fine registration between the model point cloud and the current view reconstructed point cloud. Finally, model overlay is applied to realize the VAT-AR navigation.
PNG
media_image1.png
441
934
media_image1.png
Greyscale
[Methods Preprocessing of the preoperative data] 3D Slicer is used to segment the kidney of the patient from the CT\MRI dataset and reconstruct the mesh model (triangle mesh). According to the surgical view, the model surface that is directly exposed to the laparoscope is extracted for the image registration, denoted by M. For all the models used for registration, we use their vertex information, i.e., point cloud. [Feasibility evaluation: in vivo] We validated the feasibility of our framework by processing the clinical surgical video).
PNG
media_image2.png
271
1003
media_image2.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Zhang to generate a superimposed image by superimposing the deformed region analysis result on the third medical image because "Due to the intraoperative deformation of the kidney, especially in complex surgical scenarios (such as subcutaneous tumors or tumor close to the blood vessels), surgeons generally need to imagine the localization of the tumor through viewing the preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images. It increases the risk of intraoperative bleeding (turn to open surgery), inaccurate tumor margin localization and damage to healthy tissue. Video see-through augmented reality (VST-AR) surgical navigation has been proposed to overcome the above-mentioned difficulties…It can enhance intraoperative perception by visualizing surgical targets and critical structures in the form of a video see-through overlay" [Introduction].
Regarding claim 2, He and Zhang teach the device of claim 1. He further teaches wherein the region analysis result and the deformed region analysis result include information specifying a shape of the region ([0235] In 4611, a first preliminary deformation field may be determined based on a registration between elements. [0236] In some embodiments, the elements may be element contours (e.g., organ contours, vessel contours, lesion contours) of the first medical image and the second medical image. The registration between elements may refer to the registration between image regions covered by the element contours (masks). For example, in FIGS. 16 and 17, the preoperative enhanced image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-left drawing) covered by the organ contour A of the target organ, and the intraoperative scanning image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-right drawing) covered by the organ contour B of the target organ).
He does not explicitly teach the generation unit generates the superimposed image by superimposing the information specifying the shape of the region on the third medical image.
Zhang teaches the generation unit generates the superimposed image by superimposing the information specifying the shape of the region on the third medical image
PNG
media_image1.png
441
934
media_image1.png
Greyscale
([Evaluation and Results] The proposed framework was implemented using MATLAB 2017 with the help of the open-source library OpenCV and TensorFlow. A computer workstation with a multicore Intel Xeon CPU (E5-2609 v3, 1.90 GHz, 16 GB RAM) and an Nvidia GTX 1080Ti GPU was used for the evaluation).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Zhang to superimpose information specifying the shape of the region because "Due to the intraoperative deformation of the kidney, especially in complex surgical scenarios (such as subcutaneous tumors or tumor close to the blood vessels), surgeons generally need to imagine the localization of the tumor through viewing the preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images. It increases the risk of intraoperative bleeding (turn to open surgery), inaccurate tumor margin localization and damage to healthy tissue. Video see-through augmented reality (VST-AR) surgical navigation has been proposed to overcome the above-mentioned difficulties…It can enhance intraoperative perception by visualizing surgical targets and critical structures in the form of a video see-through overlay" [Introduction].
Regarding claim 3, He and Zhang teach the method of claim 2. He further teaches wherein the information specifying the shape of the region is a contour of the region ([0236] In some embodiments, the elements may be element contours (e.g., organ contours, vessel contours, lesion contours) of the first medical image and the second medical image. The registration between elements may refer to the registration between image regions covered by the element contours (masks). For example, in FIGS. 16 and 17, the preoperative enhanced image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-left drawing) covered by the organ contour A of the target organ, and the intraoperative scanning image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-right drawing) covered by the organ contour B of the target organ).
Regarding claim 4, He and Zhang teach the method of claim 2. He does not explicitly teach wherein the information specifying the shape of the region is an image in which the region is made translucent. Zhang teaches wherein the information specifying the shape of the region is an image in which the region is made translucent.
PNG
media_image2.png
271
1003
media_image2.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Zhang to have the region made translucent because "Due to the intraoperative deformation of the kidney, especially in complex surgical scenarios (such as subcutaneous tumors or tumor close to the blood vessels), surgeons generally need to imagine the localization of the tumor through viewing the preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images. It increases the risk of intraoperative bleeding (turn to open surgery), inaccurate tumor margin localization and damage to healthy tissue. Video see-through augmented reality (VST-AR) surgical navigation has been proposed to overcome the above-mentioned difficulties…It can enhance intraoperative perception by visualizing surgical targets and critical structures in the form of a video see-through overlay" [Introduction].
Regarding claim 5, He and Zhang teach the device of claim 1. He further teaches wherein the first change information is information indicating displacement of the tissue ([0093] Secondly, the processing device may perform a first registration on the first medical image and the second medical image to obtain first deformation information. The first deformation information refers to information about a morphological change of an image element (e.g., a pixel or voxel) in the second medical image relative to a corresponding image element in the first medical image. For example, geometric change information, projection change information, or the like. The first deformation information may be represented by a first deformation matrix. Exemplarily, the first deformation matrix may include a deformation matrix in the x-direction, a deformation matrix in the y-direction, and a deformation matrix in the z-direction. An element in each deformation matrix corresponds to a unit region (e.g., 1 pixel point, a 1 mm×1 mm image region, 1 voxel point, a 1 mm×1 mm×1 mm image region, etc.) of the second medical image, and the value of the element is deformation information of the unit region in the x-axis direction, y-axis direction, or z-axis direction).
Regarding claim 11, He and Zhang teach the device of claim 1. He further teaches wherein the relation analysis unit calculates second change information indicating a change related to the same tissue in the second medical image and the third medical image by registering the second medical image with the third medical image, and outputs the deformed region analysis result by deforming the deformed region analysis result based on the second change information ([0118] In some embodiments, if the breathing of the target object is not monitored by a respiratory gating device, the second medical image and the third medical image may be obtained when the target object is at different respiratory amplitude points, the organs and/or tissues in the images may move, and the processing device may perform a second registration on the second medical image and the third medical image. As shown in FIG. 25, in some embodiments, second deformation information may be obtained by performing the second registration on the second medical image and the third medical image. [0119] The second deformation information refers to morphology change information of an image element in the third medical image relative to a corresponding image element in the second medical image. For example, geometric change information, projection change information, etc. The second deformation information may be represented by a second deformation matrix. Exemplarily, the second deformation matrix may include a deformation matrix in the x-direction, a deformation matrix in the y-direction, and a deformation matrix in the z-direction. An element in each deformation matrix corresponds to a unit region (e.g., 1 pixel point, a 1 mmxl mm image region, 1 voxel point, a 1 mm×1 mm×1 mm image region, etc.) of the third medical image, and the value of the element is deformation information of the unit region in the x-axis direction, y-axis direction, or z-axis direction. [0120] In some embodiments, the processing device may apply the second deformation information to the registration result (e.g., the fourth medical image) to obtain a fifth medical image).
Regarding claim 12, He and Zhang teach the device of claim 1. He further teaches wherein the acquisition unit acquires a plurality of time- series first medical images, periodic first change information in the body of the subject at the time of capturing the first medical images, a plurality of time- series second medical images, periodic first change information in the body of the subject at the time of capturing the first medical images, and periodic second change information in the body of the subject at the time of capturing the second medical images, and the relation analysis unit registers the first medical images and the second medical images at a timing at which the first change information and the second change information are synchronized ([0072] In some embodiments, the processing device may cause the first medical image and the second medical image to be obtained when the target object is at the same, or nearly the same respiratory amplitude point via a respiratory gating device. For example, as shown in FIG. 24, the respiratory gating device may obtain a respiratory amplitude point A where the target object is located when obtaining the first medical image. During the interventional procedure and before the puncture, the respiratory gating device may monitor the breathing of the target object and cause the medical scanning device to obtain the second medical image when the target object is at a respiratory amplitude point A′. In some embodiments, the respiratory amplitude of the target object is monitored by the respiratory gating device during the interventional procedure, and a third medical image may also be obtained using the medical scanning device when the target object adjusts his or her breathing to the respiratory amplitude point A″. [0083] In 220, the second medical image and the first medical image may be registered to obtain a registration result).
Regarding claim 13, He teaches an irradiated position confirmation support method to be executed by an irradiated position confirmation support device including a processor configured to execute a program and a storage device storing the program, the irradiated position confirmation support method comprising the processor executing ([0002] The present disclosure relates to the field of image processing technology, and in particular, to medical image processing methods, systems, and devices for interventional procedures, and computer storage media thereof. [0009] One of the embodiments of the present disclosure provides a medical image processing device for an interventional procedure, comprising a processor. [0048] Under real-time CT scanning, the physician master-slave controls a robot for puncture, thus the efficiency and accuracy of puncture are greatly improved, and the radiation irradiation dose on patients is reduced. However, due to the limitations of radiation dose, imaging time, etc., the range of real-time CT scanning is small. [0099] The spatial position of elements in the registration result (e.g., the target organ, the lesion, the blood vessels within the target organ, the non-interventional region, all the vital organs) provides a comprehensive and accurate reflection of the current status of the target object (e.g., the patient)):
an acquisition processing of acquiring a first medical image acquired by imaging an inside of a body of a subject, a second medical image acquired by imaging the inside of the body of the subject after imaging the first medical image, and a third medical image acquired by imaging the inside of the body of the subject after imaging the second medical image ([0008] When executing the operating instructions, the at least one processor is directed to cause the system to perform operations including obtaining a first medical image, a second medical image, and a third medical image of a target object, respectively, at different times. [0056] For example, the processing device 140 may obtain image data from the medical scanning device 110 via the network 120. [0054] The scanning object may be biological or non-biological. Merely by way of example, the scanning object may include patients, man-made objects (e.g., man-made phantoms), or the like. As another example, the scanning object may include specific parts, organs, and/or tissues of the patient);
a region analysis processing of analyzing a region included in the first medical image acquired by the acquisition processing ([0157] In some embodiments, segmentation of a target organ of the intraoperative scanning image by a processing device may be implemented in the following manner. [0236] For example, in FIGS. 16 and 17, the preoperative enhanced image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-left drawing) covered by the organ contour A of the target organ, and the intraoperative scanning image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-right drawing) covered by the organ contour B of the target organ);
a relation analysis processing of calculating first change information indicating a change related to the same tissue in the first medical image and the second medical image by registering the first medical image with the second medical image acquired by the acquisition processing, and outputting a deformed region analysis result by deforming a region analysis result acquired by the region analysis processing based on the first change information ([0093] Secondly, the processing device may perform a first registration on the first medical image and the second medical image to obtain first deformation information. The first deformation information refers to information about a morphological change of an image element (e.g., a pixel or voxel) in the second medical image relative to a corresponding image element in the first medical image. [0094] Finally, the processing device may actuate the first deformation information on the interventional procedure planning information image to obtain the registration result. [0085] The registration result refers to an image obtained after registering the second medical image and the first medical image. In some embodiments, the registration result may also be referred to as a fourth medical image);
and a generation processing of generating an image by mapping the deformed region analysis result acquired by the relation analysis processing on the third medical image acquired by the acquisition processing ([0008] registering the first medical image and the second medical image to obtain a fourth medical image, the fourth medical image including registered interventional procedure planning information; and mapping the fourth medical image to the third medical image to guide the interventional procedure. [0117] In some embodiments, the processing device may map the fourth medical image to the third medical image to guide the interventional procedure).
He does not explicitly teach generating a superimposed image by superimposing the deformed region analysis result acquired by the relation analysis processing on the third medical image.
Zhang, in the same field of endeavor of medical image analysis, teaches generating a superimposed image by superimposing the deformed region analysis result acquired by the relation analysis processing on the third medical image ([Abstract] A coarse-to-fine deformable registration is performed to achieve a precise automatic registration between the intraoperative point cloud and the preoperative model using the iterative closest point algorithm followed by the coherent point drift algorithm. [Methods Overview] Figure 1 shows an overview of our proposed registration framework. After accomplishing the reconstruction of preoperative 3D model and the intraoperative 3D stitched point cloud, the ICP registration is carried out to obtain a coarse registration between the preoperative model point cloud and the current view point cloud patch. With the coarse registration result, the CPD algorithm is used to achieve a fine registration between the model point cloud and the current view reconstructed point cloud. Finally, model overlay is applied to realize the VAT-AR navigation.
PNG
media_image1.png
441
934
media_image1.png
Greyscale
[Methods Preprocessing of the preoperative data] 3D Slicer is used to segment the kidney of the patient from the CT\MRI dataset and reconstruct the mesh model (triangle mesh). According to the surgical view, the model surface that is directly exposed to the laparoscope is extracted for the image registration, denoted by M. For all the models used for registration, we use their vertex information, i.e., point cloud. [Feasibility evaluation: in vivo] We validated the feasibility of our framework by processing the clinical surgical video).
PNG
media_image2.png
271
1003
media_image2.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of He with the teachings of Zhang to generate a superimposed image by superimposing the deformed region analysis result on the third medical image because "Due to the intraoperative deformation of the kidney, especially in complex surgical scenarios (such as subcutaneous tumors or tumor close to the blood vessels), surgeons generally need to imagine the localization of the tumor through viewing the preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images. It increases the risk of intraoperative bleeding (turn to open surgery), inaccurate tumor margin localization and damage to healthy tissue. Video see-through augmented reality (VST-AR) surgical navigation has been proposed to overcome the above-mentioned difficulties…It can enhance intraoperative perception by visualizing surgical targets and critical structures in the form of a video see-through overlay" [Introduction].
Regarding claim 14, He teaches an irradiated position confirmation support program for causing a processor to execute ([0010] One of the embodiments of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions. When a computer reads the computer instructions in the storage medium, the computer performs the method as described in any of the embodiments.[0048] Under real-time CT scanning, the physician master-slave controls a robot for puncture, thus the efficiency and accuracy of puncture are greatly improved, and the radiation irradiation dose on patients is reduced. However, due to the limitations of radiation dose, imaging time, etc., the range of real-time CT scanning is small. [0099] The spatial position of elements in the registration result (e.g., the target organ, the lesion, the blood vessels within the target organ, the non-interventional region, all the vital organs) provides a comprehensive and accurate reflection of the current status of the target object (e.g., the patient)):
an acquisition processing of acquiring a first medical image acquired by imaging an inside of a body of a subject, a second medical image acquired by imaging the inside of the body of the subject after imaging the first medical image, and a third medical image acquired by imaging the inside of the body of the subject after imaging the second medical image ([0008] When executing the operating instructions, the at least one processor is directed to cause the system to perform operations including obtaining a first medical image, a second medical image, and a third medical image of a target object, respectively, at different times. [0056] For example, the processing device 140 may obtain image data from the medical scanning device 110 via the network 120. [0054] The scanning object may be biological or non-biological. Merely by way of example, the scanning object may include patients, man-made objects (e.g., man-made phantoms), or the like. As another example, the scanning object may include specific parts, organs, and/or tissues of the patient);
a region analysis processing of analyzing a region included in the first medical image acquired by the acquisition processing ([0157] In some embodiments, segmentation of a target organ of the intraoperative scanning image by a processing device may be implemented in the following manner. [0236] For example, in FIGS. 16 and 17, the preoperative enhanced image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-left drawing) covered by the organ contour A of the target organ, and the intraoperative scanning image is segmented to obtain the image region (the region with the same or essentially the same grayscale within the dashed line region in the lower-right drawing) covered by the organ contour B of the target organ);
a relation analysis processing of calculating first change information indicating a change related to the same tissue in the first medical image and the second medical image by registering the first medical image with the second medical image acquired by the acquisition processing, and outputting a deformed region analysis result by deforming a region analysis result acquired by the region analysis processing based on the first change information ([0093] Secondly, the processing device may perform a first registration on the first medical image and the second medical image to obtain first deformation information. The first deformation information refers to information about a morphological change of an image element (e.g., a pixel or voxel) in the second medical image relative to a corresponding image element in the first medical image. [0094] Finally, the processing device may actuate the first deformation information on the interventional procedure planning information image to obtain the registration result. [0085] The registration result refers to an image obtained after registering the second medical image and the first medical image. In some embodiments, the registration result may also be referred to as a fourth medical image);
and a generation processing of generating an image by mapping the deformed region analysis result acquired by the relation analysis processing on the third medical image acquired by the acquisition processing ([0008] registering the first medical image and the second medical image to obtain a fourth medical image, the fourth medical image including registered interventional procedure planning information; and mapping the fourth medical image to the third medical image to guide the interventional procedure. [0117] In some embodiments, the processing device may map the fourth medical image to the third medical image to guide the interventional procedure).
He does not explicitly teach generating a superimposed image by superimposing the deformed region analysis result acquired by the relation analysis processing on the third medical image.
Zhang, in the same field of endeavor of medical image analysis, teaches generating a superimposed image by superimposing the deformed region analysis result acquired by the relation analysis processing on the third medical image ([Abstract] A coarse-to-fine deformable registration is performed to achieve a precise automatic registration between the intraoperative point cloud and the preoperative model using the iterative closest point algorithm followed by the coherent point drift algorithm. [Methods Overview] Figure 1 shows an overview of our proposed registration framework. After accomplishing the reconstruction of preoperative 3D model and the intraoperative 3D stitched point cloud, the ICP registration is carried out to obtain a coarse registration between the preoperative model point cloud and the current view point cloud patch. With the coarse registration result, the CPD algorithm is used to achieve a fine registration between the model point cloud and the current view reconstructed point cloud. Finally, model overlay is applied to realize the VAT-AR navigation.
PNG
media_image1.png
441
934
media_image1.png
Greyscale
[Methods Preprocessing of the preoperative data] 3D Slicer is used to segment the kidney of the patient from the CT\MRI dataset and reconstruct the mesh model (triangle mesh). According to the surgical view, the model surface that is directly exposed to the laparoscope is extracted for the image registration, denoted by M. For all the models used for registration, we use their vertex information, i.e., point cloud. [Feasibility evaluation: in vivo] We validated the feasibility of our framework by processing the clinical surgical video).
PNG
media_image2.png
271
1003
media_image2.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the medium of He with the teachings of Zhang to generate a superimposed image by superimposing the deformed region analysis result on the third medical image because "Due to the intraoperative deformation of the kidney, especially in complex surgical scenarios (such as subcutaneous tumors or tumor close to the blood vessels), surgeons generally need to imagine the localization of the tumor through viewing the preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images. It increases the risk of intraoperative bleeding (turn to open surgery), inaccurate tumor margin localization and damage to healthy tissue. Video see-through augmented reality (VST-AR) surgical navigation has been proposed to overcome the above-mentioned difficulties…It can enhance intraoperative perception by visualizing surgical targets and critical structures in the form of a video see-through overlay" [Introduction].
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over He in view of Zhang and Hendriks (US20230102865A1).
Regarding claim 6, He and Zhang teach the device of claim 1. He does not explicitly teach wherein the first change information is information indicating a change in size of the tissue.
Hendriks, in the same field of endeavor of medical image analysis, teaches wherein the first change information is information indicating a change in size of the tissue ([0006] In deformable image registration (DIR), two or more images are geometrically mapped onto each other with the use of a deformation model. It can be applied to find corresponding voxels or regions in two or more medical images (for example, an inhale and an exhale CT of the lung), or to construct a deformation map by showing the relative volume changes of the corresponding tissue elements (i.e., the volumetric strain)).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Hendriks for the first change information to indicate a change in size of the tissue because "DIR can provide a mapping of the regional deformations of the structures and tissues in the lung, which provides useful diagnostic information for the prevention of VILI in critical-care patients. For example, estimates of (volumetric) strain have been correlated with lung inflammation and injury in mechanically ventilated lungs" [0006].
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over He in view of Zhang and Murase (US20250009329A1).
Regarding claim 7, He and Zhang teach the device of claim 1. He does not explicitly teach wherein the region analysis result and the deformed region analysis result include a character string indicating a name of the region, and the generation unit generates the superimposed image by superimposing the character string on the third medical image.
Murase, in the same field of endeavor of medical image analysis, teaches wherein the region analysis result and the deformed region analysis result include a character string indicating a name of the region, and the generation unit generates the superimposed image by superimposing the character string on the third medical image ([0020] based on a correspondence relationship between a plurality of types of the sites and a lesion corresponding to each of the sites, determine whether a combination of the first image region and the second image region is correct for each of the frames; and based on the display mode corresponding to one of the frames used as a determination target if it is determined that the combination of the first image region and the second image region is correct, correct the display mode corresponding to one of the frames used as a determination target if it is determined that the combination of the first image region and the second image region is not correct. [0164] In this case, the control unit 104E acquires the site name information 118B from the site region information 118, and displays the information indicating the name of the site identified from the site name information 118B so as to be superimposed on the site region 116A on the first screen 22. In the example illustrated in FIG. 16, characters “pancreas” are displayed so as to be superimposed on the site region 116A as the information indicating the name of the site).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Murase to generate a superimposed image by superimposing the character string on the image because "by displaying the information indicating the name of the site in association with the site region 116A, a user or the like can grasp the name of the site indicated by the site region 116A displayed on the first screen 22" [0164].
Regarding claim 8, He, Zhang, and Murase teach the device of claim 7. He does not explicitly teach wherein the generation unit generates the superimposed image by superimposing the character string on a position of the region indicated by the character string on the third medical image.
Murase, in the same field of endeavor of medical image analysis, teaches wherein the generation unit generates the superimposed image by superimposing the character string on a position of the region indicated by the character string on the third medical image ([0164] In this case, the control unit 104E acquires the site name information 118B from the site region information 118, and displays the information indicating the name of the site identified from the site name information 118B so as to be superimposed on the site region 116A on the first screen 22. In the example illustrated in FIG. 16, characters “pancreas” are displayed so as to be superimposed on the site region 116A as the information indicating the name of the site).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Murase to superimpose the character string on the position of the region because "by displaying the information indicating the name of the site in association with the site region 116A, a user or the like can grasp the name of the site indicated by the site region 116A displayed on the first screen 22" [0164].
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over He in view of Zhang and Li (US20220284606A1).
Regarding claim 9, He and Zhang teach the method of claim 1. He does not explicitly teach wherein the relation analysis unit calculates, based on the first change information, an estimated likelihood indicating a likelihood of a change in the region caused by the registration between the first medical image and the second medical image, and the generation unit generates the superimposed image by superimposing the estimated likelihood on the third medical image.
Li, in the same field of endeavor of medical image registration, teaches wherein the relation analysis unit calculates, based on the first change information, an estimated likelihood indicating a likelihood of a change in the region caused by the registration between the first medical image and the second medical image, and the generation unit generates the superimposed image by superimposing the estimated likelihood on the third medical image ([0142] A variety of methods may be used to estimate the accuracy of the co-registration. For example, the error between predicted locations of detected anatomical landmark and the corresponding visualized anatomical landmarks can be measured, either individually, in groups, or as a whole over the entire co-registration. The magnitude these measurements may indicate an estimated level of accuracy of parts or all of the co-registration. For example, if the magnitude of measured error is large and/or exceeds certain predetermined thresholds, the predicted level of accuracy of the co-registration may be low. [0143] Once an estimated level of accuracy is determined, the estimated level of accuracy for all, or portions of co-registration be displayed. For example, the system may include software or hardware that is configured to generating a visual indicator representing the estimated accuracy of the imaging co-registration. In some embodiments, the visual indicator may be displayed and/or overlaid on all or portions of the illustrated blood vessel on the extravascular imaging data. The visual characteristic may include color, symbol, intensity, or the like, and may be overlaid/superimposed on or associated with segments of the vessel shown).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Li to calculate a likelihood of change in the region caused by the registration and superimpose the estimated likelihood on the image because "there may be some misalignment/error/discrepancy between the predicted and marked location(s) of detected anatomical landmark(s) on the extravascular imaging data, and the corresponding visualized anatomical landmark on the extravascular imaging data. As disclosed herein, one aspect of the method for vascular imaging co-registration may include aligning the predicted location of the detected anatomical landmark with the visualized anatomical landmark, and this may help alleviate some of this misalignment/error/discrepancy in the co-registration. However, the fact that this misalignment/error/discrepancy occurred in the first place (e.g. prior to aligning the predicted location of the detected anatomical landmark with the visualized anatomical landmark) may suggest a desire, and in some cases may provide a mechanism, to estimate the accuracy of the image co-registration. The system may include software or hardware that is configured to estimate the accuracy of the co-registration, and in some cases, display and/or otherwise indicate an estimated level of accuracy for portions of or all of the co-registration" [0141].
Regarding claim 10, He, Zhang, and Li teach the method of claim 9. He does not explicitly teach wherein the generation unit generates the superimposed image by superimposing the estimated likelihood on a position of the region corresponding to the estimated likelihood on the third medical image.
Li, in the same field of endeavor of medical image registration, teaches wherein the generation unit generates the superimposed image by superimposing the estimated likelihood on a position of the region corresponding to the estimated likelihood on the third medical image ([0143] Once an estimated level of accuracy is determined, the estimated level of accuracy for all, or portions of co-registration be displayed. For example, the system may include software or hardware that is configured to generating a visual indicator representing the estimated accuracy of the imaging co-registration. In some embodiments, the visual indicator may be displayed and/or overlaid on all or portions of the illustrated blood vessel on the extravascular imaging data. The visual characteristic may include color, symbol, intensity, or the like, and may be overlaid/superimposed on or associated with segments of the vessel shown).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of He with the teachings of Li to superimpose the estimated likelihood on the image because "there may be some misalignment/error/discrepancy between the predicted and marked location(s) of detected anatomical landmark(s) on the extravascular imaging data, and the corresponding visualized anatomical landmark on the extravascular imaging data. As disclosed herein, one aspect of the method for vascular imaging co-registration may include aligning the predicted location of the detected anatomical landmark with the visualized anatomical landmark, and this may help alleviate some of this misalignment/error/discrepancy in the co-registration. However, the fact that this misalignment/error/discrepancy occurred in the first place (e.g. prior to aligning the predicted location of the detected anatomical landmark with the visualized anatomical landmark) may suggest a desire, and in some cases may provide a mechanism, to estimate the accuracy of the image co-registration. The system may include software or hardware that is configured to estimate the accuracy of the co-registration, and in some cases, display and/or otherwise indicate an estimated level of accuracy for portions of or all of the co-registration" [0141].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dorman (US20240197411A1) teaches medical image registration and superimposition.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jacqueline R Zak whose telephone number is (571)272-4077. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACQUELINE R ZAK/Examiner, Art Unit 2666
/Molly Wilburn/Primary Examiner, Art Unit 2666
Prosecution Insights