DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is responsive to Applicant’s remarks received on December 18, 2025. Claims 1, 2 and 4-21 are pending.
Claim Interpretation
The previous 112(f) interpretation has been withdrawn in light of Applicant’s amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 4-6, 11-16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Matsuzaki et al. (JP2018068982, utilizing a machine translation) and Fei et al. (US 2012/0039519).
Regarding claim 1, Matsuzaki et al. discloses a method of generating a modified X-ray image, the method comprising:
obtaining an X-ray image of a subject (“Then, the image acquisition unit 111 acquires the two captured X-ray images (S101)” at paragraph 0026, line 1);
obtaining a CT image of the subject corresponding to the X-ray image (“Then, around the time when the two captured X-ray images are acquired in step S101, the image acquisition unit 111 acquires a three-dimensional image that has been captured in advance from the medical image server 2 (S102)” at paragraph 0026, line 2; “Here, as an example, a case where a CT image is used as a three-dimensional image will be described” at paragraph 0027, line 4);
determining a mapping between the X-ray image and the CT image (“Then, the calculated projection image generating unit 112 generates a two-dimensional calculated projection image from the acquired multiple three-dimensional images (CT images) based on the positions of the X-ray source 201 and the detector 202” at paragraph 0028, line 1; “Next, the image registration unit 113 performs registration between the calculated projection image and the captured X-ray image (S112)” at paragraph 0032, line 2);
identifying a structure of interest in the CT image (“Next, the rigid region attenuator 114 extracts a bone region from the three-dimensional image (S202)” at paragraph 0038, line 5);
generating an attenuation map from the CT image, the attenuation map indicative of attenuation due to the structure of interest or everything but the structure of interest on the X-ray image (“Then, the rigid region attenuation unit 114 generates a calculated projection image corresponding to the same position of the X-ray source 201 as the input captured X-ray image, based on the three-dimensional image from which only the bone region has been extracted. That is, the rigid region attenuation unit 114 generates a bone region calculated projection image, which is a calculated projection image in which only the bone region is projected (S203)” at paragraph 0039, line 1); and
generating the modified X-ray image by subtracting the attenuation map from the X-ray image (“Next, the rigid region attenuation unit 114 generates a bone region attenuated X-ray image by subtracting the bone region calculated projection image from the captured X-ray image (captured X-ray image - bone region calculated projection image) (S204)” at paragraph 0040, line 1),
wherein determining a mapping between the X-ray image and the CT image includes generating a plurality of simulated X-ray images from the CT image and using parameters corresponding to the simulated X-ray image among the plurality of X-ray images most closely matching the X-ray image (“In step S112, the image registration unit 113 moves and rotates the calculated projection image 501a in FIG. 4, for example, and calculates the similarity between the calculated projection image 502a and the captured X-ray image 501a. As the similarity, for example, the mutual information between the captured X-ray image 501a and the calculated projection image 502a is used. Here, the mutual information is the mutual information using pixel values of the captured X-ray image 501a and the calculated projection image 502a. Then, the image positioning unit 113 determines the translation and rotation parameters that maximize the similarity” at paragraph 0033, line 1).
Matsuzaki et al. does not explicitly disclose that each of the plurality of simulated X-ray images is generated using a different simulated focal point and detector position and using parameters corresponding to the simulated X-ray image among the plurality of simulated X-ray images that most closely matches the obtained X-ray image.
Fei et al. teaches a method in the same field of endeavor of x-ray and CT image processing, wherein determining a mapping between the obtained X-ray image and the obtained CT image comprises: (i) generating a plurality of simulated X-ray images from the obtained CT image (“At 64, the CT image volume is projected into an image plane. A digitally reconstructed radiography (DRR) image can be provided using projection methods, such as a Gaussian weighted projection method or an averaged-based projection method” at paragraph 0023, line 1), wherein each of the plurality of simulated X-ray images is generated using a different simulated focal point and detector position, and (ii) using parameters corresponding to a simulated X-ray image among the plurality of simulated X-ray images that most closely matches the obtained X-ray image (“The projection parameters utilized can be the same as for the real dual energy DR image acquisitions. The parameters include the distance between the X-ray tube's focus and the detector plane. After a coordinate system is setup for the projection, the location of the X-ray tube and the principal view axis can be determined. These parameters are used to compute the perspective transformation matrix” at paragraph 0025).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the projection parameters as taught by Fei et al. to generate the projection images of Matsuzaki et al. to optimize the similarity between projected DRR and real x-ray image for further processing.
Regarding claim 13, Matsuzaki et al. discloses a system for generating a modified X-ray image, the system comprising:
a memory structured to store a routine (“In addition to being stored on a hard disk (HD), information such as programs, tables, and files that realize each function can also be stored in a memory, a recording device such as a solid state drive (SSD), or a recording medium” at paragraph 0083, line 6); and
a processor configured to execute the routine stored in the memory (“As shown in FIG. 2, the above-described configurations, functions, etc. may be realized by software, in which a processor such as the CPU 102 interprets and executes a program that realizes each function” at paragraph 0083, line 4), the execution of which causes the processor to:
determine a mapping between an X-ray image and a CT image (“Then, the calculated projection image generating unit 112 generates a two-dimensional calculated projection image from the acquired multiple three-dimensional images (CT images) based on the positions of the X-ray source 201 and the detector 202” at paragraph 0028, line 1; “Next, the image registration unit 113 performs registration between the calculated projection image and the captured X-ray image (S112)” at paragraph 0032, line 2);
identify a structure of interest in the CT image (“Next, the rigid region attenuator 114 extracts a bone region from the three-dimensional image (S202)” at paragraph 0038, line 5);
generate an attenuation map from the CT image, the attenuation map indicative of attenuation due to the structure of interest or everything but the structure of interest on the X-ray image (“Then, the rigid region attenuation unit 114 generates a calculated projection image corresponding to the same position of the X-ray source 201 as the input captured X-ray image, based on the three-dimensional image from which only the bone region has been extracted. That is, the rigid region attenuation unit 114 generates a bone region calculated projection image, which is a calculated projection image in which only the bone region is projected (S203)” at paragraph 0039, line 1); and
generate the modified X-ray image by subtracting the attenuation map from the X-ray image (“Next, the rigid region attenuation unit 114 generates a bone region attenuated X-ray image by subtracting the bone region calculated projection image from the captured X-ray image (captured X-ray image - bone region calculated projection image) (S204)” at paragraph 0040, line 1),
wherein determining a mapping between the X-ray image and the CT image includes generating a plurality of simulated X-ray images from the CT image and using parameters corresponding to the simulated X-ray image among the plurality of X-ray images most closely matching the X-ray image (“In step S112, the image registration unit 113 moves and rotates the calculated projection image 501a in FIG. 4, for example, and calculates the similarity between the calculated projection image 502a and the captured X-ray image 501a. As the similarity, for example, the mutual information between the captured X-ray image 501a and the calculated projection image 502a is used. Here, the mutual information is the mutual information using pixel values of the captured X-ray image 501a and the calculated projection image 502a. Then, the image positioning unit 113 determines the translation and rotation parameters that maximize the similarity” at paragraph 0033, line 1).
Matsuzaki et al. does not explicitly disclose that each of the plurality of simulated X-ray images is generated using a different simulated focal point and detector position and using parameters corresponding to the simulated X-ray image among the plurality of simulated X-ray images that most closely matches the obtained X-ray image.
Fei et al. teaches a system in the same field of endeavor of x-ray and CT image processing, wherein determining a mapping between the obtained X-ray image and the obtained CT image comprises: (i) generating a plurality of simulated X-ray images from the obtained CT image (“At 64, the CT image volume is projected into an image plane. A digitally reconstructed radiography (DRR) image can be provided using projection methods, such as a Gaussian weighted projection method or an averaged-based projection method” at paragraph 0023, line 1), wherein each of the plurality of simulated X-ray images is generated using a different simulated focal point and detector position, and (ii) using parameters corresponding to a simulated X-ray image among the plurality of simulated X-ray images that most closely matches the obtained X-ray image (“The projection parameters utilized can be the same as for the real dual energy DR image acquisitions. The parameters include the distance between the X-ray tube's focus and the detector plane. After a coordinate system is setup for the projection, the location of the X-ray tube and the principal view axis can be determined. These parameters are used to compute the perspective transformation matrix” at paragraph 0025).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the projection parameters as taught by Fei et al. to generate the projection images of Matsuzaki et al. to optimize the similarity between projected DRR and real x-ray image for further processing.
Regarding claim 14, Matsuzaki et al. discloses a non-transitory computer readable medium (“In addition to being stored on a hard disk (HD), information such as programs, tables, and files that realize each function can also be stored in a memory, a recording device such as a solid state drive (SSD), or a recording medium” at paragraph 0083, line 6) storing one or more programs, including instructions, which when executed by a computer (“As shown in FIG. 2, the above-described configurations, functions, etc. may be realized by software, in which a processor such as the CPU 102 interprets and executes a program that realizes each function” at paragraph 0083, line 4), causes the computer to perform a method of generating a modified X-ray image, comprising:
obtaining an X-ray image of a subject (“Then, the image acquisition unit 111 acquires the two captured X-ray images (S101)” at paragraph 0026, line 1);
obtaining a CT image of the subject corresponding to the X-ray image (“Then, around the time when the two captured X-ray images are acquired in step S101, the image acquisition unit 111 acquires a three-dimensional image that has been captured in advance from the medical image server 2 (S102)” at paragraph 0026, line 2; “Here, as an example, a case where a CT image is used as a three-dimensional image will be described” at paragraph 0027, line 4);
determining a mapping between the X-ray image and the CT image (“Then, the calculated projection image generating unit 112 generates a two-dimensional calculated projection image from the acquired multiple three-dimensional images (CT images) based on the positions of the X-ray source 201 and the detector 202” at paragraph 0028, line 1; “Next, the image registration unit 113 performs registration between the calculated projection image and the captured X-ray image (S112)” at paragraph 0032, line 2);
identifying a structure of interest in the CT image (“Next, the rigid region attenuator 114 extracts a bone region from the three-dimensional image (S202)” at paragraph 0038, line 5);
generating an attenuation map from the CT image, the attenuation map indicative of attenuation due to the structure of interest or everything but the structure of interest on the X-ray image (“Then, the rigid region attenuation unit 114 generates a calculated projection image corresponding to the same position of the X-ray source 201 as the input captured X-ray image, based on the three-dimensional image from which only the bone region has been extracted. That is, the rigid region attenuation unit 114 generates a bone region calculated projection image, which is a calculated projection image in which only the bone region is projected (S203)” at paragraph 0039, line 1); and
generating the modified X-ray image by subtracting the attenuation map from the X-ray image (“Next, the rigid region attenuation unit 114 generates a bone region attenuated X-ray image by subtracting the bone region calculated projection image from the captured X-ray image (captured X-ray image - bone region calculated projection image) (S204)” at paragraph 0040, line 1),
wherein determining a mapping between the X-ray image and the CT image includes generating a plurality of simulated X-ray images from the CT image and using parameters corresponding to the simulated X-ray image among the plurality of X-ray images most closely matching the X-ray image (“In step S112, the image registration unit 113 moves and rotates the calculated projection image 501a in FIG. 4, for example, and calculates the similarity between the calculated projection image 502a and the captured X-ray image 501a. As the similarity, for example, the mutual information between the captured X-ray image 501a and the calculated projection image 502a is used. Here, the mutual information is the mutual information using pixel values of the captured X-ray image 501a and the calculated projection image 502a. Then, the image positioning unit 113 determines the translation and rotation parameters that maximize the similarity” at paragraph 0033, line 1).
Matsuzaki et al. does not explicitly disclose that each of the plurality of simulated X-ray images is generated using a different simulated focal point and detector position and using parameters corresponding to the simulated X-ray image among the plurality of simulated X-ray images that most closely matches the obtained X-ray image.
Fei et al. teaches a non-transitory computer readable medium in the same field of endeavor of x-ray and CT image processing, wherein determining a mapping between the obtained X-ray image and the obtained CT image comprises: (i) generating a plurality of simulated X-ray images from the obtained CT image (“At 64, the CT image volume is projected into an image plane. A digitally reconstructed radiography (DRR) image can be provided using projection methods, such as a Gaussian weighted projection method or an averaged-based projection method” at paragraph 0023, line 1), wherein each of the plurality of simulated X-ray images is generated using a different simulated focal point and detector position, and (ii) using parameters corresponding to a simulated X-ray image among the plurality of simulated X-ray images that most closely matches the obtained X-ray image (“The projection parameters utilized can be the same as for the real dual energy DR image acquisitions. The parameters include the distance between the X-ray tube's focus and the detector plane. After a coordinate system is setup for the projection, the location of the X-ray tube and the principal view axis can be determined. These parameters are used to compute the perspective transformation matrix” at paragraph 0025).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the projection parameters as taught by Fei et al. to generate the projection images of Matsuzaki et al. to optimize the similarity between projected DRR and real x-ray image for further processing.
Regarding claim 2, Matsuzaki et al. discloses a method wherein determining a mapping between the X-ray image and the CT image is based at least on a position of an X-ray source and a detector used to obtain the X-ray image (“For example, the calculated projection image generating unit 112 generates the calculated projection image 502 by calculating a numerical value obtained by adding pixel values of a CT image, which is the three-dimensional image 503, in the ray direction using the ray tracing method described above. Furthermore, the calculated projection image generating unit 112 generates a calculated projection image 502 for each position of the X-ray source 201 that captured the captured X-ray image 501” at paragraph 0030, line 5).
Regarding claims 4, 15 and 19, Matsuzaki et al. discloses a method, system and medium wherein identifying the structure of interest in the CT image includes identifying extents of a volume encompassed by the structure of interest (“based on the three-dimensional image from which only the bone region has been extracted” at paragraph 0039, line 3).
Regarding claim 5, Matsuzaki et al. discloses a method wherein the structure of interest is an organ (“Next, the rigid region attenuator 114 extracts a bone region from the three-dimensional image (S202)” at paragraph 0038, line 5).
Regarding claims 6, 16 and 20, Matsuzaki et al. discloses a method, system and medium wherein the attenuation map is indicative of attenuation due to the structure of interest, and wherein the structure of interest is removed from the modified X-ray image (“Next, the rigid region attenuation unit 114 generates a bone region attenuated X-ray image by subtracting the bone region calculated projection image from the captured X-ray image (captured X-ray image - bone region calculated projection image) (S204)” at paragraph 0040, line 1).
Regarding claim 11, Matsuzaki et al. discloses a method wherein the X-ray image has a higher resolution than the CT image (the spatial resolution of a radiograph is higher than CT).
Regarding claims 12 and 18, Matsuzaki et al. discloses a method and system further wherein execution of the routine stored in memory further causes the processor to display the X-ray image and the modified X-ray image (“That is, the display device 401 displays the captured X-ray images acquired by the data acquisition unit 302 and the results of processing by the data processing device 100” at paragraph 0019, line 9).
Claim(s) 7-10, 17 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Matsuzaki et al. and Fei et al. as applied to claims 1, 6, 16 and 20 above, and further in view of Sattarivand et al. (US 2021/0267563).
Regarding claims 7, 17 and 21, the Matsuzaki et al. and Fei et al. combination discloses a method, system and medium as described in claims 6, 16 and 20 as described above.
The Matsuzaki et al. and Fei et al. combination does not explicitly disclose that the attenuation map is generated by creating a modified CT image in which HU values of everything but the structure of interest are set equal to an HU value of air and integrating attenuation of X-rays through the modified CT image.
Sattarivand et al. teaches a method in the same field of endeavor of DRR image processing, wherein the attenuation map is generated by creating a modified CT image in which HU values of everything but the structure of interest are set equal to an HU value of air and integrating attenuation of X-rays through the modified CT image (“In other embodiments mask images contain density values for voxels that correspond to a tissue type that the mask relates to and voxels that correspond to other tissue types may be set to a suitable value such as a density of air or a vacuum in the density units being used” at paragraph 0228, last sentence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the masking as taught by Sattarivand et al. in the DRR generation of the Matsuzaki et al. and Fei et al. combination as an alternative way of isolating the different tissue types in the image data (see Sattarivand et al. at paragraph 0229).
Regarding claim 8, the Matsuzaki et al. and Fei et al. combination discloses a method as described in claim 6 as described above.
The Matsuzaki et al. and Fei et al. combination does not explicitly disclose that a boundary of the structure of interest in the modified X-ray image is coarser than a remainder of the X-ray image.
Sattarivand et al. teaches a method in the same field of endeavor of DRR image processing, wherein a boundary of the structure of interest in the modified X-ray image is coarser than a remainder of the X-ray image (“The user may be able to adjust the thickness of the boundary regions (margins) and/or the level of smoothness in a linear or non-linear fashion (as described elsewhere herein).” at paragraph 0327, last sentence; “ A control or controls that allow a user to select a mechanism for generating weighting factors. For example, such a control or controls may allow a user to select among: [0329] selecting among various goals for the weighting factors (e.g. deemphasize bone, deemphasize soft tissue, deemphasize some particular type of soft tissue, enhance contrast of some type of tissue e.g. tumor tissue, etc.)” at paragraph 0328).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a boundary weighting as taught by Sattarivand et al. on the modified x-ray of the Matsuzaki et al. and Fei et al. combination to allow the tissue of interest to be better differentiated from the surrounding image data.
Regarding claim 9, the Matsuzaki et al. and Fei et al. combination discloses a method as described in claim 1 as described above.
The Matsuzaki et al. and Fei et al. combination does not explicitly disclose that the attenuation map is indicative of attenuation due to everything but the structure of interest, and wherein everything but the structure of interest is removed from the modified X-ray image.
Sattarivand et al. teaches a method in the same field of endeavor of DRR image processing, wherein the attenuation map is indicative of attenuation due to everything but the structure of interest, and wherein everything but the structure of interest is removed from the modified X-ray image (“Once a DRR 242 for a tissue type is generated, each pixel value of the DRR 242 corresponds to a thickness of that tissue type. FIGS. 4A, 4B, 4C illustrate example DRRs 242 of soft tissue 52, bone tissue 54, and combined soft tissue 52 and bone tissue 54 respectively of patient P's thorax” at paragraph 0229, second to last sentence; “In some embodiments, weighting factor values 142 may be selected to de-emphasize more specific tissue types such as, for example, lung tissue, ribs, muscle tissue, or the like” at paragraph 0195, line 4; therefore, one is able to generate a DRR corresponding to a specific tissue type to be either be eliminated or preserved).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to create a variety of tissue DRRs as taught by Sattarivand et al. in the system of the Matsuzaki et al. and Fei et al. combination to be able to control what tissues of interest are to remain for display.
Regarding claim 10, the Matsuzaki et al. and Fei et al. combination discloses a method as described in claim 1 as described above.
The Matsuzaki et al. and Fei et al. combination does not explicitly disclose that the attenuation map is generated by creating a modified CT image in which HU values of the structure of interest are set equal to an HU value of air and integrating attenuation of X-rays through the modified CT image.
Sattarivand et al. teaches a method in the same field of endeavor of DRR image processing, wherein the attenuation map is generated by creating a modified CT image in which HU values of the structure of interest are set equal to an HU value of air and integrating attenuation of X-rays through the modified CT image (“In other embodiments mask images contain density values for voxels that correspond to a tissue type that the mask relates to and voxels that correspond to other tissue types may be set to a suitable value such as a density of air or a vacuum in the density units being used” at paragraph 0228, last sentence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the masking as taught by Sattarivand et al. in the DRR generation of the Matsuzaki et al. and Fei et al. combination as an alternative way of isolating the different tissue types in the image data (see Sattarivand et al. at paragraph 0229).
Response to Arguments
Summary of Remarks (@ response page labeled 9): Matsuzaki et al. does not teach or disclose determining a mapping between the obtained X-ray image and the obtained CT image comprises: (i) generating a plurality of simulated X-ray images from the obtained CT image, wherein each of the plurality of simulated X-ray images is generated using a different simulated focal point and detector position, and (ii) using parameters corresponding to a simulated X-ray image among the plurality of simulated X-ray images that most closely matches the obtained X-ray image.
Examiner’s Response: This argument is moot in view of the newly utilized Fei et al. reference.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 9:30-5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/Primary Examiner, Art Unit 2672