Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1, 11, 21, and 26 have been amended. Claims 1-5, and 7-31 are pending. This application has undergone a restriction, claims 4-5, 14-19, 24-25, and 28-30 have been withdrawn from consideration.
Response to arguments
Applicants’ arguments with respect to claim(s) 1-5 and 7-31 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument except for the new limitation in regard to HUA. Applicant asserts that “generating a plurality of variation images by applying direct image-space transformations to the reference medical image to provide variation among the plurality of variation images from a same projection viewpoint in the CBCT projection space” is not taught by HUA.
The examiner respectfully disagrees; HUA explicitly discusses augmenting the reference image data:
([0024]: “In order to increase an amount of data available for training, an initial training set is augmented/enlarged by at least one of interpolating between views and rotating pieces of data to produce additional training data”. These interpolations and rotations of data would augment the medical data to provide variation among each other from a same perspective viewpoint. For example, viewpoint 1 can have 2 images, viewpoint 2 also has 2 images, viewpoint 2’s image is interpolated to viewpoint 1 and rotated, this would provide viewpoint 1 with another image from the viewpoint which varies from the current images in viewpoint 1).
The rotations applied to the reference medical images would constitute direct-image-space transformations. Therefore, HUA does teach the amended limitation.
As for the deformations limitation new art has been found and used in the rejection that follows.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 7-8, 11-13, 21-23, 26-27, and 31 are rejected under 35 U.S.C. 103 as being unpatentable over HUA et al. (US 20230274473 A1 Hereinafter “HUA”) in view of Han (US 20190030371 A1 Hereinafter “Han”).
Regarding claim 1, HUA teaches a computer-implemented method for training a regression model for cone- beam computed tomography (CBCT) data processing, the method comprising: obtaining a reference medical image of an anatomical area of a human patient (Fig. 5, [0047]: “First, 3D volume data 409 is obtained. The 3D volume data 409 can be acquired from phantoms and/or clinical sites. For example, the 3D volume data 409 can be clinical head data or abdomen data”);
generating a set of CBCT projections, the set of CBCT projections generated from the reference medical image at each of a plurality of projection viewpoints in a CBCT projection space (Fig. 5, [0048]: “Step 411 is to simulate a CBCT projection data using the 3D volume data 409 to generate a simulated CBCT projection dataset 413”. This generated data can be obtained at a plurality of viewpoints, [0022]: “For each volume, numerous sets (e.g., 50-300) of various-angle data can be simulated”);
wherein generating the set of CBCT projections comprises:
generating a plurality of variation images by applying direct image-space transformations to the reference medical image to provide variation among the plurality of variation images from a same projection viewpoint in the CBCT projection space ([0024]: “In order to increase an amount of data available for training, an initial training set is augmented/enlarged by at least one of interpolating between views and rotating pieces of data to produce additional training data”. These interpolations and rotations of data would augment the medical data to provide variation among each other from a same perspective viewpoint. For example, viewpoint 1 can have 2 images, viewpoint 2 also has 2 images, viewpoint 2’s image is interpolated to viewpoint 1 and rotated, this would provide viewpoint 1 with another image from the viewpoint which varies from the current images in viewpoint 1. Additionally, rotations applied to the reference medical images are direct-image-space transformations);
wherein the plurality of variation images include simulated ([0024]: “In order to increase an amount of data available for training, an initial training set is augmented/enlarged by at least one of interpolating between views and rotating pieces of data to produce additional training data”. These interpolations and rotations of data would augment the medical data to provide variation among each other from a same perspective viewpoint. This would provide variation to the target organ depicted from the same projection viewpoint);
generating a set of simulated scatter data, the set of simulated scatter data to represent effects of scatter from CBCT imaging in each respective projection in the set of CBCT projections (Fig. 5, [0049]: “Step 415 is to simulate a CBCT scatter dataset using the 3D volume data 409 to generate a simulated CBCT scatter dataset 417. The simulated CBCT scatter dataset 417 indicates a scatter profile from the 3D volume data 409, and can be generated using Monte Carlo techniques (e.g. GEANT4) and/or radiative transfer equation (RTE) techniques”); and
training the regression model using the set of CBCT projections and the set of simulated scatter data ([0050]: “In step 423, a neural network is trained using the simulated CBCT projection dataset 413 as input learning data, and the smooth simulated CBCT scatter dataset 421 as target learning data”. Neural network acts as the regression model).
HUA does not expressly disclose the augments to the image data being deformation of the data to provide variation in image training data of a same perspective.
However, Han teaches augmenting medical image data using deformation of the training data to provide variation in image training data ([0095]: “For the experimental results herein, training data were augmented by applying random deformations to the input images and their segmentation maps to create new training data”. These input images are medical images “For example, according to illustrative examples herein, such techniques may be evaluated by using them to segment five OARs (e.g., the left and right lungs, the heart, the esophagus, and the spinal cord) from thoracic CT images of lung cancer patients” [0026]).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify HUA’s training data to include Han’s deformation of medical data for additional training data because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Han’s deformation of medical data for additional training data permits generation of additional training data for neural networks being trained. This known benefit in Han is applicable to HUA’s training data as they both share characteristics and capabilities, namely, they are directed to training neural networks using augmented medical training data. Therefore, it would have been recognized that modifying HUA’s training data to include Han’s deformation of medical data for additional training data would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Han’s deformation of medical data for additional training data in training neural networks using augmented medical training data and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art
Regarding claim 2, the combination of HUA and Han teaches the method of claim 1, in addition, HUA further teaches wherein the trained regression model is configured to infer scatter from newly captured CBCT projections (Fig. 5, [0050]: “The output of step 423 is a scatter estimation neural network 403, which can estimate the scatter profile of a projection dataset input”), and wherein training the regression model includes training with pairs of generated simulated scatter data that represents simulated scatter and generated CBCT projections that include effects based on the simulated scatter (Fig. 5, [0050]: “In step 423, a neural network is trained using the simulated CBCT projection dataset 413 as input learning data, and the smooth simulated CBCT scatter dataset 421 as target learning data”. These training pairs include data that represents the simulated scatter (smooth simulated CBCT scatter dataset) and the generated CBCT projections that include effects based on the simulated scatter (simulated CBCT projection dataset 413 that contains the scatter effects).
Regarding claim 3, the combination of HUA and Han teaches the method of claim 2, in addition, HUA further teaches wherein the trained regression model is configured to receive a newly captured CBCT projection that includes effects from scatter as input (Fig. 5, [0050]: “The output of step 423 is a scatter estimation neural network 403, which can estimate the scatter profile of a projection dataset input”. These projection dataset inputs contain effects from scatter), and wherein the trained regression model is configured to provide an identification of the effects from the scatter as output (Fig. 5, [0050]: “The output of step 423 is a scatter estimation neural network 403, which can estimate the scatter profile of a projection dataset input”. These projection dataset inputs contain effects from scatter).
Regarding claim 7, the combination of HUA and Han teaches the method of claim 1, in addition, HUA further teaches wherein the plurality of variation images are generated by geometrical augmentations or geometric changes to representations of the anatomical area obtained from the reference medical image ([0024]: “In order to increase an amount of data available for training, an initial training set is augmented/enlarged by at least one of interpolating between views and rotating pieces of data to produce additional training data”. These interpolations and rotations of data would be considered geometric changes, and they would affect the representations of the anatomical area obtained from the reference medical image), and wherein the plurality of projection viewpoints correspond to a plurality of projection angles used for capturing CBCT projections ([0022]: “For each volume, numerous sets (e.g., 50-300) of various-angle data can be simulated”. This differing angle data is the plurality of projection angles for the plurality of projection viewpoints).
Regarding claim 8, the combination of HUA and Han teaches the method of claim 1, in addition, HUA further teaches, wherein the reference medical image is a 3D image provided from a computed tomography (CT) scan ([0039]: “Alternatively, a high resolution CT dataset may be created by simulating a digital phantom that is converted to 3D volume data that can be processed as if it were acquired 3D data”. The 3D volume data can act as a high-resolution CT dataset), and wherein the method further comprises training of the regression model using a plurality of reference medical images from the CT scan (Fig. 5, [0047]: “First, 3D volume data 409 is obtained. The 3D volume data 409 can be acquired from phantoms and/or clinical sites. For example, the 3D volume data 409 can be clinical head data or abdomen data. This 3D volume data 409 can be used to generate synthetic pairs of projection and scatter datasets for training”. There can be a plurality of reference images (3D volume data), [0021]: “In one embodiment, numerous sets of different 3D/volume data 120 can be acquired from phantoms and clinical sites and can be used to generate the synthetic pair of projection images and scatter images”).
Regarding claim 11, HUA teaches a computer-implemented method for using a trained regression model for cone-beam computed tomography (CBCT) data processing, the method comprising:
accessing a trained regression model configured for processing CBCT projection data, wherein the trained regression model is trained using corresponding sets of simulated scatter data and CBCT projections ([0050]: “In step 423, a neural network is trained using the simulated CBCT projection dataset 413 as input learning data, and the smooth simulated CBCT scatter dataset 421 as target learning data”),
wherein the CBCT projections used in training are produced from at least one reference medical image at respective projection viewpoints of an anatomical region in a CBCT projection space (Fig. 5, [0048]: “Step 411 is to simulate a CBCT projection data using the 3D volume data 409 to generate a simulated CBCT projection dataset 413”. This generated data can be obtained at a plurality of viewpoints, [0022]: “For each volume, numerous sets (e.g., 50-300) of various-angle data can be simulated”. This data is also of anatomical regions “For example, the 3D volume data 409 can be clinical head data or abdomen data” ([0047])),
wherein the CBCT projections used in training are generated from a plurality of variation images, the plurality of variation images being generated by applying direct image-space transformations to the at least one reference medical image, and wherein the plurality of variation images ([0024]: “In order to increase an amount of data available for training, an initial training set is augmented/enlarged by at least one of interpolating between views and rotating pieces of data to produce additional training data”. These interpolations and rotations of data would augment the medical data to provide variation among each other from a same perspective viewpoint. For example, viewpoint 1 can have 2 images, viewpoint 2 also has 2 images, viewpoint 2’s image is interpolated to viewpoint 1 and rotated, this would provide viewpoint 1 with another image from the viewpoint which varies from the current images in viewpoint 1. Additionally, rotations applied to the reference medical images are direct-image-space transformations) include simulated ([0024]: “In order to increase an amount of data available for training, an initial training set is augmented/enlarged by at least one of interpolating between views and rotating pieces of data to produce additional training data”. These interpolations and rotations of data would augment the medical data to provide variation among each other from a same perspective viewpoint. This would provide variation to the target organ depicted from the same projection viewpoint), and
wherein the simulated scatter data used in training represents respective effects of scatter from CBCT imaging in the CBCT projections (Fig. 5, [0049]: “Step 415 is to simulate a CBCT scatter dataset using the 3D volume data 409 to generate a simulated CBCT scatter dataset 417. The simulated CBCT scatter dataset 417 indicates a scatter profile from the 3D volume data 409, and can be generated using Monte Carlo techniques (e.g. GEANT4) and/or radiative transfer equation (RTE) techniques”);
providing a newly captured CBCT projection as an input to the trained regression model, wherein the newly captured CBCT projection includes effects from scatter (Fig. 5, [0050]: “The output of step 423 is a scatter estimation neural network 403, which can estimate the scatter profile of a projection dataset input”. The projection dataset input is a newly captured CBCT projection which includes the effects from scatter); and
obtaining data as an output of the trained regression model, wherein the data is based on identified effects from scatter (Fig. 5, [0050]: “The output of step 423 is a scatter estimation neural network 403, which can estimate the scatter profile of a projection dataset input”. The output is the scatter profile of the projection dataset, which contains the identified effects from scatter).
HUA does not expressly disclose the augments to the image data being deformation of the data to provide variation in image training data of a same perspective.
However, Han teaches augmenting medical image data using deformation of the training data to provide variation in image training data ([0095]: “For the experimental results herein, training data were augmented by applying random deformations to the input images and their segmentation maps to create new training data”. These input images are medical images “For example, according to illustrative examples herein, such techniques may be evaluated by using them to segment five OARs (e.g., the left and right lungs, the heart, the esophagus, and the spinal cord) from thoracic CT images of lung cancer patients” [0026]).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify HUA’s training data to include Han’s deformation of medical data for additional training data because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Han’s deformation of medical data for additional training data permits generation of additional training data for neural networks being trained. This known benefit in Han is applicable to HUA’s training data as they both share characteristics and capabilities, namely, they are directed to training neural networks using augmented medical training data. Therefore, it would have been recognized that modifying HUA’s training data to include Han’s deformation of medical data for additional training data would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Han’s deformation of medical data for additional training data in training neural networks using augmented medical training data and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art
Regarding claim 12, the combination of HUA and Han teaches the method of claim 11, in addition, HUA further teaches wherein the trained regression model is configured to infer scatter from the newly captured CBCT projection (Fig. 5, [0050]: “The output of step 423 is a scatter estimation neural network 403, which can estimate the scatter profile of a projection dataset input”. The projection dataset input is a newly captured CBCT projection which includes the effects from scatter), and wherein the data generated by the trained regression model comprises scatter data that represents the identified effects from scatter (Fig. 5, [0050]: “The output of step 423 is a scatter estimation neural network 403, which can estimate the scatter profile of a projection dataset input”. The output is the scatter profile of the projection dataset, which contains the identified effects from scatter)..
Regarding claim 13, the combination of HUA and Han teaches the method of claim 12, in addition, HUA further teaches wherein the trained regression model is trained with pairs of simulated scatter data that represents simulated scatter and generated CBCT projections that include effects based on the simulated scatter (Fig. 5, [0050]: “In step 423, a neural network is trained using the simulated CBCT projection dataset 413 as input learning data, and the smooth simulated CBCT scatter dataset 421 as target learning data”. These training pairs include data that represents the simulated scatter (smooth simulated CBCT scatter dataset) and the generated CBCT projections that include effects based on the simulated scatter (simulated CBCT projection dataset 413 that contains the scatter effects).
Regarding claim 21, the content of claim 21 is similar to the content of claim 1, with the additional teachings of a non-transitory computer-readable storage medium. HUA also discloses this information ([0106]: “Alternatively, the CPU in the reconstruction device 514 can execute a computer program including a set of computer-readable instructions that perform the functions described herein, the program being stored in any of the above-described non-transitory electronic memories and/or a hard disk drive, CD, DVD, FLASH drive or any other known storage media”). Therefore, claim 21 is rejected for the same reasons of obviousness as claim 1, along with the additional teachings above.
Regarding claim 22, the content of claim 22 is similar to the content of claim 2, therefore it is rejected for the same reasons of obviousness as claim 2.
Regarding claim 23, the content of claim 23 is similar to the content of claim 3, therefore it is rejected for the same reasons of obviousness as claim 3.
Regarding claim 26, the content of claim 26 is similar to the content of claim 11, therefore it is rejected for the same reasons of obviousness as claim 11.
Regarding claim 27, the content of claim 27 is similar to the content of claim 12, therefore it is rejected for the same reasons of obviousness as claim 12.
Regarding claim 31, the content of claim 31 is similar to the content of claim 7, therefore it is rejected for the same reasons of obviousness as claim 7.
Claims 9-10 and 20 rejected under 35 U.S.C. 103 as being unpatentable over HUA et al. (US 20230274473 A1 Hereinafter “HUA”) in view of Han (US 20190030371 A1 Hereinafter “Han”) in further view of Xu et al. (US 20200151922 A1 Hereinafter “Xu”).
Regarding claim 9, the combination of HUA and Han teaches the method of claim 1,
The combination of HUA and Han does not expressly disclose wherein the trained regression model is used for radiotherapy treatment of the human patient, and wherein the anatomical area corresponds to an area of the radiotherapy treatment.
However, Xu teaches wherein a trained regression model is used for radiotherapy treatment of the human patient, and wherein the anatomical area corresponds to an area of the radiotherapy treatment (Fig. 1, [0034]: “FIG. 1 illustrates an exemplary radiotherapy system 10 for providing radiation therapy to a patient. The radiotherapy system 10 includes an image processing device, 12”. This Radiotherapy system uses a trained Deep Convolutional Neural network to remove scatter, [0084]: “FIG. 9C illustrates a method for generating artifact reduced, reconstructed 3D images using a trained DCNN, such as a DCNN that can be trained according to the method described above with respect to FIG. 9B. The DCNN can reduce artifacts such as noise, scatter, extinction artifacts, beam hardening artifacts, exponential edge gradient effects, aliasing effects, ring artifacts, motion artifacts, or misalignment effects”).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of HUA and Han’s scatter correction system for improved CBCT images to include Xu’s use of the improved CBCT images in Radiotherapy because such a modification is based on the use of known techniques to improve similar devices in the same way. More specifically, Xu’s CBCT scatter correction is comparable to the combination of HUA and Han’s scatter correction system for improved CBCT images because they both are systems meant to reduce the scatter in CBCT images using trained models. Therefore, it would be obvious to one of ordinary skill in the art to modify the combination of HUA and Han’s scatter correction system for improved CBCT images to include Xu’s use of the improved CBCT images in Radiotherapy in order to obtain the predictable result of using the combination of HUA and Han’s image improving process to aid in radiotherapy
Regarding claim 10, the combination of HUA, Han, and Xu teaches the method of claim 9, in addition, Xu further teaches wherein the method further includes training of the regression model using a plurality of reference medical images from one or more prior computed tomography (CT) scans or one or more prior CBCT scans of the human patient ([0038]: “Patient data 45 may include information such as (1) functional organ modeling data (e.g., serial versus parallel organs, appropriate dose response models, etc.); (2) radiation dosage data (e.g., dose-volume histogram (DVH) information; or (3) other clinical information about the patient and course of treatment (e.g., other surgeries, chemotherapy, previous radiotherapy, etc.)”. Previous reference medical images would be encompassed in the previous radiotherapy of the same method (Emphasis added)).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of HUA, Han, and Xu’s CBCT scatter correction system to include Xu’s use of prior patient data for training because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Xu’s use of prior patient data for training permits training a model using data associated with the patient, improving the diversity of training data for the model. This known benefit in Xu is applicable to the combination of HUA, Han, and Xu’s CBCT scatter correction system as they both share characteristics and capabilities, namely, they are directed to scatter correction for CBCT data. Therefore, it would have been recognized that modifying the combination of HUA, Han, and Xu’s CBCT scatter correction system to include Xu’s use of prior patient data for training would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Xu’s use of prior patient data for training in scatter correction for CBCT data and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Regarding claim 20, the combination of HUA and Han teaches the method of claim 11,
The combination of HUA and Han does not expressly wherein the at least one reference medical image comprises multiple images captured from one or more prior computed tomography (CT) scans or one or more prior CBCT scans of a human patient.
However, Xu teaches wherein the at least one reference medical image comprises multiple images captured from one or more prior computed tomography (CT) scans or one or more prior CBCT scans of a human patient ([0038]: “Patient data 45 may include information such as (1) functional organ modeling data (e.g., serial versus parallel organs, appropriate dose response models, etc.); (2) radiation dosage data (e.g., dose-volume histogram (DVH) information; or (3) other clinical information about the patient and course of treatment (e.g., other surgeries, chemotherapy, previous radiotherapy, etc.)”. Previous reference medical images would be encompassed in the previous radiotherapy of the same method (Emphasis added)).
The rationale for the combination of claim 20 is similar to the rationale for the combination of claim 10, due to similar methods of combination and benefits.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
MYLONAS et al. (US 20250360339 A1) teaches deforming training data to produce more training data.
Chen (US 20220059229 A1) teaches deforming training data to produce more training data.
Kearney et al. (US 20210118132 A1) teaches deforming training data to produce more training data.
Applicants’ amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEFANO A DARDANO whose telephone number is (703)756-4543. The examiner can normally be reached Monday - Friday 11:00 - 7:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEFANO ANTHONY DARDANO/ Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights