DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1 , 2, 4, 5, 7-17, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 (U.S. PG-PUB NO. 2019/0104940) in view of D2 (U.S. PG-PUB NO. 2020/0327370).
-Regarding claim 1, D1 discloses a method for performing image enhancement using a neural network in a medical imaging system (reconstruction methods for medical images, [0021]), the method comprising: acquiring an image of an imaging object ([0032]); and applying the acquired image to a trained neural network to generate an image-enhanced image of the imaging object (the reconstructed image is denoised using the DL network 135, [0035]), wherein the neural network was trained by: receiving a training image pair including a first image and a second image (the offline DL training process 110 trains the DL network 135 using a large number of noisy reconstructed images 115 that are paired with corresponding high-image-quality images 120 to train the DL network 135, [0031]), and training the neural network using the preprocessed first image as an input image and the preprocessed second image as a target image (train the DL network 135, [0060]).
D1 is silent to teaching that performing an image intensity preprocessing on the received image pair to generate a preprocessed first image and a preprocessed second image, such that the preprocessed first image has a first intensity range covering a portion of an intensity range of the first image, and the preprocessed second image has a second intensity range covering a portion of an intensity range of the second image. However, the claimed limitation is well known in the art as evidenced by D2.
In the same field of endeavor, D2 teaches performing an image intensity preprocessing on the received image pair to generate a preprocessed first image and a preprocessed second image, such that the preprocessed first image has a first intensity range covering a portion of an intensity range of the first image, and the preprocessed second image has a second intensity range covering a portion of an intensity range of the second image (Data 54 and 55 input to the paths C1 and C2 have a size similar to that of the data 51 of the first path A1, but differs at a point that the CT value is clipped (restricted) within a predetermined range, [0066]; a range 18 is a range of a lower side of the CT value with an upper limit of about 500 HU, and a range 19 is a range of a higher side of the CT value with a lower limit of about 300 HU, [0068]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of D1 with the teaching of D2 in order to provide efficiency of learning.
-Regarding claim 2, the combination further discloses the step of performing the image intensity preprocessing further comprises: based on knowledge with respect to a pathology or physiology structure of interest, determining at least one intensity bound value (D2, The specific value is not restricted in particular, and is to be set appropriately according to diagnosis type and a body part of interest of a patient etc, [0068]), clipping, based on the determined at least one intensity bound value, the first image to generate a clipped first image, as the preprocessed first image (D2, Data 54 and 55 input to the paths C1 and C2 have a size similar to that of the data 51 of the first path A1, but differs at a point that the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and clipping, based on the determined at least one intensity bound value, the second image to generate a clipped second image, as the preprocessed second image (D2, [0066], [0088]).
-Regarding claim 3, the combination further discloses the determining step further comprises determining a lower bound value (D2, a range 18 is a range of a lower side of the CT value with an upper limit of about 500 HU, and a range 19 is a range of a higher side of the CT value with a lower limit of about 300 HU, [0068]), the step of clipping the first image further comprises, when a voxel in the first image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or a predefined constant value (D2, the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and the step of clipping the second image further comprises, when a voxel in the second image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or the predefined constant value (D1, [0033]; D2, [0066]).
-Regarding claim 4, the combination further discloses the determining step further comprises determining an upper bound value (D2, a range 18 is a range of a lower side of the CT value with an upper limit of about 500 HU, and a range 19 is a range of a higher side of the CT value with a lower limit of about 300 HU, [0068]), the step of clipping the first image further comprises, when a voxel in the first image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or a predefined constant value (D2, the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and the step of clipping the second image further comprises, when a voxel in the second image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or the predefined constant value (D1, [0031]; D2, [0066]).
-Regarding claim 5, the combination further discloses the determining step further comprises determining a lower bound value and an upper bound value, the step of clipping the first image (D2, specific examples of the range 18 include a range 0 to 500 HU or 0 to 400 HU. Specific examples of the range 19 include a range 300 to 2000 HU or a range 300 to 1700 HU, [0068]) further comprises: when a voxel in the first image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or a predefined constant value, and when a voxel in the first image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or the predefined constant value (D2, the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and the step of clipping the second image further comprises: when a voxel in the second image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or the predefined constant value, and when a voxel in the second image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or the predefined constant value (D1, [0031]; D2, [0066]).
-Regarding claim 7, the combination further discloses the step of performing the image intensity preprocessing further comprises: based on knowledge with respect to a pathology or physiology structure of interest, determining a plurality of intensity segments (D2, specific examples of the range 18 include a range 0 to 500 HU or 0 to 400 HU. Specific examples of the range 19 include a range 300 to 2000 HU or a range 300 to 1700 HU, [0068]), decomposing, based on the determined plurality of intensity segments, the first image into a plurality of first sub-images, as the preprocessed first image (D2, input data reduced to the range of the lower side of the CT value to the path C1 upon dividing the medical volume data 601 into the range of the CT value in such manner, and to input data reduced to the range of the higher side of the CT value to the path C2, [0069]), and decomposing, based on the determined plurality of intensity segments, the second image into a plurality of second sub-images, as the preprocessed second image (D1, [0031]; D2, [0069], [0088]).
-Regarding claim 8, the combination further discloses the neural network includes a plurality of neural networks, and the training step further comprises, for each neural network of the plurality of neural networks, performing training by using one sub-image of the plurality of first sub- images as an input image and a corresponding one sub-image of the plurality of second sub- images as a target image, to obtain a plurality of trained neural networks (D2, From a view point of an efficiency of learning, in a case in which the two paths C1 and C2 for which the range of CT values is let to differ are provided as in the present embodiment, an ability to carry out learning with even lesser number of training data can be anticipated, [0091]; D1, In the offline training process 110, the noisy images are classified based on their noise level ranges. For each class/range, a separate network of the DL networks 135 is trained, [0051]).
-Regarding claim 9, the combination further discloses the applying step further comprises: decomposing, based on the determined plurality of intensity segments, the acquired image into a plurality of sub-images (D2, arrangement is made to input data reduced to the range of the lower side of the CT value to the path C1 upon dividing the medical volume data 601 into the range of the CT value in such manner, and to input data reduced to the range of the higher side of the CT value to the path C2, [0069]), inputting the decomposed plurality of sub-image into the plurality of trained neural networks in a one-to-one manner, to infer a plurality of sub-images at a plurality of outputs of the plurality of trained neural networks (D1, the reconstructed image is denoised using the DL network 135. The result of which is a high-quality image 175, [0035]), and combining, based on a plurality of weights, the inferred plurality of sub-images to obtain a combined image, as the generated image-enhanced image (D2, regarding the fully connected layers, parameters indicating connecting strength of the nodes (not shown) are included, [0087]).
-Regarding claim 10, the combination further discloses the neural network includes a plurality of channels with a network parameter shared thereamong, and the training step further comprises, for each channel of the plurality of channels, performing training by using one3 sub-image of the plurality of first sub-images as an input image and a corresponding one sub- image of the plurality of second sub-images as a target image (D1, a slice and the adjacent slices (i.e., the slice above and below the central slice) are identified as a three-channel input for the network, [0048]; shared weight in convolutional layers, which means that the same filter (weights bank) is used as the coefficients for each pixel in the layer, [0046]; D2, arrangement is made to input data reduced to the range of the lower side of the CT value to the path C1 upon dividing the medical volume data 601 into the range of the CT value in such manner, and to input data reduced to the range of the higher side of the CT value to the path C2, [0069]).
-Regarding claim 11, the combination further discloses the applying step further comprises: decomposing, based on the determined plurality of intensity segments, the acquired image into a plurality of sub-images (D2, arrangement is made to input data reduced to the range of the lower side of the CT value to the path C1 upon dividing the medical volume data 601 into the range of the CT value in such manner, and to input data reduced to the range of the higher side of the CT value to the path C2, [0069]), inputting the decomposed plurality of sub-image into the plurality of channels of the trained neural network in a one-to-one manner, to infer a plurality of sub-images at a plurality of outputs of the plurality of channels of the trained neural network (D1, a W×W×3 kernel is applied M times to generate M values for the convolutional layer, which are then used for the following network layers/hierarchies (e.g., a pooling layer), [0048]), and combining, based on a plurality of weights, the inferred plurality of sub-images to obtain a combined image, as the generated image-enhanced image (D2, regarding the fully connected layers, parameters indicating connecting strength of the nodes (not shown) are included, [0087]).
-Regarding claim 12, the combination further discloses the determining step further comprises determining the plurality of intensity segments, such that at least two intensity segments of the determined plurality of intensity segments have an overlap between each other (D2, specific examples of the range 18 include a range 0 to 500 HU or 0 to 400 HU. Specific examples of the range 19 include a range 300 to 2000 HU or a range 300 to 1700 HU, [0068]).
-Regarding claim 13, D1 discloses an apparatus for performing image enhancement using a neural network in a medical imaging system (reconstruction methods for medical images, [0021], [0092]), the apparatus comprising processing circuitry (CPU, [0092]) configured to: acquire an image of an imaging object ([0032]), and apply the acquired image to a trained neural network to generate an image-enhanced image of the imaging object (the reconstructed image is denoised using the DL network 135, [0035]), wherein the neural network was trained by: receiving a training image pair including a first image and a second image (the offline DL training process 110 trains the DL network 135 using a large number of noisy reconstructed images 115 that are paired with corresponding high-image-quality images 120 to train the DL network 135, [0031]), and training the neural network using the preprocessed first image as an input image and the preprocessed second image as a target image (train the DL network 135, [0060]).
D1 is silent to teaching that performing an image intensity preprocessing on the received image pair to generate a preprocessed first image and a preprocessed second image, such that the preprocessed first image has a first intensity range covering a portion of an intensity range of the first image, and the preprocessed second image has a second intensity range covering a portion of an intensity range of the second image. However, the claimed limitation is well known in the art as evidenced by D2.
In the same field of endeavor, D2 teaches performing an image intensity preprocessing on the received image pair to generate a preprocessed first image and a preprocessed second image, such that the preprocessed first image has a first intensity range covering a portion of an intensity range of the first image, and the preprocessed second image has a second intensity range covering a portion of an intensity range of the second image (Data 54 and 55 input to the paths C1 and C2 have a size similar to that of the data 51 of the first path A1, but differs at a point that the CT value is clipped (restricted) within a predetermined range, [0066]; a range 18 is a range of a lower side of the CT value with an upper limit of about 500 HU, and a range 19 is a range of a higher side of the CT value with a lower limit of about 300 HU, [0068]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of D1 with the teaching of D2 in order to provide efficiency of learning.
-Regarding claim 14, the combination further discloses the step of performing the image intensity preprocessing further comprises: based on knowledge with respect to a pathology or physiology structure of interest, determining at least one intensity bound value (D2, The specific value is not restricted in particular, and is to be set appropriately according to diagnosis type and a body part of interest of a patient etc, [0068]), clipping, based on the determined at least one intensity bound value, the first image to generate a clipped first image, as the preprocessed first image (D2, Data 54 and 55 input to the paths C1 and C2 have a size similar to that of the data 51 of the first path A1, but differs at a point that the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and clipping, based on the determined at least one intensity bound value, the second image to generate a clipped second image, as the preprocessed second image (D2, [0066], [0088]).
-Regarding claim 15, the combination further discloses the determining step further comprises determining a lower bound value (D2, a range 18 is a range of a lower side of the CT value with an upper limit of about 500 HU, and a range 19 is a range of a higher side of the CT value with a lower limit of about 300 HU, [0068]), the step of clipping the first image further comprises, when a voxel in the first image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or a predefined constant value (D2, the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and the step of clipping the second image further comprises, when a voxel in the second image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or the predefined constant value (D1, [0033]; D2, [0066]).
-Regarding claim 16, the combination further discloses the determining step further comprises determining an upper bound value (D2, a range 18 is a range of a lower side of the CT value with an upper limit of about 500 HU, and a range 19 is a range of a higher side of the CT value with a lower limit of about 300 HU, [0068]), the step of clipping the first image further comprises, when a voxel in the first image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or a predefined constant value (D2, the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and the step of clipping the second image further comprises, when a voxel in the second image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or the predefined constant value (D1, [0031]; D2, [0066]).
-Regarding claim 17, the combination further discloses the determining step further comprises determining a lower bound value and an upper bound value, the step of clipping the first image (D2, specific examples of the range 18 include a range 0 to 500 HU or 0 to 400 HU. Specific examples of the range 19 include a range 300 to 2000 HU or a range 300 to 1700 HU, [0068]) further comprises: when a voxel in the first image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or a predefined constant value, and when a voxel in the first image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or the predefined constant value (D2, the CT value is clipped (restricted) within a predetermined range. In other words, only data within that range of the CT value is input, [0066]), and the step of clipping the second image further comprises: when a voxel in the second image has an intensity value lower than the determined lower bound value, setting the intensity value of the voxel to the determined lower bound value or the predefined constant value, and when a voxel in the second image has an intensity value higher than the determined upper bound value, setting the intensity value of the voxel to the determined upper bound value or the predefined constant value (D1, [0031]; D2, [0066]).
-Regarding claim 19, the combination further discloses the step of performing the image intensity preprocessing further comprises: based on knowledge with respect to a pathology or physiology structure of interest, determining a plurality of intensity segments (D2, specific examples of the range 18 include a range 0 to 500 HU or 0 to 400 HU. Specific examples of the range 19 include a range 300 to 2000 HU or a range 300 to 1700 HU, [0068]), decomposing, based on the determined plurality of intensity segments, the first image into a plurality of first sub-images, as the preprocessed first image (D2, input data reduced to the range of the lower side of the CT value to the path C1 upon dividing the medical volume data 601 into the range of the CT value in such manner, and to input data reduced to the range of the higher side of the CT value to the path C2, [0069]), and decomposing, based on the determined plurality of intensity segments, the second image into a plurality of second sub-images, as the preprocessed second image (D1, [0031]; D2, [0069], [0088]).
-Regarding claim 20, D1 discloses method for training a neural network to perform image enhancement in a medical imaging system (D1, process 110 of method 100 performs offline training of the DL network 135, [0031]), the method comprising: receiving a training image pair including a first image and a second image (the offline DL training process 110 trains the DL network 135 using a large number of noisy reconstructed images 115 that are paired with corresponding high-image-quality images 120 to train the DL network 135, [0031]); and using the preprocessed first image as an input image and the preprocessed second image as a target image, training the neural network to obtain a trained neural network (train the DL network 135, [0060]).
D1 is silent to teaching that performing an image intensity preprocessing on the received image pair to generate a preprocessed first image and a preprocessed second image, such that the preprocessed first image has a first intensity range covering a portion of an intensity range of the first image, and the preprocessed second image has a second intensity range covering a portion of an intensity range of the second image. However, the claimed limitation is well known in the art as evidenced by D2.
In the same field of endeavor, D2 teaches performing an image intensity preprocessing on the received image pair to generate a preprocessed first image and a preprocessed second image, such that the preprocessed first image has a first intensity range covering a portion of an intensity range of the first image, and the preprocessed second image has a second intensity range covering a portion of an intensity range of the second image (Data 54 and 55 input to the paths C1 and C2 have a size similar to that of the data 51 of the first path A1, but differs at a point that the CT value is clipped (restricted) within a predetermined range, [0066]; a range 18 is a range of a lower side of the CT value with an upper limit of about 500 HU, and a range 19 is a range of a higher side of the CT value with a lower limit of about 300 HU, [0068]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of D1 with the teaching of D2 in order to provide efficiency of learning.
Claim(s) 3, 6 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 (U.S. PG-PUB NO. 2019/0104940) in view of D2 (U.S. PG-PUB NO. 2020/0327370) and further in view of D3 (U.S. PG-PUB NO. 2007/0280519).
-Regarding claim 3, the combination is silent to teaching that based on the determined at least one intensity bound value, the acquired image to generate a clipped image,2 inputting the clipped image into the trained neural network to infer an image at an output of the trained neural network, and for a voxel in the acquired image that has an intensity value outside of the determined at least one intensity bound value, backfilling the voxel in the acquired image into the inferred image to obtain a backfilled image, as the generated image-enhanced image. However, the claimed limitation is well known in the art as evidenced by D3.
In the same field of endeavor, D3 teaches based on the determined at least one intensity bound value, the acquired image to generate a clipped image (a CT image copy may be a copy of only those elements of the original CT image, which have intensity values within a certain, predetermined intensity value range, [0035]), inputting the clipped image into the trained neural network to infer an image at an output of the trained neural network (the original CT image and/or the CT image copies are subjected to enhancement processing, whereby an enhancement processed CT image is obtained, [0037]), and for a voxel in the acquired image that has an intensity value outside of the determined at least one intensity bound value, backfilling the voxel in the acquired image into the inferred image to obtain a backfilled image, as the generated image-enhanced image (the enhancement processed images and possibly also the original CT image (or a copy thereof) are combined or merged such that a combined CT image is obtained, [0043]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of the combination with the teaching of D3 in order to provide increase the overall execution speed.
-Regarding claim 6, the combination further discloses clipping, based on the determined at least one intensity bound value, the acquired image to generate a clipped image (D2, Data 54 and 55 input to the paths C1 and C2 have a size similar to that of the data 51 of the first path A1, but differs at a point that the CT value is clipped (restricted) within a predetermined range, [0066]), inputting the clipped image into the trained neural network to infer an image at an output of the trained neural network (D1, the reconstructed image is denoised using the DL network 135. The result of which is a high-quality image 175, [0035]; D3, In step 103, the original CT image and/or the CT image copies are subjected to enhancement processing, whereby an enhancement processed CT image is obtained, [0037]), and for a voxel in the acquired image that has an intensity value outside of the determined at least one intensity bound value, backfilling the voxel in the acquired image into the inferred image to obtain a backfilled image, as the generated image-enhanced image (D3, the enhanced combined CT image and wherein resultX etc indicates the output of the processing step 104a-c for CT image copy number X: enhIm=maskBorgIm+mask1result1+ . . . +maskNresultN, [0054]).
-Regarding claim 18, the combination further discloses the applying step further comprises: clipping, based on the determined at least one intensity bound value, the acquired image to generate a clipped image (D2, Data 54 and 55 input to the paths C1 and C2 have a size similar to that of the data 51 of the first path A1, but differs at a point that the CT value is clipped (restricted) within a predetermined range, [0066]), inputting the clipped image into the trained neural network to infer an image at an output of the trained neural network (D1, the reconstructed image is denoised using the DL network 135. The result of which is a high-quality image 175, [0035]; D3, In step 103, the original CT image and/or the CT image copies are subjected to enhancement processing, whereby an enhancement processed CT image is obtained, [0037]), and for a voxel in the acquired image that has an intensity value outside of the determined at least one intensity bound value, backfilling the voxel in the acquired image into the inferred image to obtain a backfilled image, as the generated image-enhanced image (D3, the enhanced combined CT image and wherein resultX etc indicates the output of the processing step 104a-c for CT image copy number X: enhIm=maskBorgIm+mask1result1+ . . . +maskNresultN, [0054]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PING Y HSIEH whose telephone number is (571)270-3011. The examiner can normally be reached Monday-Friday, 9am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PING Y HSIEH/ Primary Examiner, Art Unit 2664
Prosecution Insights