DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1, 7, and 10-13 are amended. Claims 14-16 are added. Claims 1, 3-4, and 6-20 are pending in this application.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3-4, 6-13 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Hirsch et al., “Segmentation of MRI head anatomy using deep volumetric networks and multiple spatial priors”.
Regarding claim 1, Hirsch discloses an image diagnosis apparatus, comprising:
an acquirer to acquire a tomographic image including a region to be diagnosed of a subject (Section 1 Introduction; first paragraph and Section 2.8 Training and Testing Data; first paragraph; Clinical and basic research require segmentation of magnetic resonance images (MRIs) of human heads, including abnormal anatomies such as tumors or lesions; The training data consist of T1-weighted MRI scans from 4 healthy subjects and 43 individuals who suffered a stroke. The strokes occurred at least 6 month prior to the MRI scan, at which point the lesion is largely replaced by CSF. MRI scans from normal subjects were obtained on a 3T Siemens Trio scanner (Erlangen, Germany). The stroke scans were collected at Georgetown University and the University of North Carolina, Chapel Hill, also on a 3T Siemens Trio scanner. The trained network was also applied to MRI images of 47 patients with disorders of consciousness collected at the Pitié-Salpêtrière University Hospital in Paris, on a 3T General Electric Signa system (Milwaukee, Wisconsin)); and
a drawer to draw, based on the tomographic image acquired by the acquirer, a labeled image including the region to be diagnosed, the labeled image being partitioned according to classes each of which indicates one of a lesion area, a cavity area, a soft tissue area, a bone area, and a background area, the labeled image having a unique pixel value with respect to each class (Section 2.8 Training and Testing Data; second paragraph and Figure 5; Specifically, the 43 stroke heads are first segmented automatically and then manually corrected for errors in particular around the stroke lesions and boundaries between CSF, gray matter, and skull, resulting in seven classes (background, air cavities, skin, bone, CSF, white matter, and gray matter); The manual segmentation is on the first column, followed by the T1-weighted MRI that is used as an input for the network, next are segmentations from (a), (b) the detail + context network and (c) SPM8 compared to the segmentation from the Multiprior. Each color represents one of each of seven tissue-classes used for classification: black, background; brown, skin; yellow, bone/skull; green, air/sinus cavities; light blue, CSF; white, white matter; and gray, gray matter. Notice the large CSF-filled lesion in (a)), wherein
the drawer estimates, based on a model that is generated by machine learning and that, with respect to input of a pixel value of each pixel in a tomographic image, outputs a pixel value of each pixel in a labeled image, a pixel value of each pixel in a labeled image from the tomographic image acquired by the acquirer (Section 2.1 Detail CNN and Figure 1; Multiprior network structure. The detail network (black path) consists of a 3D CNN with eight layers. During training, this network takes as input a patch of 253 voxels around a target patch of 93 to be classified (green cube). The size of the convolutional kernels mapping between layer is indicated by numbers to the right. For instance, 33 × 50 × 30 indicates a 3D convolution kernel of size 33 transforming 30 features to 50 features. The “context” network (red path) is identical in structure to the “detail” network, except that it processes a downsampled version of a larger FOV of 573 voxels during training. It includes an upsampling layer at the end to merge features at the same scale as the detail network. Prior probabilities for the target patch are extracted from a TPM and added as input to the final classification (blue arrow). The “classification” network (purple) takes the concatenated output of all three pathways as input and classifies the target patch with three fully connected layers and no additional spatial mixing (kernel of size 13). After the entire image has been segmented, a 3D CRF processes the resulting output segmentation while taking the original input image into account (green arrow). Arrows indicate copying).
Regarding claim 3, the image diagnosis apparatus according to claim 1, Hirsch further discloses wherein the lesion area is a tumor area in a brain, the tomographic image is an image of a cross section of a brain of a subject, the cross section being obtained by slicing the brain in a transverse plane direction at a plurality of points, and the drawer draws a labeled image corresponding to each tomographic image acquired by the acquirer (Figures 3, 5-7, and 11-13).
Regarding claim 4, the image diagnosis apparatus according to claim 3, Hirsch further discloses wherein the tumor area is an area where a metastatic brain tumor has developed (Figure 5, last sentence).
Regarding claim 6, the image diagnosis apparatus according to claim 1, Hirsch further discloses wherein the image diagnosis apparatus further includes a trainer to generate the model by machine learning (Section 1 Introduction, last paragraph; Section 2.3 Classification Network; and Section 2.8 Training and Testing Data).
Regarding claim 7, Hirsch discloses an image diagnosis apparatus, comprising:
an acquirer to acquire a tomographic image including a region to be diagnosed of a subject (Section 1 Introduction; first paragraph and Section 2.8 Training and Testing Data; first paragraph; Clinical and basic research require segmentation of magnetic resonance images (MRIs) of human heads, including abnormal anatomies such as tumors or lesions; The training data consist of T1-weighted MRI scans from 4 healthy subjects and 43 individuals who suffered a stroke. The strokes occurred at least 6 month prior to the MRI scan, at which point the lesion is largely replaced by CSF. MRI scans from normal subjects were obtained on a 3T Siemens Trio scanner (Erlangen, Germany). The stroke scans were collected at Georgetown University and the University of North Carolina, Chapel Hill, also on a 3T Siemens Trio scanner. The trained network was also applied to MRI images of 47 patients with disorders of consciousness collected at the Pitié-Salpêtrière University Hospital in Paris, on a 3T General Electric Signa system (Milwaukee, Wisconsin));
a drawer to draw, based on the tomographic image acquired by the acquirer, a labeled image including the region to be diagnosed, the labeled image being partitioned according to classes each of which indicates one of at least a lesion area, a normal tissue, and a background area, the labeled image having a unique pixel value with respect to each class (Section 2.8 Training and Testing Data; second paragraph and Figure 5; Specifically, the 43 stroke heads are first segmented automatically and then manually corrected for errors in particular around the stroke lesions and boundaries between CSF, gray matter, and skull, resulting in seven classes (background, air cavities, skin, bone, CSF, white matter, and gray matter); The manual segmentation is on the first column, followed by the T1-weighted MRI that is used as an input for the network, next are segmentations from (a), (b) the detail + context network and (c) SPM8 compared to the segmentation from the Multiprior. Each color represents one of each of seven tissue-classes used for classification: black, background; brown, skin; yellow, bone/skull; green, air/sinus cavities; light blue, CSF; white, white matter; and gray, gray matter. Notice the large CSF-filled lesion in (a)), and
a trainer to generate a model that is generated by machine learning and that, with respect to input of a pixel value of each pixel in a tomographic image, outputs a pixel value of each pixel in a labeled image, wherein the drawer estimates, based on the model generated by the trainer, a pixel value of each pixel in a labeled image from a pixel value of each pixel in the tomographic image acquired by the acquirer (Section 2.1 Detail CNN and Figure 1; Multiprior network structure. The detail network (black path) consists of a 3D CNN with eight layers. During training, this network takes as input a patch of 253 voxels around a target patch of 93 to be classified (green cube). The size of the convolutional kernels mapping between layer is indicated by numbers to the right. For instance, 33 × 50 × 30 indicates a 3D convolution kernel of size 33 transforming 30 features to 50 features. The “context” network (red path) is identical in structure to the “detail” network, except that it processes a downsampled version of a larger FOV of 573 voxels during training. It includes an upsampling layer at the end to merge features at the same scale as the detail network. Prior probabilities for the target patch are extracted from a TPM and added as input to the final classification (blue arrow). The “classification” network (purple) takes the concatenated output of all three pathways as input and classifies the target patch with three fully connected layers and no additional spatial mixing (kernel of size 13). After the entire image has been segmented, a 3D CRF processes the resulting output segmentation while taking the original input image into account (green arrow). Arrows indicate copying), and
the trainer generates the model, using teacher data that include a pixel value of each pixel in a tomographic image as input data and a pixel value of each pixel in a labeled image as output data, the labeled image being generated based on the tomographic image and having a weight of each class adjusted based on a number of counted pixels of the class (Section 2.7 Cost Function and Section 2.8 Training and Testing Data; third paragraph; Training was set to reduce the generalized Dice loss between the predicted segmentation by the network and the ground-truth provided by the manual segmentations…The inner product · sums over all elements of the 3D volume. C is the total number of classes (C = 7 in this case); During network training, four heads were kept out for validation purposes, measuring generalization performance during training epochs and used to define the stopping point, i.e., the epoch with maximum Dice score on the validation set. This procedure was used for training all convolutional neural networks (CNNs) (Multiprior, DeepMedic, and U-Net variants)).
Regarding claim 8, the image diagnosis apparatus according to claim 6, Hirsch further discloses wherein the trainer generates the model, using teacher data generated based on a plurality of tomographic images that has cross sections obtained by slicing a brain of each of a plurality of subjects in a transverse plane direction at a plurality of points (Section 1 Introduction, last paragraph; Section 2.3 Classification Network; and Section 2.8 Training and Testing Data).
Regarding claim 9, the image diagnosis apparatus according to claim 1, Hirsch further discloses wherein the image diagnosis apparatus further includes an outputter to color-code a labeled image with respect to each class, the labeled image being drawn by the drawer (Figures 5-7 and 11-13).
Regarding claim 10, this claim recites substantially the same limitations that are performed by claim 1 above, and it is rejected for the same reasons.
Regarding claim 11, this claim recites substantially the same limitations that are performed by claim 7 above, and it is rejected for the same reasons.
Regarding claim 12, this claim recites substantially the same limitations that are performed by claim 1 above, and it is rejected for the same reasons.
Regarding claim 13, this claim recites substantially the same limitations that are performed by claim 7 above, and it is rejected for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 14-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hirsch et al., “Segmentation of MRI head anatomy using deep volumetric networks and multiple spatial priors” in view of Floyd et al., “Artificial Neural Networks for SPECT Image Reconstruction with Optimized Weighted Backprojection”.
Regarding claim 14, the image diagnosis apparatus of claim 6, Hirsch discloses wherein the drawer draws, using a neural network (Section 2.1 Detail CNN and Figure 1).
Hirsch discloses claim 14 as enumerated above, but Hirsch does not explicitly disclose an input layer including a plurality of neurons for input, an output layer including a plurality of neurons for output, and at least one intermediate layer, including a plurality of intermediate neurons, positioned between the input layer and the output layer, wherein connection states exist between the input layer, the output layer, and the intermediate layer as claimed.
However, Floyd discloses artificial neural network includes the input layer, the hidden layer, and the output layer. The input layer connected to the hidden layer and the hidden later connected to the output layer (Fig. 1; Section: Method, First and Second paragraphs).
Therefore, taking the combined disclosures of Hirsch and Floyd as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate artificial neural network includes the input layer, the hidden layer, and the output layer. The input layer connected to the hidden layer and the hidden later connected to the output layer as taught by Floyd into the invention of Hirsch for the benefit of representing the tomographic reconstruction process using the artificial neural network architecture (Floyd: Section: Discussion, First paragraph).
Regarding claim 15, the image diagnosis apparatus of claim 14, Floyd in the combination further disclose wherein a number of the plurality of neurons in the input layer corresponds to a number of pieces of input data (Fig. 1; Section: Method, First and Second paragraphs).
Regarding claim 16, the image diagnosis apparatus of claim 14, Floyd in the combination further disclose wherein a number of the plurality of neurons in the output layer corresponds to a number of pieces of output data (Fig. 1; Section: Method, First and Second paragraphs).
Regarding claim 17, the image diagnosis apparatus of claim 14, Floyd in the combination further disclose wherein each piece of input data in the input layer is input to respective intermediate neurons in the intermediate layer (Fig. 1; Section: Method, First and Second paragraphs).
Regarding claim 18, the image diagnosis apparatus of claim 14, Floyd in the combination further disclose wherein the trainer uses a plurality of data sets to adjust weighting coefficients that indicate the connection states between the input layer, the output layer, and the intermediate layer (Section: Introduction).
Regarding claim 19, the image diagnosis of claim 18, Floyd in the combination further disclose wherein based on the pixel values of the respective pixels in the labeled image (estimated values of output data) output from the neurons for output by inputting the pixel values of the respective pixels in the tomographic image (input data) to the neurons for input in the input layer and the pixel values of the respective pixels in the labeled image (set values of output data), a mean squared error (MSE) between the estimated values and the set values of the output data is calculated (Sections: Method and Result).
Regarding claim 20, the image diagnosis of claim 19, Floyd in the combination further disclose wherein by minimizing the MSE the weighting coefficients of the connection states between the input layer, the output layer, and the intermediate layer of the neural network are optimized (Sections: Introduction, Method, and Result).
Response to Arguments
Applicant's arguments filed 04/21/2026 have been fully considered but they are not persuasive.
Regarding independent claim 1, Applicant argues that the CSF class in the MRI images of Hirsh includes not only CSF regions altered by the stroke lesion but also CSF regions that normally exist in a healthy head (for example, a region between the skull and the brain), and these two types of regions cannot be distinguished. Examiner respectfully disagrees. As stated in the rejection above, Hirsch discloses specifically, the 43 stroke heads are first segmented automatically and then manually corrected for errors in particular around the stroke lesions and boundaries between CSF, gray matter, and skull, resulting in seven classes (background, air cavities, skin, bone, CSF, white matter, and gray matter); The manual segmentation is on the first column, followed by the T1-weighted MRI that is used as an input for the network, next are segmentations from (a), (b) the detail + context network and (c) SPM8 compared to the segmentation from the Multiprior. Each color represents one of each of seven tissue-classes used for classification: black, background; brown, skin; yellow, bone/skull; green, air/sinus cavities; light blue, CSF; white, white matter; and gray, gray matter. Notice the large CSF-filled lesion in (a)) (Section 2.8 Training and Testing Data; second paragraph and Figure 5). Hence, Hirsch discloses segmentation classes that include lesion-containing regions and output labeled images having unique pixel values corresponding to anatomical/lesion-related regions. Claim 1 does not expressly require the lesion region area to be mutually exclusive from other regions or uniquely isolate.
Therefore, the claimed limitation reads on the disclosure of Hirsch.
Regarding independent claim 7, Applicant argues that claim 7 recites "using teacher data that include a pixel value of each pixel in a tomographic image as input data and a pixel value of each pixel in a labeled image as output data, the labeled image being generated based on the tomographic image and having a weight of each class adjusted based on a number of counted pixels of the class" and Hirsch does not disclose "using teacher data that include a pixel value of each pixel in a tomographic image as input data and a pixel value of each pixel in a labeled image as output data, the labeled image being generated based on the tomographic image and having a weight of each class adjusted based on a number of counted pixels of the class".
However, Applicant fails to point out any specific error in the Examiner’s findings or provide any explanation as to why the disclosure in Hirsch (see Sections 2.7-2.8) does not meet this limitation. Under MPEP 714.04, a mere allegation that the prior art lacks a claim limitation, without distinctly pointing out the errors in the Examiner’s Action, is insufficient to overcome the rejection. Accordingly the rejection of claim 7 is maintained.
In view of the above arguments, the Examiner believes all rejections are proper and are maintained.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN D HUYNH whose telephone number is (571)270-1937. The examiner can normally be reached 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN D HUYNH/Primary Examiner, Art Unit 2665
Prosecution Insights