Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/06/2024 and 02/11/2025 are being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 8-12, 15, 17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chowdary et al. (A Multi-Task Learning Framework for Automated Segmentation and Classification of Breast Tumors From Ultrasound Images, published 2022), in view of Huang et al. (Automatic diagnosis of cardiac magnetic resonance images based on semi-supervised learning, published 23 May 2024).
Regarding Claim 1, Chowdary teaches “A method for
applying a multi-task deep learning model to the one or more(Chowdary, Materials and Methods; Residual-U-Net, paragraph 1 discloses “The proposed multi-task learning approach for segmentation and classification is shown in Figure 2”; where proposed multi-task learning approach is a multi-task deep learning model), “the multi-task deep learning model configured to simultaneously perform segmentation and disease detection” (Chowdary, Materials and Methods; Residual U-Net, paragraph 1 discloses “The proposed multi-task learning approach can perform segmentation and classification simultaneously”; where classification in Fig. 2 is a classification of benign or malignant, thus Chowdary teaches disease detection in the classification branch), “wherein the multi-task deep learning model comprises an encoder-decoder structure” (Chowdary, Materials and Methods; Residual U-Net, paragraph 2 discloses “The Residual-U-Net proposed in this work is a 9 level architecture consisting of 3 parts namely encoder, decoder, and a bridge”) “and a classification network” (Chowdary, Materials and Methods; Residual U-Net, paragraph 3 discloses “Whereas in our classification branch, the extracted features from the last block of encoder, bridge, and the first block of decoder is used for classification (Figure 2)”; where classification branch is a classification network), “wherein a compressed representation extracted by an encoder of the encoder-decoder structure is used for reconstructing a segmentation mask by the decoder of the encoder-decoder structure and as an input for the classification network for a classification of one or more diseases” (Chowdary, Materials and Methods; Residual U-Net, paragraph 3 discloses “For classifying the tumor as benign or malignant, the feature map from the last layer of the encoder block is taken and passed through a single dense layer for classification. Whereas in our classification branch, the extracted features from the last block of encoder, bridge, and the first block of decoder is used for classification (Figure 2)”; where cancer is one or more diseases; where a feature map is a compressed representation; where output in Figure 2, rightmost side, is a segmentation mask); “and
outputting, by the multi-task deep learning model the segmentation mask and the classification” (Chowdary, Materials and Methods; Residual U-Net, paragraph 2 discloses “After the last encoding unit, there is a 3×3 convolutional layer and a sigmoid activation layer to project the desired segmented image”; where projecting a segmented image is outputting a segmentation mask. Chowdary, Materials and Methods; Residual U-Net, paragraph 4 discloses “The first dense layer that receives the fused features consists of 256 units and is activated with ReLU function and the last dense layer contains 2 units and is activated with softmax function to predict the class of the input ultrasound image as malignant or benign”; where predicting the class is outputting a classification).
PNG
media_image1.png
581
938
media_image1.png
Greyscale
Figure 2 of Chowdary
Chowdary does not explicitly teach “acquiring one or more MR images of a patient” and “applying a multi-task deep learning model to the one or more MR images” (emphasis added).
However, in an analogous field of endeavor, “acquiring one or more MR images of a patient” (Huang, Section 2.1, paragraph 1 discloses “Cardiac magnetic resonance images (comprising end-diastolic and end-systolic phases) and partially annotated labels serve as inputs”) and “applying a multi-task deep learning model to the one or more MR images” (Huang, Section 2.1, paragraph 1 discloses “The optimal model from the training phase is utilized for classification inference on the testing set, resulting in disease predictions. Finally, segmentation evaluation and diagnostic assessment can be performed based on the segmentation and classification results.”)
PNG
media_image2.png
634
986
media_image2.png
Greyscale
Figure 1 of Huang
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Chowdary to incorporate the teachings of Huang by inputting cardiac magnetic resonance images to a machine learning model. The prior art contained a ‘base’ method upon which the claimed invention can be seen as an ‘improvement.’ That is, Huang teaches a method of automatic segmentation of cardiac images and auxiliary diagnosis. The claimed invention can be seen as an improvement because the segmentation and diagnosis steps are simultaneous. The prior art of Chowdary contained a known technique that is applicable to the base method; Chowdary teaches a multi-task learning approach for automatic segmentation and classification of a benign or malignant state of breast ultrasound images. One of ordinary skill in the art would have recognized that applying the known technique of simultaneous automatic segmentation and classification of ultrasound images to cardiac magnetic resonance images, as in Chowdary, to the cardiac MR images of Huang would have yielded predictable results and resulted in an improved system. Finally, one of ordinary skill in the art would be motivated to combine the references in order to improve efficiency: Chowdary, Conclusion, paragraph 1 discloses “Experimental results show that the proposed model is more efficient than the existing segmentation and classification methods in terms of accuracy and recall.”) Accordingly, the combination of Chowdary and Huang discloses the invention of Claim 1.
Regarding Claim 2, the combination of Chowdary and Huang teaches “The method of claim 1, wherein the one or more MR images comprise cardiac MR images of the patient, wherein the multi-task deep learning model is configured to perform myocardial segmentation and cardiac disease classification” (Huang, Section 1, paragraph 7 discloses “The proposed semi-supervised cardiac image automatic segmentation and auxiliary diagnosis model requires only a small amount of annotated image data and can achieve fully automated, high-precision cardiac image segmentation, clinical index calculation, feature extraction for classification, and disease prediction.”) The proposed combination as well as the motivation for combining the Chowdary and Huang references presented in the rejection of Claim 1, apply to Claim 2 and are incorporated herein by reference. Thus, the apparatus recited in Claim 2 is met by Chowdary and Huang.
Regarding Claim 3, the combination of Chowdary and Huang teaches “The method of claim 1, wherein the encoder-decoder structure comprises a DenseUNet architecture” (Chowdary, Materials and Methods, paragraph 2 discloses “As shown in Figure 1(b), shortcut connections or skip connections are the ones that skip one or more layers in the neural network.”)
Regarding Claim 4, the combination of Chowdary and Huang teach “The method of claim 1, wherein the classification network additionally uses one or more statistical features derived from the segmentation mask as an input” (Huang, Section 2.3, paragraph 6 discloses “By computing the average or standard deviation of MWT for each short-axis slice and aggregating them in the long-axis (LA) direction, it is feasible to estimate the variation in myocardial wall thickness. Consequently, there are four features describing the variation in myocardial wall thickness at both end-diastole and end-systole. Table 1 illustrates all cardiac disease classification features and their meanings employed in this study”; where average and standard deviation of MWT (myocardial wall thickness), obtained from myocardial segmentation, are statistical features derived from the segmentation masks; where classification features are inputs to the classifier structure of Huang; see Figure 3). The proposed combination as well as the motivation for combining the Chowdary and Huang references presented in the rejection of Claim 1, apply to Claim 4 and are incorporated herein by reference. Thus, the apparatus recited in Claim 4 is met by Chowdary and Huang.
PNG
media_image3.png
602
632
media_image3.png
Greyscale
Figure 3 of Huang
Regarding Claim 8, the combination of Chowdary and Huang teaches “The method of claim 1, further comprising: displaying the segmentation mask and/or the classification” (Chowdary, Materials and Methods; Residual U-Net, paragraph 2 discloses “After the last encoding unit, there is a 3×3 convolutional layer and a sigmoid activation layer to project the desired segmented image”; where projecting a desired segmented image is displaying the segmentation mask; see also Figure 2, where the output on the rightmost side is displayed).
Regarding Claim 9, Chowdary teaches “A system for
a memory configured to store a multi-task deep learning model configured to simultaneously perform segmentation and disease detection” (Chowdary, Materials and Methods; Residual U-Net, paragraph 1 discloses “The proposed multi-task learning approach can perform segmentation and classification simultaneously”; where classification in Fig. 2 is a classification of benign or malignant, thus Chowdary teaches disease detection in the classification branch), “wherein the multi-task deep learning model comprises an encoder-decoder structure” (Chowdary, Materials and Methods; Residual U-Net, paragraph 2 discloses “The Residual-U-Net proposed in this work is a 9 level architecture consisting of 3 parts namely encoder, decoder, and a bridge”) “and a classification network” (Chowdary, Materials and Methods; Residual U-Net, paragraph 3 discloses “Whereas in our classification branch, the extracted features from the last block of encoder, bridge, and the first block of decoder is used for classification (Figure 2)”; where classification branch is a classification network), “wherein a latent space extracted by an encoder of the encoder-decoder structure is used for reconstructing one or more segmentation masks by the decoder of the encoder-decoder structure and as an input for the classification network for a classification of one or more diseases” (Chowdary, Materials and Methods; Residual U-Net, paragraph 3 discloses “For classifying the tumor as benign or malignant, the feature map from the last layer of the encoder block is taken and passed through a single dense layer for classification. Whereas in our classification branch, the extracted features from the last block of encoder, bridge, and the first block of decoder is used for classification (Figure 2)”; where cancer is one or more diseases; where a feature map is a feature space where output in Figure 2, rightmost side, is a segmentation mask; where a feature map is in a latent space); “and
a processor configured to generate the one or more segmentation masks and the classification by inputting the cardiac image into the multi-task deep learning model” (Chowdary, Materials and Methods; Residual U-Net, paragraph 2 discloses “After the last encoding unit, there is a 3×3 convolutional layer and a sigmoid activation layer to project the desired segmented image”; where projecting a segmented image is outputting a segmentation mask. Chowdary, Materials and Methods; Residual U-Net, paragraph 4 discloses “The first dense layer that receives the fused features consists of 256 units and is activated with ReLU function and the last dense layer contains 2 units and is activated with softmax function to predict the class of the input ultrasound image as malignant or benign”; where predicting the class is outputting a classification).
Chowdary does not explicitly teach “A system for magnetic resonance (MR) image analysis” and “a medical imaging device configured to acquire a cardiac image of a patient” (emphasis added).
However, in an analogous field of endeavor, “A system for magnetic resonance (MR) image analysis” and “a medical imaging device configured to acquire a cardiac image of a patient” (Huang, Section 2.1, paragraph 1 discloses “Cardiac magnetic resonance images (comprising end-diastolic and end-systolic phases) and partially annotated labels serve as inputs.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Chowdary to incorporate the teachings of Huang by inputting cardiac magnetic resonance images to a machine learning model. The prior art contained a ‘base’ method upon which the claimed invention can be seen as an ‘improvement.’ That is, Huang teaches a method of automatic segmentation of cardiac images and auxiliary diagnosis. The claimed invention can be seen as an improvement because the segmentation and diagnosis steps are simultaneous. The prior art of Chowdary contained a known technique that is applicable to the base method; Chowdary teaches a multi-task learning approach for automatic segmentation and classification of a benign or malignant state of breast ultrasound images. One of ordinary skill in the art would have recognized that applying the known technique of simultaneous automatic segmentation and classification of ultrasound images to cardiac magnetic resonance images, as in Huang, would have yielded predictable results and resulted in an improved system. Finally, one of ordinary skill in the art would be motivated to combine the references in order to improve efficiency: Huang, Conclusion, paragraph 1 discloses “Experimental results show that the proposed model is more efficient than the existing segmentation and classification methods in terms of accuracy and recall.”) Accordingly, the combination of Chowdary and Huang discloses the invention of Claim 9.
Regarding Claim 10, the combination of Chowdary and Huang teaches “The system of claim 9, further comprising: a display configured to display the one or more segmentation masks and/or the classification” (Chowdary, Materials and Methods; Residual U-Net, paragraph 2 discloses “After the last encoding unit, there is a 3×3 convolutional layer and a sigmoid activation layer to project the desired segmented image”; where projecting a desired segmented image is displaying the segmentation mask; see also Figure 2, where the output on the rightmost side is displayed).
Regarding Claim 11, the combination of Chowdary and Huang teaches “The system of claim 9, wherein the multi-task deep learning model comprises a DenseUNet architecture with dense blocks comprising multiple convolutional layers where each layer receives inputs from all previous layers” (Chowdary, Materials and Methods, paragraph 2 discloses “As shown in Figure 1(b), shortcut connections or skip connections are the ones that skip one or more layers in the neural network.”)
Regarding Claim 12, the combination of Chowdary and Huang teaches “The system of claim 9, wherein the classification network further takes as input one or more statistical features derived from the one or more segmentation masks” (Huang, Section 2.3, paragraph 6 discloses “By computing the average or standard deviation of MWT for each short-axis slice and aggregating them in the long-axis (LA) direction, it is feasible to estimate the variation in myocardial wall thickness. Consequently, there are four features describing the variation in myocardial wall thickness at both end-diastole and end-systole. Table 1 illustrates all cardiac disease classification features and their meanings employed in this study”; where average and standard deviation of MWT (myocardial wall thickness), obtained from myocardial segmentation, are statistical features derived from the segmentation masks; where classification features are inputs to the classifier structure of Huang; see Figure 3). The proposed combination as well as the motivation for combining the Chowdary and Huang references presented in the rejection of Claim 1, apply to Claim 12 and are incorporated herein by reference. Thus, the apparatus recited in Claim 12 is met by Chowdary and Huang.
Regarding Claim 15, the combination of Chowdary and Huang teaches “The system of claim 9, wherein the classification network comprises a plurality of linear layers, with first layers of the plurality of linear layers followed by a ReLU activation function and a last layer followed by a softmax layer for multi-class classification” (Chowdary, Materials and Methods; Residual-U-Net, paragraph 4 discloses “The first dense layer that receives the fused features consists of 256 units and is activated with ReLU function and the last dense layer contains 2 units and is activated with softmax function to predict the class of the input ultrasound image as malignant or benign”).
Regarding Claim 17, Chowdary teaches “A method for configuring a multi-task deep learning model, the method comprising:
acquiring training data comprising a plurality of (Chowdary, Experimental Setting; paragraph 1 discloses “the same experimental set-up is followed, and only benign and malignant cases are considered for training.” Chowdary, Experimental Setting; Evaluation Metrics discloses “In equations (6) and (7), A and B are the ground truth and segmentation results”);
“inputting (Chowdary, Materials and Methods; Residual U-Net, paragraph 1 discloses “The proposed multi-task learning approach can perform segmentation and classification simultaneously”; where classification in Fig. 2 is a classification of benign or malignant, thus Chowdary teaches a disease classification branch);
“outputting, by the multi-task deep learning model, a segmentation mask and a disease classification” (Chowdary, Materials and Methods; Residual U-Net, paragraph 2 discloses “After the last encoding unit, there is a 3×3 convolutional layer and a sigmoid activation layer to project the desired segmented image”; where projecting a segmented image is outputting a segmentation mask. Chowdary, Materials and Methods; Residual U-Net, paragraph 4 discloses “The first dense layer that receives the fused features consists of 256 units and is activated with ReLU function and the last dense layer contains 2 units and is activated with softmax function to predict the class of the input ultrasound image as malignant or benign”; where predicting the class is outputting a classification);
“adjusting weights of the segmentation branch based on a comparison of the segmentation mask to the related ground truth segmentation mask” (Chowdary, Materials and Methods; Loss Function, paragraph 2 discloses “In the above equation, PS represents the predicted segmentation result, YS represents the actual segmented result, and Segloss represents the segmentation loss”; where training using segmentation loss is a comparison of segmentation mask to a ground truth segmentation mask; where training by segmentation loss is adjusting weights of the segmentation branch);
“adjusting weights of the disease classification branch based on a comparison of the disease classification to the related ground truth disease classification” (Chowdary, Materials and Methods; Loss Function, paragraph 3 discloses “To deal with this imbalancing problem, the weighted focal loss45 is used for the classification task and is shown in equation (3)”; where a weighted focal loss is a comparison of disease classification to ground truth disease classification; Chowdary, Materials and Methods; Loss Function, also discloses:
PNG
media_image4.png
285
456
media_image4.png
Greyscale
Excerpt of Chowdary
; where training by classification loss is adjusting weights of disease classification branch)
“repeating inputting, outputting, adjusting, and adjusting for a plurality of iterations” (Chowdary, Experimental Setting, paragraph 1 discloses “For each fold, the model is trained for 500 epochs with batch size of 16 and the learning rate is set to 0.0001 which further decreases by one tenth after every 20 epochs”); “and
outputting a trained multi-task deep learning model” (see Figure 2 of Chowdary; where proposed multi-task learning approach is a trained multi-task deep learning model).
Chowdary does not explicitly teach “acquiring training data comprising a plurality of cardiac magnetic resonance (MR) images” and “inputting a cardiac MR image into the multi-task deep learning model” (emphasis added).
However, in an analogous field of endeavor, “acquiring training data comprising a plurality of cardiac magnetic resonance (MR) images” (Huang, Section 2.1, paragraph 1 discloses “Cardiac magnetic resonance images (comprising end-diastolic and end-systolic phases) and partially annotated labels serve as inputs”) and “inputting a cardiac MR image into the multi-task deep learning model” (Huang, Section 2.1, paragraph 1 discloses “Cardiac magnetic resonance images (comprising end-diastolic and end-systolic phases) and partially annotated labels serve as inputs.” Huang, Section 2.1, paragraph 1 discloses “The optimal model from the training phase is utilized for classification inference on the testing set, resulting in disease predictions. Finally, segmentation evaluation and diagnostic assessment can be performed based on the segmentation and classification results.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Chowdary to incorporate the teachings of Huang by inputting cardiac magnetic resonance images to a machine learning model. The prior art contained a ‘base’ method upon which the claimed invention can be seen as an ‘improvement.’ That is, Huang teaches a method of automatic segmentation of cardiac images and auxiliary diagnosis. The claimed invention can be seen as an improvement because the segmentation and diagnosis steps are simultaneous. The prior art of Chowdary contained a known technique that is applicable to the base method; Chowdary teaches a multi-task learning approach for automatic segmentation and classification of a benign or malignant state of breast ultrasound images. One of ordinary skill in the art would have recognized that applying the known technique of simultaneous automatic segmentation and classification of ultrasound images to cardiac magnetic resonance images, as in Huang, would have yielded predictable results and resulted in an improved system. Finally, one of ordinary skill in the art would be motivated to combine the references in order to improve efficiency: Huang, Conclusion, paragraph 1 discloses “Experimental results show that the proposed model is more efficient than the existing segmentation and classification methods in terms of accuracy and recall.”) Accordingly, the combination of Chowdary and Huang discloses the invention of Claim 17.
Regarding Claim 19, the combination of Chowdary and Huang teaches “The method of claim 17, wherein the segmentation branch comprises a DenseUNet architecture” (Chowdary, Materials and Methods, paragraph 2 discloses “As shown in Figure 1(b), shortcut connections or skip connections are the ones that skip one or more layers in the neural network.”)
Regarding Claim 20, the combination of Chowdary and Huang teaches “The method of claim 17, wherein the disease classification branch further takes as input one or more statistical features derived from the segmentation mask” (Huang, Section 2.3, paragraph 6 discloses “By computing the average or standard deviation of MWT for each short-axis slice and aggregating them in the long-axis (LA) direction, it is feasible to estimate the variation in myocardial wall thickness. Consequently, there are four features describing the variation in myocardial wall thickness at both end-diastole and end-systole”; where average and standard deviation of MWT (myocardial wall thickness), obtained from myocardial segmentation, are statistical features derived from the segmentation masks; where classification features are inputs to the classifier structure of Huang; see Figure 3). The proposed combination as well as the motivation for combining the Chowdary and Huang references presented in the rejection of Claim 1, apply to Claim 20 and are incorporated herein by reference. Thus, the apparatus recited in Claim 20 is met by Chowdary and Huang.
Claims 6, 7, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Chowdary et al. (A Multi-Task Learning Framework for Automated Segmentation and Classification of Breast Tumors From Ultrasound Images, published 2022), in view of Huang et al. (Automatic diagnosis of cardiac magnetic resonance images based on semi-supervised learning, published 23 May 2024), further in view of Punn et al. (Modality specific U-Net variants for biomedical image segmentation: A survey, published 2022).
Regarding Claim 6, the combination of Chowdary and Huang does not explicitly teach “The method of claim 1, wherein the multi-task deep learning model is trained using an alternating weight update strategy for the encoder-decoder structure and the classification network.”
However, in an analogous field of endeavor, Punn teaches “The method of claim 1, wherein the multi-task deep learning model is trained using an alternating weight update strategy for the encoder-decoder structure and the classification network” (Punn, page 33, paragraph 1 discloses “For the training phase, most models employed a hybrid loss function that combines the binary cross entropy loss with dice similarity coefficient loss or with Jaccard loss, which tends to better penalize the false positive and false negative predictions”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Chowdary and Huang to incorporate the teachings of Punn by using a hybrid loss function. Punn is also directed to U-Net architectures for biomedical image segmentation: Punn, page 33, paragraph 1 discloses “In the most recent developments of U-Net based biomedical image segmentation models following observations are drawn.” One of ordinary skill in the art would be motivated to combine the references in order to better penalize false positive and false negative predictions: Punn, page 33, paragraph 1 discloses “which tends to better penalize the false positive and false negative predictions.” Accordingly, the combination of Chowdary, Huang, and Punn discloses the invention of Claim 6.
Regarding Claim 7, the combination of Chowdary, Huang, and Punn teaches “The method of claim 6, wherein the alternating weight update strategy uses a Jaccard loss for the encoder-decoder structure and then a binary cross-entropy loss for the classification network” (Punn, page 33, paragraph 1 discloses “For the training phase, most models employed a hybrid loss function that combines the binary cross entropy loss with dice similarity coefficient loss or with Jaccard loss, which tends to better penalize the false positive and false negative predictions”). The proposed combination as well as the motivation for combining the Chowdary and Huang references presented in the rejection of Claim 6, apply to Claim 7 and are incorporated herein by reference. Thus, the apparatus recited in Claim 20 is met by Chowdary, Huang, and Punn.
Regarding Claim 14, the combination of Chowdary and Huang does not explicitly teach “The system of claim 9, wherein the multi-task deep learning model is trained using an alternating weight update strategy for the encoder-decoder structure and the classification network.”
However, in an analogous field of endeavor, Punn teaches “The system of claim 9, wherein the multi-task deep learning model is trained using an alternating weight update strategy for the encoder-decoder structure and the classification network” (Punn, page 33, paragraph 1 discloses “For the training phase, most models employed a hybrid loss function that combines the binary cross entropy loss with dice similarity coefficient loss or with Jaccard loss, which tends to better penalize the false positive and false negative predictions”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Chowdary and Huang to incorporate the teachings of Punn by using a hybrid loss function. Punn is also directed to U-Net architectures for biomedical image segmentation: Punn, page 33, paragraph 1 discloses “In the most recent developments of U-Net based biomedical image segmentation models following observations are drawn.” One of ordinary skill in the art would be motivated to combine the references in order to better penalize false positive and false negative predictions: Punn, page 33, paragraph 1 discloses “which tends to better penalize the false positive and false negative predictions.” Accordingly, the combination of Chowdary, Huang, and Punn discloses the invention of Claim 14.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Chowdary et al. (A Multi-Task Learning Framework for Automated Segmentation and Classification of Breast Tumors From Ultrasound Images, published 2022), in view of Huang et al. (Automatic diagnosis of cardiac magnetic resonance images based on semi-supervised learning, published 23 May 2024), further in view of Wilson et al. (US 2024/0169525 A1).
Regarding Claim 13, the combination of Chowdary and Huang does not explicitly teach “The system of claim 12, wherein the statistical features comprise at least one of a mean intensity, a median intensity, or lower and upper quartile intensity that are derived from an image grey value histogram of the one or more segmentation masks.”
However, in an analogous field of endeavor, Wilson teaches “The system of claim 12, wherein the statistical features comprise at least one of a mean intensity, a median intensity, or lower and upper quartile intensity that are derived from an image grey value histogram of the one or more segmentation masks” (Wilson, [0061] discloses “In some embodiments, the plurality of CCTA PCAT features 523 are extracted from voxels within one or more binary masks in the one or more CCTA images 502… In some embodiments, the plurality of CCTA PCAT features 523 may be extracted in 3 categories: intensity features, traditional morphology features, and texture features. The intensity features may include histogram related features, such as minimum, mean, standard deviation, median, maximum value, skewness, kurtosis, entropy, potentially large bin histograms within fat region, and/or the like”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Chowdary and Huang to incorporate the teachings of Wilson by determining histogram related features. Wilson is also directed to medical image segmentation; it would be obvious to one of ordinary skill in the art that the teachings of Wilson with respect to computed tomography angiography (CCTA) images would be applicable to the magnetic resonance images of Huang; that is, using a known technique of intensity feature determination to improve a similar method in a similar way would be obvious to one of ordinary skill in the art. One of ordinary skill in the art would be motivated to combine the references in order to select relevant features: Wilson, [0061] discloses “the plurality of non-confounding PCAT features 110 may be selected at least in-part based upon a plurality of CCTA PCAT features 523 identified from the one or more CCTA images 502 to improve selection of relevant features.” Accordingly, the combination of Chowdary, Huang, and Wilson discloses the invention of Claim 13.
Allowable Subject Matter
Claims 5, 16, and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 5, none of the previously cited prior art explicitly teaches “The method of claim 4, wherein the one or more statistical features are integrated at the compressed representation of the encoder-decoder structure.” That is, although Chowdary and Huang teach compressed representations (see Chowdary, Fig. 2, above, and see Figure 2 of Huang, below), and Huang teaches determining statistical features of determined mask images (Huang, Section 2.3, paragraph 6 discloses “By computing the average or standard deviation of MWT for each short-axis slice and aggregating them in the long-axis (LA) direction, it is feasible to estimate the variation in myocardial wall thickness. Consequently, there are four features describing the variation in myocardial wall thickness at both end-diastole and end-systole”), none of the previously cited references explicitly teaches integrating the statistical features derived from the segmentation mask at the compressed representation, or in a latent representation, or in feature space. Huang teaches determining statistical features after cardiac image segmentation and does not explicitly teach integrating the features at a compressed representation. Thus, none of the previously cited references, alone or in combination, provide a motivation to teach the ordered combination of “The method of claim 4, wherein the one or more statistical features are integrated at the compressed representation of the encoder-decoder structure.”
PNG
media_image5.png
507
977
media_image5.png
Greyscale
Figure 2 of Huang
Regarding Claim 16, the combination of Chowdary, Huang, and Punn does not explicitly teach “The system of claim 15, wherein the classification network includes a number of inputs equal to a number of features available from the latent space at a bottleneck of the encoder-decoder structure plus a number of statistical features derived from the one or more segmentation masks.”
That is, although Chowdary and Huang teach a latent space and classification networks (see Chowdary, Fig. 2, above, and see Figure 2 of Huang, below), and Huang teaches determining statistical features of determined mask images (Huang, Section 2.3, paragraph 6 discloses “By computing the average or standard deviation of MWT for each short-axis slice and aggregating them in the long-axis (LA) direction, it is feasible to estimate the variation in myocardial wall thickness. Consequently, there are four features describing the variation in myocardial wall thickness at both end-diastole and end-systole”), none of the previously cited references explicitly teaches the number of inputs to a classification network being equal to a number of latent space features plus a number of statistical features. That is, although the prior art teaches both a number of latent space features and a number of statistical features, none of the previously cited prior art explicitly teaches a number of inputs to a classification network equal to the sum of these numbers.
Thus, none of the previously cited prior art references, alone or in combination, provide a motivation to teach the ordered combination of “The system of claim 15, wherein the classification network includes a number of inputs equal to a number of features available from the latent space at a bottleneck of the encoder-decoder structure plus a number of statistical features derived from the one or more segmentation masks.”
Regarding Claim 18, the combination of Chowdary, Huang, and Punn does not explicitly teach “The method of claim 17, wherein the comparison of the segmentation mask to the related ground truth segmentation mask provides a Jaccard loss for segmentation and the comparison of the disease classification to the related ground truth disease classification provides a binary cross-entropy loss for classification.”
Punn, page 33, paragraph 1 discloses “For the training phase, most models employed a hybrid loss function that combines the binary cross entropy loss with dice similarity coefficient loss or with Jaccard loss, which tends to better penalize the false positive and false negative predictions,” however Punn does not explicitly teach a Jaccard loss for segmentation and comparison of disease classification and a binary cross-entropy loss for classification. That is, although Jaccard loss and binary cross-entropy loss are known in the art of U-Net architectures for biomedical image processing, none of the prior art explicitly teaches the respective losses for segmentation and classification purposes separately.
Thus, none of the previously cited prior art references, alone or in combination, provide a motivation to teach the ordered combination of “The method of claim 17, wherein the comparison of the segmentation mask to the related ground truth segmentation mask provides a Jaccard loss for segmentation and the comparison of the disease classification to the related ground truth disease classification provides a binary cross-entropy loss for classification.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Wang et al. (Simultaneous Segmentation and Classification of Bone Surfaces from Ultrasound Using a Multi-feature Guided CNN, published 2018) discloses a method of segmenting and classifying ultrasound bone scans using an encoder-decoder U-Net framework and bone type classification stage (see Fig. 1 of Wang).
PNG
media_image6.png
411
758
media_image6.png
Greyscale
Fig. 1 of Wang
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLINE TABANCAY DUFFY whose telephone number is (703)756-1859. The examiner can normally be reached Monday - Friday 8:00 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at 5712723382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLINE TABANCAY DUFFY/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights