DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 4, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Invoked despite absence of “means”
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a computing device” in claims 14-15
NOTE: claim 16 provides structure thus, is not interpreted under 35 USC 112(f)
“an input module” in claims 14 and 16-17
NOTE: claim 15 provides structure thus, is not interpreted under 35 USC 112(f)
“an output module” in claims 14-17
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 9, 13-14 and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by CHENGJIA WANG ET AL: "A two-stage 3D Unet framework for multi-class segmentation on full resolution image", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 12 April 2018 (2018-04-12), XP080870068, (hereinafter Wang).
Regarding independent claim 1, Wang discloses A computer-implemented method (page 8, “the pro-posed method can directly make prediction for data with original resolution due to its SRCNN-inspired architecture;” neural networks are well known to be computer-implemented) for generating at least one at least 3- dimensional medical image segmentation for at least one structure of a human heart (page 2, section 1, “The purpose of this study is to develop a DCNN which can perform multi-class segmentation on full-resolution volumetric CT;” Table 2-3 display the networks as success in segmenting various heart structures (Page 6 has abbreviations coordinating to the table)), comprising the steps of:
providing a first n-dimensional medical image comprising the at least one structure of the human heart, wherein n=3 or n=4 (page 3, “The input of this network is a two-channel 4D volume”);
generating a segmentation of at least part of the provided first n-dimensional medical image using at least one first trained artificial neural network (page 3, “The architecture of Net2 is inspired by the deep Super-Resolution Convolu-tional Neural Network (SRCNN) [15] with skip connections and recursive units [16]. The input of this network is a two-channel 4D volume composed by the output of Net1 and the original data;” page 5, “The output of Net2 is the segmentation of the (K+1)/2th slice of a input subvolume;” Net2 is referenced as the neural network that takes an input 4D volume to perform the segmentation; Net2 is the trained ANN: page 4, “The two U-Net-like DCNNs of the proposed model are flexible enough to be trained either separately or end-to-end with changing sizes of input data.”), wherein the at least one first trained artificial neural network is configured as a convolutional processing network with U-net architecture (page 2, “we propose a two-stage DCNN framework which is built by concatenating two U-Net-like networks;” Net1 and Net2 are each U-net networks; DCNN is read as “deep convolutional neural network (see also page 2)”), wherein each convolutional processing network with U-net architecture comprises:
a down-sampling path comprising at least two processing convolutional blocks and at least two down-sampling blocks (down-sampling is also known as pooling layers; as seen in Figure 1, Network2 has more than 2 pooling layers (which are further read as blocks));
an up-sampling path comprising at least two processing convolutional blocks and at least two up-sampling blocks (up-sampling is also known as unpooling layer; as seen in Figure 1, Network 2 has more than 2 unpooling layers (which are further read as blocks));
wherein the down-sampling path generates a direct input and/or an indirect input for the up-sampling path (Figure 1, the pooling flows to the unpooling which is read as an input for upsampling); and
generating at least one at least 3-dimensional medical image segmentation for the at least one structure of the human heart based at least on the segmentation generated by the at least one first trained artificial neural network (page 5, “The output of Net2 is the segmentation of the (K+1)/2th slice of a input subvolume;” Net2 is referenced as the neural network that takes an input 4D volume; page 3, “The architecture of Net2 is inspired by the deep Super-Resolution Convolu-tional Neural Network (SRCNN) [15] with skip connections and recursive units [16]. The input of this network is a two-channel 4D volume composed by the output of Net1 and the original data.” ).
Regarding dependent claim 3, the rejection of claim 1 is incorporated herein. Additionally, Wang further discloses wherein the at least one structure comprises at least one valve structure, at least one blood cavity (page 6, section 3, “anatomical structures which are manually delineated include, the left ventricle blood cavity (LV),…, the right ventricle blood cavity (RV),”), at least one muscle tissue structure (page 6, section 3, “anatomical structures which are manually delineated include, the left ventricle blood cavity (LV), the myocardium of the left ventricle (Myo), the right ventricle blood cavity (RV), the left atrium blood cavity (LA), the right atrium blood cavity (RA), the ascending aorta (AA) and the pulmonary artery (PA);” these structures are of the heart, which is inherently a muscle tissue structure; the abbreviations correspond to Table 2-3 which show how Net2 has performed in segmenting the various structures), at least one implant and/or at least one anomaly (page 1, section 1, “Segmenting the whole heart structures from CT and MRI data is a necessary step for pre-precedural planing of cardiovascular diseases;” page 6, section 3, “anatomical structures which are manually delineated include, the left ventricle blood cavity (LV), the myocardium of the left ventricle (Myo), the right ventricle blood cavity (RV), the left atrium blood cavity (LA), the right atrium blood cavity (RA), the ascending aorta (AA) and the pulmonary artery (PA);” diseases require that there is some sort of difference or anomaly as compared to a healthy patient; as such, one of the segmented structures could contain an anomaly).
Regarding dependent claim 9, the rejection of claim 1 is incorporated herein. Additionally, Wang further discloses wherein at least one processing block in at least one of the trained artificial neural networks comprises a bottleneck structure in which a r x r x r 3-dimensional convolution layer follows an q x q x q 3-dimensional convolution layer with a reduced number output channels compared to a number of input channels of the at least one processing block, wherein rand q are integers and wherein q is smaller than r (page 3, “The convolutional kernel size in the contracting path is 3x3x3, and 5x5x5 in the expansive path.” q < r is read as 3 < 5 – the expansive path is read as following the contracting path).
Regarding dependent claim 13, the rejection of claim 1 is incorporated herein. Additionally, Wang further discloses wherein a plurality of first trained artificial neural networks which are differently configured and/or differently trained is provided (Wang discloses two different types of neural networks (see Network 1 and Network 2 in Figure 1), they operate on different data sets, and thus must be trained differently.);
wherein at least two of the plurality of the first trained artificial neural networks are configured as a respective convolutional processing network with U-net architecture (figure 1, “The concatenated U-Net architecture proposed in this work”);
wherein a respective candidate segmentation of at least part of the provided first n- dimensional medical image is generated by each of the plurality of provided first trained artificial neural networks (page 5, “The output of Net2 is the segmentation of the (K+1)/2th slice of a input subvolume;” Net2 is referenced as the neural network that takes an input 4D volume; page 3, “The architecture of Net2 is inspired by the deep Super-Resolution Convolu-tional Neural Network (SRCNN) [15] with skip connections and recursive units [16]. The input of this network is a two-channel 4D volume composed by the output of Net1 and the original data.”); and
wherein the at least 3-dimensional medical image segmentation is generated based on the generated candidate segmentations (page 5, “The output of Net2 is the segmentation of the (K+1)/2th slice of a input subvolume;”).
Regarding dependent claim 14, the rejection of claim 1 applies directly. Additionally, Wang further discloses A system for generating at least one at least 3-dimensional medical image segmentation for at least one structure of a human heart (page 8, “the pro-posed method can directly make prediction for data with original resolution due to its SRCNN-inspired architecture;” neural networks are well known to be computer-implemented and involve a system (i.e. the computer); page 2, section 1, “The purpose of this study is to develop a DCNN which can perform multi-class segmentation on full-resolution volumetric CT;” Table 2-3 display the networks as success in segmenting various heart structures (Page 6 has abbreviations coordinating to the table)), comprising:
a computing device configured to implement (abstract, “we developed a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution;” Unets are types of neural networks which are well known to be implemented on computing devices; computing device is read as the implementation of a neural network on a computer):
an input module configured to provide a first n-dimensional medical image of the at least one structure of the human heart, wherein n=3 or n=4 (input module is further read as the entity receiving the two-channel volume (see Figure 1 and page 3)see claim 1 analysis)); and
a controller implementing at least one first trained artificial neural network, wherein the trained artificial neural network is configured as a convolutional processing network with U-net architecture, wherein each convolutional processing network with U-net architecture comprises (a controller is read as the computer the neural network is implemented on; see claim 1 analysis):
a down-sampling path comprising at least two processing convolutional blocks and at least two down-sampling blocks (see claim 1 analysis);
an up-sampling path comprising at least two processing convolutional blocks and at least two up-sampling blocks (see claim 1 analysis);
wherein the down-sampling path generates a direct input and/or an indirect input for the up-sampling path (see claim 1 analysis);
wherein the first trained artificial neural network is trained and configured to generate a segmentation of at least part of the provided first n-dimensional medical image; and
an output module configured to generate at least one 3-dimensional medical image segmentation based at least on the segmentation generated by the at least one first trained artificial neural network (the output module is read as the images seen generated from Network 2; since this system is used for segmentation, which allows for diagnosis, the images must be output for display; see also Figure 2; see claim 1 analysis).
Regarding dependent claim 18, the rejection of claim 1 is incorporated herein. Additionally, Wang further discloses A non-transitory computer-readable data storage medium comprising executable program code configured to, when executed, perform the method according to claim 1 (page 2, section 1, “The purpose of this study is to develop a DCNN which can perform multi-class segmentation on full-resolution volumetric CT;” neural networks are well known to be coded and implemented on computers, as such, requiring non-transitory computer-readable storage media; see claim 1 analysis).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4-7 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Wang as applied to claim 1 above, and further in view of Mahendra Khened, Varghese Alex Kollerathu, Ganapathy Krishnamurthi, Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers, Medical Image Analysis, Volume 51, Published online October 19, 2018, Pages 21-45, ISSN 1361-8415, https://doi.org/10.1016/j.media.2018.10.004. (https://www.sciencedirect.com/science/article/pii/S136184151830848X) (hereinafter Mahendra).
Regarding dependent claim 4, the rejection of claim 1 is incorporated herein. Additionally, Wang discloses wherein:
as the at least one at least 3-dimensional medical image segmentation, a 4-dimensional medical image segmentation is generated (page 3, “The input of this network is a two-channel 4D volume” page 2, section 1, “The purpose of this study is to develop a DCNN which can perform multi-class segmentation on full-resolution volumetric CT;” page 8, table 2, “Comparison of CT segmentation results obtained by 3D U-Net, and our proposed Net1 and Net2.”);
the first n-dimensional medical image comprises a plurality of 3-dimensional medical images, each for a different effective time point (page 3, “The input of this network is a two-channel 4D volume”). Wang fails to explicitly disclose as further recited.
However, Mahendra discloses wherein:
a respective segmentation for at least two of the plurality of 3-dimensional medical images is independently generated by the at least one first trained artificial neural network (page 5, “We extensively validated our proposed network on two cardiac segmentation tasks: (i) segmentation of left ventricle (LV), right ventricle (RV) and myocardium (MYO) from multi-slice cine MR images for both end-diastolic (ED) and end-systolic (ES) phase instances;” page 7, “ROI detection involved spatio-temporal statistical analysis of cardiac phases and circular Hough transform (Duda, Hart, 1972, Korshunova, Burms, Degrave, Dambre) to delineate the heart structures from the surrounding tissues;” the phases are their own segmentations (i.e. the ED and ES phases are segmented separately)); and
the 4-dimensional medical image segmentation comprises at least two of the independently generated segmentations for the plurality of 3-dimensional medical images (page 5, “We extensively validated our proposed network on two cardiac segmentation tasks: (i) segmentation of left ventricle (LV), right ventricle (RV) and myocardium (MYO) from multi-slice cine MR images for both end-diastolic (ED) and end-systolic (ES) phase instances;” the slices of the cine (i.e. 3D MRI over time) are read as the 4D segmentation (i.e. the 3D segmentations across the phases ).
Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). Mahendra is directed toward “computational efficacy of incorporating conventional computer vision techniques for region of interest detection in an end-to-end deep learning based segmentation framework. From the segmentation maps we extract clinically relevant cardiac parameters and hand-craft features which reflect the clinical diagnostic analysis and train an ensemble system for cardiac disease classification (abstract).” As such, both Wang and Mahendra are directed toward similar methods of endeavor of using deep learning to segment structures in cardiac systems and further providing information required for diagnosis. Further, it is well known that the heart changes positions and moves throughout time series imaging. A segmentation at one phase, may not be similar or reflect a segmentation at another phase. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Mahendra in order to ensure segmentations of one phase do not affect segmentations of another phase.
Regarding dependent claim 5, the rejection of claim 1 is incorporated herein. Additionally, Wang discloses wherein:
as the at least one at least 3-dimensional medical image segmentation, a 4-dimensional medical image segmentation is generated (page 3, “The input of this network is a two-channel 4D volume” page 2, section 1, “The purpose of this study is to develop a DCNN which can perform multi-class segmentation on full-resolution volumetric CT;” page 8, table 2, “Comparison of CT segmentation results obtained by 3D U-Net, and our proposed Net1 and Net2.”). However, Wang fails to explicitly disclose as further recited.
However, Mahendra discloses wherein:
the method further comprises the step of generating a 3-dimensional medical image segmentation corresponding to a first effective time point within the 4-dimensional medical image segmentation to be generated (page 5, “We extensively validated our proposed network on two cardiac segmentation tasks: (i) segmentation of left ventricle (LV), right ventricle (RV) and myocardium (MYO) from multi-slice cine MR images for both end-diastolic (ED) and end-systolic (ES) phase instances;” first effective time point is read as end-diastolic phase; page 25, “For a typical 4D MR sequence with data dimension of 256 × 256 × 10 × 30 (Height × Width × Slices × Phases) our approach took approximately 3 seconds for ROI detection and 7 seconds for segmentation.”),
wherein at least one 3-dimensional medical image segmentation corresponding to at least one second effective time point within the 4-dimensional medical image segmentation to be generated is generated by the at least one first trained artificial neural network based on at least one output (page 5, “We extensively validated our proposed network on two cardiac segmentation tasks: (i) segmentation of left ventricle (LV), right ventricle (RV) and myocardium (MYO) from multi-slice cine MR images for both end-diastolic (ED) and end-systolic (ES) phase instances;” second effective time point is read as end-systolic phase; page 25, “For a typical 4D MR sequence with data dimension of 256 × 256 × 10 × 30 (Height × Width × Slices × Phases) our approach took approximately 3 seconds for ROI detection and 7 seconds for segmentation.”) and/or at least one latent representation and/or at least one hidden feature generated by the at least one first trained artificial neural network when generating the 3-dimensional medical image segmentation corresponding to the first effective time point.
Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). Mahendra is directed toward “computational efficacy of incorporating conventional computer vision techniques for region of interest detection in an end-to-end deep learning based segmentation framework. From the segmentation maps we extract clinically relevant cardiac parameters and hand-craft features which reflect the clinical diagnostic analysis and train an ensemble system for cardiac disease classification (abstract).” As such, both Wang and Mahendra are directed toward similar methods of endeavor of using deep learning to segment structures in cardiac systems and further providing information required for diagnosis. Further, it is well known that the heart changes positions and moves throughout time series imaging. A segmentation at one phase, may not be similar or reflect a segmentation at another phase. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Mahendra in order to ensure segmentations of one phase do not affect segmentations of another phase.
Regarding dependent claim 6, the rejection of claim 5 is incorporated herein. Additionally, Wang in the combination further discloses wherein the at least one output and/or at least one latent representation and/or at least one hidden feature generated by the at least one first trained artificial neural network when generating the 3-dimensional medical image segmentation corresponding to the first effective time point is used when generating 3-dimensional medical image segmentations for the at least one structure of the human heart for a plurality of other effective time points that are adjacent to the first effective time point (page 2, “The proposed DCNN model classify all the voxels within an axial slice based on a pre-defined neighborhood of axial slices around it;” different time series are further disclosed in Mahendra (see claim 5)).
Regarding dependent claim 7, the rejection of claim 5 is incorporated herein. Additionally, Mahendra in the combination further discloses wherein the first and the second effective time point correspond to different stages of a cardiac cycle (page 5, “We extensively validated our proposed network on two cardiac segmentation tasks: (i) segmentation of left ventricle (LV), right ventricle (RV) and myocardium (MYO) from multi-slice cine MR images for both end-diastolic (ED) and end-systolic (ES) phase instances”).
One of ordinary skill in the art before the effective filing date of the claimed invention would be easily aware the heart moves over time, and the key cycle phases are often relevant for diagnosis and analysis. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Mahendra in order to be sure the most relevant data is obtained for accurate diagnosis.
Regarding dependent claim 11, the rejection of claim 1 is incorporated herein. Additionally, Wang fails to explicitly disclose wherein a 3-dimensional medical image is generated from a multi-slice medical image by generating 3-dimensional voxels taking into account the values of the pixels of slices of the multi-slice medical image as well as distances between the pixels and distances between the slices.
However, Mahendra discloses wherein a 3-dimensional medical image is generated from a multi-slice medical image by generating 3-dimensional voxels taking into account the values of the pixels of slices of the multi-slice medical image as well as distances between the pixels and distances between the slices (page 25, “For a typical 4D MR sequence with data dimension of 256 × 256 × 10 × 30 (Height × Width × Slices × Phases) our approach took approximately 3 seconds for ROI detection and 7 seconds for segmentation;” 4D MRI is read as 3D MRI over time, further obtains 3D voxels over time and further determines slices within the volume as a whole; the slices inherently are split by a specific width (i.e. distance)).
Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). Mahendra is directed toward “computational efficacy of incorporating conventional computer vision techniques for region of interest detection in an end-to-end deep learning based segmentation framework. From the segmentation maps we extract clinically relevant cardiac parameters and hand-craft features which reflect the clinical diagnostic analysis and train an ensemble system for cardiac disease classification (abstract).” As such, both Wang and Mahendra are directed toward similar methods of endeavor of using deep learning to segment structures in cardiac systems and further providing information required for diagnosis. Further, it is well known that analyzing heart data in three dimensions provides key details of spatial relationships. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Mahendra in order to ensure key details of the heart can be seen accurately, and further in relation to each other to aid in further diagnosis and analysis.
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Wang as applied to claim 1 above, and further in view of WO 2017/091833 (hereinafter WO ‘833). Regarding dependent claim 8, the rejection of claim 1 is incorporated herein. Additionally, Wang fails to explicitly disclose wherein at least one processing block in at least one of the trained artificial neural networks comprises a plurality of convolutional filter layers, wherein at least two of the plurality of convolutional filter layers apply different strides.
However, WO ‘833 discloses wherein at least one processing block in at least one of the trained artificial neural networks comprises a plurality of convolutional filter layers, wherein at least two of the plurality of convolutional filter layers apply different strides. (page 26, line 12, “In at least some implementations, the network 600 includes two convolutional layers 606 before every pooling layer 608, with convolution kernels of size 3x3 and stride 1. Different combinations of these parameters (number of layers, convolution kernel size, convolution stride) may also be used.”).
As noted above, Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). WO ‘833 is directed toward Systems and methods for automated segmentation of anatomical structures, such as the human heart (abstract). As such, both Wang and WO ‘833 are directed toward methods of segmenting cardiac images using neural networks. Stride is well known in the art to affect neural networks performance and efficiency. Thus, knowing that there is a tradeoff of the two, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of incorporate the teaching of WO ‘833 in order to optimize a neural network system by modifying stride.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over Wang as applied to claim 1 above, and further in view of WO 2016/207875 (hereinafter WO ‘875).
Regarding dependent claim 10, the rejection of claim 1 is incorporated herein. Additionally, Wang fails to explicitly disclose wherein at least one up-sampling block and/or at least one down-sampling block in at least one of the trained artificial neural networks applies a nearest neighbor and/or trilinear interpolation method.
However, WO ‘875 discloses wherein at least one up-sampling block and/or at least one down-sampling block in at least one of the trained artificial neural networks applies a nearest neighbor and/or trilinear interpolation method (page 67, line 9, “The downsampling step may consist of, may be based on, may use, or may include, adaptive or non-adaptive interpolation. The non-adaptive interpolation may consist of, may be based on, may use, or may include, a nearest-neighbor replacement, bilinear interpolation, bicubic interpolation, Lanczos interpolation, spline interpolation, or filtering-based approach.”).
As noted above, Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). WO ‘875 is directed toward processing images using neural networks (abstract, page 60, lines 12-21). Further, it is well known in the art that one of the simplest methods of downsampling is nearest neighbor. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of WO ‘875 in order to downsample through nearest neighbor techniques which allows for simplicity and low computation (page 37, line 1-4)
Claim(s) 12 is rejected under 35 U.S.C. 103 as being unpatentable over Wang as applied to claim 1 above, and further in view of U.S. Publication No. 2017/0109881 to Avendi et al. (hereinafter Avendi).
Regarding dependent claim 12, the rejection of claim 1 is incorporated herein. Additionally, Wang fails to explicitly disclose further comprising the steps of:
providing a third trained artificial neural network configured as an autoencoder or a variational autoencoder;
inputting the at least part of the provided first n-dimensional medical image into the third trained artificial neural network; and
outputting, based on an output of the third trained artificial neural network, an output signal indicating a quality confidence score of the segmentation generated by the at least one first trained artificial neural network and/or indicating an anomaly.
However, Avendi discloses further comprising the steps of:
providing a third trained artificial neural network configured as an autoencoder or a variational autoencoder (paragraph 0054-0055, “detect the body chamber in the images using deep convolutional networks trained to locate the body chamber, infer a shape of a body chamber using a stacked auto-encoder trained to delineate the body chamber,”);
inputting the at least part of the provided first n-dimensional medical image into the third trained artificial neural network (paragraph 0054, “detect the body chamber in the images using deep convolutional networks trained to locate the body chamber” in order for the images to be processed, they have to be input; further this process occurs on MRI stacks, and there could be time resolved MRI); and
outputting, based on an output of the third trained artificial neural network, an output signal indicating a quality confidence score of the segmentation generated by the at least one first trained artificial neural network and/or indicating an anomaly(NOTE: and/or requires that only one limitation is read on; paragraph 0058-0059, “measuring clinical indicia at two time points, and detecting whether the body chamber shows an abnormal functional motion, wherein an abnormal functional motion is indicated by an abnormal change in the clinical indicia.”).
As noted above, Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). Avendi is directed toward systems and methods for automatically segmenting a heart chamber from medical images of a patient (abstract). Thus, both Wang and Avendi are directed toward similar methods of endeavor of segmenting and visualizing structures of the heart. Further, it can be easily understood that having neural networks be task specific allows for less training (i.e. trained for one task instead of two), and allows for easier error detection. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Avendi in order to connect neural networks serially allowing for different tasks to be completed more efficiently.
Claim(s) 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Wang as applied to claim 14 above, and further in view of U.S. Patent No. 7,860,283 to Begelman et al. (hereinafter Begelman).
Regarding dependent claim 15, the rejection of claim 14 is incorporated herein. Additionally, Wang fails to explicitly disclose wherein the input module is configured as an interface for receiving the 4-dimensional medical image from a picture archiving and communication system of a hospital or directly from a medical computed tomography imaging device.
However, Begelman discloses wherein the input module is configured as an interface for receiving the 4-dimensional medical image from a picture archiving and communication system of a hospital or directly from a medical computed tomography imaging device(paragraph 0026, “In an exemplary embodiment, computing device 102 is connected to a hospital computer network and a picture archive, and communication system (PACS) receives a CT study acquired on CT apparatus 101 in an ER;” see also Figure 1).
As noted above, Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). Begelman is directed toward a method of presenting information associated with blood vessels to a user (abstract). Further, it is well known in the art that image data within hospitals are stored in PACs systems, both for security as well as for ease of access. Thus it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Begelman in order to obtain the imaging data in a secure manner, and input the images into the system.
Regarding dependent claim 16, the rejection of claim 14 is incorporated herein. Additionally, Wang fails to explicitly disclose wherein the computing device is configured as a cloud computing platform or as a remote server which is operatively remotely connected to a graphical user interface configured to display information to a user and to receive input from the user.
However, Begelman discloses wherein the computing device is configured as a cloud computing platform or as a remote server which is operatively remotely connected to a graphical user interface configured to display information to a user and to receive input from the user (paragraph 0019, “Components of image processing system 100 may be positioned in a single location, a single facility, and/or may be remote from one another;” Figure 1, element 106, “input interface;” elements 106 and 104 are both within the system 100; paragraph 0021, “Input interface 106 may provide both an input and an output interface. For example, a touch screen both allows user input and presents output to the user”).
As noted above, Wang is directed toward a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract) and more specifically related to segmenting whole heart structures (see page 1). Begelman is directed toward a method of presenting information associated with blood vessels to a user (abstract). Further, it is well known in the art that image data within hospitals needs to be accessed from a variety of locations, units, hospitals, etc. rather than just the location it was obtained. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Begelman in order to allow for users in a variety of locations to access the desired information.
Regarding dependent claim 17, the rejection of claim 16 is incorporated herein. Additionally, Begelman in the combination further discloses wherein the graphical user interface is run on a local machine in a hospital environment (paragraph 0019, “Components of image processing system 100 may be positioned in a single location, a single facility, and/or may be remote from one another;” Figure 1, element 106, “input interface;” paragraph 0026, “In an exemplary embodiment, computing device 102 is connected to a hospital computer network and a picture archive, and communication system (PACS) receives a CT study acquired on CT apparatus 101 in an ER;” the user interface is within the computing device, which is connected to a hospital network; this could mean that the device is within the hospital connecting to the network (for example the ER where the images were acquired) or external on a remote connection).
One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand having a GUI on a local device for users to review would make the interaction much easier as opposed to the GUI being run somewhere else in a hospital. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Begelman in order to ensure the GUI can be presented to users throughout the hospital for review of relevant information.
Double Patenting
Non-statutory
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-18 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-13 and 18-21 of U.S. Patent No. 12,136,220 (hereinafter US ‘220). Although the claims at issue are not identical, they are not patentably distinct from each other because the instant application is more broad in scope than US ‘220.
Claim 1: Regarding claim 1, claim 1 compares to claim 1 of the US ‘220 patent as indicated below:
Current application - Claim 1
US Patent '220 - Claim 1
Notes
A computer-implemented method for generating at least one at least 3- dimensional medical image segmentation for at least one structure of a human heart, comprising the steps of:
A computer-implemented method for generating a 4-dimensional medical image segmentation for at least one structure of a human heart, comprising the steps of:
Instant application more broad
4D images are simply 3D images over time, thus included in a 4D image is already 3D images. Thus the instant application is more broad
providing a first n-dimensional medical image comprising the at least one structure of the human heart, wherein n=3 or n=4;
providing a first 4-dimensional medical image comprising the at least one structure of the human heart, the medical image being based on a computed tomography scan image;
Instant application more broad
generating a segmentation of at least part of the provided first n-dimensional medical image using at least one first trained artificial neural network, wherein the at least one first trained artificial neural network is configured as a convolutional processing network with U-net architecture, wherein each convolutional processing network with U-net architecture comprises:
generating a segmentation of at least part of the provided first 4-dimensional medical image using at least one first trained artificial neural network, wherein the at least one first trained artificial neural network is configured as a convolutional processing network with U-net architecture, wherein each convolutional processing network with U-net architecture comprises:
Instant application more broad
a down-sampling path comprising at least two processing convolutional blocks and at least two down-sampling blocks;
a down-sampling path comprising at least two processing convolutional blocks and at least two down-sampling blocks;
Verbatim
an up-sampling path comprising at least two processing convolutional blocks and at least two up-sampling blocks;
an up-sampling path comprising at least two processing convolutional blocks and at least two up-sampling blocks;
Verbatim
wherein the down-sampling path generates a direct input and/or an indirect input for the up-sampling path; and
wherein the down-sampling path generates a direct input and/or an indirect input for the up-sampling path;
Verbatim
generating at least one at least 3-dimensional medical image segmentation for the at least one structure of the human heart based at least on the segmentation generated by the at least one first trained artificial neural network.
generating at least one 4-dimensional medical image segmentation for the at least one structure of the human heart based at least on the segmentation generated by the at least one first trained artificial neural network;
Instant application more broad
providing a computed tomography scan image as a second 4-dimensional medical image;
Instant application more broad
generating a segmentation for at least part of the second 4-dimensional medical image using a second trained artificial neural network configured as a convolutional processing network with U-net architecture,
Instant application more broad
determining a portion of the segmentation for the second 4-dimensional medical image which comprises the at least one structure of the human heart;
Instant application more broad
extracting a portion of the provided second 4-dimensional medical image corresponding to the determined portion of the segmentation for the second 4-dimensional medical image; and
Instant application more broad
providing the extracted portion as the first 4-dimensional medical image to the at least one first trained artificial neural network.
Instant application more broad
As can be seen above, claim 1 of the current application is more broad in scope than claim 1 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 1 of the US ‘220 patent.
Claim 2: Regarding claim 2, claim 2 compares to claim 1 of the US ‘220 patent as indicated below:
Current application - Claim 2
US Patent '220 - Claim 1
Notes
The method according to claim 1, further comprising the steps of:
providing a second n-dimensional medical image;
providing a computed tomography scan image as a second 4-dimensional medical image;
Instant application more broad
generating a segmentation for at least part of the second n-dimensional medical image using a second trained artificial neural network configured as a convolutional processing network with Unet architecture,
generating a segmentation for at least part of the second 4-dimensional medical image using a second trained artificial neural network configured as a convolutional processing network with U-net architecture,
Instant application more broad
determining a portion of the segmentation for the second n-dimensional medical image which comprises the at least one structure of the human heart;
determining a portion of the segmentation for the second 4-dimensional medical image which comprises the at least one structure of the human heart;
Instant application more broad
extracting a portion of the provided second n-dimensional medical image corresponding to the determined portion of the segmentation for the second n-dimensional medical image; and
extracting a portion of the provided second 4-dimensional medical image corresponding to the determined portion of the segmentation for the second 4-dimensional medical image; and
Instant application more broad
providing the extracted portion as the first n-dimensional medical image to the at least one first trained artificial neural network.
providing the extracted portion as the first 4-dimensional medical image to the at least one first trained artificial neural network.
Instant application more broad
As can be seen above, claim 2 of the current application is more broad in scope (when incorporating claim dependency) than claim 1 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 1 of the US ‘220 patent.
Claim 3: Regarding claim 3, claim 3 compares to claim 2 of the US ‘220 patent as indicated below:
Current application - Claim 3
US Patent '220 - Claim 2
Notes
The method according to claim 1, wherein the at least one structure comprises at least one valve structure, at least one blood cavity, at least one muscle tissue structure, at least one implant and/or at least one anomaly.
The method of claim 1, wherein the at least one structure comprises at least one valve structure, at least one blood cavity, at least one muscle tissue structure, at least one implant and/or at least one anomaly.
Verbatim
As can be seen above, claim 3 of the current application is more broad in scope (when incorporating claim dependency) than claim 2 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 2 of the US ‘220 patent.
Claim 4: Regarding claim 4, claim 4 compares to claim 3 of the US ‘220 patent as indicated below:
Current application - Claim 4
US Patent '220 - Claim 3
Notes
The method according to claim 1, wherein:
The method according to claim 1,
as the at least one at least 3-dimensional medical image segmentation, a 4-dimensional medical image segmentation is generated;
… (claim 1):generating at least one 4-dimensional medical image segmentation
Substantially the same
the first n-dimensional medical image comprises a plurality of 3-dimensional medical images, each for a different effective time point;
wherein the first 4-dimensional medical image comprises a plurality of 3-dimensional medical images, each for a different effective time point;
Instant application more broad
a respective segmentation for at least two of the plurality of 3-dimensional medical images is independently generated by the at least one first trained artificial neural network; and
wherein a respective segmentation for at least two of the plurality of 3-dimensional medical images is independently generated by the at least one first trained artificial neural network; and
Verbatim
the 4-dimensional medical image segmentation comprises at least two of the independently generated segmentations for the plurality of 3-dimensional medical images.
wherein the 4-dimensional medical image segmentation comprises at least two of the independently generated segmentations for the plurality of 3-dimensional medical images.
Verbatim
As can be seen above, claim 4 of the current application is more broad in scope (when incorporating claim dependency) than claim 3 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 3 of the US ‘220 patent.
Claim 5: Regarding claim 5, claim 5 compares to claim 5 of the US ‘220 patent as indicated below:
Current application - Claim 5
US Patent '220 - Claim 5
Notes
The method according to claim 1, wherein:
The method according to claim 1, further comprising the step of:
Substantially the same
as the at least one at least 3-dimensional medical image segmentation, a 4-dimensional medical image segmentation is generated; and
… (claim 1):generating at least one 4-dimensional medical image segmentation
Substantially the same
the method further comprises the step of generating a 3-dimensional medical image segmentation corresponding to a first effective time point within the 4-dimensional medical image segmentation to be generated,
generating a 3-dimensional medical image segmentation corresponding to a first effective time point within the 4-dimensional medical image segmentation to be generated; and
Substantially the same
wherein at least one 3-dimensional medical image segmentation corresponding to at least one second effective time point within the 4-dimensional medical image segmentation to be generated is generated by the at least one first trained artificial neural network based on at least one output and/or at least one latent representation and/or at least one hidden feature generated by the at least one first trained artificial neural network when generating the 3-dimensional medical image segmentation corresponding to the first effective time point.
wherein at least one 3-dimensional medical image segmentation corresponding to at least one second effective time point within the 4-dimensional medical image segmentation to be generated is generated by the at least one first trained artificial neural network based on at least one output and/or at least one latent representation and/or at least one hidden feature generated by the at least one first trained artificial neural network when generating the 3-dimensional medical image segmentation corresponding to the first effective time point.
verbatim
As can be seen above, claim 5 of the current application is more broad in scope (when incorporating claim dependency) than claim 5 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 5 of the US ‘220 patent.
Claim 6: Regarding claim 6, claim 6 compares to claim 6 of the US ‘220 patent as indicated below:
Current application - Claim 6
US Patent '220 - Claim 6
Notes
The method of claim 5, wherein the at least one output and/or at least one latent representation and/or at least one hidden feature generated by the at least one first trained artificial neural network when generating the 3-dimensional medical image segmentation corresponding to the first effective time point is used when generating 3-dimensional medical image segmentations for the at least one structure of the human heart for a plurality of other effective time points that are adjacent to the first effective time point.
The method of claim 5, wherein the at least one output and/or at least one latent representation and/or at least one hidden feature generated by the at least one first trained artificial neural network when generating the 3-dimensional medical image segmentation corresponding to the first effective time point is used when generating 3-dimensional medical image segmentations for the at least one structure of the human heart for a plurality of other effective time points that are adjacent to the first effective time point.
Verbatim
As can be seen above, claim 6 of the current application is more broad in scope (when incorporating claim dependency) than claim 6 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 6 of the US ‘220 patent.
Claim 7: Regarding claim 7, claim 7 compares to claim 7 of the US ‘220 patent as indicated below:
Current application - Claim 7
US Patent '220 - Claim 7
Notes
The method according to claim 5, wherein the first and the second effective time point correspond to different stages of a cardiac cycle.
The method according to claim 5, wherein the first and the second effective time point correspond to different stages of a cardiac cycle.
Verbatim
As can be seen above, claim 7 of the current application is more broad in scope (when incorporating claim dependency) than claim 7 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 7 of the US ‘220 patent.
Claim 8: Regarding claim 8, claim 8 compares to claim 8 of the US ‘220 patent as indicated below:
Current application - Claim 8
US Patent '220 - Claim 8
Notes
The method according to claim 1, wherein at least one processing block in at least one of the trained artificial neural networks comprises a plurality of convolutional filter layers, wherein at least two of the plurality of convolutional filter layers apply different strides.
The method according to claim 1, wherein at least one processing block in at least one of the trained artificial neural networks comprises a plurality of convolutional filter layers, wherein at least two of the plurality of convolutional filter layers apply different strides.
Verbatim
As can be seen above, claim 8 of the current application is more broad in scope (when incorporating claim dependency) than claim 8 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 8 of the US ‘220 patent.
Claim 9: Regarding claim 9, claim 9 compares to claim 9 of the US ‘220 patent as indicated below:
Current application - Claim 9
US Patent '220 - Claim 9
Notes
The method according to claim 1, wherein at least one processing block in at least one of the trained artificial neural networks comprises a bottleneck structure in which a r x r x r 3-dimensional convolution layer follows an q x q x q 3-dimensional convolution layer with a reduced number output channels compared to a number of input channels of the at least one processing block, wherein rand q are integers and wherein q is smaller than r.
The method according to claim 1, wherein at least one processing block in at least one of the trained artificial neural networks comprises a bottleneck structure in which a r×r×r 3-dimensional convolution layer follows an q×q×q 3-dimensional convolution layer with a reduced number output channels compared to a number of input channels of the at least one processing block, wherein r and q are integers and wherein q is smaller than r.
Verbatim
As can be seen above, claim 9 of the current application is more broad in scope (when incorporating claim dependency) than claim 9 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 9 of the US ‘220 patent.
Claim 10: Regarding claim 10, claim 10 compares to claim 10 of the US ‘220 patent as indicated below:
Current application - Claim 10
US Patent '220 - Claim 10
Notes
The method according to claim 1, wherein at least one up-sampling block and/or at least one down-sampling block in at least one of the trained artificial neural networks applies a nearest neighbor and/or trilinear interpolation method.
The method according to claim 1, wherein at least one up-sampling block and/or at least one down-sampling block in at least one of the trained artificial neural networks applies a nearest neighbor and/or trilinear interpolation method.
Verbatim
As can be seen above, claim 10 of the current application is more broad in scope (when incorporating claim dependency) than claim 10 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 10 of the US ‘220 patent.
Claim 11: Regarding claim 11, claim 11 compares to claim 4 of the US ‘220 patent as indicated below:
Current application - Claim 11
US Patent '220 - Claim 4
Notes
The method according to claim 1, wherein a 3-dimensional medical image is generated from a multi-slice medical image by generating 3-dimensional voxels taking into account the values of the pixels of slices of the multi-slice medical image as well as distances between the pixels and distances between the slices.
The method according to claim 3, wherein each 3-dimensional medical image of the plurality of 3-dimensional medical images of the first 4-dimensional medical image is generated from a multi-slice computed tomography scan image by generating 3-dimensional voxels taking into account the values of the pixels of slices of the multi-slice medical image as well as distances between the pixels and distances between the slices.
Instant application more broad
As can be seen above, claim 11 of the current application is more broad in scope (when incorporating claim dependency) than claim 4 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 4 of the US ‘220 patent.
Claim 12: Regarding claim 12, claim 12 compares to claim 11 of the US ‘220 patent as indicated below:
Current application - Claim 12
US Patent '220 - Claim 11
Notes
The method according to claim 1, further comprising the steps of:
The method according to claim 1, further comprising the steps of:
Verbatim
providing a third trained artificial neural network configured as an autoencoder or a variational autoencoder;
providing a third trained artificial neural network configured as an autoencoder or a variational autoencoder;
Verbatim
inputting the at least part of the provided first n-dimensional medical image into the third trained artificial neural network; and
inputting the at least part of the provided first 4-dimensional medical image into the third trained artificial neural network; and
Instant application more broad
outputting, based on an output of the third trained artificial neural network, an output signal indicating a quality confidence score of the segmentation generated by the at least one first trained artificial neural network and/or indicating an anomaly.
outputting, based on an output of the third trained artificial neural network, an output signal indicating a quality confidence score of the segmentation generated by the at least one first trained artificial neural network and/or indicating an anomaly.
verbatim
As can be seen above, claim 12 of the current application is more broad in scope (when incorporating claim dependency) than claim 11 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 11 of the US ‘220 patent.
Claim 13: Regarding claim 13, claim 13 compares to claim 12 of the US ‘220 patent as indicated below:
Current application - Claim 13
US Patent '220 - Claim 12
Notes
The method according to claim 1, wherein a plurality of first trained artificial neural networks which are differently configured and/or differently trained is provided;
The method of claim 1,
wherein a plurality of first trained artificial neural networks which are differently configured and/or differently trained is provided;
Verbatim
wherein at least two of the plurality of the first trained artificial neural networks are configured as a respective convolutional processing network with U-net architecture;
wherein at least two of the plurality of the first trained artificial neural networks are configured as a respective convolutional processing network with U-net architecture;
Verbatim
wherein a respective candidate segmentation of at least part of the provided first n- dimensional medical image is generated by each of the plurality of provided first trained artificial neural networks; and
wherein a respective candidate segmentation of at least part of the provided first 4-dimensional medical image is generated by each of the plurality of provided first trained artificial neural networks; and
Instant application more broad
wherein the at least 3-dimensional medical image segmentation is generated based on the generated candidate segmentations.
wherein the 4-dimensional medical image segmentation is generated based on the generated candidate segmentations.
Instant application more broad
As can be seen above, claim 13 of the current application is more broad in scope (when incorporating claim dependency) than claim 12 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 12 of the US ‘220 patent.
Claim 14: Regarding claim 14, claim 14 compares to claim 18 of the US ‘220 patent as indicated below:
Current application - Claim 14
US Patent '220 - Claim 18
Notes
A system for generating at least one at least 3-dimensional medical image segmentation for at least one structure of a human heart, comprising:
A system for generating at least one 4-dimensional medical image segmentation for at least one structure of a human heart, comprising:
Instant application more broad
4D images are simply 3D images over time, thus included in a 4D image is already 3D images. Thus the instant application is more broad
a computing device configured to implement:
a computing device including:
Substantially the same
an input module configured to provide a first n-dimensional medical image of the at least one structure of the human heart, wherein n=3 or n=4; and
an input module configured to receive a first 4-dimensional medical image of the at least one structure of the human heart, the medical image being based on a computed tomography scan image;
Instant application more brad
a controller implementing at least one first trained artificial neural network, wherein the trained artificial neural network is configured as a convolutional processing network with U-net architecture, wherein each convolutional processing network with U-net architecture comprises:
a controller configured and operable to implement at least one first trained artificial neural network, wherein the trained artificial neural network is configured as a convolutional processing network with U-net architecture, wherein each convolutional processing network with U-net architecture includes;
verbatim
a down-sampling path comprising at least two processing convolutional blocks and at least two down-sampling blocks;
a down-sampling path comprising at least two processing convolutional blocks and at least two down-sampling blocks;
verbatim
an up-sampling path comprising at least two processing convolutional blocks and at least two up-sampling blocks;
an up-sampling path comprising at least two processing convolutional blocks and at least two up-sampling blocks;
Verbatim
wherein the down-sampling path generates a direct input and/or an indirect input for the up-sampling path;
wherein the down-sampling path generates a direct input and/or an indirect input for the up-sampling path;
verbatim
wherein the first trained artificial neural network is trained and configured to generate a segmentation of at least part of the provided first n-dimensional medical image; and
wherein the first trained artificial neural network is trained and configured to generate a segmentation of at least part of the provided first 4-dimensional medical image; and
Instant application more broad
an output module configured to generate at least one 3-dimensional medical image segmentation based at least on the segmentation generated by the at least one first trained artificial neural network.
an output module configured to generate at least one 4-dimensional medical image segmentation based at least on the segmentation generated by the at least one first trained artificial neural network;
Instant application more broad
wherein the input module is further configured to receive a second 4-dimensional medical image based on a computer tomography scan image;
Instant application more broad
wherein said controller is further configured an operable to;
Instant application more broad
generate a segmentation for at least part of the second 4-dimensional medical image using a second trained artificial neural network configured as a convolutional processing network with U-net architecture,
Instant application more broad
determine a portion of the segmentation for the second 4-dimensional medical image which comprises the at least one structure of the human heart;
Instant application more broad
extract a portion of the provided second 4-dimensional medical image corresponding to the determined portion of the segmentation for the second 4-dimensional medical image, and
Instant application more broad
provide the extracted portion as the first 4-dimensional medical image to the at least one first trained artificial neural network.
Instant application more broad
As can be seen above, claim 14 of the current application is more broad in scope (when incorporating claim dependency) than claim 18 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 18 of the US ‘220 patent.
Claim 15: Regarding claim 15, claim 15 compares to claim 19 of the US ‘220 patent as indicated below:
Current application - Claim 15
US Patent '220 - Claim 19
Notes
The system of claim 14, wherein the input module is configured as an interface for receiving the 4-dimensional medical image from a picture archiving and communication system of a hospital or directly from a medical computed tomography imaging device.
The system of claim 18, wherein the input module is configured as an interface for receiving the 4-dimensional medical image from a picture archiving and communication system of a hospital or directly from a medical computed tomography imaging device.
Verbatim
As can be seen above, claim 15 of the current application is more broad in scope (when incorporating claim dependency) than claim 19 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 19 of the US ‘220 patent.
Claim 16: Regarding claim 16, claim 16 compares to claim 20 of the US ‘220 patent as indicated below:
Current application - Claim 16
US Patent '220 - Claim 20
Notes
The system of claim 14, wherein the computing device is configured as a cloud computing platform or as a remote server which is operatively remotely connected to a graphical user interface configured to display information to a user and to receive input from the user.
The system of claim 18, wherein the computing device is configured as a cloud computing platform or as a remote server which is operatively remotely connected to a graphical user interface configured to display information to a user and to receive input from the user.
Verbatim
As can be seen above, claim 16 of the current application is more broad in scope (when incorporating claim dependency) than claim 20 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 20 of the US ‘220 patent.
Claim 17: Regarding claim 17, claim 17vcompares to claim 21 of the US ‘220 patent as indicated below:
Current application - Claim 17
US Patent '220 - Claim 21
Notes
The system of claim 16, wherein the graphical user interface is run on a local machine in a hospital environment.
The system of claim 20, wherein the graphical user interface is run on a local machine in a hospital environment.
Verbatim
As can be seen above, claim 17 of the current application is more broad in scope (when incorporating claim dependency) than claim 21 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 21 of the US ‘220 patent.
Claim 18: Regarding claim 18, claim 18 compares to claim 13 of the US ‘220 patent as indicated below:
Current application - Claim 18
US Patent '220 - Claim 13
Notes
A non-transitory computer-readable data storage medium comprising executable program code configured to, when executed, perform the method according to claim 1.
A non-transitory computer-readable data storage medium comprising executable program code configured to, when executed, perform the method according to claim 1.
Verbatim
As can be seen above, claim 18 of the current application is more broad in scope (when incorporating claim dependency) than claim 13 of the US ’220 patent. Therefore, any patent granted on the current application would result in the unjustifiable timewise extension of the monopoly granted on claim 13 of the US ‘220 patent.
Allowable Subject Matter
Claim 2 would be allowable if rewritten to overcome the double patenting rejections set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach methods of processing 3D and 4D medical images of the heart, and segmenting key features of interest from the images.
However, none of them alone or in any combination teaches using a second 3D or 4D image, and segmenting the second image using an additional convolutional U-net neural network, locating heart structures, extracting a portion of the image corresponding to the structure and providing the extracted portion as the first 3D or 4D medical image to the first neural network.
The closest prior art, Wang, discloses, “a two-stage modified Unet framework that simultaneously learns to detect a ROI within the full volume and to classify voxels without losing the original resolution (abstract).” The architecture is shown in Figure 1, and one can see the inputs are all determined in the beginning, which are input into Network1 and Network2.
However, Wang fails to disclose using a second 3D or 4D image, and segmenting the second image using an additional convolutional U-net neural network, locating heart structures, extracting a portion of the image corresponding to the structure and providing the extracted portion as the first 3D or 4D medical image to the first neural network.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Publication No. 2018/0240235 to Mazo discloses, “a method for segmentation of an image of a target patient (abstract)”
U.S. Publication No. 2007/0276214 to Dachille et al. discloses, “An imaging system for automated segmentation and visualization of medical images (abstract)”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COURTNEY JOAN NELSON/Primary Examiner, Art Unit 2661
Prosecution Insights