Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 28 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because the computer program is not eligible patented subject matter. Claim 28 are directed to, a computer-readable medium. However, one of ordinary skilled in the art would broadly interpret, the claimed computer program to encompass ineligible transitory signal embodiments as "signal" per se and software per se, which does not fall within one of the statutory categories of the invention (i.e. process, machine, manufacture or a composition of matter) and broadly interpret the computer readable medium to typically cover forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media. Therefore, a transitory computer readable medium would reasonably be interpreted by one of ordinary skill in the art as signal, per se. Thus, subject matter " a computer-readable medium" is not limited to that which falls within a statutory category of invention. Therefore, the claim contains ineligible non-statutory subject matter. “ The applicant can add non-transitory in the preamble to overcome the prior art. Products that do not have a physical or tangible form, such as information (often referred to as "data per se") or a computer program per se (often referred to as "software per se") when claimed as a product without any structural recitations see MPEP 2106.03.
Claim Objections
Claims 14, 27-29 are objected to because of the following informalities: Claim 14, the limitation “the Euclidean distance” should be “a Euclidean distance”, please correct these type of errors in the rest of the claims as well that contain similar limitations. Claim 27 needs to look like most of the claims without all the terms in capital letters, and the examiner has made the best effort to find repetitive terms, please double check this claim accordingly. Claim 28, the preamble needs to have the limitation “non-transitory”. Claim 29 appears to be a type the way it is written, as the final limitations claims “a computer system….claim 1.” Please correct in order for these claims to be properly examined, as it is claiming 2 systems. Further the entire set of claims must be doubled checked for all the terms in the formulas that are defined as well as anteceded basis. The claims are still not written in standard USPTO common standards, please consider rewriting in order to advance prosecution. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 7-8, 10-15, 20 AND 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over Feng (US 2020/0380696) in view of Putha (US 2022/0245795).
As per claim 1, Feng teaches, a teaches a method and a computer-readable medium of detection of a landmark (104, 206) in a volume (101, 201) of medical images (Feng, FIG. 6B shows segmented contours in first stage (blue), second stage (yellow) along with ground truth(green). Which would represent the landmark, as there are no definitions of this landmark in the claimed subject matter or in the specification, which then must be in the claim language to be considered), wherein the method comprises generating at least volume of medical images (Feng, fig.8 represents heatmap and fig.14B is a four-layer U-Net dense block, according to an example of the present disclosure. And ¶[0091] “Furthermore, the U-Net structure is compared with two other structures, a fully convolutional network with fusion predictions (FCN-8s) and SegNet which use deeper networks (5 resolution level) for both encoder and decoder. All the comparison studies were conducted to segment the target muscle adductor brevis, which has a relatively small volume and thus is difficult to segment.” ), wherein the volume of medical images comprises a computational tomography volume (101, 201) or a magnetic resonance volume (Feng, ¶[0093] “The segmented high-resolution 3D image can also be used to determine quantitative metrics of muscle volume. The volumes can be derived by summing up the number of voxels within the boundary and multiplying it by the voxel size. Additionally, a superior-inferior length can be derived by calculating the number of axial slices and multiplying by the slice thickness.” ¶[0095] “FIG. 4A-4E shows a workflow of the automated segmentation framework for one target muscle. FIG. 4A shows a high-resolution 3D sub-image of an MRI scanned pair of legs. FIG. 4B shows a high-resolution 3D sub-image of an MRI scanned pair of legs that has been pre-processed. Pre-processing was used to correct the image inhomogeneity due to radiofrequency field (B1) variations was corrected using improved nonparametric nonuniform intensity normalization (N3) bias correction.” ), wherein the volume of medical images comprises voxels, wherein said at least one heatmap comprises a probability of finding localization of said landmark (104, 206) in voxels (Feng, ¶[0108] “When combining all ROIs, since each network makes a voxel-by-voxel prediction of whether it belongs to the target ROI, it is possible that different networks predict the same voxel to be different ROIs in the second stage.” This represents that the images contain voxels), characterized in that the trained U- Net is a trained residual U-Net (203) that comprises a contracting path and an expansive path (Feng, ¶[0115] “Overall, considering the minimal differences and the fact that the network with residual block needs more parameters and computations, the plain U-Net with three resolution levels, two convolutional layers per level, and 96 filters on the first convolutional layer was used to train all the target muscles in the second stage.” The stages represent a contracting path and an expansive path), and wherein the contracting path comprises encoding residual block (302, 303, 304, 305) for encoding the volume of medical images and the expansive path comprises decoding residual block (307, 308, 309, 310) for decoding the volume of medical images encoded by the contracting path (Feng, ¶[0091] “Furthermore, the U-Net structure is compared with two other structures, a fully convolutional network with fusion predictions (FCN-8s) and SegNet which use deeper networks (5 resolution level) for both encoder and decoder.” This represents having the coding and encoding), wherein the method additionally comprises transforming said at least one heatmap to coordinates of at least one landmark (104,206) using a 3D Differentiable Spatial to Numerical Transform (103, 206) (Feng, ¶[0093] “The segmented high-resolution 3D image can also be used to determine quantitative metrics of muscle volume. The volumes can be derived by summing up the number of voxels within the boundary and multiplying it by the voxel size. Additionally, a superior-inferior length can be derived by calculating the number of axial slices and multiplying by the slice thickness.” This represents a 3D Differentiable Spatial to Numerical Transform).
Feng doesn’t clearly teach, one heatmap using a trained U-Net applied on the medical image.
However, Putha teaches, one heatmap using a trained U-Net applied on the medical image (Putha, ¶[0019] “wherein the patient rotation classifier comprises a U-net based segmentation model trained to give heatmaps of clavicles and spinous processes” This represents heatmap using a trained U-Net applied on the medical image).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Feng with those of Putha to be able to have one heatmap using a trained U-Net applied on the medical image.
The motivation would have been to improve efficiency as taught by Putha ¶[003] “There is an emerging need for automated approaches to improve the efficiency, accuracy and cost effectiveness of the medical imaging evaluation.”
2. (Canceled)
As per claim 3, Feng in view of Putha teaches, the method of claim 1, wherein the volume of medical images is a pre-processed volume (102) of medical images (Feng, fig.9C medical image that has been pre-processed).
As per claim 4, Feng in view of Putha teaches, the method of claim 3, wherein the pre-processed volume of medical images is obtained using at least one of central cropping, clipping and setting values of selected voxels to defined values, normalizing (204) the values of voxels and downsizing the volume of medical images and augmenting (202) (Feng, fig.6A central cropping and centering as seen by the bounding box as well as downsizing by 4 by a quarter).
As per claim 7, Feng in view of Putha teaches, the method of claim 4, wherein said downsizing includes decreasing the volume of the volume of medical images 3 to 4 times (Feng, fig.6A showing downsizing by 2-3 times).
As per claim 8, Feng in view of Putha teaches, the method of claim 7, wherein said downsizing includes decreasing the volume of the volume of medical images 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8 or 3.9 times (Feng, fig.6A this shows 3-4 downsizing therefore including this downsizing ).
As per claim 10, Feng in view of Putha teaches, the method of claim 1, wherein encoding comprises downsampling of the volume of medical images and increasing the number of channels to produce a downsampled output and sending the downsampled output to a corresponding encoding residual block (302, 303, 304, 305) and to a corresponding decoding residual block (307, 308, 309, 310) ( Feng, ¶[0091] “Furthermore, the U-Net structure is compared with two other structures, a fully convolutional network with fusion predictions (FCN-8s) and SegNet which use deeper networks (5 resolution level) for both encoder and decoder. All the comparison studies were conducted to segment the target muscle adductor brevis, which has a relatively small volume and thus is difficult to segment.” And this would be standard practice rather than independently innovative. Downsampling spatial or volumetric dimensions while increasing channels (e.g., via strided convolution or pooling followed by feature expansion) and routing signals into Residual Networks (ResNet) or encoder-decoder architectures (like V-Net or Residual Encoder-Decoder Networks) are fundamental, well-established building blocks in deep learning, and as this current prior art Feng teaches).
As per claim 11, Feng in view of Putha teaches, the method of claim 10, wherein downsampling is done by a factor of two (Feng, fig.6A this shows downsizing therefore including this downsizing by a factor of 2 if required ).
As per claim 12, Feng in view of Putha teaches, the method of claim 1, wherein decoding comprises upsampling of an input to a residual decoding block of the volume of medical images to produce an upsampled output with a decreased number of channels (Feng, ¶[0084] “The modified 3D-UNet network follows the structure of 3D-UNet which consisted of an encoder and a decoder, each with four resolution levels.” If there are 4 resolution levels then this represents up sampling. This again with the channels being standard practice ).
As per claim 13, Feng in view of Putha teaches, the method of claim 12, wherein up sampling is done by a factor of two (Feng, ¶[0084] “The modified 3D-UNet network follows the structure of 3D-UNet which consisted of an encoder and a decoder, each with four resolution levels.” If there are 4 resolution levels then this represents up sampling. This again with the channels being standard practice ).
As per claim 14, Feng in view of Putha teaches, the method of claim 1, wherein training of the trained residual U-Net (203) comprises at least one of:- minimization of the divergence between said at least one heatmap and an annotated heatmap comprising at least one annotation, wherein said at least annotation points to localization of the landmark (104, 206) on said at least one heatmap, and- minimization of the Euclidean distance between the localization on said at least one heatmap and the localization on the heatmap comprising at least one annotation (Feng, ¶ [0080] “The splitting process can include the whole leg images split into three parts: abdomen, upper leg and lower leg based on the superior-inferior coordinates and the estimated ratios of the lengths of the three parts.” To be able to get the distances this represents “the Euclidean distance”, which must be corrected to “a Euclidean distance” and the claim says at least one of the different options).
As per claim 15, Feng in view of Putha teaches, the method of claim 1, wherein the probability of finding the landmark (104, 206) has a normal distribution (Feng, ¶[0086] “For simplicity, the uniform distribution was used in sampling the ratio.” Uniform distribution a normal distribution).
As per claim 20, Feng in view of Putha teaches, the method of any of claim 14, wherein said at least one heatmap and said annotated heatmap are produced using the same volume of medical images or using a different volumes (Feng, ¶ [0093] “The segmented high-resolution 3D image can also be used to determine quantitative metrics of muscle volume. The volumes can be derived by summing up the number of voxels within the boundary and multiplying it by the voxel size. Additionally, a superior-inferior length can be derived by calculating the number of axial slices and multiplying by the slice thickness.” The option to multiply the voxel size represents using the same or different volumes).
As claim 29, Feng in view of Putha teaches, a system for detection of a landmark in a volume of medical images, wherein the system comprises: a measuring means for collection a computational tomography volume or a magnetic resonance volume, for a human patient, and a computer system adapted to perform the steps of a method defined in any of claims 1 (Feng, fig.9C are of a human, and the means are as seen in the fig.6-9).
Allowable Subject Matter
Claims 5-6, 9, 16-19, 21-27, and 30-32 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. As long as all of the claimed objections are fixed, and claims are rechecked for these common errors.
Conclusion
Note: A 101 analysis for usefulness and significantly more and the conclusion was made that U-Net to find anatomical landmarks in 3D scans like CT or MRI and converts probability heatmaps directly into precise 3D coordinates using a differentiable spatial-to-numerical transform are useful transformation of viewing those images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SANTIAGO GARCIA whose telephone number is (571)270-5182. The examiner can normally be reached Monday-Friday 9:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SANTIAGO GARCIA/Primary Examiner, Art Unit 2673
/SG/
Prosecution Insights