DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 05/15/2026 and 09/27/2024 are being considered by the examiner.
Claim Objections
In claim 19, line 1, the term “according to claim 17” should be changed to, “according to claim 18” in order to avoid an insufficient antecedent issue with the term “the generated stack.”
In claim 26, line 1, the term “according to claim 24” should be changed to, “according to claim 25” in order to avoid an insufficient antecedent issue with the term “the generated stack.”
Appropriate correction is required.
Double Patenting
The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A non-statutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 17, 18, 22, 24, 25, 29, and 31, are rejected on the ground of non-statutory double patenting as being unpatentable over claims 20, 22, 26, 27, 29, and 33-34, of Co-Pending Application No. 18/851,737 in view of ZHOU et al. (US 20170256090 A1).
Although the claims 17-31 of this Application No. 18/851,719 and claims at issue are not identical, they are not patentably distinct from each other because the instant application and the conflicting Patent are claiming common subject matter, as follows:
This Application No. 18/851,719
Co-Pending Application No. 18/851,737
Claim 17: An apparatus for processing medical images, comprising:
a memory that stores a plurality of instructions; and
at least one processor coupled to the memory and configured to execute the plurality of instructions to:
detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation;
map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image;
interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and
sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image,
wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
Claim 24: A computer implemented method for processing medical images, comprising:
detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation;
mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image;
interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and
sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image,
wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
Claim 31: A non-transitory computer-readable medium for storing executable instructions, which cause a method to be performed to process medical images, the method comprising:
detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation;
mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image;
interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and
sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image,
wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
Claim 20: An apparatus for detecting a fracture in medical images, comprising:
a memory that stores a plurality of instructions; and
at least one processor coupled to the memory and configured to execute the plurality of instructions to:
detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation;
map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image;
interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image;
sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image;
predict a fracture via a trained fracture detection model using machine learning or deep learning; and
revise the reformatted 2D manifold image to show the predicted fracture according to the trained fracture detection model.
Claim 27: A computer implemented method for detecting a fracture in medical images, the method comprising:
detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation;
mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image;
interpolating coordinates of each 3D position missing on the 2D manifold image, such
that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image;
sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image;
predicting the fracture via a trained fracture detection model using machine learning or deep learning; and
revising the reformatted 2D manifold image to show the predicted fracture according to the trained fracture detection model.
Claim 34: A non-transitory computer-readable medium for storing executable instructions, which cause a method to be performed to detect a fracture in medical images, the method comprising:
detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation;
mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image;
interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image;
sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image;
predicting the fracture via a trained fracture detection model using machine learning or deep learning; and
revising the reformatted 2D manifold image to show the predicted fracture according to the trained fracture detection model.
Although Co-Pending Application No. 18/851,737 claim 20 teaches an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image, Co-Pending Application No. 18/851,737 claim 20 as stated in the table above with respect to claim 17, fails to clearly disclose wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
However, ZHOU et al. (US 20170256090 A1) explicitly teaches wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine (Fig. 3, illustrates a reformatted 2D manifold image with continuous and straightened visualization of a rib cage and a spine. Paragraph [0030]-ZHOU discloses FIG. 3 illustrates an example of a rendering of an unfolded two-dimensional slice of a volume (wherein an unfolded two-dimensional slice of a volume is a 2D manifold image). Please see annotated Fig. 3 below.).
PNG
media_image1.png
267
764
media_image1.png
Greyscale
Annotated diagram of ZHOU’s Fig. 3 illustrating a 2D manifold image with a rib cage and a spine. The blue boxes indicate various ribs of a rib cage in the image, while the red box indicates the spine.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 20 of having an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image, with the teachings of ZHOU et al. (US 20170256090 A1), of having wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
Wherein having Co-Pending Application No. 18/851,737 claim 20 having wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Although Co-Pending Application No. 18/851,737 claim 27 teaches A computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image, Co-Pending Application No. 18/851,737 claim 27 as stated in the table above with respect to claim 24, fails to clearly disclose wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
However, ZHOU et al. (US 20170256090 A1) explicitly teaches wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine (Fig. 3, illustrates a reformatted 2D manifold image with continuous and straightened visualization of a rib cage and a spine. Paragraph [0030]-ZHOU discloses FIG. 3 illustrates an example of a rendering of an unfolded two-dimensional slice of a volume (wherein an unfolded two-dimensional slice of a volume is a 2D manifold image). Please see annotated Fig. 3 below.).
PNG
media_image1.png
267
764
media_image1.png
Greyscale
Annotated diagram of ZHOU’s Fig. 3 illustrating a 2D manifold image with a rib cage and a spine. The blue boxes indicate various ribs of a rib cage in the image, while the red box indicates the spine.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 27 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of ZHOU et al. (US 20170256090 A1), of having wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
Wherein having Co-Pending Application No. 18/851,737 claim 27 having wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Although Co-Pending Application No. 18/851,737 claim 34 teaches a non-transitory computer-readable medium for storing executable instructions, which cause a method to be performed to process medical images, the method comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image, Co-Pending Application No. 18/851,737 claim 34 as stated in the table above with respect to claim 31, fails to clearly disclose wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
However, ZHOU et al. (US 20170256090 A1) explicitly teaches wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine (Fig. 3, illustrates a reformatted 2D manifold image with continuous and straightened visualization of a rib cage and a spine. Paragraph [0030]-ZHOU discloses FIG. 3 illustrates an example of a rendering of an unfolded two-dimensional slice of a volume (wherein an unfolded two-dimensional slice of a volume is a 2D manifold image). Please see annotated Fig. 3 below.).
PNG
media_image1.png
267
764
media_image1.png
Greyscale
Annotated diagram of ZHOU’s Fig. 3 illustrating a 2D manifold image with a rib cage and a spine. The blue boxes indicate various ribs of a rib cage in the image, while the red box indicates the spine.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 34 of having an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image, with the teachings of ZHOU et al. (US 20170256090 A1), of having wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
Wherein having Co-Pending Application No. 18/851,737 claim 34 having wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
The further limitations of the dependent claims are similar as indicated below:
This Application No. 18/851,719
Co-Pending Application No. 18/851,737
Claim 18: wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction and repeat the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine.
Claim 22: wherein the coordinates are interpolated using thin plate spline techniques.
Claim 25: further comprising shifting the reformatted 2D manifold image along a normal direction and repeating the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine.
Claim 29: wherein the coordinates are interpolated using thin plate spline techniques.
Claim 22: wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction and repeat the sampling to generate a stack of manifold slices covering a complete 3D visualization of a rib cage and a spine.
Claim 26: wherein interpolating the missing coordinates on the manifold is performed via interpolation techniques including thin plate splines.
Claim 29: further comprising shifting the reformatted 2D manifold image along a normal direction and repeating the sampling to generate a stack of manifold slices covering a complete 3D visualization of a rib cage and a spine
Claim 33: wherein interpolating the missing coordinates on the manifold is performed via interpolation techniques including thin plate splines.
Claims 18, 22, 25, and 29 contain the same limitations as Co-Pending Application No. 18/851,737 claims 22, 26, 29, and 33 respectively. Therefore, given that claims 18, 22, 25, and 29 depend from claims 17 and 24 respectively and claims 22, 26, 29, and 33 depend from claims 20 and 27 respectively. Claim 18, 22, 25, and 29 are rejected for the same reasons set forth in the rejection of the independent claim above.
Claims 19 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 20 in view of ZHOU et al. (US 20170256090 A1) and further in view of KIRALY et al. (US 20080287796 A1).
Regarding claim 19, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) teaches the apparatus according to claim 17, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
However, KIRALY et al. (US 20080287796 A1) explicitly teaches wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs) (Figs. 5 and 6. Paragraph [0026]-KIRALY discloses the method of FIG. 5 can be performed to implement step 106 of FIG. 1, and generates an MRP based volume that aligns the spine along the natural curvature of the spine. This method reformats the image volume based on a curved MPR automatically defined through the spine centerline (wherein the image with a reformatted image volume based on a curved MPR is an interpolated 2D multiplanar reconstruction). Paragraph [0029]-KIRALY discloses the 2D image at each centerline point can be generated by sampling the original image volume using tri-linear interpolation or other well known sampling methods. Further in Paragraph [0030]-KIRALY discloses at step 508, the series of 2D images are stacked, with each 2D image shifted in the x-direction from the average of the x-coordinates by the shift value of the corresponding centerline point (wherein the images stacked are 2D MPRs).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 20 of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of KIRALY et al. (US 20080287796 A1), of having wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
Wherein having Co-Pending Application No. 18/851,737 claim 20 having wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim 20 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 20 in view of ZHOU et al. (US 20170256090 A1) and further in view of KIRALY et al. (US 20080287796 A1) and further in view of RAI et al. (US 20160180529 A1).
Regarding claim 20, Co-Pending Application No. 18/851,737 claim 20 in view of in view of ZHOU et al. (US 20170256090 A1) explicitly teaches the apparatus according to claim 19, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
However, RAI et al. (US 20160180529 A1) explicitly teaches wherein the at least one processor is further configured to generate a 3D image of the rib cage and the spine based on the stack of 2D MPRs (Fig. 2. [0050]-RAI discloses with reference to FIG. 2, step 36 recites to compute a 3D-projected location of the anatomical structure based on a 3D data set. Available 3D image data from the subject includes without limitation high resolution computed tomography (HRCT) scans, MRI, PET, 3D angiographic, and X-ray data sets. Optionally, Ribs, spine, and bones are segmented out from 3D image and their centerlines may be computed. Further in paragraph [0051]-RAI discloses in a method, or system, the workstation receives a 3D image file, 3D image data set, or a set of 2D images of the organ from which a 3D model of the organ may be computed (wherein the set 2D images of the organ is a stack of 2D MPRs and wherein the 3D model is a 3D image).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 20 of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of RAI et al. (US 20160180529 A1), of having wherein the at least one processor is further configured to generate a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
Wherein having Co-Pending Application No. 18/851,737 claim 20 having wherein the at least one processor is further configured to generate a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim 21 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 20 in view of ZHOU et al. (US 20170256090 A1) and further in view of RAI et al. (US 20160180529 A1).
Regarding claim 21, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) explicitly teaches the apparatus according to claim 17, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
However, GEORGESCU et al. (US 20200193594 A1) explicitly teaches wherein rib segmentation and spine segmentation (Fig. 3. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region.) are performed with machine learning or deep learning techniques (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses the anatomical objects are automatically segmented from the medical image data using an adversarial deep image-to-image network. The adversarial deep image-to-image network comprises a generator network and a discriminator network. The generator network may be a deep image-to-image (DI2I) network that receives the medical image data and anatomical landmark locations (detected at step 104) as input and outputs a probability map indicating a probability score of voxels belonging to the anatomical objects (wherein the deep image-to-image network is a neural network and this incorporates machine learning and deep learning techniques).),
the machine learning or deep learning techniques including at least one of neural networks (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region (wherein the adversarial network is a neural network).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 20 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of GEORGESCU et al. (US 20200193594 A1), of having wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
Wherein having Co-Pending Application No. 18/851,737 claim 20 having wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) and in view of GEORGESCU et al. (US 20200193594 A1) fail to explicitly teach logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
However, SUGIMOTO (US 20210310053 A1) explicitly teaches logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis (Fig. 1. Paragraph [0099]-SUGIMOTO discloses the machine learning algorithm may comprise one or more of regression analysis, regularization, classification, dimensionality reduction, ensemble learning, meta learning, reinforcement learning, association rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning. The machine learning algorithm may comprise, but is not limited to: k-means, k-means clustering, k-nearest neighbors, learning vector quantization, linear regression, non-linear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principle component regression, least absolute shrinkage and selection operation, least angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal components analysis, principal coordinates analysis, projection pursuit, Sammon mapping, t-distributed stochastic neighbor embedding, AdaBoosting, boosting, gradient boosting, bootstrap aggregation, ensemble averaging, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stacked generalization, Bayesian networks, Bayesian belief networks, naïve Bayes, Gaussian naïve Bayes, multinomial naïve Bayes, hidden Markov models, hierarchical hidden Markov models, support vector machines, encoders, decoders, auto-encoders, stacked auto-encoders, perceptrons, multi-layer perceptrons, artificial neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, long short-term memory, deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, or generative adversarial networks.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 20 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of SUGIMOTO (US 20210310053 A1), of having logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
Wherein having Co-Pending Application No. 18/851,737 claim 20 having logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim 23 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 20 in view of ZHOU et al. (US 20170256090 A1) and further in view of REYNOLDS (US 20170262978 A1).
Regarding claim 23, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) explicitly teaches the apparatus according to claim 17, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
However, REYNOLDS (US 20170262978 A1) explicitly teaches wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs. Paragraph [0049]-REYNOLDS discloses some data sampling paths 220 may pass through gaps between ribs.) and a region of tissue bordering the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs and a region of tissue bordering the ribs. Paragraph [0070]-REYNOLDS discloses at stage 120 the image generation circuitry 28 determines a respective pixel value for each of the plurality of ray paths 500 using the voxel intensity values sampled at stage 118 (wherein the voxel intensity values are image intensities at each coordinate of the 2D manifold image).) Further in paragraph [0075]-REYNOLDS discloses at stage 122 the image generation circuitry 28 generates the two-dimensional output image for display on display screen 16 using the pixel values determined at stage 120. Each pixel of the two-dimensional image corresponds to a respective ray path 500 (wherein the ray path represents the location of the sampling). Please see annotated Figs. 5A-B below.).
PNG
media_image2.png
687
720
media_image2.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5A illustrates the sampling region of tissue between the ribs, as the lungs indicated by the red arrow are made up of tissue and are located between the ribs indicated by the blue arrows.
PNG
media_image3.png
447
455
media_image3.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5B illustrates the sampling region of tissue bordering the ribs, as the lungs indicated by the red arrow are made up of tissue and border the ribs indicated by the blue arrows.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 20 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of REYNOLDS (US 20170262978 A1) of having wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
Wherein having Co-Pending Application No. 18/851,737 claim 20 having wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim 26 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 27 in view of ZHOU et al. (US 20170256090 A1) and further in view of KIRALY et al. (US 20080287796 A1).
Regarding claim 26, Co-Pending Application No. 18/851,737 claim 27 in view of ZHOU et al. (US 20170256090 A1) explicitly teaches the apparatus according to claim 24, Co-Pending Application No. 18/851,737 claim 27 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
However, KIRALY et al. (US 20080287796 A1) explicitly teaches wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs) (Figs. 5 and 6. Paragraph [0026]-KIRALY discloses the method of FIG. 5 can be performed to implement step 106 of FIG. 1, and generates an MRP based volume that aligns the spine along the natural curvature of the spine. This method reformats the image volume based on a curved MPR automatically defined through the spine centerline (wherein the image with a reformatted image volume based on a curved MPR is an interpolated 2D multiplanar reconstruction). Paragraph [0029]-KIRALY discloses the 2D image at each centerline point can be generated by sampling the original image volume using tri-linear interpolation or other well known sampling methods. Further in Paragraph [0030]-KIRALY discloses at step 508, the series of 2D images are stacked, with each 2D image shifted in the x-direction from the average of the x-coordinates by the shift value of the corresponding centerline point (wherein the images stacked are 2D MPRs).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 27 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of KIRALY et al. (US 20080287796 A1), of having wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
Wherein having Co-Pending Application No. 18/851,737 claim 27 having wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim 27 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 27 in view of ZHOU et al. (US 20170256090 A1) and further in view of KIRALY et al. (US 20080287796 A1) and further in view of RAI et al. (US 20160180529 A1).
Regarding claim 27, Co-Pending Application No. 18/851,737 claim 27 in view of ZHOU et al. (US 20170256090 A1) and further in view of KIRALY et al. (US 20080287796 A1) explicitly teaches the apparatus according to claim 24, Co-Pending Application No. 18/851,737 claim 27 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach further comprising generating a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
However, RAI et al. (US 20160180529 A1) explicitly teaches further comprising generating a 3D image of the rib cage and the spine based on the stack of 2D MPRs (Fig. 2. [0050]-RAI discloses with reference to FIG. 2, step 36 recites to compute a 3D-projected location of the anatomical structure based on a 3D data set. Available 3D image data from the subject includes without limitation high resolution computed tomography (HRCT) scans, MRI, PET, 3D angiographic, and X-ray data sets. Optionally, Ribs, spine, and bones are segmented out from 3D image and their centerlines may be computed. Further in paragraph [0051]-RAI discloses in a method, or system, the workstation receives a 3D image file, 3D image data set, or a set of 2D images of the organ from which a 3D model of the organ may be computed (wherein the set 2D images of the organ is a stack of 2D MPRs and wherein the 3D model is a 3D image).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 27 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of RAI et al. (US 20160180529 A1) of having further comprising generating a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
Wherein having Co-Pending Application No. 18/851,737 claim 27 having further comprising generating a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim 28 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 27 in view of ZHOU et al. (US 20170256090 A1) and further in view of KIRALY et al. (US 20080287796 A1) and further in view of RAI et al. (US 20160180529 A1).
Regarding claim 28, Co-Pending Application No. 18/851,737 claim 27 teaches the apparatus according to claim 24, Co-Pending Application No. 18/851,737 claim 27 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
However, GEORGESCU et al. (US 20200193594 A1) explicitly teaches wherein rib segmentation and spine segmentation (Fig. 3. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region.) are performed with machine learning or deep learning techniques (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses the anatomical objects are automatically segmented from the medical image data using an adversarial deep image-to-image network. The adversarial deep image-to-image network comprises a generator network and a discriminator network. The generator network may be a deep image-to-image (DI2I) network that receives the medical image data and anatomical landmark locations (detected at step 104) as input and outputs a probability map indicating a probability score of voxels belonging to the anatomical objects (wherein the deep image-to-image network is a neural network and this incorporates machine learning and deep learning techniques).),
the machine learning or deep learning techniques including at least one of neural networks (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region (wherein the adversarial network is a neural network).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 27 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of GEORGESCU et al. (US 20200193594 A1), of having wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
Wherein having Co-Pending Application No. 18/851,737 claim 27 having wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Co-Pending Application No. 18/851,737 claim 27 in view of ZHOU et al. (US 20170256090 A1) and in view of GEORGESCU et al. (US 20200193594 A1) fail to explicitly teach logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
However, SUGIMOTO (US 20210310053 A1) explicitly teaches logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis (Fig. 1. Paragraph [0099]-SUGIMOTO discloses the machine learning algorithm may comprise one or more of regression analysis, regularization, classification, dimensionality reduction, ensemble learning, meta learning, reinforcement learning, association rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning. The machine learning algorithm may comprise, but is not limited to: k-means, k-means clustering, k-nearest neighbors, learning vector quantization, linear regression, non-linear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principle component regression, least absolute shrinkage and selection operation, least angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal components analysis, principal coordinates analysis, projection pursuit, Sammon mapping, t-distributed stochastic neighbor embedding, AdaBoosting, boosting, gradient boosting, bootstrap aggregation, ensemble averaging, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stacked generalization, Bayesian networks, Bayesian belief networks, naïve Bayes, Gaussian naïve Bayes, multinomial naïve Bayes, hidden Markov models, hierarchical hidden Markov models, support vector machines, encoders, decoders, auto-encoders, stacked auto-encoders, perceptrons, multi-layer perceptrons, artificial neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, long short-term memory, deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, or generative adversarial networks.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 27 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of SUGIMOTO (US 20210310053 A1), of having logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
Wherein having Co-Pending Application No. 18/851,737 claim 27 having logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim 30 is rejected on the ground of non-statutory double patenting as being unpatentable over claim 27 in view of ZHOU et al. (US 20170256090 A1) and further in view of REYNOLDS (US 20170262978 A1).
Regarding claim 30, Co-Pending Application No. 18/851,737 claim 27 in view of ZHOU et al. (US 20170256090 A1) explicitly teaches the apparatus according to claim 24, Co-Pending Application No. 18/851,737 claim 20 in view of ZHOU et al. (US 20170256090 A1) fail to explicitly teach wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
However, REYNOLDS (US 20170262978 A1) explicitly teaches wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs. Paragraph [0049]-REYNOLDS discloses some data sampling paths 220 may pass through gaps between ribs.) and a region of tissue bordering the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs and a region of tissue bordering the ribs. Paragraph [0070]-REYNOLDS discloses at stage 120 the image generation circuitry 28 determines a respective pixel value for each of the plurality of ray paths 500 using the voxel intensity values sampled at stage 118 (wherein the voxel intensity values are image intensities at each coordinate of the 2D manifold image).) Further in paragraph [0075]-REYNOLDS discloses at stage 122 the image generation circuitry 28 generates the two-dimensional output image for display on display screen 16 using the pixel values determined at stage 120. Each pixel of the two-dimensional image corresponds to a respective ray path 500 (wherein the ray path represents the location of the sampling). Please see annotated Figs. 5A-B below.).
PNG
media_image2.png
687
720
media_image2.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5A illustrates the sampling region of tissue between the ribs, as the lungs indicated by the red arrow are made up of tissue and are located between the ribs indicated by the blue arrows.
PNG
media_image3.png
447
455
media_image3.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5B illustrates the sampling region of tissue bordering the ribs, as the lungs indicated by the red arrow are made up of tissue and border the ribs indicated by the blue arrows.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of the Co-Pending Application No. 18/851,737 claim 27 of having a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image, with the teachings of REYNOLDS (US 20170262978 A1) of having wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
Wherein having Co-Pending Application No. 18/851,737 claim 27 having wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 17, 23-24, and 30-31 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU et al. (US 20170256090 A1), hereinafter referenced as ZHOU in view of REYNOLDS (US 20170262978 A1), hereinafter referenced as REYNOLDS.
Regarding claim 17, ZHOU explicitly teaches an apparatus for processing medical images (Fig. 7. Paragraph [0021]-ZHOU discloses the server or workstation performs imaging processing on the three-dimensional scan data to identify and extract one or more three-dimensional structures of the input volume from the three-dimensional scan data.), comprising:
a memory (Fig. 7, #201 called a server. Paragraph [0041]-ZHOU discloses [0041] The server 201 is a server computer platform having hardware such as one or more central processing units (CPU), a system memory, a random access memory (RAM) and input/output (I/O) interface(s).) that stores a plurality of instructions (Fig. 7. Paragraph [0042]-ZHOU discloses the server 201 is configured to execute an application to receive an input volume from the scanner 207 over the network 203. The server 201 is further configured to execute an application (e.g., an image processing module or image processing engine) to perform image processing to the input volume, such as to extract one or more structures from the input volume. The server 201 is further configured to execute an application (e.g., another image processing module, such as an unfolding module, or another image processing engine) to unfold the extracted structures (wherein the applications are instructions).); and
at least one processor coupled to the memory and configured to execute the plurality of instructions to (Fig. 7, #201 called a server. Paragraph [0041]-ZHOU discloses the server 201 is a server computer platform having hardware such as one or more central processing units (CPU), a system memory, a random access memory (RAM) and input/output (I/O) interface(s). The server 201 also includes a graphics processor unit (GPU) to accelerate image rendering. Further in paragraph [0042]-ZHOU discloses the server 201 includes image processor 209 and renderer 211. The image processor 209 and renderer 211 may be implemented in the same or separate hardware or devices. In another alternative, the image processor and/or renderer 211 may be part of the workstation 205 or the scanner 207. In other alternative embodiments, the extraction, rendering, and/or transmission are performed by separate processors or devices.):
detect and label (Fig. 2. Paragraph [0035]-ZHOU discloses segmentation is performed to identify voxels belonging to or representing the specific structure or structures. Any number of separate volumes (e.g., one for each structure segmented from the scan voxels) may be created (wherein detect and label is identifying voxels and creating separate volumes).) rib centerlines and vertebra body center landmarks (Fig. 2-3. Paragraph [0023]-ZHOU discloses the extracted three-dimensional structures include at least a portion of a patient's skeleton. For example, a portion of the patent's skeleton may include a rib centerline and a vertebra disk localization (wherein vertebra disk localization is vertebra body center landmarks).) based on, respectively, rib segmentation and spine segmentation (Figs. 2-3 and 6. Paragraph [0035]-ZHOU discloses at act 205, image processing is applied to the plurality of voxels to extract a plurality of three-dimensional volumes. Segmentation is performed to identify voxels belonging to or representing the specific structure or structures. Any number of separate volumes (e.g., one for each structure segmented from the scan voxels) may be created.);
map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image (Fig. 3, illustrates a result of mapping each 3D position to its 2D position on a 2D manifold image. Paragraph [0036]-ZHOU discloses at act 207, the extracted three-dimensional volumes are unfolded. Geometric transformations are applied to the voxels representing each of the extracted plurality of three-dimensional volumes. For example, due to complex and twisted geometries, the extracted three-dimensional volumes, or parts thereof, may be in different planes (i.e., multi-planar structures). The extracted three-dimensional volumes are unfolded and aligned to be in the same plane, and the unfolding may involve untwisting each of the plurality of volumes (wherein mapping is unfolding and applying geometric transformations and wherein the three-dimensional volumes represent the rib centerlines and vertebra body center landmarks). Please see annotated Fig. 3 below.);
wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine (Fig. 3, illustrates a reformatted 2D manifold image with continuous and straightened visualization of a rib cage and a spine. Paragraph [0030]-ZHOU discloses FIG. 3 illustrates an example of a rendering of an unfolded two-dimensional slice of a volume (wherein an unfolded two-dimensional slice of a volume is a 2D manifold image). Please see annotated Fig. 3 below.).
PNG
media_image1.png
267
764
media_image1.png
Greyscale
Annotated diagram of ZHOU’s Fig. 3 illustrating a 2D manifold image with a rib cage and a spine. The blue boxes indicate various ribs of a rib cage in the image, while the red box indicates the spine.
ZHOU fails to explicitly teach interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
However, REYNOLDS explicitly teaches interpolate coordinates of each 3D position missing on the 2D manifold image (Fig. 2, #112 called interpolate missing sections. Paragraph [0050]-REYNOLDS discloses the manifold determination circuitry 24 determines an estimated manifold position for each of the data sampling paths 220 that does not intersect a rib, by interpolating or extrapolating from the estimated manifold positions that were determined as midpoints at stage 110 (wherein a 3D position missing on the 2D manifold image is a position that does not intersect a rib). Therefore, it would have been obvious to one of ordinary skill of the art at the time the invention was made to interpolate coordinates of each 3D position missing on the 2D manifold image since REYNOLDS clearly discloses interpolating coordinates of each 3D position missing in a manifold image. Thus, in order to have a more accurate representation of the imaging subject), such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image (Fig. 2. Paragraph [0051]-REYNOLDS discloses the manifold determination circuitry 24 may perform the interpolation by fitting a curve to midpoints that were determined at stage 110. Further in paragraph [0053]-REYNOLDS discloses the determined manifold 400 is a full manifold, which may comprise a continuous surface. In other embodiments, the manifold 400 is a partial manifold, which may comprise at least one discontinuous surface. For example, the manifold 400 may be defined on the ribs and may be not defined for regions between ribs. In the present embodiment, the manifold 400 is a surface that intersects all of the ribs and represents a shape of at least part of the ribcage. In other embodiments, the three-dimensional manifold 400 may intersect any appropriate anatomical structure (wherein a manifold that intersects all the ribs or any appropriate anatomical structure is one that aligns with the detected rib centerlines and vertebra center landmarks in the 3D image).); and
sample image intensities at each coordinate of the 2D manifold image (Fig. 2, #118 called sample points at which rays intersect manifold. Paragraph [0070]-REYNOLDS discloses at stage 120 the image generation circuitry 28 determines a respective pixel value for each of the plurality of ray paths 500 using the voxel intensity values sampled at stage 118 (wherein the voxel intensity values are image intensities at each coordinate of the 2D manifold image).) to reformat the 2D manifold image (Fig. 2, #122 called output two-dimensional image (wherein the two-dimensional image is the 2D manifold image). Paragraph [0075]-REYNOLDS discloses at stage 122 the image generation circuitry 28 generates the two-dimensional output image for display on display screen 16 using the pixel values determined at stage 120. Each pixel of the two-dimensional image corresponds to a respective ray path 500. Since the ray paths 500 are defined in a tilted cylindrical configuration, the two-dimensional image provides an unfolded view of the ribs in which at least some of the ribs may appear to be substantially horizontal (wherein the image generated is the 2D manifold image). Therefore, it would have been obvious to one of ordinary skill of the art at the time the invention was made to sample image intensities at each coordinate of the 2D manifold image since REYNOLDS clearly discloses sampling the intensities of a 3D manifold image. Thus, in order to have image intensities that match in the manifold image and the reformatted manifold image.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of REYNOLDS of interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine having interpolate coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sample image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and REYNOLDS analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while REYNOLDS there is a need for clinicians to be able to identify and count displaced and non-displaced fractures or bone lesions when performing chest examinations. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and REYNOLDS (US 20170262978 A1), Paragraph [0002].
Regarding claim 23, ZHOU in view of REYNOLDS explicitly teach the apparatus according to claim 17,
ZHOU fails to explicitly teach wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
However, REYNOLDS explicitly teaches wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs. Paragraph [0049]-REYNOLDS discloses some data sampling paths 220 may pass through gaps between ribs.) and a region of tissue bordering the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs and a region of tissue bordering the ribs. Paragraph [0070]-REYNOLDS discloses at stage 120 the image generation circuitry 28 determines a respective pixel value for each of the plurality of ray paths 500 using the voxel intensity values sampled at stage 118 (wherein the voxel intensity values are image intensities at each coordinate of the 2D manifold image).) Further in paragraph [0075]-REYNOLDS discloses at stage 122 the image generation circuitry 28 generates the two-dimensional output image for display on display screen 16 using the pixel values determined at stage 120. Each pixel of the two-dimensional image corresponds to a respective ray path 500 (wherein the ray path represents the location of the sampling). Please see annotated Figs. 5A-B below.).
PNG
media_image2.png
687
720
media_image2.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5A illustrates the sampling region of tissue between the ribs, as the lungs indicated by the red arrow are made up of tissue and are located between the ribs indicated by the blue arrows.
PNG
media_image3.png
447
455
media_image3.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5B illustrates the sampling region of tissue bordering the ribs, as the lungs indicated by the red arrow are made up of tissue and border the ribs indicated by the blue arrows.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of REYNOLDS of wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and REYNOLDS analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while REYNOLDS there is a need for clinicians to be able to identify and count displaced and non-displaced fractures or bone lesions when performing chest examinations. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and REYNOLDS (US 20170262978 A1), Paragraph [0002].
Regarding claim 24, ZHOU explicitly teaches a computer implemented method for processing medical images (Fig. 7. Paragraph [0021]-ZHOU discloses the server or workstation performs imaging processing on the three-dimensional scan data to identify and extract one or more three-dimensional structures of the input volume from the three-dimensional scan data.), comprising:
detecting and labeling (Fig. 2. Paragraph [0035]-ZHOU discloses segmentation is performed to identify voxels belonging to or representing the specific structure or structures. Any number of separate volumes (e.g., one for each structure segmented from the scan voxels) may be created (wherein detect and label is identifying voxels and creating separate volumes).) rib centerlines and vertebra body center landmarks (Fig. 2-3. Paragraph [0023]-ZHOU discloses the extracted three-dimensional structures include at least a portion of a patient's skeleton. For example, a portion of the patent's skeleton may include a rib centerline and a vertebra disk localization (wherein vertebra disk localization is vertebra body center landmarks).) based on, respectively, rib segmentation and spine segmentation (Figs. 2-3 and 6. Paragraph [0035]-ZHOU discloses at act 205, image processing is applied to the plurality of voxels to extract a plurality of three-dimensional volumes. Segmentation is performed to identify voxels belonging to or representing the specific structure or structures. Any number of separate volumes (e.g., one for each structure segmented from the scan voxels) may be created.);
mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image (Fig. 3, illustrates a result of mapping each 3D position to its 2D position on a 2D manifold image. Paragraph [0036]-ZHOU discloses at act 207, the extracted three-dimensional volumes are unfolded. Geometric transformations are applied to the voxels representing each of the extracted plurality of three-dimensional volumes. For example, due to complex and twisted geometries, the extracted three-dimensional volumes, or parts thereof, may be in different planes (i.e., multi-planar structures). The extracted three-dimensional volumes are unfolded and aligned to be in the same plane, and the unfolding may involve untwisting each of the plurality of volumes (wherein mapping is unfolding and applying geometric transformations and wherein the three-dimensional volumes represent the rib centerlines and vertebra body center landmarks). Please see annotated Fig. 3 below.);
wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine (Fig. 3, illustrates a reformatted 2D manifold image with continuous and straightened visualization of a rib cage and a spine. Paragraph [0030]-ZHOU discloses FIG. 3 illustrates an example of a rendering of an unfolded two-dimensional slice of a volume (wherein an unfolded two-dimensional slice of a volume is a 2D manifold image). Please see annotated Fig. 3 below.).
PNG
media_image1.png
267
764
media_image1.png
Greyscale
Annotated diagram of ZHOU’s Fig. 3 illustrating a 2D manifold image with a rib cage and a spine. The blue boxes indicate various ribs of a rib cage in the image, while the red box indicates the spine.
ZHOU fails to explicitly teach interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
However, REYNOLDS explicitly teaches interpolating coordinates of each 3D position missing on the 2D manifold image (Fig. 2, #112 called interpolate missing sections. Paragraph [0050]-REYNOLDS discloses the manifold determination circuitry 24 determines an estimated manifold position for each of the data sampling paths 220 that does not intersect a rib, by interpolating or extrapolating from the estimated manifold positions that were determined as midpoints at stage 110 (wherein a 3D position missing on the 2D manifold image is a position that does not intersect a rib). Therefore, it would have been obvious to one of ordinary skill of the art at the time the invention was made to interpolate coordinates of each 3D position missing on the 2D manifold image since REYNOLDS clearly discloses interpolating coordinates of each 3D position missing in a manifold image. Thus, in order to have a more accurate representation of the imaging subject), such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image (Fig. 2. Paragraph [0051]-REYNOLDS discloses the manifold determination circuitry 24 may perform the interpolation by fitting a curve to midpoints that were determined at stage 110. Further in paragraph [0053]-REYNOLDS discloses the determined manifold 400 is a full manifold, which may comprise a continuous surface. In other embodiments, the manifold 400 is a partial manifold, which may comprise at least one discontinuous surface. For example, the manifold 400 may be defined on the ribs and may be not defined for regions between ribs. In the present embodiment, the manifold 400 is a surface that intersects all of the ribs and represents a shape of at least part of the ribcage. In other embodiments, the three-dimensional manifold 400 may intersect any appropriate anatomical structure (wherein a manifold that intersects all the ribs or any appropriate anatomical structure is one that aligns with the detected rib centerlines and vertebra center landmarks in the 3D image).); and
sampling image intensities at each coordinate of the 2D manifold image (Fig. 2, #118 called sample points at which rays intersect manifold. Paragraph [0070]-REYNOLDS discloses at stage 120 the image generation circuitry 28 determines a respective pixel value for each of the plurality of ray paths 500 using the voxel intensity values sampled at stage 118 (wherein the voxel intensity values are image intensities at each coordinate of the 2D manifold image).) to reformat the 2D manifold image (Fig. 2, #122 called output two-dimensional image (wherein the two-dimensional image is the 2D manifold image). Paragraph [0075]-REYNOLDS discloses at stage 122 the image generation circuitry 28 generates the two-dimensional output image for display on display screen 16 using the pixel values determined at stage 120. Each pixel of the two-dimensional image corresponds to a respective ray path 500. Since the ray paths 500 are defined in a tilted cylindrical configuration, the two-dimensional image provides an unfolded view of the ribs in which at least some of the ribs may appear to be substantially horizontal (wherein the image generated is the 2D manifold image). Therefore, it would have been obvious to one of ordinary skill of the art at the time the invention was made to sample image intensities at each coordinate of the 2D manifold image since REYNOLDS clearly discloses sampling the intensities of a 3D manifold image. Thus, in order to have image intensities that match in the manifold image and the reformatted manifold image.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of REYNOLDS of interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine having interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and REYNOLDS analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while REYNOLDS there is a need for clinicians to be able to identify and count displaced and non-displaced fractures or bone lesions when performing chest examinations. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and REYNOLDS (US 20170262978 A1), Paragraph [0002].
Regarding claim 30, ZHOU in view of REYNOLDS explicitly teach the method according to claim 24,
ZHOU fails to explicitly teach wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
However, REYNOLDS explicitly teaches wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs. Paragraph [0049]-REYNOLDS discloses some data sampling paths 220 may pass through gaps between ribs.) and a region of tissue bordering the ribs (Figs. 5A-B, illustrate a region of tissue between the ribs and a region of tissue bordering the ribs. Paragraph [0070]-REYNOLDS discloses at stage 120 the image generation circuitry 28 determines a respective pixel value for each of the plurality of ray paths 500 using the voxel intensity values sampled at stage 118 (wherein the voxel intensity values are image intensities at each coordinate of the 2D manifold image).) Further in paragraph [0075]-REYNOLDS discloses at stage 122 the image generation circuitry 28 generates the two-dimensional output image for display on display screen 16 using the pixel values determined at stage 120. Each pixel of the two-dimensional image corresponds to a respective ray path 500 (wherein the ray path represents the location of the sampling). Please see annotated Figs. 5A-B below.).
PNG
media_image2.png
687
720
media_image2.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5A illustrates the sampling region of tissue between the ribs, as the lungs indicated by the red arrow are made up of tissue and are located between the ribs indicated by the blue arrows.
PNG
media_image3.png
447
455
media_image3.png
Greyscale
Annotated diagram of REYNOLDS’ Figs. 5B illustrates the sampling region of tissue bordering the ribs, as the lungs indicated by the red arrow are made up of tissue and border the ribs indicated by the blue arrows.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of REYNOLDS of wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the sampled image intensities at each position of the reformatted 2D manifold image correspond to at least one of a region of tissue between the ribs and a region of tissue bordering the ribs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and REYNOLDS analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while REYNOLDS there is a need for clinicians to be able to identify and count displaced and non-displaced fractures or bone lesions when performing chest examinations. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and REYNOLDS (US 20170262978 A1), Paragraph [0002].
Regarding claim 31, ZHOU explicitly teaches a non-transitory computer-readable medium (Fig. 7, #201 called a server. Paragraph [0041]-ZHOU discloses [0041] The server 201 is a server computer platform having hardware such as one or more central processing units (CPU), a system memory, a random access memory (RAM) and input/output (I/O) interface(s).) for storing executable instructions, which cause a method to be performed to process medical images (Fig. 7. Paragraph [0042]-ZHOU discloses the server 201 is configured to execute an application to receive an input volume from the scanner 207 over the network 203. The server 201 is further configured to execute an application (e.g., an image processing module or image processing engine) to perform image processing to the input volume, such as to extract one or more structures from the input volume. The server 201 is further configured to execute an application (e.g., another image processing module, such as an unfolding module, or another image processing engine) to unfold the extracted structures (wherein the applications are instructions).), the method comprising:
detecting and labeling (Fig. 2. Paragraph [0035]-ZHOU discloses segmentation is performed to identify voxels belonging to or representing the specific structure or structures. Any number of separate volumes (e.g., one for each structure segmented from the scan voxels) may be created (wherein detect and label is identifying voxels and creating separate volumes).) rib centerlines and vertebra body center landmarks (Fig. 2-3. Paragraph [0023]-ZHOU discloses the extracted three-dimensional structures include at least a portion of a patient's skeleton. For example, a portion of the patent's skeleton may include a rib centerline and a vertebra disk localization (wherein vertebra disk localization is vertebra body center landmarks).) based on, respectively, rib segmentation and spine segmentation (Figs. 2-3 and 6. Paragraph [0035]-ZHOU discloses at act 205, image processing is applied to the plurality of voxels to extract a plurality of three-dimensional volumes. Segmentation is performed to identify voxels belonging to or representing the specific structure or structures. Any number of separate volumes (e.g., one for each structure segmented from the scan voxels) may be created.);
mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image (Fig. 3, illustrates a result of mapping each 3D position to its 2D position on a 2D manifold image. Paragraph [0036]-ZHOU discloses at act 207, the extracted three-dimensional volumes are unfolded. Geometric transformations are applied to the voxels representing each of the extracted plurality of three-dimensional volumes. For example, due to complex and twisted geometries, the extracted three-dimensional volumes, or parts thereof, may be in different planes (i.e., multi-planar structures). The extracted three-dimensional volumes are unfolded and aligned to be in the same plane, and the unfolding may involve untwisting each of the plurality of volumes (wherein mapping is unfolding and applying geometric transformations and wherein the three-dimensional volumes represent the rib centerlines and vertebra body center landmarks). Please see annotated Fig. 3 below.);
wherein the reformatted 2D manifold image provides a continuous and straightened visualization of a rib cage and a spine (Fig. 3, illustrates a reformatted 2D manifold image with continuous and straightened visualization of a rib cage and a spine. Paragraph [0030]-ZHOU discloses FIG. 3 illustrates an example of a rendering of an unfolded two-dimensional slice of a volume (wherein an unfolded two-dimensional slice of a volume is a 2D manifold image). Please see annotated Fig. 3 below.).
PNG
media_image1.png
267
764
media_image1.png
Greyscale
Annotated diagram of ZHOU’s Fig. 3 illustrating a 2D manifold image with a rib cage and a spine. The blue boxes indicate various ribs of a rib cage in the image, while the red box indicates the spine.
ZHOU fails to explicitly teach interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
However, REYNOLDS explicitly teaches interpolating coordinates of each 3D position missing on the 2D manifold image (Fig. 2, #112 called interpolate missing sections. Paragraph [0050]-REYNOLDS discloses the manifold determination circuitry 24 determines an estimated manifold position for each of the data sampling paths 220 that does not intersect a rib, by interpolating or extrapolating from the estimated manifold positions that were determined as midpoints at stage 110 (wherein a 3D position missing on the 2D manifold image is a position that does not intersect a rib). Therefore, it would have been obvious to one of ordinary skill of the art at the time the invention was made to interpolate coordinates of each 3D position missing on the 2D manifold image since REYNOLDS clearly discloses interpolating coordinates of each 3D position missing in a manifold image. Thus, in order to have a more accurate representation of the imaging subject), such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image (Fig. 2. Paragraph [0051]-REYNOLDS discloses the manifold determination circuitry 24 may perform the interpolation by fitting a curve to midpoints that were determined at stage 110. Further in paragraph [0053]-REYNOLDS discloses the determined manifold 400 is a full manifold, which may comprise a continuous surface. In other embodiments, the manifold 400 is a partial manifold, which may comprise at least one discontinuous surface. For example, the manifold 400 may be defined on the ribs and may be not defined for regions between ribs. In the present embodiment, the manifold 400 is a surface that intersects all of the ribs and represents a shape of at least part of the ribcage. In other embodiments, the three-dimensional manifold 400 may intersect any appropriate anatomical structure (wherein a manifold that intersects all the ribs or any appropriate anatomical structure is one that aligns with the detected rib centerlines and vertebra center landmarks in the 3D image).); and
sampling image intensities at each coordinate of the 2D manifold image (Fig. 2, #118 called sample points at which rays intersect manifold. Paragraph [0070]-REYNOLDS discloses at stage 120 the image generation circuitry 28 determines a respective pixel value for each of the plurality of ray paths 500 using the voxel intensity values sampled at stage 118 (wherein the voxel intensity values are image intensities at each coordinate of the 2D manifold image).) to reformat the 2D manifold image (Fig. 2, #122 called output two-dimensional image (wherein the two-dimensional image is the 2D manifold image). Paragraph [0075]-REYNOLDS discloses at stage 122 the image generation circuitry 28 generates the two-dimensional output image for display on display screen 16 using the pixel values determined at stage 120. Each pixel of the two-dimensional image corresponds to a respective ray path 500. Since the ray paths 500 are defined in a tilted cylindrical configuration, the two-dimensional image provides an unfolded view of the ribs in which at least some of the ribs may appear to be substantially horizontal (wherein the image generated is the 2D manifold image). Therefore, it would have been obvious to one of ordinary skill of the art at the time the invention was made to sample image intensities at each coordinate of the 2D manifold image since REYNOLDS clearly discloses sampling the intensities of a 3D manifold image. Thus, in order to have image intensities that match in the manifold image and the reformatted manifold image.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU of a non-transitory computer-readable medium for storing executable instructions, which cause a method to be performed to process medical images, the method comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image of REYNOLDS of interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image; and sampling image intensities at each coordinate of the 2D manifold image to reformat the 2D manifold image.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and REYNOLDS analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while REYNOLDS there is a need for clinicians to be able to identify and count displaced and non-displaced fractures or bone lesions when performing chest examinations. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and REYNOLDS (US 20170262978 A1), Paragraph [0002].
Claims 18 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU et al. (US 20170256090 A1), hereinafter referenced as ZHOU in view of REYNOLDS (US 20170262978 A1), hereinafter referenced as REYNOLDS, and further in view of GNANAMANI et al. (US 20150131881 A1), hereinafter referenced as GNANAMANI.
Regarding claim 18, ZHOU in view of REYNOLDS explicitly teach the apparatus according to claim 17,
ZHOU in view of REYNOLDS fail to explicitly teach wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction and repeat the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine.
However, GNANAMANI explicitly teaches wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction (Fig. 9. Paragraph [0069]-GNANAMANI discloses FIG. 9 shows an example of the 2D transformed image shifted in posterior direction (wherein a posterior direction is a normal direction and wherein the 2D transformed image is a 2D manifold image).) and repeat the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine (Figs. 4, 5, and 7. Paragraph [0066]-GNANAMANI discloses along such a fitted curve intensity values are sampled from the 3D image data and mapped into the straightened configuration, e.g. along the vertical direction, in order to generate the portion of a spine in a 2D transformed image. The procedure is repeated within the radius of the spinal column for the left and right half of the rib cage. Further in paragraph [0069]-GNANAMANI discloses a multitude of 2D transformed images can be saved as a stack, in particular in DICOM format, so that the stack can be stored on a PACS server and accessed and transferred easily. In such a case a single 2D transformed image is also a slice of a stack (wherein the 2D transformed images are manifold slices and wherein covering a 3D visualization of the rib cage and the spine is using the 3D image data of a rib cage and spine to generate the 2D transformed image).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of GNANAMANI of wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction and repeat the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction and repeat the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and GNANAMANI analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while GNANAMANI as a result the invention increases the speed and reliability of the reading workflow. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and GNANAMANI et al. (US 20150131881 A1), Paragraph [0006].
Regarding claim 25, ZHOU in view of REYNOLDS explicitly teach the method according to claim 24,
However, GNANAMANI explicitly teaches further comprising shifting the reformatted 2D manifold image along a normal direction (Fig. 9. Paragraph [0069]-GNANAMANI discloses FIG. 9 shows an example of the 2D transformed image shifted in posterior direction (wherein a posterior direction is a normal direction and wherein the 2D transformed image is a 2D manifold image).) and repeating the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine (Figs. 4, 5, and 7. Paragraph [0066]-GNANAMANI discloses along such a fitted curve intensity values are sampled from the 3D image data and mapped into the straightened configuration, e.g. along the vertical direction, in order to generate the portion of a spine in a 2D transformed image. The procedure is repeated within the radius of the spinal column for the left and right half of the rib cage. Further in paragraph [0069]-GNANAMANI discloses a multitude of 2D transformed images can be saved as a stack, in particular in DICOM format, so that the stack can be stored on a PACS server and accessed and transferred easily. In such a case a single 2D transformed image is also a slice of a stack (wherein the 2D transformed images are manifold slices and wherein covering a 3D visualization of the rib cage and the spine is using the 3D image data of a rib cage and spine to generate the 2D transformed image).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of GNANAMANI of wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction and repeat the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the at least one processor is further configured to shift the reformatted 2D manifold image along a normal direction and repeat the sampling to generate a stack of manifold slices covering a 3D visualization of the rib cage and the spine.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and GNANAMANI analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while GNANAMANI as a result the invention increases the speed and reliability of the reading workflow. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and GNANAMANI et al. (US 20150131881 A1), Paragraph [0006].
Claims 19 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU et al. (US 20170256090 A1), hereinafter referenced as ZHOU in view of REYNOLDS (US 20170262978 A1), hereinafter referenced as REYNOLDS, and further in view of KIRALY et al. (US 20080287796 A1), hereinafter referenced as KIRALY.
Regarding claim 19, ZHOU in view of REYNOLDS explicitly teach the apparatus according to claim 17,
ZHOU in view of REYNOLDS fail to explicitly teach wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
However, KIRALY explicitly teaches wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs) (Figs. 5 and 6. Paragraph [0026]-KIRALY discloses the method of FIG. 5 can be performed to implement step 106 of FIG. 1, and generates an MRP based volume that aligns the spine along the natural curvature of the spine. This method reformats the image volume based on a curved MPR automatically defined through the spine centerline (wherein the image with a reformatted image volume based on a curved MPR is an interpolated 2D multiplanar reconstruction). Paragraph [0029]-KIRALY discloses the 2D image at each centerline point can be generated by sampling the original image volume using tri-linear interpolation or other well known sampling methods. Further in Paragraph [0030]-KIRALY discloses at step 508, the series of 2D images are stacked, with each 2D image shifted in the x-direction from the average of the x-coordinates by the shift value of the corresponding centerline point (wherein the images stacked are 2D MPRs).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of KIRALY of wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and KIRALY analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while KIRALY generate improved views of the spine that can be used for further analysis or segmentation of the spine. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and KIRALY et al. (US 20080287796 A1), Paragraph [0005].
Regarding claim 26, ZHOU in view of REYNOLDS explicitly teach the method according to claim 24,
ZHOU in view of REYNOLDS fail to explicitly teach wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
However, KIRALY explicitly teaches wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs) (Figs. 5 and 6. Paragraph [0026]-KIRALY discloses the method of FIG. 5 can be performed to implement step 106 of FIG. 1, and generates an MRP based volume that aligns the spine along the natural curvature of the spine. This method reformats the image volume based on a curved MPR automatically defined through the spine centerline (wherein the image with a reformatted image volume based on a curved MPR is an interpolated 2D multiplanar reconstruction). Paragraph [0029]-KIRALY discloses the 2D image at each centerline point can be generated by sampling the original image volume using tri-linear interpolation or other well known sampling methods. Further in Paragraph [0030]-KIRALY discloses at step 508, the series of 2D images are stacked, with each 2D image shifted in the x-direction from the average of the x-coordinates by the shift value of the corresponding centerline point (wherein the images stacked are 2D MPRs).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of KIRALY of wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the generated stack of manifold slices is a stack of interpolated 2D multiplanar reconstructions (MPRs).
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and KIRALY analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while KIRALY generate improved views of the spine that can be used for further analysis or segmentation of the spine. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and KIRALY et al. (US 20080287796 A1), Paragraph [0005].
Claims 20 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU et al. (US 20170256090 A1), hereinafter referenced as ZHOU in view of REYNOLDS (US 20170262978 A1), hereinafter referenced as REYNOLDS, and further in view of KIRALY et al. (US 20080287796 A1), hereinafter referenced as KIRALY, and further in view of RAI et al. (US 20160180529 A1), hereinafter referenced as RAI.
Regarding claim 20, ZHOU in view of REYNOLDS and further in view of KIRALY explicitly teach the apparatus according to claim 19,
ZHOU in view of REYNOLDS and further in view of KIRALY fail to explicitly teach wherein the at least one processor is further configured to generate a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
However, RAI explicitly teaches wherein the at least one processor is further configured to generate a 3D image of the rib cage and the spine based on the stack of 2D MPRs (Fig. 2. [0050]-RAI discloses with reference to FIG. 2, step 36 recites to compute a 3D-projected location of the anatomical structure based on a 3D data set. Available 3D image data from the subject includes without limitation high resolution computed tomography (HRCT) scans, MRI, PET, 3D angiographic, and X-ray data sets. Optionally, Ribs, spine, and bones are segmented out from 3D image and their centerlines may be computed. Further in paragraph [0051]-RAI discloses in a method, or system, the workstation receives a 3D image file, 3D image data set, or a set of 2D images of the organ from which a 3D model of the organ may be computed (wherein the set 2D images of the organ is a stack of 2D MPRs and wherein the 3D model is a 3D image).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of RAI of wherein the at least one processor is further configured to generate a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the at least one processor is further configured to generate a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and RAI analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while RAI more accurately registers images without biasing towards certain anatomical features. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and RAI et al. (US 20160180529 A1), Paragraph [0002-0004].
Regarding claim 27, ZHOU in view of REYNOLDS and further in view of KIRALY explicitly teach the method according to claim 26,
However, RAI explicitly teaches further comprising generating a 3D image of the rib cage and the spine based on the stack of 2D MPRs (Fig. 2. [0050]-RAI discloses with reference to FIG. 2, step 36 recites to compute a 3D-projected location of the anatomical structure based on a 3D data set. Available 3D image data from the subject includes without limitation high resolution computed tomography (HRCT) scans, MRI, PET, 3D angiographic, and X-ray data sets. Optionally, Ribs, spine, and bones are segmented out from 3D image and their centerlines may be computed. Further in paragraph [0051]-RAI discloses in a method, or system, the workstation receives a 3D image file, 3D image data set, or a set of 2D images of the organ from which a 3D model of the organ may be computed (wherein the set 2D images of the organ is a stack of 2D MPRs and wherein the 3D model is a 3D image).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS and further in view of KIRALY of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of RAI of further comprising generating a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine further comprising generating a 3D image of the rib cage and the spine based on the stack of 2D MPRs.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and RAI analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while RAI more accurately registers images without biasing towards certain anatomical features. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and RAI et al. (US 20160180529 A1), Paragraph [0002-0004].
Claims 21 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU et al. (US 20170256090 A1), hereinafter referenced as ZHOU in view of REYNOLDS (US 20170262978 A1), hereinafter referenced as REYNOLDS, and further in view of GEORGESCU et al. (US 20200193594 A1), hereinafter referenced as GEORGESCU, and further in view of SUGIMOTO (US 20210310053 A1), hereinafter referenced as SUGIMOTO.
Regarding claim 21, ZHOU in view of REYNOLDS explicitly teach the apparatus according to claim 17,
ZHOU in view of REYNOLDS fail to explicitly teach wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
However, GEORGESCU explicitly teaches wherein rib segmentation and spine segmentation (Fig. 3. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region.) are performed with machine learning or deep learning techniques (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses the anatomical objects are automatically segmented from the medical image data using an adversarial deep image-to-image network. The adversarial deep image-to-image network comprises a generator network and a discriminator network. The generator network may be a deep image-to-image (DI2I) network that receives the medical image data and anatomical landmark locations (detected at step 104) as input and outputs a probability map indicating a probability score of voxels belonging to the anatomical objects (wherein the deep image-to-image network is a neural network and this incorporates machine learning and deep learning techniques).),
the machine learning or deep learning techniques including at least one of neural networks (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region (wherein the adversarial network is a neural network).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of GEORGESCU of wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and GEORGESCU analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while GEORGESCU more accurately registers images without biasing towards certain anatomical features. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and GEORGESCU et al. (US 20200193594 A1), Paragraph [0002-0004].
ZHOU in view of REYNOLDS and further in view of GEORGESCU fail to explicitly teach logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
However, SUGIMOTO explicitly teaches logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis (Fig. 1. Paragraph [0099]-SUGIMOTO discloses the machine learning algorithm may comprise one or more of regression analysis, regularization, classification, dimensionality reduction, ensemble learning, meta learning, reinforcement learning, association rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning. The machine learning algorithm may comprise, but is not limited to: k-means, k-means clustering, k-nearest neighbors, learning vector quantization, linear regression, non-linear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principle component regression, least absolute shrinkage and selection operation, least angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal components analysis, principal coordinates analysis, projection pursuit, Sammon mapping, t-distributed stochastic neighbor embedding, AdaBoosting, boosting, gradient boosting, bootstrap aggregation, ensemble averaging, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stacked generalization, Bayesian networks, Bayesian belief networks, naïve Bayes, Gaussian naïve Bayes, multinomial naïve Bayes, hidden Markov models, hierarchical hidden Markov models, support vector machines, encoders, decoders, auto-encoders, stacked auto-encoders, perceptrons, multi-layer perceptrons, artificial neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, long short-term memory, deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, or generative adversarial networks.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS and further in view of GEORGESCU of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of SUGIMOTO of logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine having logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and SUGIMOTO analyzing biomedical imaging data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while SUGIMOTO image-based pooled screening may provide many advantages over the pooled screening and image-based screening methods described above. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and SUGIMOTO (US 20210310053 A1), Paragraph [0082].
Regarding claim 28, ZHOU in view of REYNOLDS explicitly teach the method according to claim 24,
ZHOU in view of REYNOLDS fail to explicitly teach wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
However, GEORGESCU explicitly teaches wherein rib segmentation and spine segmentation (Fig. 3. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region.) are performed with machine learning or deep learning techniques (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses the anatomical objects are automatically segmented from the medical image data using an adversarial deep image-to-image network. The adversarial deep image-to-image network comprises a generator network and a discriminator network. The generator network may be a deep image-to-image (DI2I) network that receives the medical image data and anatomical landmark locations (detected at step 104) as input and outputs a probability map indicating a probability score of voxels belonging to the anatomical objects (wherein the deep image-to-image network is a neural network and this incorporates machine learning and deep learning techniques).),
the machine learning or deep learning techniques including at least one of neural networks (Fig. 3, illustrates a U-net. Paragraph [0022]-GEORGESCU discloses adversarial deep image-to-image network was trained to segment the following anatomical objects: all five lung lobes, airways, bone regions, ribs, spine, femur heads, brain, esophagus, heart, aorta, liver, spline, pancreas, bladder, prostate, rectum, left and right kidney, abdominal region, mediastinal region, and axillary region (wherein the adversarial network is a neural network).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of GEORGESCU of wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein rib segmentation and spine segmentation are performed with machine learning or deep learning techniques, the machine learning or deep learning techniques including at least one of neural networks.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and GEORGESCU analyzing and visualizing medical image data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while GEORGESCU more accurately registers images without biasing towards certain anatomical features. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and GEORGESCU et al. (US 20200193594 A1), Paragraph [0002-0004].
ZHOU in view of REYNOLDS and further in view of GEORGESCU fail to explicitly teach logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
However, SUGIMOTO explicitly teaches logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis (Fig. 1. Paragraph [0099]-SUGIMOTO discloses the machine learning algorithm may comprise one or more of regression analysis, regularization, classification, dimensionality reduction, ensemble learning, meta learning, reinforcement learning, association rule learning, cluster analysis, anomaly detection, deep learning, or ultra-deep learning. The machine learning algorithm may comprise, but is not limited to: k-means, k-means clustering, k-nearest neighbors, learning vector quantization, linear regression, non-linear regression, least squares regression, partial least squares regression, logistic regression, stepwise regression, multivariate adaptive regression splines, ridge regression, principle component regression, least absolute shrinkage and selection operation, least angle regression, canonical correlation analysis, factor analysis, independent component analysis, linear discriminant analysis, multidimensional scaling, non-negative matrix factorization, principal components analysis, principal coordinates analysis, projection pursuit, Sammon mapping, t-distributed stochastic neighbor embedding, AdaBoosting, boosting, gradient boosting, bootstrap aggregation, ensemble averaging, decision trees, conditional decision trees, boosted decision trees, gradient boosted decision trees, random forests, stacked generalization, Bayesian networks, Bayesian belief networks, naïve Bayes, Gaussian naïve Bayes, multinomial naïve Bayes, hidden Markov models, hierarchical hidden Markov models, support vector machines, encoders, decoders, auto-encoders, stacked auto-encoders, perceptrons, multi-layer perceptrons, artificial neural networks, feedforward neural networks, convolutional neural networks, recurrent neural networks, long short-term memory, deep belief networks, deep Boltzmann machines, deep convolutional neural networks, deep recurrent neural networks, or generative adversarial networks.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS and further in view of GEORGESCU of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of SUGIMOTO of logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine having logistic regression, random forests, nearest neighbors, and cluster or multivariate analysis.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and SUGIMOTO analyzing biomedical imaging data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while SUGIMOTO image-based pooled screening may provide many advantages over the pooled screening and image-based screening methods described above. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and SUGIMOTO (US 20210310053 A1), Paragraph [0082].
Claims 22 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over ZHOU et al. (US 20170256090 A1), hereinafter referenced as ZHOU in view of REYNOLDS (US 20170262978 A1), hereinafter referenced as REYNOLDS, and further in view GRASS et al. (US 20090141935 A1), hereinafter referenced as GRASS.
Regarding claim 22, ZHOU in view of REYNOLDS explicitly teach the apparatus according to claim 17,
ZHOU in view of REYNOLDS fails to explicitly teach wherein the coordinates are interpolated using thin plate spline techniques.
However, GRASS explicitly teaches wherein the coordinates are interpolated using thin plate spline techniques (Fig. 1. Paragraph [0061]-GRASS discloses the complete volume may be reconstructed motion compensated after a spatial extra-/interpolation of the motion vectors at the different volume positions. A possible interpolation method may be the thin plate spline interpolation or a simple tri-linear interpolation.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of an apparatus for processing medical images, comprising: a memory that stores a plurality of instructions; and at least one processor coupled to the memory and configured to execute the plurality of instructions to: detect and label rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; map each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image with the teachings of GRASS of wherein the coordinates are interpolated using thin plate spline techniques.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the coordinates are interpolated using thin plate spline techniques.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and GRASS analyzing biomedical imaging data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while GRASS It may be desirable to provide for an improved motion compensated reconstruction of an object of interest, in particular for an improved motion compensated CT reconstruction of high contrast objects. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and GRASS et al. (US 20090141935 A1), Paragraph [0005].
Regarding claim 29, ZHOU in view of REYNOLDS explicitly teach the method according to claim 24,
ZHOU in view of REYNOLDS fails to explicitly teach wherein the coordinates are interpolated using thin plate spline techniques.
However, GRASS explicitly teaches wherein the coordinates are interpolated using thin plate spline techniques (Fig. 1. Paragraph [0061]-GRASS discloses the complete volume may be reconstructed motion compensated after a spatial extra-/interpolation of the motion vectors at the different volume positions. A possible interpolation method may be the thin plate spline interpolation or a simple tri-linear interpolation.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of ZHOU in view of REYNOLDS of a computer implemented method for processing medical images, comprising: detecting and labeling rib centerlines and vertebra body center landmarks based on, respectively, rib segmentation and spine segmentation; mapping each three-dimensional (3D) position of the rib centerlines and the vertebra body center landmarks to a respective two-dimensional (2D) position on a 2D manifold image; interpolating coordinates of each 3D position missing on the 2D manifold image, such that the 2D manifold image aligns with the detected rib centerlines and vertebra center landmarks in the 3D image with the teachings of GRASS of wherein the coordinates are interpolated using thin plate spline techniques.
Wherein having ZHOU’s method of generating a visualization of the ribs and spine wherein the coordinates are interpolated using thin plate spline techniques.
The motivation behind the modification would have been to obtain a method of generating a visualization of the ribs and spine that enhances the quality and readability of a visualization. Since both ZHOU and GRASS analyzing biomedical imaging data, wherein ZHOU the inventive method provides 2D transformed images which can be read fast and interpreted easily, while GRASS It may be desirable to provide for an improved motion compensated reconstruction of an object of interest, in particular for an improved motion compensated CT reconstruction of high contrast objects. Please see ZHOU et al. (US 20170256090 A1), Paragraph [0031], and GRASS et al. (US 20090141935 A1), Paragraph [0005].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant’s disclosure.
Krishnan et al. (US 20230124879 A1) - A computer-implemented method for visualization of an elongated anatomical structure (20), for example of a fetal spine using ultrasound is provided. The method comprising the steps of: receiving a plurality of 3D ultrasound image volumes, each image volume depicting at least a portion of an elongated anatomical structure (20); on each 3D ultrasound image volume, automatically or semi-automatically fitting a parametric curve (30) to the depicted portion of the elongated anatomical structure, the parametric curve being defined by curve parameters; reformatting each 3D ultrasound image volume by applying a transformation which straightens the parametric curve along at least one axis, so as to generate a plurality of reformatted image volumes and reformatted parametric curves (32, 34); registering the reformatted image volumes with one another by determining the joining point of their respective parametric curves; and fusing the reformatted image volumes with one another to yield a fused image depicting the whole elongated anatomical structure or a larger portion thereof than the 3D ultrasound image volumes… Abstract, Fig. 4.
Krishnan et al. (US 20120172700 A1) - Systems and methods for supporting a diagnostic workflow from a computer system are disclosed herein. In accordance with one implementation, a set of pre-identified anatomical landmarks associated with one or more structures of interest within one or more medical images are presented to a user. In response to a user input selecting at least one or more regions of interest including one or more of the pre-identified anatomical landmarks, the user is automatically navigated to the selected region of interest. In another implementation, a second user input selecting one or more measurement tools is received. An evaluation may be automatically determined based on one or more of the set of anatomical landmarks in response to the second user input…Abstract, Fig. 1.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ETHAN N WOLFSON whose telephone number is (571)272-1898. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ETHAN N WOLFSON/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights