DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-21 are currently pending and under examination herein.
Claims 1-21 are rejected.
Priority
The instant application claims priority to US provisional applications 63347916 filed 01 June 2022, 63348316 filed 02 June 2022, 63348299 filed 02 June 2022, 63348306 filed 02 June 2022, 63348304 filed 02 June 2022, 63396934 filed 10 August 2022, and 63396932 field 10 June 2022. In this action, claims 1-21 are examined as though they had an effective filing date of 01 June 2022. In future actions, the effective filing date of one or more claims may change, due to amendments to the claims, or further analysis of the disclosure(s) of the priority application(s).
Information Disclosure Statement
No information disclosure statement (IDS) is located amongst the submitted documents.
Drawings
The drawings filed 01 June 2023 are accepted.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. In accordance with MPEP 2106, claims found to recite statutory subject matter (Step 1: YES) are then analyzed to determine if the claims recite any concepts that equate to an abstract idea (Step 2A, Prong 1). Claims 1-12 and 18-21 are directed to methods and Claims 13-17 are directed to a system. In the instant application, the claims recite the following limitations that equate to an abstract idea:
Claim 1 recites the limitation - determining training data including a plurality of lumen centerlines; determining a voxel data set corresponding to a voxel of the lumen model; determining a centerline of the lumen model using the centerline status. Based on the broadest reasonable interpretation, determining data could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea. The claim also recites training a neural network using the training data. Based on the broadest reasonable interpretation, training a generic neural network encompasses equations. This draws the limitation to a mathematical concept, which classifies the limitation as an abstract idea.
Claim 2 recites the limitation - determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model. Based on the broadest reasonable interpretation, determining data by modeling rays encompasses equations and could practically be done by the human mind. This draws the limitation to a mathematical concept and a mental process, which classifies the limitation as an abstract idea.
Claim 3 recites the limitation - wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels. This limitation specifies the data determined in the judicial exception of claims 1 and 2. The refined determination indicated by this limitation still represents a judicial expectation.
Claim 4 recites the limitation - wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline. This limitation specifies the centerline determined in the judicial exception of claims 1. The refined determination indicated by this limitation still represents a judicial expectation.
Claim 5 recites the limitation - determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model; and updating the centerline of the lumen model based on the second centerline statuses. Based on the broadest reasonable interpretation, determining and updating data could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea.
Claim 6 recites the limitation - wherein updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold. Based on the broadest reasonable interpretation, comparing data to a threshold encompasses equations and could practically be done by the human mind. This draws the limitation to a mathematical concept and a mental process, which classifies the limitation as an abstract idea.
Claim 7 recites the limitation - casting a plurality of rays from a voxel of a lumen model to a surface of the lumen model. Based on the broadest reasonable interpretation, modeling rays encompasses equations and could practically be done by the human mind. This draws the limitation to a mathematical concept and a mental process, which classifies the limitation as an abstract idea. The claim also recites determining a voxel data set after casting the plurality of rays; and determining a centerline of the lumen model including the voxel using the centerline status. Based on the broadest reasonable interpretation, determining information could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea.
Claim 8 recites the limitation - determining training data including a plurality of lumen centerlines. Based on the broadest reasonable interpretation, determining data could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea. The claim also recites training the neural network using the training data. Based on the broadest reasonable interpretation, training a generic neural network encompasses equations. This draws the limitation to a mathematical concept, which classifies the limitation as an abstract idea.
Claim 9 recites the limitation - wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels. This limitation specifies the determining data in the judicial exception of claim 7. The refined determination indicated by this limitation still represents a judicial expectation.
Claim 10 recites the limitation - wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline. This limitation specifies the determining in the judicial exception of claim 7. The refined determination indicated by this limitation still represents a judicial expectation.
Claim 11 recites the limitation - determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model; and updating the centerline of the lumen model using the second centerline statuses. Based on the broadest reasonable interpretation, determining and updating data could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea.
Claim 12 recites the limitation - wherein updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold. Based on the broadest reasonable interpretation, comparing data to a threshold encompasses equations and could practically be done by the human mind. This draws the limitation to a mathematical concept and a mental process, which classifies the limitation as an abstract idea.
Claim 13 recites the limitation - determining training data including a plurality of lumen centerlines; determining a voxel data set corresponding to a voxel of a lumen model; determining a centerline of the lumen model using the centerline status. Based on the broadest reasonable interpretation, determining information could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea. The claim also recites training a neural network using the training data. Based on the broadest reasonable interpretation, training a generic neural network encompasses equations. This draws the limitation to a mathematical concept, which classifies the limitation as an abstract idea.
Claim 14 recites the limitation - wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model. Based on the broadest reasonable interpretation, modeling rays encompasses equations and could practically be done by the human mind. This draws the limitation to a mathematical concept and a mental process, which classifies the limitation as an abstract idea.
Claim 15 recites the limitation - wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels. This limitation specifies the determining in the judicial exception of claim 13 and 14. The refined determination indicated by this limitation still represents a judicial expectation.
Claim 16 recites the limitation - wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline. This limitation specifies the determining in the judicial exception of claim 13. The refined determination indicated by this limitation still represents a judicial expectation.
Claim 17 recites the limitation - determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model. Based on the broadest reasonable interpretation, determining data could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea. The claim also recites updating the centerline of the lumen model including comparing the second centerline statuses to a centerline threshold. Based on the broadest reasonable interpretation, comparing data to a threshold encompasses equations and could practically be done by the human mind. This draws the limitation to a mathematical concept and a mental process, which classifies the limitation as an abstract idea.
Claim 18 recites the limitation - determining training data including a plurality of lumen centerlines; determining a voxel data set corresponding to a voxel of a lumen model; determining a centerline of the lumen model using the centerline status. Based on the broadest reasonable interpretation, determining information could practically be done by the human mind. This draws the limitation to a mental process, which classifies the limitation as an abstract idea. The claim also recites training a neural network using the training data. Based on the broadest reasonable interpretation, training a generic neural network encompasses equations. This draws the limitation to a mathematical concept, which classifies the limitation as an abstract idea.
Claim 19 recites the limitation - wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model. Based on the broadest reasonable interpretation, modeling rays encompasses equations and could practically be done by the human mind. This draws the limitation to a mathematical concept and a mental process, which classifies the limitation as an abstract idea.
Claim 20 recites the limitation - wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels. This limitation specifies the data determined in the judicial exception of claims 18 and 19. The refined determination indicated by this limitation still represents a judicial expectation.
Claim 21 recites the limitation - wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline. This limitation specifies the data determined in the judicial exception of claims 18. The refined determination indicated by this limitation still represents a judicial expectation.
These limitations recite concepts of determining data, modeling rays, and training a generic neural network that are so generically recited that they can be practically performed in the human mind as claimed, which falls under the “Mental processes” and “Mathematical concepts” grouping of abstract ideas. A mathematical concept need not be expressed in mathematical symbols, because words used in a claim operating on data to solve a problem can serve the same purpose as a formula (MPEP 2106.04(a)(2)). Additionally, both product claims and process claims may recite mental processes, which can include a claim that requires a computer (MPEP 2106.04(a)(2)). Therefore, these limitations fall under the “Mental process” and “Mathematical concepts” groupings of abstract ideas. As such, claims 1-21 recite an abstract idea (Step 2A, Prong 1: YES).
Claims found to recite a judicial exception under Step 2A, Prong 1 are then further analyzed to determine if the claims as a whole integrate the recited judicial exception into a practical application or not (Step 2A, Prong 2). These judicial exceptions are not integrated into a practical application because the claims do not recite an additional element that reflects an improvement to technology (MPEP 2106.04(d)(1)) or particular treatment (MPEP 2106.04(d)(2)). Rather, the claims provide insignificant extra-solution activity (MPEP § 2106.05(g)) and provide mere instructions to apply a judicial exception (MPEP § 2106.05(f)). Specifically, the claims recite the following additional elements:
Claim 1 recites inputting the voxel data set into the neural network; and outputting a centerline status for the voxel from the neural network.
Claim 5 recites inputting a plurality of voxel data sets into the neural network; outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; and for each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network.
Claim 7 recites inputting the voxel data set into a neural network; and outputting a centerline status from the neural network.
Claim 11 recites inputting a plurality of voxel data sets into the neural network; outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; and for each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network.
Claim 13 recites computer usable medium having computer readable program code thereon; program code for inputting the voxel data set into the neural network; program code for outputting a centerline status for the voxel from the neural network.
Claim 17 recites program code for inputting a plurality of voxel data sets into the neural network; program code for outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; and for each voxel data set of the portion of the plurality of voxel data sets, program code for inputting the voxel data set into the neural network and outputting a second centerline status from the neural network.
Claim 18 recites inputting the voxel data set into the neural network; and outputting a centerline status for the voxel from the neural network.
There are no limitations that indicate that the claimed determining data, modeling rays, and training a generic neural network require anything other than generic computing systems. As such, these limitations equate to mere instructions to implement the abstract idea on a generic computer or as computer code that the courts have stated does not render an abstract idea eligible. There is no indication that these steps are affected by the judicial exception in any way and thus do not integrate the recited judicial exception into a practical application. As such, claims 1-21 are directed to an abstract idea (Step 2A, Prong 2: NO).
Claims found to be directed to a judicial exception are then further evaluated to determine if the claims recite an inventive concept that provides significantly more than the judicial exception itself (Step 2B). The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claims recite conventional additional elements that equate to mere instructions to apply the recited exception in a generic way or in a generic computing environment. The claims also recite conventional additional elements that represent insignificant extra-solution activities.
As discussed above, there are no additional limitations to indicate that the claimed determining data, modeling rays, and training a generic neural network require anything other than generic computer components in order to carry out the recited abstract idea in the claims. Claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea or natural law eligible. MPEP 2106.05(f) discloses that mere instructions to apply the judicial exception cannot provide an inventive concept to the claims. As specified in MPEP 2106.05(g), extra-solution activities can be understood as incidental to the primary process or product that are merely a nominal or tangential addition to the claim. Insignificant extra-solution activities include mere data gathering, selecting a particular data source or type of data to be manipulated, and displaying information. Additionally, Chen at al. (2020, Frontiers in Cardiovascular Medicine, Vol. 7: 1-33) teach the use of computers and neural networks for lumen modeling and generating centerline data is well understood, routine, and conventional (Page 15, Table 3).
The additional elements do not comprise an inventive concept when considered individually or as an ordered combination that transforms the claimed judicial exception into a patent-eligible application of the judicial exception. Therefore, the claims do not amount to significantly more than the judicial exception itself (Step 2B: No). As such, Claims 1-21 are not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-6 and 18-21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Alblas et al. (2021, arxiv, 1-9).
Applicable claims include:
Claim 1. A method for determining a lumen model, comprising: (Claim 1.i) determining training data including a plurality of lumen centerlines; (Claim 1.ii) training a neural network using the training data; (Claim 1.iii) determining a voxel data set corresponding to a voxel of the lumen model; (Claim 1.iv) inputting the voxel data set into the neural network; (Claim 1.v) outputting a centerline status for the voxel from the neural network; and (Claim 1.vi) determining a centerline of the lumen model using the centerline status.
Claim 2. The method of claim 1, wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model.
Claim 3. The method of claim 2, wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels.
Claim 4. The method of claim 2, wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline.
Claim 5. The method of claim 1, comprising (Claim 5.i) inputting a plurality of voxel data sets into the neural network; and (Claim 5.ii) outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; (Claim 5.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model; (Claim 5.iv) for each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network; and (Claim 5.v) updating the centerline of the lumen model based on the second centerline statuses.
Claim 6. The method of claim 5, wherein updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold.
Claim 18. A method for determining an airway lumen model, comprising: (Claim 18.i) determining training data including a plurality of lumen centerlines; (Claim 18.ii) training a neural network using the training data; (Claim 18.iii) determining a voxel data set corresponding to a voxel of a lumen model; (Claim 18.iv) inputting the voxel data set into the neural network; (Claim 18.v) outputting a centerline status for the voxel from the neural network; and (Claim 18.vi) determining a centerline of the lumen model using the centerline status.
Claim 19. The method of claim 18, wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model.
Claim 20. The method of claim 19, wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels.
Claim 21. The method of claim 19, wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline.
Regarding Claim 1 and 18, Alblas et al. teach (Claim 1.i) determining training data including a plurality of lumen centerlines (Page 2 Paragraph 3: To develop our method, we used 26 black-blood MR images that are provided as a training set; Page 2 Paragraph 4: Since the axial slices were sparsely annotated, we manually annotated continuous lumen centerlines of the internal and external carotid arteries for all 26 patients). Alblas et al. teach (Claim 1.ii) training a neural network using the training data (Page 2, Paragraph 2: In this work, we explore the idea of using polar coordinate systems for local ROIs and training CNNs to perform regression in these images; Page 2 Paragraph 4: Since the axial slices were sparsely annotated, we manually annotated continuous lumen centerlines of the internal and external carotid arteries for all 26 patients). Alblas et al. teach (Claim 1.iii) determining a voxel data set corresponding to a voxel of the lumen model (Page 2, Paragraph 3: Each MR image consists of 640 or 720 axial slices, with an isotropic voxel spacing; Page 3, Paragraph 3: This function describes the proximity of each voxel to the centerlines of the external carotid artery). The 3D models are composed on voxel. Therefore that are interpreted as voxel datasets that correspond to a voxel and many voxels. Alblas et al. teach (Claim 1.iv) inputting the voxel data set into the neural network (Page 3, Paragraph 6: The input to the CNN is a polar image, constructed by taking the centerline point intersecting an axial slice as the origin. vector containing the voxel intensities the particular ray passes in the Cartesian image. Rays are stacked to form a 2D polar image. Subsequently, this image is used as input to a CNN). Alblas et al. teach (Claim 1.v) outputting a centerline status for the voxel from the neural network (Page 4, Paragraph 4: This network has a two-channel output; Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. To this end, all images were resampled to an isotropic voxel spacing. These centerlines served as input to the second step). Each voxel that is determined to be a centerline in output with a designated status of centerline. Alblas et al. teach (Claim 1.vi) determining a centerline of the lumen model using the centerline status (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. These centerlines served as input to the second step). The voxel was determined to be a centerline by the model through the generation of the status. Claim 18 recites the limitations of claim 1 directed to another method. A definition for lumen provided by the specification (Paragraph 0003 of the published specification (US 20230394202 A1): inner cavities of anatomical tubular structures, also known as lumens, are present in humans and other organisms. Lumens may be hollow, such as airways, or filled with another substance such a blood vessel filled with blood or a bone filled with bone marrow) indicates a lumen can be an airway leading to no difference in interpretation between a method of determining a lumen model (Claim 1) and a method of determining an airway lumen model (Claim 18).
Regarding Claim 2 and 19, Alblas et al. teach determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model (Page 2, Figure 1: local 3D regions-of-interest are extracted around the centerlines and transformed to a polar representation through ray-casting; Page 3, Paragraph 6: At each angle, a ray with a fixed length is cast, resulting in a vector containing the voxel intensities the particular ray passes in the Cartesian image). Claim 19 recites the limitations of claim 2 directed to another method.
Regarding Claim 3 and 20, Alblas et al. teach the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels (Page 3, Paragraph 6: At each angle, a ray with a fixed length is cast, resulting in a vector containing the voxel intensities the particular ray passes in the Cartesian image. Rays are stacked to form a 2D polar image. Subsequently, this image is used as input to a CNN that predicts the aforementioned radii values). Claim 20 recites the limitations of claim 3 directed to another method.
Regarding Claim 4 and 21, Alblas et al. teach the centerline status indicates whether the voxel of the lumen model corresponds to the centerline (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. Coordinates of the resulting centerlines were expressed in world coordinates). Claim 21 recites the limitations of claim 4 directed to another method.
Regarding Claim 5, Alblas et al. teach (Claim 5.i) inputting a plurality of voxel data sets into the neural network (Page 7, Paragraph 2: Here, we have presented a systematic comparison of experimental conditions on the training set. In addition, we evaluated our method on the test set). Alblas et al. teach (Claim 5.ii) outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets (Page 5 Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization; We used randomly sampled patches of 368 x 64x 320 voxels mini-batches of eight samples). Alblas et al. teach (Claim 5.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. These centerlines served as input to the second step). This is interpreted as equivalent to determining which voxel compose the centerline which is equivalent to identifying the centerline within a 3d model composed of voxels. Alblas et al. teach (Claim 5.iv) for each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network (Page 4, Paragraph 3: for new and unseen data, limitations in the centerline localization step may lead to incorrectly centered polar images. To compensate for this and improve robustness, we simulate this phenomenon in the training data by randomly sampling the centerpoint of the polar image within a small radius of the center of mass. This results in a number of slightly different polar images, which are used to train the CNN; Page 7, Paragraph 1: The automatically identified centerline, marked by the white cross, is far off-center. The CNN trained on augmented images shows segmentations that match the underlying image better than the non-augmented one). This could also be interpreted as training and then testing the neural network to identify the centerline from a three dimensional voxel based model which is taught by Alblas et al. as indicated above. Alblas et al. teach (Claim 5.v) updating the centerline of the lumen model based on the second centerline statuses (Page 4, Paragraph 3: for new and unseen data, limitations in the centerline localization step may lead to incorrectly centered polar images. To compensate for this and improve robustness, we simulate this phenomenon in the training data by randomly sampling the centerpoint of the polar image within a small radius of the center of mass. This results in a number of slightly different polar images, which are used to train the CNN; Page 7, Paragraph 1: The automatically identified centerline, marked by the white cross, is far off-center. The CNN trained on augmented images shows segmentations that match the underlying image better than the non-augmented one). This could also be interpreted as training and then testing the neural network to identify the centerline from a three dimensional voxel based model which is taught by Alblas et al. as indicated above.
Regarding Claim 6, Alblas et al. teach updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold (Page 3, Paragraph 3: To locate the centerlines in the first step, we define a cost-function. The function is nonzero within a predefined radius of the centerline). The function that determines the centerline involved comparison with within or beyond a predefined radius, which is interpreted as a threshold.
Claims 13-17 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Gildea et al. (WO 2021007570 A1).
Applicable claims include:
Claim 13. A computer program product for use on a computer system for designing a stent, the computer program product comprising a tangible, non-transient computer usable medium having computer readable program code thereon, the computer readable program code comprising: (Claim 13.i) program code for determining training data including a plurality of lumen centerlines; (Claim 13.ii) program code for training a neural network using the training data; (Claim 13.iii) program code for determining a voxel data set corresponding to a voxel of a lumen model; (Claim 13.iv) program code for inputting the voxel data set into the neural network; (Claim 13.v) program code for outputting a centerline status for the voxel from the neural network; and (Claim 13.vi) program code for determining a centerline of the lumen model using the centerline status.
Claim 14. The computer program product of claim 13, wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model.
Claim 15. The computer program product of claim 14, wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels.
Claim 16. The computer program product of claim 14, wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline.
Claim 17. The computer program product of claim 13, comprising: (Claim 17.i) program code for inputting a plurality of voxel data sets into the neural network; (Claim 17.ii) program code for outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; (Claim 17.iii) program code determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model; (Claim 17.vi) for each voxel data set of the portion of the plurality of voxel data sets, program code for inputting the voxel data set into the neural network and outputting a second centerline status from the neural network; and (Claim 17.v) program code for updating the centerline of the lumen model including comparing the second centerline statuses to a centerline threshold.
Regarding Claim 13-17, the claims are directed to computer readable media storing program code. The limitations recite program code and the intended use/function of the program code. The claims do not recite the code is executed by a processor or other device to perform a function. Therefore, the claims are equivalent to nonfunctional descriptive material that are anticipated by any instructions on a computer readable medium (see MPEP 2111.05). Gildea et al. teach a system includes a processor and a non-transitory memory storing computer executable instructions for designing a stent for placement within an airway of a patient (Paragraph 0005). Therefore, Gildea et al. anticipates claims 13-17.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-21 are rejected under 35 U.S.C. 103 as being unpatentable over Alblas et al. (2021, arxiv, 1-9), as applied to claims 1-6 and 18-21 in a 102 rejection above, in view of Gildea et al. (WO 2021007570 A1), as applied to claims 13-17 in a 102 rejection above. Italicized text from reference art.
Applicable claims include:
Claim 1. A method for determining a lumen model, comprising: (Claim 1.i) determining training data including a plurality of lumen centerlines; (Claim 1.ii) training a neural network using the training data; (Claim 1.iii) determining a voxel data set corresponding to a voxel of the lumen model; (Claim 1.iv) inputting the voxel data set into the neural network; (Claim 1.v) outputting a centerline status for the voxel from the neural network; and (Claim 1.vi) determining a centerline of the lumen model using the centerline status.
Claim 2. The method of claim 1, wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model.
Claim 3. The method of claim 2, wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels.
Claim 4. The method of claim 2, wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline.
Claim 5. The method of claim 1, comprising (Claim 5.i) inputting a plurality of voxel data sets into the neural network; and (Claim 5.ii) outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; (Claim 5.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model; (Claim 5.iv) for each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network; and (Claim 5.v) updating the centerline of the lumen model based on the second centerline statuses.
Claim 6. The method of claim 5, wherein updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold.
Claim 7. A method for designing a stent, comprising: (Claim 7.i) casting a plurality of rays from a voxel of a lumen model to a surface of the lumen model; (Claim 7.ii) determining a voxel data set after casting the plurality of rays; (Claim 7.iii) inputting the voxel data set into a neural network; (Claim 7.iv) outputting a centerline status from the neural network; and (Claim 7.v) determining a centerline of the lumen model including the voxel using the centerline status.
Claim 8. The method of claim 7, comprising: (Claim 8.i) determining training data including a plurality of lumen centerlines; and (Claim 8.ii) training the neural network using the training data.
Claim 9. The method of claim 7, wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels.
Claim 10. The method of claim 7, wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline.
Claim 11. The method of claim 7, comprising: (Claim 11.i) inputting a plurality of voxel data sets into the neural network; and (Claim 11.ii) outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; (Claim 11.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model; (Claim 11.vi) for each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network; and (Claim 11.v) updating the centerline of the lumen model using the second centerline statuses.
Claim 12. The method of claim 11, wherein updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold.
Claim 13. A computer program product for use on a computer system for designing a stent, the computer program product comprising a tangible, non-transient computer usable medium having computer readable program code thereon, the computer readable program code comprising: (Claim 13.i) program code for determining training data including a plurality of lumen centerlines; (Claim 13.ii) program code for training a neural network using the training data; (Claim 13.iii) program code for determining a voxel data set corresponding to a voxel of a lumen model; (Claim 13.iv) program code for inputting the voxel data set into the neural network; (Claim 13.v) program code for outputting a centerline status for the voxel from the neural network; and (Claim 13.vi) program code for determining a centerline of the lumen model using the centerline status.
Claim 14. The computer program product of claim 13, wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model.
Claim 15. The computer program product of claim 14, wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels.
Claim 16. The computer program product of claim 14, wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline.
Claim 17. The computer program product of claim 13, comprising: (Claim 17.i) program code for inputting a plurality of voxel data sets into the neural network; (Claim 17.ii) program code for outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets; (Claim 17.iii) program code determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model; (Claim 17.vi) for each voxel data set of the portion of the plurality of voxel data sets, program code for inputting the voxel data set into the neural network and outputting a second centerline status from the neural network; and (Claim 17.v) program code for updating the centerline of the lumen model including comparing the second centerline statuses to a centerline threshold.
Claim 18. A method for determining an airway lumen model, comprising: (Claim 18.i) determining training data including a plurality of lumen centerlines; (Claim 18.ii) training a neural network using the training data; (Claim 18.iii) determining a voxel data set corresponding to a voxel of a lumen model; (Claim 18.iv) inputting the voxel data set into the neural network; (Claim 18.v) outputting a centerline status for the voxel from the neural network; and (Claim 18.vi) determining a centerline of the lumen model using the centerline status.
Claim 19. The method of claim 18, wherein determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model.
Claim 20. The method of claim 19, wherein the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels.
Claim 21. The method of claim 19, wherein the centerline status indicates whether the voxel of the lumen model corresponds to the centerline.
Regarding Claim 1, 13, and 18, Alblas et al. teach (Claim 1.i) determining training data including a plurality of lumen centerlines (Page 2 Paragraph 3: To develop our method, we used 26 black-blood MR images that are provided as a training set; Page 2 Paragraph 4: Since the axial slices were sparsely annotated, we manually annotated continuous lumen centerlines of the internal and external carotid arteries for all 26 patients). Alblas et al. teach (Claim 1.ii) training a neural network using the training data (Page 2, Paragraph 2: In this work, we explore the idea of using polar coordinate systems for local ROIs and training CNNs to perform regression in these images; Page 2 Paragraph 4: Since the axial slices were sparsely annotated, we manually annotated continuous lumen centerlines of the internal and external carotid arteries for all 26 patients). Alblas et al. teach (Claim 1.iii) determining a voxel data set corresponding to a voxel of the lumen model (Page 2, Paragraph 3: Each MR image consists of 640 or 720 axial slices, with an isotropic voxel spacing; Page 3, Paragraph 3: This function describes the proximity of each voxel to the centerlines of the external carotid artery). The 3D models are composed on voxel. Therefore that are interpreted as voxel datasets that correspond to a voxel and many voxels. Alblas et al. teach (Claim 1.iv) inputting the voxel data set into the neural network (Page 3, Paragraph 6: The input to the CNN is a polar image, constructed by taking the centerline point intersecting an axial slice as the origin. vector containing the voxel intensities the particular ray passes in the Cartesian image. Rays are stacked to form a 2D polar image. Subsequently, this image is used as input to a CNN). Alblas et al. teach (Claim 1.v) outputting a centerline status for the voxel from the neural network (Page 4, Paragraph 4: This network has a two-channel output; Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. To this end, all images were resampled to an isotropic voxel spacing. These centerlines served as input to the second step). Each voxel that is determined to be a centerline in output with a designated status of centerline. Alblas et al. teach (Claim 1.vi) determining a centerline of the lumen model using the centerline status (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. These centerlines served as input to the second step). The voxel was determined to be a centerline by the model through the generation of the status. Claim 13 recites the limitations of claim 1 directed to a computer readable medium and Claim 18 recites the limitations of claim 1 directed to another method. A definition for lumen provided by the specification (Paragraph 0003 of the published specification (US 20230394202 A1): inner cavities of anatomical tubular structures, also known as lumens, are present in humans and other organisms. Lumens may be hollow, such as airways, or filled with another substance such a blood vessel filled with blood or a bone filled with bone marrow) indicates a lumen can be an airway leading to no difference in interpretation between a method of determining a lumen model (Claim 1) and a method of determining an airway lumen model (Claim 18).
Regarding Claim 2, 14, and 19, Alblas et al. teach determining the voxel data set includes casting a plurality of rays from the voxel of the lumen model to a surface of the lumen model (Page 2, Figure 1: local 3D regions-of-interest are extracted around the centerlines and transformed to a polar representation through ray-casting; Page 3, Paragraph 6: At each angle, a ray with a fixed length is cast, resulting in a vector containing the voxel intensities the particular ray passes in the Cartesian image).
Regarding Claim 3, 9, 15, and 20, Alblas et al. teach the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels (Page 3, Paragraph 6: At each angle, a ray with a fixed length is cast, resulting in a vector containing the voxel intensities the particular ray passes in the Cartesian image. Rays are stacked to form a 2D polar image. Subsequently, this image is used as input to a CNN that predicts the aforementioned radii values).
Regarding Claim 4, 10, 16, and 21, Alblas et al. teach the centerline status indicates whether the voxel of the lumen model corresponds to the centerline (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. Coordinates of the resulting centerlines were expressed in world coordinates).
Regarding Claim 5 and 11, Alblas et al. teach (Claim 5.i) inputting a plurality of voxel data sets into the neural network (Page 7, Paragraph 2: Here, we have presented a systematic comparison of experimental conditions on the training set. In addition, we evaluated our method on the test set). Alblas et al. teach (Claim 5.ii) outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets (Page 5 Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization; We used randomly sampled patches of 368 x 64x 320 voxels mini-batches of eight samples). Alblas et al. teach (Claim 5.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. These centerlines served as input to the second step). This is interpreted as equivalent to determining which voxel compose the centerline which is equivalent to identifying the centerline within a 3d model composed of voxels. Alblas et al. teach (Claim 5.iv) for each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network (Page 4, Paragraph 3: for new and unseen data, limitations in the centerline localization step may lead to incorrectly centered polar images. To compensate for this and improve robustness, we simulate this phenomenon in the training data by randomly sampling the centerpoint of the polar image within a small radius of the center of mass. This results in a number of slightly different polar images, which are used to train the CNN; Page 7, Paragraph 1: The automatically identified centerline, marked by the white cross, is far off-center. The CNN trained on augmented images shows segmentations that match the underlying image better than the non-augmented one). This could also be interpreted as training and then testing the neural network to identify the centerline from a three dimensional voxel based model which is taught by Alblas et al. as indicated above. Alblas et al. teach (Claim 5.v) updating the centerline of the lumen model based on the second centerline statuses (Page 4, Paragraph 3: for new and unseen data, limitations in the centerline localization step may lead to incorrectly centered polar images. To compensate for this and improve robustness, we simulate this phenomenon in the training data by randomly sampling the centerpoint of the polar image within a small radius of the center of mass. This results in a number of slightly different polar images, which are used to train the CNN; Page 7, Paragraph 1: The automatically identified centerline, marked by the white cross, is far off-center. The CNN trained on augmented images shows segmentations that match the underlying image better than the non-augmented one). This could also be interpreted as training and then testing the neural network to identify the centerline from a three dimensional voxel based model which is taught by Alblas et al. as indicated above.
Regarding Claim 6 and 12, Alblas et al. teach updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold (Page 3, Paragraph 3: To locate the centerlines in the first step, we define a cost-function. The function is nonzero within a predefined radius of the centerline). The function that determines the centerline involved comparison with within or beyond a predefined radius, which is interpreted as a threshold.
Regarding Claim 7, Alblas et al. teach (Claim 7.i) casting a plurality of rays from a voxel of a lumen model to a surface of the lumen model ((Page 2, Figure 1: local 3D regions-of-interest are extracted around the centerlines and transformed to a polar representation through ray-casting; Page 3, Paragraph 6: At each angle, a ray with a fixed length is cast, resulting in a vector containing the voxel intensities the particular ray passes in the Cartesian image). Alblas et al. teach (Claim 7.ii) determining a voxel data set after casting the plurality of rays (Page 3, Paragraph 6: This polar image can be easily constructed from the original Cartesian data using ray-casting at N angles. At each angle, a ray with a fixed length is cast, resulting in a vector containing the voxel intensities the particular ray passes in the Cartesian image; Page 4, Paragraph 2: In the multi-slice version, 2D polar images are stacked with polar images constructed from adjacent slices in both directions to obtain a 3D polar image). Alblas et al. teach (Claim 7.iii) inputting the voxel data set into a neural network (Page 3, Paragraph 6: The input to the CNN is a polar image, constructed by taking the centerline point intersecting an axial slice as the origin. vector containing the voxel intensities the particular ray passes in the Cartesian image. Rays are stacked to form a 2D polar image. Subsequently, this image is used as input to a CNN). Alblas et al. teach (Claim 7.iv) outputting a centerline status from the neural network (Page 4, Paragraph 4: This network has a two-channel output; Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. To this end, all images were resampled to an isotropic voxel spacing. These centerlines served as input to the second step). Alblas et al. teach (Claim 7.v) determining a centerline of the lumen model including the voxel using the centerline status (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. These centerlines served as input to the second step).
Regarding Claim 8, Alblas et al. teach (Claim 8.i) determining training data including a plurality of lumen centerlines (Page 2 Paragraph 3: To develop our method, we used 26 black-blood MR images that are provided as a training set; Page 2 Paragraph 4: Since the axial slices were sparsely annotated, we manually annotated continuous lumen centerlines of the internal and external carotid arteries for all 26 patients). Alblas et al. teach (Claim 8.ii) training the neural network using the training data (Page 2, Paragraph 2: In this work, we explore the idea of using polar coordinate systems for local ROIs and training CNNs to perform regression in these images; Page 2 Paragraph 4: Since the axial slices were sparsely annotated, we manually annotated continuous lumen centerlines of the internal and external carotid arteries for all 26 patients).
Regarding Claim 17, Alblas et al. teach (Claim 17.i) inputting a plurality of voxel data sets into the neural network (Page 7, Paragraph 2: Here, we have presented a systematic comparison of experimental conditions on the training set. In addition, we evaluated our method on the test set). Alblas et al. teach (Claim 17.ii) outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets (Page 5 Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization; We used randomly sampled patches of 368 x 64x 320 voxels mini-batches of eight samples). Alblas et al. teach (Claim 17.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model (Page 5, Paragraph 2: In the first step of our method, a 3D U-Net was trained to predict the cost-image f(I) for centerline localization. These centerlines served as input to the second step). Alblas et al. teach (Claim 17.iv) each voxel data set of the portion of the plurality of voxel data sets, inputting the voxel data set into the neural network and outputting a second centerline status from the neural network (Page 4, Paragraph 3: for new and unseen data, limitations in the centerline localization step may lead to incorrectly centered polar images. To compensate for this and improve robustness, we simulate this phenomenon in the training data by randomly sampling the centerpoint of the polar image within a small radius of the center of mass. This results in a number of slightly different polar images, which are used to train the CNN; Page 7, Paragraph 1: The automatically identified centerline, marked by the white cross, is far off-center. The CNN trained on augmented images shows segmentations that match the underlying image better than the non-augmented one). This could also be interpreted as training and then testing the neural network to identify the centerline from a three dimensional voxel based model which is taught by Alblas et al. as indicated above. Alblas et al. teach (Claim 17.v) updating the centerline of the lumen model including comparing the second centerline statuses to a centerline threshold (Page 4, Paragraph 3: for new and unseen data, limitations in the centerline localization step may lead to incorrectly centered polar images. To compensate for this and improve robustness, we simulate this phenomenon in the training data by randomly sampling the centerpoint of the polar image within a small radius of the center of mass. This results in a number of slightly different polar images, which are used to train the CNN; Page 7, Paragraph 1: The automatically identified centerline, marked by the white cross, is far off-center. The CNN trained on augmented images shows segmentations that match the underlying image better than the non-augmented one; Page 3, Paragraph 3: To locate the centerlines in the first step, we define a cost-function. The function is nonzero within a predefined radius of the centerline).
Alblas et al. does not teach a method related to a designing a stent (Claim 7). Alblas et al. does not teach computer readable media related to a designing a stent (Claim 13).
Regarding Claim 1, 13, and 18, Gildea et al. teach (Claim 1.i) determining training data including a plurality of lumen centerlines (Paragraph 0023: The model can include a calculated centerline for the airway in the three-dimensional model for reference by the user). Gildea et al. teach (Claim 1.iii) determining a voxel data set corresponding to a voxel of the lumen model (Paragraph 0022: generate the three-dimensional model of the airway; Paragraph 0023: select a plurality of locations). Locations within a three dimensional model are interpreted as equivalent to selecting a voxel. Therefore, the used selected locations are equivalent to generating a voxel dataset. Gildea et al. teach (Claim 1.vi) determining a centerline of the lumen model using the centerline status (Paragraph 0024: a centerline of the stent is selected to track a centerline of the model of the patient’s airway). The use of a neural network to automate the process of centerline determination is interpreted as an obvious variant of Gildea et al. teachings on manually setting the centerline of a 3D lumen model, especially in combination with the other art presented. Additionally, Gildea et al. teach (Claim 13) their methods are conducted by a computer that inherently has memory, computer readable memory, and at least one processor (Paragraph 0034: These computer program instructions can be stored in memory and provided to a processor of a general purpose computer). Additionally, Gildea et al. teach (Claim 18) the application of the methods to airways (Paragraph 0024: a centerline of the stent is selected to track a centerline of the model of the patient’s airway). Additionally, Gildea et al. teach (Claim 13) the application of centerlines to designing a stent (Paragraph 0020: The systems and methods taught herein can be used to model a stent and determine an appropriate placement for the stent).
Regarding Claim 4, 10, 16, and 21, Gildea et al. teach the centerline status indicates whether the voxel of the lumen model corresponds to the centerline (Paragraph 0024: a centerline of the stent is selected to track a centerline of the model of the patient’s airway).
Regarding Claim 5 and 11, Gildea et al. teach (Claim 5.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model (Paragraph 0031: the cylindrical mesh follows the centerline of the airway, although other algorithms can be applied to a set of points selected on the three-dimensional airway model to generate the centerline). This is interpreted as equivalent to determining which voxel compose the centerline which is equivalent to identifying the centerline within a 3d model composed of voxels. Gildea et al. teach (Claim 5.v) updating the centerline of the lumen model based on the second centerline statuses (Paragraph 0032: Once the user has finished editing the stent model, the user can approve the model via the graphic user interface). The model contains a centerline.
Regarding Claim 7, Gildea et al. teach (Claim 7.v) determining a centerline of the lumen model including the voxel using the centerline status (Paragraph 0024: a centerline of the stent is selected to track a centerline of the model of the patient’s airway). Additionally, Gildea et al. teach the application of centerlines to designing a stent (Paragraph 0020: The systems and methods taught herein can be used to model a stent and determine an appropriate placement for the stent).
Regarding Claim 8, Gildea et al. teach (Claim 8.i) determining training data including a plurality of lumen centerlines (Paragraph 0023: The model can include a calculated centerline for the airway in the three-dimensional model for reference by the user).
Regarding Claim 17, Gildea et al. teach (Claim 17.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model (Paragraph 0031: the cylindrical mesh follows the centerline of the airway, although other algorithms can be applied to a set of points selected on the three-dimensional airway model to generate the centerline). Gildea et al. teach (Claim 17.v) updating the centerline of the lumen model including comparing the second centerline statuses to a centerline threshold (Paragraph 0032: Once the user has finished editing the stent model, the user can approve the model via the graphic user interface).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to combine Alblas et al. and Gildea et al. All teach utilizing 3d lumen models incorporating centerline prediction related to human health. Gildea et al. teach their methods for modeling lumens is an improvement over prior methods and are capable of improving patient health through personalized medicine (Paragraph 0020: The present disclosure addresses the issues with existing airway stents using model-based design and placement of patient-specific airway stents. A patient-specific stent may minimize complications, improve quality of life, and reduce the need for repeated procedures). Alblas et al. teach their methods have increased accuracy and performance of other technique for modeling lumens (Page 7, Paragraph 2: method that accurately segments the carotid artery vessel; more effective use of training data; improved segmentation performance). Furthermore, one of ordinary skill in the art would predict that the methods could be readily combined with a reasonable expectation of success because both are within the same technical field – utilizing neural networks and three dimensional lumen models related to image analysis incorporating the specific modeling of centerlines and lumen surfaces.
Claims 1-21 are rejected under 35 U.S.C. 103 as being unpatentable over Alblas et al., as applied to claims 1-21 above, in view of Gildea et al., as applied to claims 1-21 above, and in further view of Tetteh et al. (2020, Frontiers in Neuroscience, Vol. 14: 1-17). Italicized text from reference art.
Regarding Claims 1-21, Alblas et al. and Gildea et al. teach the limitations of claims 1-21 as presented above.
Similarly to Alblas et al., Tetteh et al. teaches methods and device for training and utilizing neural networks on three dimensional voxel based models of lumens to identify and assess centerlines.
Regarding Claim 1, 13, and 18, Tetteh et al. teach (Claim 1.i) determining training data including a plurality of lumen centerlines (Page 8, Column 1, Paragraph 2: Training convolutional networks from scratch typically requires significant amounts of training data. To overcome this problem, we generate synthetic data. We generate 136 volumes of size 325 × 304 × 600 with corresponding labels for vessel segmentation, centerlines). Tetteh et al. teach (Claim 1.ii) training a neural network using the training data (Page 7, Column 2, Paragraph 2: We now use this generator to simulate physiologically plausible vascular trees that we can use in training our CNN algorithms). Tetteh et al. teach (Claim 1.iii) determining a voxel data set corresponding to a voxel of the lumen model (Page 8, Column 1, Paragraph 2: initialize the processes with different random seeds and scale the resulting vessel sizes in voxels to match the sizes of vessels in clinical datasets). The 3D models are composed on voxel. Therefore that are interpreted as voxel datasets that correspond to a voxel and many voxels. Tetteh et al. teach (Claim 1.iv) inputting the voxel data set into the neural network (Page 7, Column 2, Paragraph 4: use three different datasets to train and test the networks; Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction, and bifurcation detection; Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA datasets). Tetteh et al. teach (Claim 1.v) outputting a centerline status for the voxel from the neural network (Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction; Page 9, Column 2, Paragraph 2: we carry out the convolutions in a way that the output image is of the same size as the input image). Each voxel that is determined to be a centerline in output with a designated status of centerline. Tetteh et al. teach (Claim 1.vi) determining a centerline of the lumen model using the centerline status (Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction; Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA datasets). The voxel was determined to be a centerline by the model through the generation of the status. Additionally, Tetteh et al. teach their methods are conducted using a computer that inherently has memory, computer readable memory, and at least one processor (Page 10, Column 2, Paragraph 1: We implement our algorithm using the THEANO Python framework and train on a machine with 64GB of RAM and Nvidia TITAN X 12GB GPU). Claims 13 recites the limitations of claim 1 directed to CRM and claim 18 recites the limitations of claim 1 directed to another method.
Regarding Claim 3, 9, 15, and 20, Tetteh et al. teach the voxel data set includes at least one of a centricity value, a mean radius of the plurality of rays, a minimum radius of the plurality of rays, position coordinates of the voxel, ray length values for each of the plurality of rays, voxel density, or a number of neighboring voxels (Page 4, Column 2, Paragraph 1: where {aijk} is a position element of matrix A). Also see figure 2 (Page 4) the position of each voxel is known and maintained within the dataset.
Regarding Claim 4, 10, 16, and 21, Tetteh et al. teach the centerline status indicates whether the voxel of the lumen model corresponds to the centerline (Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction; Page 14, Column 2, Table 4: Results for centerline prediction tasks).
Regarding Claim 5 and 11, Tetteh et al. teach (Claim 5.i) inputting a plurality of voxel data sets into the neural network (Page 7, Column 2, Paragraph 4: In this work, we use three different datasets to train and test the networks). The data sets are interpreted as voxel data sets because they represent three dimensional models and are composed of voxels. Tetteh et al. teach (Claim 5.ii) outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets (Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA dataset). Tetteh et al. teach (Claim 5.iii) determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model (Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction; Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA datasets). This is interpreted as equivalent to determining which voxel compose the centerline which is equivalent to identifying the centerline within a 3d model composed of voxels. The limitations of claims 5.iv and 5.v could also be interpreted as training and then testing the neural network to identify the centerline from a three dimensional voxel based model which is taught by Tetteh et al. as indicated above.
Regarding Claim 6 and 12, Tetteh et al. teach updating the centerline of the lumen model includes comparing the second centerline statuses to a centerline threshold (Page 7, Column 1, Paragraph 3: This allows us to penalize false predictions, which are very far from the central point. The false predictions are obtained through a probability threshold).
Regarding Claim 7, Tetteh et al. teach (Claim 7.iii) inputting the voxel data set into a neural network (Page 7, Column 2, Paragraph 4: use three different datasets to train and test the networks; Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction, and bifurcation detection; Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA datasets). Tetteh et al. teach (Claim 7.iv) outputting a centerline status from the neural network (Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction; Page 9, Column 2, Paragraph 2: we carry out the convolutions in a way that the output image is of the same size as the input image). Tetteh et al. teach (Claim 7.v) determining a centerline of the lumen model including the voxel using the centerline status (Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction; Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA datasets).
Regarding Claim 8, Tetteh et al. teach (Claim 8.i) determining training data including a plurality of lumen centerlines (Page 8, Column 1, Paragraph 2: Training convolutional networks from scratch typically requires significant amounts of training data. To overcome this problem, we generate synthetic data. We generate 136 volumes of size 325 × 304 × 600 with corresponding labels for vessel segmentation, centerlines). Tetteh et al. teach (Claim 8.ii) training the neural network using the training data (Page 7, Column 2, Paragraph 2: We now use this generator to simulate physiologically plausible vascular trees that we can use in training our CNN algorithms).
Regarding Claim 17, Tetteh et al. teach (Claim 17.i) program code for inputting a plurality of voxel data sets into the neural network (Page 7, Column 2, Paragraph 4: In this work, we use three different datasets to train and test the networks). Tetteh et al. teach (Claim 17.ii) program code for outputting a plurality of centerline statuses from the neural network for the plurality of voxel data sets (Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA dataset). Tetteh et al. teach (Claim 17.iii) program code determining a portion of the plurality of voxel data sets correspond to the centerline of the lumen model (Page 9, Column 1, Paragraph 3: In this study we focus on the use of artificial neural networks for the tasks of vessel segmentation, centerline prediction; Page 13, Column 1, Paragraph 2: For centerline prediction, we train DeepVesselNet on the synthetic dataset, test it on synthetic dataset and present visualizations on synthetic and clinical MRA datasets). The limitations of claim 17.iv could also be interpreted as training and then testing the neural network to identify the centerline from a three dimensional voxel based model which is taught by Tetteh et al. as indicated above. Tetteh et al. teach (Claim 17.v) program code for updating the centerline of the lumen model including comparing the second centerline statuses to a centerline (Page 7, Column 1, Paragraph 3: This allows us to penalize false predictions, which are very far from the central point. The false predictions are obtained through a probability threshold).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to combine Tetteh et al. with Alblas et al. and Gildea et al. All teach utilizing 3d lumen models incorporating centerline prediction related to human health. Tetteh et al. teach their methods for modeling lumens has improved efficiency and accuracy over other modeling techniques (Page 2, Column 1, Paragraph 1: DeepVesselNet deals with challenges that result from speed and memory requirements, unbalanced class labels, and the difficulty of obtaining well-annotated data for curvilinear volumetric structures by addressing the following three key limitations). Furthermore, one of ordinary skill in the art would predict that the methods could be readily combined with a reasonable expectation of success because both are within the same technical field – utilizing neural networks and three dimensional lumen models related to image analysis incorporating the specific modeling of centerlines and lumen surfaces.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 1, 13, and 18 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, and 7 of U.S. Patent No. 12462379 (reference patent) in view of Alblas et al., Tetteh et al., and Gildea et al., applied in the 35 USC 102/103 rejections above.
U.S. Patent No. 12462379 utilizes 3D lumen models (reference patent claim 1), including centerlines (reference patent claim 2), user selected locations within 3D lumen models (reference patent claim 1), and applies a trained neural networks to generate 3d models that can include centerlines (reference patent claim 7).
U.S. Patent No. 12462379 does not teach a voxel based modeling system but this would be obvious based on Alblas et al. and Tetteh et al. (see 35 USC rejections above). See the 35 USC 103 rejections above for the relationship between Gildea et al. and Alblas et al. and Tetteh et al., for obvious teachings of the instant claims (US application 17626344 (U.S. Patent No. 12462379) is related as a 371 to Gildea et al.). Therefore, any deficiencies of reference patent claims 1, 2, and 7 in regards to their teachings of instant application claims 1, 13, and 18 are obvious given the art presented above.
Conclusion
No claims are allowed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BLAKE ELKINS whose telephone number is (571)272-2649. The examiner can normally be reached Monday-Thursday 8-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Karlheinz Skowronek can be reached at (571) 272-9047. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.H.E./Examiner, Art Unit 1687
/Karlheinz R. Skowronek/Supervisory Patent Examiner, Art Unit 1687
Prosecution Insights