DETAILED ACTION
Claims 1-20 are pending.
Specification
Amendment to the title of the invention filed on April 2, 2026 is acknowledged, but the amended title of the invention is still not descriptive. In particular, the title of the invention was amended to recite “METHODS AND APPARATUS FOR TRAINING DEEP LEARNING BASED IMAGE RECONSTRUCTION MODELS USING LOSS VALUES FOR FORWARD PROJECTED DATA”. However, “IMAGE RECONSTRUCTION MODELS” are not currently being recited in the claims.
Therefore, based on above, a new title is still required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: “METHODS AND APPARATUS FOR TRAINING DEEP LEARNING USING LOSS VALUES BASED ON FORWARD PROJECTED DATA AND PROJECTION DATA”.
Response to Arguments
Applicant’s arguments filed on April 2, 2026 have been fully considered but are respectfully found unconvincing for the following reasons.
Regarding claim 1, and similar independent claim(s), Applicant asserts that “Cao does not disclose training of a machine learning model through comparison of forward projected image data to originally received image data as required by claim 1… Cao does not disclose using a loss value based on a forward projected image data and projection data for determining that a machine learning process is completed” (Remarks, Pg. 8-9).
Applicant’s arguments above with respect to “training of a machine learning model through comparison of forward projected image data to originally received image” and “using a loss value based on a forward projected image data and projection data for determining that a machine learning process is completed” are understood, but they are not relevant to the claims since these features are not being recited in the claims.
Claim 1 instead recites “determining a loss value based on the forward projected image data and the projection data … determining the machine learning process is trained based on the loss value” in lines 8-9 of the claim.
Cao discloses “first loss function may be related to a difference between the forward projection data associated with the image to be processed and originally acquired projection data associated with the initial image… processing device 120 may iteratively optimize the image to be processed until a value of the first loss function satisfies a termination condition in the iteration step”, in Par. [0076, 86], as previously set forth in the last Office action (OA), Pg. 11, which recite similar concept corresponding to the claimed “determining a loss value based on the forward projected image data and the projection data” and “determining the machine learning process is trained based on the loss value”, respectively, by iteratively optimizing an “image to be processed until a value of the first loss function satisfies a termination condition” in an iteration step, for example.
Regarding claim 1, and similar independent claim(s), Applicant further asserts that the “Action appears to allege that Cao discloses "determining a loss value based on the forward projected image data and the projection data"… However, the Action lacks any citations to any portion of Cao that allegedly discloses this element and fails to include any discussion regarding how Cao allegedly meets this element… Further, Cao does not disclose storing machine learning parameters associated with the machine learning process in a data repository” (Remarks, Pg. 9).
Examiner respectfully disagrees.
As indicated above, Cao discloses “a first loss function relating to a difference between forward projection data associated with the image to be processed and the originally acquired projection data associated with the initial image”, in Par. [0076], as previously set forth in the last OA, Pg. 8, which recite similar concept corresponding to the claimed “determining a loss value based on the forward projected image data and the projection data” recited in line 8 of claim 1.
Additionally, Cao further discloses “optimizing model may be pre-trained and stored in a storage device (e.g., the storage device 150)… the optimizing model may include a machine learning model, for example, a deep learning model” and “the preliminary optimizing model may a machine learning model (e.g., a neural network model)… the preliminary optimizing model may include at least one preliminary model parameter”, as previously set forth in the last OA, Pg. 13, which recite similar concept corresponding to the claimed “storing machine learning parameters associated with the machine learning process in a data repository (e.g., the storage device 150), as indicated above, for example.
Therefore, based on above rationale, Applicant’s arguments filed on April 2, 2026 are respectfully found unconvincing and all prior rejections previously set forth in the last OA are hereby maintained as nothing prevents Cao from being applied alone, or in combination, to reject pending claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 7, 9, 11-15, and 18-20 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by CAO et al. (US PG Publication No. 2023/0085203 A1), hereafter referred to as CAO (grounds are restated below for convenience).
Regarding claim 1, CAO discloses a computer-implemented method (Par. [0002-4]: systems and methods for iterative image reconstruction… The method may be implemented on a computing device including at least one processor and a computer-readable storage device) comprising:
receiving projection data (Par. [0043]: scanning device 110 may be configured to acquire image data relating to at least one part of a subject. The scanning device 110 may scan the subject or a portion thereof that is located within its detection region and generate image data relating to the subject or the portion thereof. The image data relating to at least one part of a subject may include… projection data; Par. [0073]: processing device 120 may direct the scanning device 110 to perform a scan (e.g., a CT scan) on an object (e.g., a patient) for obtaining scanning data of the object (also referred to as originally acquired scanning data (e.g., originally acquired projection data) of the object)… The originally acquired scanning data or the portion based on which the initial image is determined may also be referred to as originally acquired scanning data (e.g., originally acquired projection data) associated with the initial image; receiving projection data (e.g. systems and methods for iterative image reconstruction include obtaining (i.e. receiving) scanning data of an object, also referred to as originally acquired projection data (i.e. receiving projection data), as indicated above), for example);
applying a machine learning process to the projection data and, based on the application of the machine learning process to the projection data, generating output image data (Par. [0075-106]: the processing device 120 (e.g., the reconstruction module 420) (e.g., the processing circuits of the processor 210) may generate a reconstructed image by performing a plurality of iteration steps on the initial image… the plurality of iteration steps may include a first optimization operation and a second optimization operation… the first optimization operation and the second optimization operation may be executed sequentially. For example, an optimized result (e.g., an output) of the first optimization operation may be an image to be optimized (e.g., an input) of the second optimization operation… The first optimizing model, the second optimizing model, and the third optimizing model may be applied in series such that the output of the first optimizing model may be used as input to the second optimizing model, and the output of the second optimizing model may be used as input to the third optimizing model… the updated image may be input to the first optimizing model, the second optimizing model, and the third optimizing model, respectively, to output a first image, a second image, and a third image, respectively; Par. [0138-151]: the image reconstruction described elsewhere in the present disclosure may include a plurality of iteration steps. Each of the iteration steps may include an optimizing model… the optimizing models in different iteration steps may correspond to a same type of machine learning model and have a same network structure, while the optimization models in different iteration steps may correspond to different values of at least one learnable parameter… the plurality of optimizing models may be trained… the processing device 120 may input the projection data to be processed and/or an initial image) into the image reconstruction model… the initial image may be an image reconstructed based on the projection data according to a reconstruction algorithm (e.g., an FBP algorithm)… the processing device 120 may determine a reconstructed image by performing a plurality of iteration steps on an initial image… an input of the image reconstruction model 1000 may include originally acquired projection data (i.e., the projection data to be processed illustrated in FIG. 9) and/or an initial image; an output of the image reconstruction model 1000 may include the reconstructed image… sub-model 1010 may be configured to receive the input (e.g., the originally acquired projection data and/or the initial image) of the image reconstruction model 1000 and generate an output. Each of the second sub-model 1020 may be configured to receive an output of a previously adjacent sub-model connected to the second sub-model 1020 and generate an output. The output of the image reconstruction model 1000 may be the output of the last second sub-model… processing layer may be configured to receive an image to be processed in the sub-model… for the first sub-model 1010, the input of the processing layer (e.g., the processing layer 1) may include the input (e.g., the originally acquired projection data and/or the initial image) of the image reconstruction model 1000; for the second sub-model 1020 (e.g., 1021), the input of the processing layer (e.g., the processing layer 2) may include a sub-reconstructed image output by a reconstruction layer (e.g., the reconstruction layer 1) of a previously adjacent sub-model (e.g., the first sub-model 1010). The processing layer may be also configured to determine a regularization result by regularizing the image to be processed based on the projection data. Specifically, the processing layer may be configured to regularize the input of the processing layer and generate a regularization result (e.g., a result matrix) corresponding to the input. For example, the processing layer 1 of the first sub-model 1010 may be configured to regularize the initial image and generate a regularization result of the initial image. As another example, the processing layer 2 of the second sub-model 1021 may be configured to regularize a sub-reconstructed image output by the reconstruction layer 1 and generate a regularization result of the sub-reconstructed image… the processing layer may include a neural network mode… for different sub-models, parameters of the processing layers may be different and may be obtained by training… reconstruction layer may be configured to determine a sub-reconstructed image based on the regularization result and a previous sub-reconstructed image determined by a previously adjacent sub-model… for the first sub-model 1010, the input of the reconstruction layer (e.g., the reconstruction layer 1) may include an output (i.e., the regularization result of the initial image) of the processing layer (e.g., the processing layer 1) of the first sub-model 1010 and the input (e.g., the projection data) of the image reconstruction model 1000… The reconstruction layer may be also configured to designate the sub-reconstructed image as a next image to be processed in a next sub-model… the output of the reconstruction layer of each of the plurality of sub-models may include a sub-reconstructed image corresponding to the sub-model. Accordingly, the output of the last reconstruction layer (e.g., the reconstruction layer n) of the last second sub-model (e.g., 1022) may be the reconstructed image; applying a machine learning process to the projection data and, based on the application of the machine learning process to the projection data, generating output image data (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including a processing device that inputs projection data to be processed, and/or an initial image, into an image reconstruction model that is based on a machine learning model (i.e. applying a machine learning process to the projection data), for example, and the initial image includes an image reconstructed based on the projection data according to a reconstruction algorithm (i.e. applying a machine learning process to the projection data and, based on the application of the machine learning process to the projection data, generating output image data), for example, and the processing device generates a reconstructed image by performing a plurality of iteration steps on the initial image, such as a first, second, third,… Nth optimization operation, respectively, and an output of the first optimization operation includes an image (i.e. output image data) to be optimized, which becomes an input of the second optimization operation, as indicated above), for example);
applying a forward projection process to the output image data and, based on the application of the forward projection process to the output image data, generating forward projected image data;
determining a loss value based on the forward projected image data and the projection data;
determining the machine learning process is trained based on the loss value (Par. [0075-89]: the processing device 120 (e.g., the reconstruction module 420) (e.g., the processing circuits of the processor 210) may generate a reconstructed image by performing a plurality of iteration steps on the initial image… the plurality of iteration steps may include a first optimization operation and a second optimization operation… the plurality of iteration steps may include a first optimization operation and a second optimization operation… the first optimization operation and the second optimization operation may be executed sequentially. For example, an optimized result (e.g., an output) of the first optimization operation may be an image to be optimized (e.g., an input) of the second optimization operation. Take a specific iteration step as an example, the first optimization operation may include receiving an image to be processed in the iteration step (e.g., for the first iteration step, the image to be processed is the initial image) and determining an updated image by preliminarily optimizing the image to be processed (e.g., according to a first loss function relating to a difference between forward projection data associated with the image to be processed and the originally acquired projection data associated with the initial image)… the processing device 120 may determine the optimized image by optimizing the updated image based on at least one optimizing model (e.g., at least one machine learning model)… the processing device 120 (e.g., the reconstruction module 420) (e.g., the processing circuits of the processor 210) may determine an updated image by preliminarily optimizing the image to be processed… the processing device 120 may determine forward projection data by performing a forward projection transformation on the image to be processed in the iteration step. According to the forward projection transformation, the processing device 120 may transform data (e.g., the image to be processed) in an image domain to data (e.g., the forward projection data) in a projection domain… the processing device 120 may transform the image to be processed into forward projection data by multiplying the image to be processed by a forward projection matrix… the processing device 120 may determine the updated image in the iteration step by optimizing the image to be processed according to a first loss function. The first loss function may be related to a difference between the forward projection data associated with the image to be processed and originally acquired projection data associated with the initial image. As used herein, the originally acquired projection data associated with the initial image refers to original projection data acquired… For example, the first operation in the iteration step itself may include an iterative operation. The processing device 120 may iteratively optimize the image to be processed until a value of the first loss function satisfies a termination condition in the iteration step… the processing device 120 may determine a difference between the forward projection data and originally acquired projection data as an error between the forward projection data and originally acquired projection data… the first loss function may include a fidelity term that relates to a difference between the forward projection data and originally acquired projection data associated with the initial image; Par. [0133-150]: processing device 120 may train the preliminary optimizing model iteratively until a termination condition is satisfied. In response to that the termination condition is satisfied, the optimizing model may be finalized… the termination condition may relate to a value of a loss function (also referred to as a second loss function), e.g., relating to a difference between a gold standard image and an estimated reconstructed image determined by inputting a sample image corresponding to the gold standard image to the in the specific iteration. For example, the termination condition may be satisfied if the value of the loss function is minimal or smaller than a predetermined threshold. As another example, the termination condition may be satisfied if the value of the loss function converges… “convergence” may refer to that the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold… “convergence” may refer to that a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold… image reconstruction model 1000 may include a plurality of sequentially connected sub- models, such as a first sub-model 1010 and one or more second sub-models 1020 (e.g., 1021 and 1022)… the first sub-model 1010 may be configured to receive the input (e.g., the originally acquired projection data and/or the initial image) of the image reconstruction model 1000 and generate an output… each of the plurality of sub-models may include a processing layer… and a reconstruction layer… The processing layer may be configured to receive an image to be processed in the sub-model… for the first sub-model 1010, the input of the processing layer (e.g., the processing layer 1) may include the input (e.g., the originally acquired projection data and/or the initial image) of the image reconstruction model 1000; applying a forward projection process to the output image data and, based on the application of the forward projection process to the output image data, generating forward projected image data;
determining a loss value based on the forward projected image data and the projection data;
determining the machine learning process is trained based on the loss value (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including a processing device that inputs projection data to be processed, and/or an initial image, into an image reconstruction model that is based on a machine learning model, for example, and the initial image includes an image reconstructed based on the projection data (i.e. the output image data) according to a reconstruction algorithm, as indicated above, for example, in which the processing device generates a reconstructed image by performing a plurality of iteration steps on the initial image, such as a first, second, third,… Nth optimization operation, respectively, including determining forward projection data (i.e. generating forward projected image data) by performing a forward projection transformation on the projection data to be processed, and/or image to be processed (i.e. the originally acquired projection data associated with the initial image, or original projection data acquired), in an iteration step (i.e. applying a forward projection process to the output image data and, based on the application of the forward projection process to the output image data, generating forward projected image data), for example, and determining an updated image by preliminarily optimizing the projection to be processed, and/or the image to be processed, according to a first loss function relating to a difference between the forward projection data and the originally acquired projection data associated with the initial image (i.e. determining a loss value based on the forward projected image data and the projection data), for example, in which the processing device trains the preliminary optimizing model iteratively until a termination condition is satisfied (i.e. determining the machine learning process is trained), and the termination condition is satisfied if the value of the loss function is minimal or smaller than a predetermined threshold, if the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold, or if a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold (i.e. determining the machine learning process is trained based on the loss value), as indicated above), for example); and
storing parameters associated with the machine learning process in a data repository (Par. [0089-99]: loss function may include a fidelity term that relates to a difference between the forward projection data and originally acquired projection data associated with the initial image. For example, the processing device 120 may determine the updated image based on the first loss function including the fidelity term… loss function may include both the classical regularization term and the machine learning regularization term… loss function (e.g., the loss function L) may include one or more additional parameters related to the fidelity term, the classical regularization term, and/or the machine learning regularization term… the processing device 120 may determine the updated image based on the first loss function including the classical regularization term and the machine learning regularization term, and one or more additional parameters… DL refers to the machine learning model… θ(k-1) refers to learnable parameters (e.g., a value of a convolution kernel of a convolutional layer of the machine learning model) in a network of the machine learning model involved in the machine learning regularization term for the kth iteration step… the optimizing model may be pre-trained and stored in a storage device (e.g., the storage device 150)… the optimizing model may include a machine learning model, for example, a deep learning model such as a neural network model; Par. [0131-153]: the preliminary optimizing model may a machine learning model (e.g., a neural network model)… the preliminary optimizing model may include at least one preliminary model parameter… the optimization models in different iteration steps may correspond to different values of at least one learnable parameter… for different sub-models, parameters of the processing layers may be different and may be obtained by training… parameters of each sub-model in the image reconstruction model 1000 may be obtained by end-to-end training; storing parameters associated with the machine learning process in a data repository (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including a processing device that inputs projection data to be processed, and/or an initial image, into an image reconstruction model that is based on a machine learning model, for example, including optimizing models that include at least one model parameter (i.e. parameters associated with the machine learning process), for example, and the optimizing models are pre-trained and stored in a storage device (i.e. storing parameters associated with the machine learning process in a data repository), as indicated above), for example).
Regarding claim 2, claim 1 is incorporated and CAO discloses the method, further comprising: comparing the loss value to a threshold value; and
determining the machine learning process is trained based on the comparison (Par. [0133]: the processing device 120 may train the preliminary optimizing model iteratively until a termination condition is satisfied. In response to that the termination condition is satisfied, the optimizing model may be finalized… the termination condition may relate to a value of a loss function (also referred to as a second loss function), e.g., relating to a difference between a gold standard image and an estimated reconstructed image determined by inputting a sample image corresponding to the gold standard image to the in the specific iteration. For example, the termination condition may be satisfied if the value of the loss function is minimal or smaller than a predetermined threshold. As another example, the termination condition may be satisfied if the value of the loss function converges… “convergence” may refer to that the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold… “convergence” may refer to that a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold… the termination condition may be satisfied when a specified count of iterations has been performed in the training process; comparing the loss value to a threshold value; and
determining the machine learning process is trained based on the comparison (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including determining an updated image by preliminarily optimizing the projection to be processed, and/or the image to be processed, according to a first loss function (i.e. the loss value) relating to a difference between the forward projection data and the originally acquired projection data associated with the initial image, for example, in which the processing device trains the preliminary optimizing model iteratively until a termination condition is satisfied (i.e. determining the machine learning process is trained), and the termination condition is satisfied if the value of the loss function is minimal or smaller than a predetermined threshold, if the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold, or if a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold (i.e. comparing the loss value to a threshold value; and determining the machine learning process is trained based on the comparison), as indicated above), for example).
Regarding claim 3, claim 1 is incorporated and CAO discloses the method, wherein the loss value is a first loss value, the computer-implemented method further comprising:
determining a second loss value based on the projection data and the output image data; and
determining the machine learning process is trained based on the first loss value and the second loss value (Par. [0039-94]: systems and methods for image reconstruction. The systems may obtain an initial image (e.g., a CT image) to be processed and generate a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. In the first optimization operation in an iteration step, the systems may receive an image to be processed (e.g., the initial image for the first iteration step) in the iteration step and determine an updated image by preliminarily optimizing the image to be processed (e.g., according to a loss function related to a first quality weight associated with a quality of originally acquired projection data, a second quality weight associated with a quality of the image to be processed and/or a third weight associated with a cone angle corresponding to at least one detector row that acquires the originally acquired projection data. In the second optimization operation in the iteration step, the systems may determine an optimized image by optimizing the updated image and designate the optimized image as a next image to be processed in a next iteration step or designate the optimized image as the reconstructed image. The systems may determine the optimized image by reducing the interference information of the updated image based on a machine learning model (e.g., a deep learning model)… a loss function (e.g., a first loss function) for image reconstruction may be optimized… the first optimization operation and the second optimization operation may be executed sequentially. For example, an optimized result (e.g., an output) of the first optimization operation may be an image to be optimized (e.g., an input) of the second optimization operation. Take a specific iteration step as an example, the first optimization operation may include receiving an image to be processed in the iteration step (e.g., for the first iteration step, the image to be processed is the initial image) and determining an updated image by preliminarily optimizing the image to be processed (e.g., according to a first loss function relating to a difference between forward projection data associated with the image to be processed and the originally acquired projection data associated with the initial image)… the processing device 120 may determine the updated image in the iteration step by optimizing the image to be processed according to a first loss function. The first loss function may be related to a difference between the forward projection data associated with the image to be processed and originally acquired projection data associated with the initial image… the originally acquired projection data associated with the initial image refers to original projection data acquired… the first operation in the iteration step itself may include an iterative operation. The processing device 120 may iteratively optimize the image to be processed until a value of the first loss function satisfies a termination condition in the iteration step… the first loss function may include a fidelity term that relates to a difference between the forward projection data and originally acquired projection data associated with the initial image…besides the fidelity term, the first loss function may include at least one of a classical regularization term or a machine learning regularization term that involves a machine learning model (e.g., a deep learning model)… the first loss function may relate to at least one of the first quality weight, the second quality weight, or the third weight… the processing device 120 may determine the updated image based on the first loss function; Par. [0133-134]: the processing device 120 may train the preliminary optimizing model iteratively until a termination condition is satisfied. In response to that the termination condition is satisfied, the optimizing model may be finalized… the termination condition may relate to a value of a loss function (also referred to as a second loss function), e.g., relating to a difference between a gold standard image and an estimated reconstructed image determined by inputting a sample image corresponding to the gold standard image to the in the specific iteration. For example, the termination condition may be satisfied if the value of the loss function is minimal or smaller than a predetermined threshold. As another example, the termination condition may be satisfied if the value of the loss function converges… “convergence” may refer to that the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold… “convergence” may refer to that a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold… the termination condition may be satisfied when a specified count of iterations has been performed in the training process… the loss function (i.e., the second loss function) may be positively related to a second quality; wherein the loss value is a first loss value, the computer-implemented method further comprising:
determining a second loss value based on the projection data and the output image data; and
determining the machine learning process is trained based on the first loss value and the second loss value (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including a processing device that inputs projection data to be processed, and/or an initial image, into an image reconstruction model that is based on a machine learning model, for example, and the initial image includes an image reconstructed based on the projection data (i.e. the output image data) according to a reconstruction algorithm, as indicated above, for example, in which the processing device generates a reconstructed image by performing a plurality of iteration steps on the initial image, such as a first, second, third,… Nth optimization operation, respectively, including determining an updated image by preliminarily optimizing the projection to be processed, and/or the image to be processed, according to a first, second, third,… Nth loss function relating to a difference between the forward projection data and the originally acquired projection data associated with the initial image (i.e. determining a first, second, third,… Nth loss value based on the projection data and the output image data), in which the processing device trains the preliminary optimizing model iteratively until a termination condition is satisfied, for example, and the termination condition is satisfied if the value of the first, second, third,… Nth loss function is minimal or smaller than a predetermined threshold, if the variation of the values of the first, second, third,… Nth loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold, or if a difference between the value of the first, second, third,… Nth loss function and a target value is equal to or smaller than a predetermined threshold (i.e. determining the machine learning process is trained based on the first loss value and the second loss value), as indicated above), for example).
Regarding claim 4, claim 3 is incorporated and CAO discloses the method, further comprising: comparing the first loss value to a first threshold value;
comparing the second loss value to a second threshold value; and
determining the machine learning process is trained based on the comparisons Par. [0039-94]: systems and methods for image reconstruction. The systems may obtain an initial image (e.g., a CT image) to be processed and generate a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. In the first optimization operation in an iteration step, the systems may receive an image to be processed (e.g., the initial image for the first iteration step) in the iteration step and determine an updated image by preliminarily optimizing the image to be processed (e.g., according to a loss function related to a first quality weight associated with a quality of originally acquired projection data, a second quality weight associated with a quality of the image to be processed and/or a third weight associated with a cone angle corresponding to at least one detector row that acquires the originally acquired projection data. In the second optimization operation in the iteration step, the systems may determine an optimized image by optimizing the updated image and designate the optimized image as a next image to be processed in a next iteration step or designate the optimized image as the reconstructed image. The systems may determine the optimized image by reducing the interference information of the updated image based on a machine learning model (e.g., a deep learning model)… a loss function (e.g., a first loss function) for image reconstruction may be optimized… the first optimization operation and the second optimization operation may be executed sequentially. For example, an optimized result (e.g., an output) of the first optimization operation may be an image to be optimized (e.g., an input) of the second optimization operation. Take a specific iteration step as an example, the first optimization operation may include receiving an image to be processed in the iteration step (e.g., for the first iteration step, the image to be processed is the initial image) and determining an updated image by preliminarily optimizing the image to be processed (e.g., according to a first loss function relating to a difference between forward projection data associated with the image to be processed and the originally acquired projection data associated with the initial image)… the processing device 120 may determine the updated image in the iteration step by optimizing the image to be processed according to a first loss function. The first loss function may be related to a difference between the forward projection data associated with the image to be processed and originally acquired projection data associated with the initial image… the originally acquired projection data associated with the initial image refers to original projection data acquired… the first operation in the iteration step itself may include an iterative operation. The processing device 120 may iteratively optimize the image to be processed until a value of the first loss function satisfies a termination condition in the iteration step… the first loss function may include a fidelity term that relates to a difference between the forward projection data and originally acquired projection data associated with the initial image…besides the fidelity term, the first loss function may include at least one of a classical regularization term or a machine learning regularization term that involves a machine learning model (e.g., a deep learning model)… the first loss function may relate to at least one of the first quality weight, the second quality weight, or the third weight… the processing device 120 may determine the updated image based on the first loss function; Par. [0133-134]: the processing device 120 may train the preliminary optimizing model iteratively until a termination condition is satisfied. In response to that the termination condition is satisfied, the optimizing model may be finalized… the termination condition may relate to a value of a loss function (also referred to as a second loss function), e.g., relating to a difference between a gold standard image and an estimated reconstructed image determined by inputting a sample image corresponding to the gold standard image to the in the specific iteration. For example, the termination condition may be satisfied if the value of the loss function is minimal or smaller than a predetermined threshold. As another example, the termination condition may be satisfied if the value of the loss function converges… “convergence” may refer to that the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold… “convergence” may refer to that a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold… the termination condition may be satisfied when a specified count of iterations has been performed in the training process… the loss function (i.e., the second loss function) may be positively related to a second quality; further comprising: comparing the first loss value to a first threshold value;
comparing the second loss value to a second threshold value; and
determining the machine learning process is trained based on the comparisons (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including a processing device that inputs projection data to be processed, and/or an initial image, into an image reconstruction model that is based on a machine learning model, for example, and the initial image includes an image reconstructed based on the projection data (i.e. the output image data) according to a reconstruction algorithm, as indicated above, for example, in which the processing device generates a reconstructed image by performing a plurality of iteration steps on the initial image, such as a first, second, third,… Nth optimization operation, respectively, including determining an updated image by preliminarily optimizing the projection to be processed, and/or the image to be processed, according to a first, second, third,… Nth loss function relating to a difference between the forward projection data and the originally acquired projection data associated with the initial image, in which the processing device trains the preliminary optimizing model iteratively until a termination condition is satisfied, for example, and the termination condition is satisfied if the value of the first, second, third,… Nth loss function is minimal or smaller than a first, second, third,… Nth predetermined threshold, if the variation of the values of the first, second, third,… Nth loss function in two or more consecutive iterations is equal to or smaller than a first, second, third,… Nth predetermined threshold, or if a difference between the value of the first, second, third,… Nth loss function and a target value is equal to or smaller than a first, second, third,… Nth predetermined threshold (i.e. comparing the first, second, third,… Nth loss value to a first, second, third,… Nth threshold value; and determining the machine learning process is trained based on the comparisons), as indicated above), for example).
Regarding claim 7, claim 1 is incorporated and CAO discloses the method, further comprising: based on determining the machine learning process is trained:
receiving additional projection data;
applying the trained machine learning process to the additional projection data; and
based on the application of the trained machine learning process to the additional projection data, generating additional output image data, the additional output image data characterizing a final image volume (Par. [0039-70]: systems and methods for image reconstruction. The systems may obtain an initial image (e.g., a CT image) to be processed and generate a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. In the first optimization operation in an iteration step, the systems may receive an image to be processed (e.g., the initial image for the first iteration step) in the iteration step and determine an updated image by preliminarily optimizing the image to be processed (e.g., according to a loss function related to a first quality weight associated with a quality of originally acquired projection data, a second quality weight associated with a quality of the image to be processed and/or a third weight associated with a cone angle corresponding to at least one detector row that acquires the originally acquired projection data. In the second optimization operation in the iteration step, the systems may determine an optimized image by optimizing the updated image and designate the optimized image as a next image to be processed in a next iteration step or designate the optimized image as the reconstructed image. The systems may determine the optimized image by reducing the interference information of the updated image based on a machine learning model (e.g., a deep learning model)… a reconstructed image is generated by performing a plurality of iteration steps each of which includes a first optimization operation (which is used to preliminarily optimize the image to be processed) and a second optimization operation (which is used to further optimize an updated image generated in the first operation) implemented via a optimizing model (e.g., a machine learning model), that is, a preliminary optimization and a further optimization via a machine learning model are used in combination, which can improve the image quality of the reconstructed image and optimize the reconstructed image (e.g., reduce the noise of the reconstructed image)… scanning device 110 may be configured to acquire image data relating to at least one part of a subject. The scanning device 110 may scan the subject or a portion thereof that is located within its detection region and generate image data relating to the subject or the portion thereof. The image data relating to at least one part of a subject may include an image (e.g., an image slice), projection data, or a combination thereof… the image data may be two-dimensional (2D) image data, three-dimensional (3D) image data, four-dimensional (4D) image data… generate a reconstructed image by performing a plurality of iteration steps on the initial image… at least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation… the first optimization operation and the second optimization operation may be executed sequentially. Take a specific iteration step as an example, the first optimization operation may include receiving an image to be processed in the iteration step (e.g., for the first iteration step, the image to be processed is the initial image) and determining an updated image by preliminarily optimizing the image to be processed. The second optimization operation may include determining an optimized image based on the updated image and designating the optimized image as a next image to be processed in a next iteration step or designating the updated image as the reconstructed image… for the second optimization operation, the reconstruction module 420 may determine the optimized image by optimizing the updated image based on one or more optimizing models (e.g., a machine learning model)… the reconstruction module 420 and the training module 430 may be combined as a single module which may both generate the reconstructed image and determine the optimizing model; Par. [0075-106]: the processing device 120 (e.g., the reconstruction module 420) (e.g., the processing circuits of the processor 210) may generate a reconstructed image by performing a plurality of iteration steps on the initial image… the plurality of iteration steps may include a first optimization operation and a second optimization operation… the first optimization operation and the second optimization operation may be executed sequentially. For example, an optimized result (e.g., an output) of the first optimization operation may be an image to be optimized (e.g., an input) of the second optimization operation. Take a specific iteration step as an example, the first optimization operation may include receiving an image to be processed in the iteration step (e.g., for the first iteration step, the image to be processed is the initial image) and determining an updated image by preliminarily optimizing the image to be processed (e.g., according to a first loss function relating to a difference between forward projection data associated with the image to be processed and the originally acquired projection data associated with the initial image). The second optimization operation may include determining an optimized image by optimizing the updated image and designating the optimized image as a next image to be processed in a next iteration step (e.g., for the last iteration step, the optimized image is designated as the reconstructed image)… the second optimization operation may include determining an optimized image using a machine learning model. For example, for the second optimization operation, the processing device 120 may determine the optimized image by optimizing the updated image based on at least one optimizing model (e.g., at least one machine learning model)… the processing device 120 may determine whether a termination condition is satisfied in the current iteration step. Exemplary termination conditions may include that a certain count of iteration steps has been performed, the optimized image in the current iteration step has reached a desired image quality (e.g., a noise rate is less than a threshold), etc... If it is determined that the termination condition is satisfied in the current iteration step, the processing device 120 may designate the optimized image determined in the second optimization operation in the current iteration step as the reconstructed image. If it is determined that the termination condition is not satisfied in the current iteration step, the processing device 120 may proceed to a next iteration step until the termination condition is satisfied… the reconstructed image of the heart may include multiple reconstructed image slices. The processing device 120 may determine multiple optimized image slices in the iteration step corresponding to the multiple reconstructed image slices respectively. The processing device 120 may determine whether a portion of the multiple optimized image slices satisfies a desired image quality. In response to determining that the portion of the multiple image slices satisfies the desired image quality, the processing device 120 may determine all of the multiple updated image slices satisfy the desired image quality. Thus, the termination condition may be satisfied, and the multiple updated image slices may be designed as the multiple reconstructed image slices to generate the reconstructed image of the heart. In such cases, an efficiency of the image reconstruction may be improved by reducing the consumed-time of the image reconstruction in comparision with evaluating all of the optimized image and/or all of the optimized image slices… The first optimizing model, the second optimizing model, and the third optimizing model may be applied in series such that the output of the first optimizing model may be used as input to the second optimizing model, and the output of the second optimizing model may be used as input to the third optimizing model… the updated image may be input to the first optimizing model, the second optimizing model, and the third optimizing model, respectively, to output a first image, a second image, and a third image, respectively; Par. [0150-151]: processing layer may be configured to receive an image to be processed in the sub-mode… for the first sub-model 1010, the input of the processing layer (e.g., the processing layer 1) may include the input (e.g., the originally acquired projection data and/or the initial image) of the image reconstruction model 1000; for the second sub-model 1020 (e.g., 1021), the input of the processing layer (e.g., the processing layer 2) may include a sub-reconstructed image output by a reconstruction layer (e.g., the reconstruction layer 1) of a previously adjacent sub-model (e.g., the first sub-model 1010). The processing layer may be also configured to determine a regularization result by regularizing the image to be processed based on the projection data. Specifically, the processing layer may be configured to regularize the input of the processing layer and generate a regularization result (e.g., a result matrix) corresponding to the input. For example, the processing layer 1 of the first sub-model 1010 may be configured to regularize the initial image and generate a regularization result of the initial image. As another example, the processing layer 2 of the second sub-model 1021 may be configured to regularize a sub-reconstructed image output by the reconstruction layer 1 and generate a regularization result of the sub-reconstructed image… the processing layer may include a neural network model… the input of the reconstruction layer (e.g., the reconstruction layer 1) may include an output (i.e., the regularization result of the initial image) of the processing layer (e.g., the processing layer 1) of the first sub-model 1010 and the input (e.g., the projection data) of the image reconstruction model 1000; for the second sub-model 1020 (e.g., 1021), the input of the reconstruction layer (e.g., the reconstruction layer 2) may include an output (i.e., the regularization result of the sub-reconstructed image) of a processing layer (e.g., the processing layer 2) of the second sub-model 1020 (e.g., 1021) and a sub-reconstructed image output by a reconstruction layer (e.g., the reconstruction layer 1) in a previously adjacent sub-model (e.g., the first sub-model 1010). The reconstruction layer may be also configured to designate the sub-reconstructed image as a next image to be processed in a next sub-model. As illustrated, the output of the reconstruction layer of each of the plurality of sub-models may include a sub-reconstructed image corresponding to the sub-model. Accordingly, the output of the last reconstruction layer (e.g., the reconstruction layer n) of the last second sub-model (e.g., 1022) may be the reconstructed image; based on determining the machine learning process is trained:
receiving additional projection data;
applying the trained machine learning process to the additional projection data; and
based on the application of the trained machine learning process to the additional projection data, generating additional output image data, the additional output image data characterizing a final image volume (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. projection data), as indicated above, for example, including a processing device that inputs (first, second, third,… Nth) projection data to be processed (i.e. receiving additional projection data), and/or an initial image, into an image reconstruction model that is based on a machine learning model, for example, in which the processing device generates a reconstructed image (i.e. output image data) by performing a plurality (i.e. first, second, third,… Nth) of iteration steps on the initial image, such as a first, second, third,… Nth optimization operation, respectively, including training optimizing models iteratively until a termination condition is satisfied, for example, and in response to that the termination condition is satisfied (i.e. based on determining the machine learning process is trained), the optimizing models are finalized and final reconstructed image (i.e. final output image data) is generated by determining multiple optimized images, including an image slice, projection data, or a combination thereof, and the image data includes two-dimensional (2D) image data, three-dimensional (3D) image data (i.e. a volume), four-dimensional (4D) image data, for example, and including multiple optimized image slices (i.e. volume) in each iteration step corresponding to multiple reconstructed image slices (i.e. applying the trained machine learning process to the additional projection data; and based on the application of the trained machine learning process to the additional projection data, generating additional output image data, the additional output image data characterizing a final image volume), as indicated above), for example).
Regarding claim 9, claim 1 is incorporated and CAO discloses the method, wherein applying the forward projection process to the output image data comprises applying a trained deep learning process to the output image data (Par. [0075-89]: the processing device 120 (e.g., the reconstruction module 420) (e.g., the processing circuits of the processor 210) may generate a reconstructed image by performing a plurality of iteration steps on the initial image… the plurality of iteration steps may include a first optimization operation and a second optimization operation… the plurality of iteration steps may include a first optimization operation and a second optimization operation… the first optimization operation and the second optimization operation may be executed sequentially. For example, an optimized result (e.g., an output) of the first optimization operation may be an image to be optimized (e.g., an input) of the second optimization operation. Take a specific iteration step as an example, the first optimization operation may include receiving an image to be processed in the iteration step (e.g., for the first iteration step, the image to be processed is the initial image) and determining an updated image by preliminarily optimizing the image to be processed (e.g., according to a first loss function relating to a difference between forward projection data associated with the image to be processed and the originally acquired projection data associated with the initial image)… the processing device 120 may determine the optimized image by optimizing the updated image based on at least one optimizing model (e.g., at least one machine learning model)… the processing device 120 (e.g., the reconstruction module 420) (e.g., the processing circuits of the processor 210) may determine an updated image by preliminarily optimizing the image to be processed… the processing device 120 may determine forward projection data by performing a forward projection transformation on the image to be processed in the iteration step. According to the forward projection transformation, the processing device 120 may transform data (e.g., the image to be processed) in an image domain to data (e.g., the forward projection data) in a projection domain… the processing device 120 may transform the image to be processed into forward projection data by multiplying the image to be processed by a forward projection matrix… the processing device 120 may determine the updated image in the iteration step by optimizing the image to be processed according to a first loss function. The first loss function may be related to a difference between the forward projection data associated with the image to be processed and originally acquired projection data associated with the initial image. As used herein, the originally acquired projection data associated with the initial image refers to original projection data acquired… For example, the first operation in the iteration step itself may include an iterative operation. The processing device 120 may iteratively optimize the image to be processed until a value of the first loss function satisfies a termination condition in the iteration step… the processing device 120 may determine a difference between the forward projection data and originally acquired projection data as an error between the forward projection data and originally acquired projection data… the first loss function may include a fidelity term that relates to a difference between the forward projection data and originally acquired projection data associated with the initial image; Par. [0133-150]: processing device 120 may train the preliminary optimizing model iteratively until a termination condition is satisfied. In response to that the termination condition is satisfied, the optimizing model may be finalized… the termination condition may relate to a value of a loss function (also referred to as a second loss function), e.g., relating to a difference between a gold standard image and an estimated reconstructed image determined by inputting a sample image corresponding to the gold standard image to the in the specific iteration. For example, the termination condition may be satisfied if the value of the loss function is minimal or smaller than a predetermined threshold. As another example, the termination condition may be satisfied if the value of the loss function converges… “convergence” may refer to that the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold… “convergence” may refer to that a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold… image reconstruction model 1000 may include a plurality of sequentially connected sub- models, such as a first sub-model 1010 and one or more second sub-models 1020 (e.g., 1021 and 1022)… the first sub-model 1010 may be configured to receive the input (e.g., the originally acquired projection data and/or the initial image) of the image reconstruction model 1000 and generate an output… each of the plurality of sub-models may include a processing layer… and a reconstruction layer… The processing layer may be configured to receive an image to be processed in the sub-model… for the first sub-model 1010, the input of the processing layer (e.g., the processing layer 1) may include the input (e.g., the originally acquired projection data and/or the initial image) of the image reconstruction model 1000; wherein applying the forward projection process to the output image data comprises applying a trained deep learning process to the output image data (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including a processing device that inputs projection data to be processed, and/or an initial image, into an image reconstruction model that is based on a machine learning model (i.e. a deep learning process), for example, and the initial image includes an image reconstructed based on the projection data according to a reconstruction algorithm, as indicated above, for example, and the processing device generates a reconstructed image (i.e. the output image data) by performing a plurality of iteration steps on the initial image, such as a first, second, third,… Nth optimization operation, respectively, including determining forward projection data by performing a forward projection transformation on the projection data to be processed, and/or image to be processed (i.e. the originally acquired projection data associated with the initial image, or original projection data acquired), in each iteration step to generate an updated output image (i.e. applying the forward projection process to the output image data), for example, in which the updated image is input into a first optimizing model, a second optimizing model, and a third optimizing model applied (i.e. applying a deep learning process to the output image data) such that the output of the first optimizing model is used as input to the second optimizing model, and the output of the second optimizing model is used as input to the third optimizing model, respectively, to output a first image, a second image, and a third image, respectively, and the processing device trains the optimizing models iteratively until a termination condition is satisfied (i.e. wherein applying the forward projection process to the output image data comprises applying a trained deep learning process to the output image data), as indicated above), for example).
Regarding claim 11, claim 1 is incorporated and CAO discloses the method, wherein the machine learning process is based on a deep learning neural network (Par. [0039]: systems may determine the optimized image by reducing the interference information of the updated image based on a machine learning model (e.g., a deep learning model); Par. [0091-105]: loss function may include at least one of a classical regularization term or a machine learning regularization term that involves a machine learning model (e.g., a deep learning model)… the optimizing model may include a machine learning model, for example, a deep learning model such as a neural network model. The neural network model may include… a deep neural network (DNN) model… the plurality of optimizing models may be deep neural network models with different numbers of neural networks or/and different numbers of neurons. As another example, the plurality of optimizing models may be deep neural network models with different activation modes and/or different model structures).
Regarding claim 12, CAO discloses a non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations (Par. [0007]: a non-transitory computer readable medium is provided. The non-transitory computer readable medium may include executable instructions that, when executed by at least one processor, direct the at least one processor to perform a method. The method may include obtaining an initial image to be processed. The method may also include generating a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. The first optimization operation may include receiving an image to be processed in the iteration step and determining an updated image by preliminarily optimizing the image to be processed. The second optimization operation may include determining, using an optimizing model, an optimized image based on the updated image and designating the optimized image as a next image to be processed in a next iteration step or designating the optimized image as the reconstructed image).
The steps of the program further recited in claim 12 correspond to claim 1 when executed and are rejected as applied to method claim 1 above.
Regarding claim 13, claim 12 is incorporated and is a corresponding computer readable medium claim rejected as applied to the method claim 2 above.
Regarding claim 14, claim 12 is incorporated and is a corresponding computer readable medium claim rejected as applied to the method claim 3 above.
Regarding claim 15, claim 14 is incorporated and is a corresponding computer readable medium claim rejected as applied to the method claim 4 above.
Regarding claim 18, CAO discloses a system (Par. [0002-6]: systems and methods for iterative image reconstruction… a system for image reconstruction is provided. The system may include at least one storage device including a set of instructions and at least one processor configured to communicate with the at least one storage device. When executing the set of instructions, the at least one processor is configured to direct the system to perform following operations. The operations may include obtaining an initial image to be processed and generating a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. The first optimization operation may include receiving an image to be processed in the iteration step and determining an updated image by preliminarily optimizing the image to be processed. The second optimization operation may include determining, using an optimizing model, an optimized image based on the updated image and designating the optimized image as a next image to be processed in a next iteration step or designating the optimized image as the reconstructed image… a system for image reconstruction is provided) comprising:
a database (Par. [0004-5]: method may be implemented on a computing device including at least one processor and a computer-readable storage device… a system for image reconstruction is provided. The system may include at least one storage device including a set of instructions and at least one processor configured to communicate with the at least one storage device; Par. [0053-60]: storage device 150 may store data, instructions, and/or any other information… the storage device 150 may store data obtained from the scanning device 110, the terminal device 130, and/or the processing device 120… the storage device 150 may store data and/or instructions that the processing device 120 may execute or use to perform exemplary methods described in the present disclosure… storage 220 may store data/information obtained from the scanning device 110, the storage device 150, the terminal device 130, and/or any other component of the medical system 100… the storage 220 may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM), or the like, or a combination thereof… the storage 220 may store one or more programs and/or instructions to perform exemplary methods described in the present disclosure. For example, the storage 220 may store a program for the processing device 120 to execute to determine an optimizing model. As another example, the storage 220 may store a program for the processing device 120 to execute to apply the optimizing model to determine an optimized image); and
at least one processor communicatively coupled to the database (Par. [0004-5]: method may be implemented on a computing device including at least one processor and a computer-readable storage device… a system for image reconstruction is provided. The system may include at least one storage device including a set of instructions and at least one processor configured to communicate with the at least one storage device; Par. [0053-54]: the storage device 150 may store data obtained from the scanning device 110, the terminal device 130, and/or the processing device 120… the storage device 150 may store data and/or instructions that the processing device 120 may execute or use to perform exemplary methods described in the present disclosure … the storage device 150 may be connected to the network 140 to communicate with one or more components (e.g., the scanning device 110, the processing device 120, the terminal device 130) of the medical system 100. One or more components of the medical system 100 may access the data or instructions stored in the storage device 150 via the network 140) and configured to (Par. [0002-7]: systems and methods for iterative image reconstruction… a system for image reconstruction is provided. The system may include at least one storage device including a set of instructions and at least one processor configured to communicate with the at least one storage device. When executing the set of instructions, the at least one processor is configured to direct the system to perform following operations. The operations may include obtaining an initial image to be processed and generating a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. The first optimization operation may include receiving an image to be processed in the iteration step and determining an updated image by preliminarily optimizing the image to be processed. The second optimization operation may include determining, using an optimizing model, an optimized image based on the updated image and designating the optimized image as a next image to be processed in a next iteration step or designating the optimized image as the reconstructed image… a non-transitory computer readable medium is provided. The non-transitory computer readable medium may include executable instructions that, when executed by at least one processor, direct the at least one processor to perform a method. The method may include obtaining an initial image to be processed. The method may also include generating a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. The first optimization operation may include receiving an image to be processed in the iteration step and determining an updated image by preliminarily optimizing the image to be processed. The second optimization operation may include determining, using an optimizing model, an optimized image based on the updated image and designating the optimized image as a next image to be processed in a next iteration step or designating the optimized image as the reconstructed image; [0053-60]: storage device 150 may store data, instructions, and/or any other information… the storage device 150 may store data obtained from the scanning device 110, the terminal device 130, and/or the processing device 120… the storage device 150 may store data and/or instructions that the processing device 120 may execute or use to perform exemplary methods described in the present disclosure… storage 220 may store data/information obtained from the scanning device 110, the storage device 150, the terminal device 130, and/or any other component of the medical system 100… the storage 220 may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM), or the like, or a combination thereof… the storage 220 may store one or more programs and/or instructions to perform exemplary methods described in the present disclosure. For example, the storage 220 may store a program for the processing device 120 to execute to determine an optimizing model. As another example, the storage 220 may store a program for the processing device 120 to execute to apply the optimizing model to determine an optimized image).
The steps of the program further recited in claim 18 correspond to claim 1 when executed and are rejected as applied to method claim 1 above.
Regarding claim 19, claim 18 is incorporated and is a corresponding apparatus claim rejected as applied to the method claim 2 above.
Regarding claim 20, claim 18 is incorporated and is a corresponding apparatus claim rejected as applied to the method claim 3 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over CAO, as applied to claim 1 above, in view of Whiteley et al. (US PG Publication No. 2021/0104079 A1), hereafter referred to as Whiteley, Applicant cited prior art as US Patent No. 11164344 (grounds are restated below for convenience).
Regarding claim 5, claim 1 is incorporated and CAO discloses the method, further comprising:
receiving positron emission tomography (PET) measurement data from an image scanning system (Par. [0035-43]: the medical system may include a medical imaging system. The medical imaging system may include a single modality system and/or a multi-modality system… The single modality system may include, for example… a positron emission tomography (PET) system… The multi-modality system may include, for example… a positron emission tomography-X-ray imaging (PET-X-ray) system… a positron emission tomography-computed tomography (PET-CT) system… a positron emission tomography-magnetic resonance imaging (PET-MR) system… scanning device 110 may be configured to acquire image data relating to at least one part of a subject. The scanning device 110 may scan the subject or a portion thereof that is located within its detection region and generate image data relating to the subject or the portion thereof. The image data relating to at least one part of a subject may include… projection data… the scanning device 110 may include a single modality imaging device. For example, the scanning device 110 may include… a positron emission tomography (PET) device…Exemplary multi-modality imaging devices may include a PET-CT device, a PET-MRI device); and
generating the projection data based on the PET measurement data (Par. [0035-43]: the medical system may include a medical imaging system. The medical imaging system may include a single modality system and/or a multi-modality system… The single modality system may include, for example… a positron emission tomography (PET) system… The multi-modality system may include, for example… a positron emission tomography-X-ray imaging (PET-X-ray) system… a positron emission tomography-computed tomography (PET-CT) system… a positron emission tomography-magnetic resonance imaging (PET-MR) system… scanning device 110 may be configured to acquire image data relating to at least one part of a subject. The scanning device 110 may scan the subject or a portion thereof that is located within its detection region and generate image data relating to the subject or the portion thereof. The image data relating to at least one part of a subject may include… projection data… the scanning device 110 may include a single modality imaging device. For example, the scanning device 110 may include… a positron emission tomography (PET) device…Exemplary multi-modality imaging devices may include a PET-CT device, a PET-MRI device… For illustration purposes, the present disclosure is described with reference to a CT device; generating the projection data based on the PET measurement data (e.g. systems and methods for iterative image reconstruction include obtaining scanning data of an object, also referred to as originally acquired projection data (i.e. the projection data), as indicated above, for example, including medical imaging systems, such as a positron emission tomography (PET) system (i.e. generating the projection data based on the PET measurement data), as indicated above), for example), but fails to teach the following as further recited in claim 5.
However, Whiteley teaches the projection data characterizing histo-images (Par. [0014-38]: provide a neural network which deconvolves a (blurred) histo-image to obtain a simulated reconstructed PET image. The histo-image may be generated from TOF sinograms using TOF back-projection, or directly from PET event data by back-projecting each recorded event. The neural network may be trained using PET images which were reconstructed from PET raw data (e.g., list mode or TOF sinograms) using conventional reconstruction techniques… System 100 includes trained network 110… Although depicted as a neural network, network 110 and all neural networks referred to herein may comprise any type of processing system to implement a function. For example, network 110 may comprise a software application programmed to implement a function generated via prior neural network training as described below. Network 110 and all neural networks described herein may comprise hardware and software specifically-intended for executing algorithms based on a specified network architecture and trained network parameters … System 200 includes trained network 230 which, similar to network 110 of system 100, operates to generate an output image based on an input histo-image. Unlike the deployment of network 110, the histo-image is generated by back-projecting each event specified in list-mode PET data 210… a PET scanner acquires list-mode data 210 as is known in the art. List-mode data 210 includes the coordinates of each detected coincidence event. Event back-projection component 220 applies a back-projection algorithm to each event of list-mode data 210 generate a histo-image. Generally, the back-projection algorithm assigns every event to an image voxel along the LOR, according to its timing information. The histo-image generated by back-projecting list-mode data 210 may be more accurate than a histo-image generated by back-projecting sinograms generated from list-mode data 210 because list mode data exhibits higher-resolution timing data than TOF sinogram data generated therefrom. The generated histo-image is then input to trained network 230… Network 230 applies its trained function to the histo-image (and optional mu-map) to generate an output image… each of the acquired PET datasets is histogrammed into TOF sinograms as is known in the art. Next, at S330, training image volumes are generated by reconstructing the raw PET data (list-mode or sinogram) using any reconstruction algorithm that is or becomes known… Each histo-image (and, optionally, each corresponding mu-map) is input into a neural network at S340 to generate a plurality of output image volumes. That is, the neural network generates a single output image volume based on each input histo-image volume. At S360, the neural network is trained based on differences between the plurality of output image volumes and corresponding ones of the plurality of PET training image volumes… Training system 440 may employ any type of neural network training that is or becomes known. Generally, training system 440 may determine a loss based on a comparison between k output image volumes generated by network 430 and corresponding ones of PET training image volumes.sub.1-k. The loss may comprise an L1 loss, and L2 loss, or any other suitable measure of total loss. An L1 loss is the sum of the absolute differences between each output image and its corresponding ground truth PET image, and an L2 loss is the sum of the squared differences between each output image and its corresponding ground truth PET image… The process of S350 and S360 may repeat until it is determined that the loss has reached an acceptable level or training otherwise terminates. At termination, network 430 may be considered trained. Conceptually, network 430 has been trained to perform the reconstruction of the target PET images based on corresponding histo-images; the projection data characterizing histo-images (e.g. system, and corresponding method, includes trained network which operates to generate an output image based on input histo-images generated from sinograms using back-projection (i.e. projection data characterizing histo-images), or directly from PET event data by back-projecting each recorded event, for example, and each histo-image is input into a neural network to generate a plurality of output image volumes in order to generate a single output image volume based on each input histo-image volume, as indicated above), for example).
CAO and Whiteley are considered to be analogous art because they pertain to medical image processing applications. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the systems and methods for iterative image reconstruction (as disclosed by CAO) with the projection data characterizing histo-images (as taught by Whiteley, Abstract, Par. [0014-38]) to generate an output image based on an input histo-image, to generate a plurality of output image volumes in order to generate a single output image volume based on each input histo-image volume, to obtain a reconstructed PET image, and to generate improved images (Whiteley, Abstract, Par. [0003, 14-38]).
Regarding claim 10, claim 1 is incorporated and CAO discloses the method (Par. [0002-4]), but fails to teach the following as further recited in claim 10.
However, Whiteley teaches the projection data characterizing histo-images (Par. [0014-38]: provide a neural network which deconvolves a (blurred) histo-image to obtain a simulated reconstructed PET image. The histo-image may be generated from TOF sinograms using TOF back-projection, or directly from PET event data by back-projecting each recorded event. The neural network may be trained using PET images which were reconstructed from PET raw data (e.g., list mode or TOF sinograms) using conventional reconstruction techniques… System 100 includes trained network 110… Although depicted as a neural network, network 110 and all neural networks referred to herein may comprise any type of processing system to implement a function. For example, network 110 may comprise a software application programmed to implement a function generated via prior neural network training as described below. Network 110 and all neural networks described herein may comprise hardware and software specifically-intended for executing algorithms based on a specified network architecture and trained network parameters … System 200 includes trained network 230 which, similar to network 110 of system 100, operates to generate an output image based on an input histo-image. Unlike the deployment of network 110, the histo-image is generated by back-projecting each event specified in list-mode PET data 210… a PET scanner acquires list-mode data 210 as is known in the art. List-mode data 210 includes the coordinates of each detected coincidence event. Event back-projection component 220 applies a back-projection algorithm to each event of list-mode data 210 generate a histo-image. Generally, the back-projection algorithm assigns every event to an image voxel along the LOR, according to its timing information. The histo-image generated by back-projecting list-mode data 210 may be more accurate than a histo-image generated by back-projecting sinograms generated from list-mode data 210 because list mode data exhibits higher-resolution timing data than TOF sinogram data generated therefrom. The generated histo-image is then input to trained network 230… Network 230 applies its trained function to the histo-image (and optional mu-map) to generate an output image… each of the acquired PET datasets is histogrammed into TOF sinograms as is known in the art. Next, at S330, training image volumes are generated by reconstructing the raw PET data (list-mode or sinogram) using any reconstruction algorithm that is or becomes known… Each histo-image (and, optionally, each corresponding mu-map) is input into a neural network at S340 to generate a plurality of output image volumes. That is, the neural network generates a single output image volume based on each input histo-image volume. At S360, the neural network is trained based on differences between the plurality of output image volumes and corresponding ones of the plurality of PET training image volumes… Training system 440 may employ any type of neural network training that is or becomes known. Generally, training system 440 may determine a loss based on a comparison between k output image volumes generated by network 430 and corresponding ones of PET training image volumes.sub.1-k. The loss may comprise an L1 loss, and L2 loss, or any other suitable measure of total loss. An L1 loss is the sum of the absolute differences between each output image and its corresponding ground truth PET image, and an L2 loss is the sum of the squared differences between each output image and its corresponding ground truth PET image… The process of S350 and S360 may repeat until it is determined that the loss has reached an acceptable level or training otherwise terminates. At termination, network 430 may be considered trained. Conceptually, network 430 has been trained to perform the reconstruction of the target PET images based on corresponding histo-images; the projection data characterizing histo-images (e.g. system, and corresponding method, includes trained network which operates to generate an output image based on input histo-images generated from sinograms using back-projection (i.e. projection data characterizing histo-images), or directly from PET event data by back-projecting each recorded event, for example, and each histo-image is input into a neural network to generate a plurality of output image volumes in order to generate a single output image volume based on each input histo-image volume, as indicated above), for example).
The same motivation to combine above-mentioned teachings applies, as previously indicated in claim 5.
Regarding claim 17, claim 12 is incorporated and is a corresponding computer readable medium claim rejected as applied to the method claim 10 above.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over CAO, in view of Whiteley, as applied to claim 5 above, in further view of Whiteley et al. (US PG Publication No. 2021/0074034 A1), hereafter referred to as Whiteley II (grounds are restated below for convenience).
Regarding claim 6, claim 5 is incorporated and the combination of CAO and Whiteley, as a whole, teaches the method (CAO, Par. [0002-4], but fails to teach the following as further recited in claim 6.
However, Whiteley II teaches further comprising: receiving an attenuation map from the image scanning system;
applying the machine learning process to the projection data and the attenuation map; and
generating the output image data based on the application of the machine learning process to the projection data and the attenuation map (Par. [0027-60]: present disclosure can employ machine learning methods or processes to provide clinical information from nuclear imaging systems. For example, the embodiments can employ machine learning methods or processes to reconstruct images based on captured measurement data, and provide the reconstructed images for clinical diagnosis… machine learning methods or processes are trained, to improve the reconstruction of images… a scanning device, such as a positron emission tomography (PET) scanner, provides measurement data, such as three-dimensional (3D) time-of-flight sinograms (e.g., projection data). The measurement data is passed through a neural network with trained parameters to generate a reconstructed image… a machine learning algorithm can be trained with an image data (e.g., sinogram data) training set (e.g., until convergence) to generate activation maps that identify how much each bin of the image data training set (e.g., as provided by a PET scanner) contributes to each pixel of the output image. As known in the art, a bin is an element of a sinogram, analogous to a pixel in an image. The masks can be generated based on the activation maps… FIG. 1A illustrates one embodiment of a nuclear imaging system 100. In this example, the nuclear imaging system 100 employs an imaging pipeline using time-of-flight (TOF) Fourier rebinned PET sinogram data and X-ray CT based attenuation maps to generate PET image volumes. As illustrated, nuclear imaging system 100 includes image scanning system 102 and image reconstruction system 104. Image scanning system 102 can be, for example, a PET/CT scanner. The image scanning system 102 generates sinogram data 103, such as TOF sinograms, as well as attenuation maps 105… Neural network 116 receives 2D sinograms 107 from sum TOF bins 114, as well as attenuation maps 105 from image scanning system 102. Attenuation maps 105 can be generated by a CT imaging system, where the attenuation maps 105 are generated based on X-ray data captured by the CT imaging system. Neural network 116 operates on 2D sinogram data 107 and attenuation maps 105 to generate final image volume 191. For example, a single forward pass of 2D sinograms 107 and attenuation maps 105 through neural network 116 of the PET/CT data produces final image volume 191, which can be a multi-slice image volume… masks 142 can be generated by forward projecting image patches 145… Referring now to FIG. 1D, refinement and scaling segment 160 obtains the initial image estimate, initial image volume 149, as well as the corresponding attenuation maps 105. Refinement and scaling segment 160 can remove noise and scale the initial image volume 149 to full size, illustrated as final image volume 121. The attenuation maps 105 provide additional image space anatomical information that can significantly boost the neural network's 116 output image quality. Refinement and scaling segment 160 can perform operations that draw on significant deep learning research in the areas of denoising and super-resolution, for example; Par. [0082]: method comprises receiving a plurality of attenuation maps from the image scanning system, wherein the plurality of attenuation maps correspond to each of a plurality of slices of the sinogram data, and applying a plurality of convolutional layers to the first image and to the plurality of attenuation maps to generate a second image (e.g., final image volume 121)… the method further comprises applying a plurality of residual blocks to the first image and to the plurality of attenuation maps after applying the plurality of convolutional layers to generate the second image; further comprising: receiving an attenuation map from the image scanning system;
applying the machine learning process to the projection data and the attenuation map; and
generating the output image data based on the application of the machine learning process to the projection data and the attenuation map (e.g. machine learning methods or processes (i.e. machine learning process) to provide clinical information from nuclear imaging systems include receiving a plurality of attenuation maps from an image scanning system (i.e. receiving an attenuation map from the image scanning system), wherein the plurality of attenuation maps correspond to each of a plurality of slices of sinogram data, or projection data (i.e. the projection data), for example, and applying a plurality of convolutional layers of a neural network to a first image and to the plurality of attenuation maps in order to generate a final output image volume (i.e. applying the machine learning process to the projection data and the attenuation map; and generating the output image data based on the application of the machine learning process to the projection data and the attenuation map), as indicated above), for example).
CAO, Whiteley, and Whiteley II are considered to be analogous art because they pertain to medical image processing applications. Therefore, the combined teachings of CAO, Whiteley, and Whiteley II, as a whole, would have rendered obvious the invention recited in claim 6 with a reasonable expectation of success in order to modify the systems and methods for iterative image reconstruction (as disclosed by CAO) with receiving an attenuation map from the image scanning system; applying the machine learning process to the projection data and the attenuation map; and generating the output image data based on the application of the machine learning process to the projection data and the attenuation map (as taught by Whiteley II, Abstract, Par. [0027-60]) to improve the reconstruction of images, to generate a final image, to generate reconstructed image, to generate PET image volumes, and to generate final image volumes (Whiteley II, Abstract, Par. [0003-9, 27-60]).
Claims 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over CAO, as applied to claim 1 above, in view of Whiteley II (grounds are restated below for convenience).
Regarding claim 8, claim 1 is incorporated and CAO discloses the method (Par. [0002-4]), but fails to teach the following as further recited in claim 8.
However, Whiteley II teaches wherein applying the forward projection process to the output image data comprises:
generating attenuation data based on attenuating the output image data;
generating forward projected data based on forward projecting the attenuation data (Par. [0031-73]: execution of the machine learning algorithm generates values (e.g., weights) identifying the impact each bin of the image training set has on each pixel of the output image. The masks can then be generated based on the weights… the masks are generated by forward projecting image patches. For example, given a patch of pixels in image space all set to a same constant non-zero value (e.g., 1), and any remaining image pixels set to zero, performing the Radon transform on the given image (e.g., forward projecting the image into sinogram space) produces a sinogram where the bins in sinogram space corresponding to the pixels of the patch in image space that are non-zero and the remaining sinogram bins are zero, effectively producing a sinogram mask for the given image patch… the nuclear imaging system 100 employs an imaging pipeline using time-of-flight (TOF) Fourier rebinned PET sinogram data and X-ray CT based attenuation maps to generate PET image volumes. As illustrated, nuclear imaging system 100 includes image scanning system 102 and image reconstruction system 104. Image scanning system 102 can be, for example, a PET/CT scanner. The image scanning system 102 generates sinogram data 103, such as TOF sinograms, as well as attenuation maps… … Neural network 116 receives 2D sinograms 107 from sum TOF bins 114, as well as attenuation maps 105 from image scanning system 102. Attenuation maps 105 can be generated by a CT imaging system, where the attenuation maps 105 are generated based on X-ray data captured by the CT imaging system. Neural network 116 operates on 2D sinogram data 107 and attenuation maps 105 to generate final image volume 191. For example, a single forward pass of 2D sinograms 107 and attenuation maps 105 through neural network 116 of the PET/CT data produces final image volume 191, which can be a multi-slice image volume… masks 142 can be generated by forward projecting image patches 145… Referring now to FIG. 1D, refinement and scaling segment 160 obtains the initial image estimate, initial image volume 149, as well as the corresponding attenuation maps 105. Refinement and scaling segment 160 can remove noise and scale the initial image volume 149 to full size, illustrated as final image volume 121. The attenuation maps 105 provide additional image space anatomical information that can significantly boost the neural network's 116 output image quality. Refinement and scaling segment 160 can perform operations that draw on significant deep learning research in the areas of denoising and super-resolution, for example… each image patch and its associated buffer are forward projected to create a mask 342 for the input sinogram 302 that isolates only the bins that contribute to the output image; wherein applying the forward projection process to the output image data comprises:
generating attenuation data based on attenuating the output image data;
generating forward projected data based on forward projecting the attenuation (e.g. machine learning methods or processes (i.e. machine learning process) to provide clinical information from nuclear imaging systems include receiving a plurality of attenuation maps from an image scanning system (i.e. receiving an attenuation map from the image scanning system), wherein the plurality of attenuation maps correspond to each of a plurality of slices of sinogram data, or projection data (i.e. the projection data), for example, and applying a plurality of convolutional layers of a neural network to a first image and to the plurality of attenuation maps in order to generate a final output image volume (i.e. generating attenuation data based on attenuating the output image data), for example, by using a Neural network that operates on 2D sinogram data and attenuation maps to generate final image volume, for example, in which a single forward pass of 2D sinograms and attenuation maps (i.e. generating forward projected data based on forward projecting the attenuation) through neural network of PET/CT data produces final image volume, as indicated above), for example);
generating normalized data based on normalizing the forward projected data (Par. [0037-39]: image reconstruction system 104 processes sinogram data 103 to generate final image volume 191. Final image volume 191 can include image data that can be provided for display and analysis, for example. In this example, random correction 106 of image reconstruction system 104 applies a correction for random measurement data, which can be a source of noise, to sinogram data 103. Image reconstruction system 104 normalizes the output of the random correction 106 by applying normalization 108 (e.g., based on calibration parameters), and then applies an arc correction 110 to the output of normalization 108… Neural network 116 receives 2D sinograms 107 from sum TOF bins 114, as well as attenuation maps 105 from image scanning system 102. Attenuation maps 105 can be generated by a CT imaging system, where the attenuation maps 105 are generated based on X-ray data captured by the CT imaging system. Neural network 116 operates on 2D sinogram data 107 and attenuation maps 105 to generate final image volume 191. For example, a single forward pass of 2D sinograms 107 and attenuation maps 105 through neural network 116 of the PET/CT data produces final image volume 191, which can be a multi-slice image volume; generating normalized data based on normalizing the forward projected data; and
generating the forward projected image data based on correcting the normalized data for scatter and random coincidences (e.g. machine learning methods or processes (i.e. machine learning process) to provide clinical information from nuclear imaging systems include receiving a plurality of attenuation maps from an image scanning system, wherein the plurality of attenuation maps correspond to each of a plurality of slices of sinogram data, or projection data (i.e. the projection data), for example, including applying a correction for random measurement data (i.e. random coincidences), which can be a source of noise (i.e. scatter, error, etc.), to sinogram data, for example, and normalizes the output of the random correction by applying normalization (i.e. generating the forward projected image data based on correcting the normalized data for scatter and random coincidences), for example, by using a Neural network that operates on 2D sinogram data and attenuation maps to generate a final image volume, for example, in which a single forward pass of 2D sinograms and attenuation maps (i.e. generating normalized data based on normalizing the forward projected data and generating the forward projected image data based on correcting the normalized data for scatter and random coincidences) through neural network of PET/CT data produces final image volume, as indicated above), for example).
CAO and Whiteley II are considered to be analogous art because they pertain to medical image processing applications. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the systems and methods for iterative image reconstruction (as disclosed by CAO) with wherein applying the forward projection process to the output image data comprises: generating attenuation data based on attenuating the output image data; generating forward projected data based on forward projecting the attenuation data; generating normalized data based on normalizing the forward projected data; and generating the forward projected image data based on correcting the normalized data for scatter and random coincidences (as taught by Whiteley II, Abstract, Par. [0027-73]) to improve the reconstruction of images, to generate a final image, to generate reconstructed image, to generate PET image volumes, and to generate final image volumes (Whiteley II, Abstract, Par. [0003-9, 27-60]).
Regarding claim 16, claim 12 is incorporated and CAO discloses the computer readable medium of claim 12 storing instructions that, when executed by at least one processor, further cause the at least one processor to perform operations (Par. [0007]: a non-transitory computer readable medium is provided. The non-transitory computer readable medium may include executable instructions that, when executed by at least one processor, direct the at least one processor to perform a method. The method may include obtaining an initial image to be processed. The method may also include generating a reconstructed image by performing a plurality of iteration steps on the initial image. At least one of the plurality of iteration steps may include a first optimization operation and a second optimization operation. The first optimization operation may include receiving an image to be processed in the iteration step and determining an updated image by preliminarily optimizing the image to be processed. The second optimization operation may include determining, using an optimizing model, an optimized image based on the updated image and designating the optimized image as a next image to be processed in a next iteration step or designating the optimized image as the reconstructed image), but fails to teach the following as further recited in claim 16.
However, Whiteley II teaches comprising:
receiving an attenuation map from the image scanning system;
applying the machine learning process to the projection data and the attenuation map; and
generating the output image data based on the application of the machine learning process to the projection data and the attenuation map (Par. [0027-60]: present disclosure can employ machine learning methods or processes to provide clinical information from nuclear imaging systems. For example, the embodiments can employ machine learning methods or processes to reconstruct images based on captured measurement data, and provide the reconstructed images for clinical diagnosis… machine learning methods or processes are trained, to improve the reconstruction of images… a scanning device, such as a positron emission tomography (PET) scanner, provides measurement data, such as three-dimensional (3D) time-of-flight sinograms (e.g., projection data). The measurement data is passed through a neural network with trained parameters to generate a reconstructed image… a machine learning algorithm can be trained with an image data (e.g., sinogram data) training set (e.g., until convergence) to generate activation maps that identify how much each bin of the image data training set (e.g., as provided by a PET scanner) contributes to each pixel of the output image. As known in the art, a bin is an element of a sinogram, analogous to a pixel in an image. The masks can be generated based on the activation maps… FIG. 1A illustrates one embodiment of a nuclear imaging system 100. In this example, the nuclear imaging system 100 employs an imaging pipeline using time-of-flight (TOF) Fourier rebinned PET sinogram data and X-ray CT based attenuation maps to generate PET image volumes. As illustrated, nuclear imaging system 100 includes image scanning system 102 and image reconstruction system 104. Image scanning system 102 can be, for example, a PET/CT scanner. The image scanning system 102 generates sinogram data 103, such as TOF sinograms, as well as attenuation maps 105… Neural network 116 receives 2D sinograms 107 from sum TOF bins 114, as well as attenuation maps 105 from image scanning system 102. Attenuation maps 105 can be generated by a CT imaging system, where the attenuation maps 105 are generated based on X-ray data captured by the CT imaging system. Neural network 116 operates on 2D sinogram data 107 and attenuation maps 105 to generate final image volume 191. For example, a single forward pass of 2D sinograms 107 and attenuation maps 105 through neural network 116 of the PET/CT data produces final image volume 191, which can be a multi-slice image volume… masks 142 can be generated by forward projecting image patches 145… Referring now to FIG. 1D, refinement and scaling segment 160 obtains the initial image estimate, initial image volume 149, as well as the corresponding attenuation maps 105. Refinement and scaling segment 160 can remove noise and scale the initial image volume 149 to full size, illustrated as final image volume 121. The attenuation maps 105 provide additional image space anatomical information that can significantly boost the neural network's 116 output image quality. Refinement and scaling segment 160 can perform operations that draw on significant deep learning research in the areas of denoising and super-resolution, for example; Par. [0082]: method comprises receiving a plurality of attenuation maps from the image scanning system, wherein the plurality of attenuation maps correspond to each of a plurality of slices of the sinogram data, and applying a plurality of convolutional layers to the first image and to the plurality of attenuation maps to generate a second image (e.g., final image volume 121)… the method further comprises applying a plurality of residual blocks to the first image and to the plurality of attenuation maps after applying the plurality of convolutional layers to generate the second image; further comprising: receiving an attenuation map from the image scanning system;
applying the machine learning process to the projection data and the attenuation map; and
generating the output image data based on the application of the machine learning process to the projection data and the attenuation map (e.g. machine learning methods or processes (i.e. machine learning process) to provide clinical information from nuclear imaging systems include receiving a plurality of attenuation maps from an image scanning system (i.e. receiving an attenuation map from the image scanning system), wherein the plurality of attenuation maps correspond to each of a plurality of slices of sinogram data, or projection data (i.e. the projection data), for example, and applying a plurality of convolutional layers of a neural network to a first image and to the plurality of attenuation maps in order to generate a final output image volume (i.e. applying the machine learning process to the projection data and the attenuation map; and generating the output image data based on the application of the machine learning process to the projection data and the attenuation map), as indicated above), for example).
CAO and Whiteley II are considered to be analogous art because they pertain to medical image processing applications. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify the systems and methods for iterative image reconstruction (as disclosed by CAO) with receiving an attenuation map from the image scanning system; applying the machine learning process to the projection data and the attenuation map; and generating the output image data based on the application of the machine learning process to the projection data and the attenuation map (as taught by Whiteley II, Abstract, Par. [0027-60]) to improve the reconstruction of images, to generate a final image, to generate reconstructed image, to generate PET image volumes, and to generate final image volumes (Whiteley II, Abstract, Par. [0003-9, 27-60]).
Conclusion
THIS ACTION IS MADE FINAL.
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GUILLERMO RIVERA-MARTINEZ whose telephone number is 571-272-4979. The examiner can normally be reached on Monday-Friday (8am - 5pm Eastern Time). If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GUILLERMO M RIVERA-MARTINEZ/ Primary Examiner, Art Unit 2677
Prosecution Insights