DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
1. The information disclosure statements (IDS) submitted on 10/18/2024 and 5/30/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements have been considered by the examiner.
Specification
2. The disclosure is objected to because of the following informalities:
In [0079], line 5-6, "processing device 230" should read "processing device 120"
Appropriate correction is required.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Adler et al. (US-6023495-A, hereinafter "Adler") in view of Li et al. (US-2014/0187932-A1, hereinafter "Li"), and further in view of Jun Jin Hwan (KR-101226479-B1).
5. As per claim 1, Adler discloses: A method for image generation, implemented on a computing device having one or more processors and one or more storage devices, the method comprising: (Adler, col. 5, lines 39-47, “The CT images 160 and the related CT projection data 170 are input to a computer 150 and, optionally, stored on an external storage medium 154. … The computer 150 includes a central processing unit (CPU) 152 and memory 156.”)
obtaining a three-dimensional (3D) base image of a target subject, the 3D base image being captured by performing a 3D scan on the target subject with a preset posture; (Adler, col. 4, lines 1-3, “According to the invention, we solve this problem by integrating three-dimensional CT scans of a patient's anatomy ...” and col. 4, lines 12-15, “In the test case of spinal examination of patients afflicted by idiopathic scoliosis, we can capture the salient features of the spinal deformation by combining CT scan data and the scout data.” And col. 6, lines 47-53, “The description of the collection of data in the preferred embodiment involves CT scan machines for supine patients as this is the current state of the art. This description would adapt readily to examinations where the patient is erect, as soon as such CT scan machines are available (the erect position might be preferable as it is closer to traditional examinations).”)
obtaining one or more two-dimensional (2D) scout images of the target subject, the one or more 2D scout images of the target subject being captured by performing a scout scan on the target subject with the preset posture [[after an internal structure of the target subject changes;]] (Adler, col. 4, lines 1-11, “According to the invention, we solve this problem by integrating three-dimensional CT scans of a patient's anatomy with three-dimensional data derived from CT scout images (or scouts). Scouts are digital two-dimensional X-ray images produced by a CT scanner. ... In an alternative embodiment of this invention standard two-dimensional X-ray images could be used. In this case, they would first need to be digitized, and their scanning geometry registered with that of the CT scan data.” and col. 4, lines 25-26, “In the preferred embodiment of this invention the 2D X-ray images are scout images.” and col. 4, lines 12-15, “In the test case of spinal examination of patients afflicted by idiopathic scoliosis, we can capture the salient features of the spinal deformation by combining CT scan data and the scout data.” And col. 6, lines 47-53, “The description of the collection of data in the preferred embodiment involves CT scan machines for supine patients as this is the current state of the art. This description would adapt readily to examinations where the patient is erect, as soon as such CT scan machines are available (the erect position might be preferable as it is closer to traditional examinations).”
generating a 3D [[predicted]] image of the subject based on the 3D base image and the one or more 2D scout images, the 3D predicted image indicating the internal structure of the target subject when the one or more 2D scout images are captured. (Adler, col. 4, lines 12-15, “In the test case of spinal examination of patients afflicted by idiopathic scoliosis, we can capture the salient features of the spinal deformation by combining CT scan data and the scout data.” and col. 4, lines 26-37, “Since scout images are produced by the same device (the CT scanner) that produced the CT slices, both types of data (CT slices and scout images) are automatically registered; i.e., their positions in 3D are described in the same coordinate frame. Since the scout data are a form of X-ray projection data, we can apply standard tomographic reconstruction algorithms to create additional 2D cross-sectional slices of the 3D object that has been scanned. These 2D slices are then combined with the original slices produced directly by the CT scanner to provide a 3D volume of data from which the geometry of 3D object is derived.”)
6. Adler doesn't explicitly disclose but Li discloses: [[obtaining one or more two-dimensional (2D) scout images of the target subject, the one or more 2D scout images of the target subject being captured by performing a scout scan on the target subject with the preset posture]] after an internal structure of the target subject changes; (Li, [0077]-[0078], “In another exemplary embodiment, the CT system 100 is initiated to perform a shuttle-mode scout scan on the subject 114 so as to carry out real-time tracking of coronary artery enhancement of the subject 114. To be specific, scan range of the shuttle-mode scout scan can be arranged via the input device 134. For instance, the scan range can be positioned above the heart of the subject 114 to monitor the aorta enhancement, or the scan range of the region of interest where the heart of the subject 114 is located can be arranged as about 300 mm to get effective enhancement.”, Examiner’s note: The scout imaging process disclosed by Li is a real-time process and is one used to monitor tracking of a coronary artery enhancement. This is monitoring changes in the internal structure of the subject.)
7. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Adler to include the disclosure of obtains scout images of the target subject after an internal structure of the target subject changes, of Li. The motivation for this modification could have been to provide an updated view of the subject’s interior as the images update. This could be useful during a medical procedure to understand the current status within the body of the subject.
8. Adler in view of Li doesn't explicitly disclose but Jun Jin Hwan discloses: [[generating a 3D]] predicted [[image of the subject based on the 3D base image and the one or more 2D scout images, the 3D predicted image indicating the internal structure of the target subject when the one or more 2D scout images are captured.]] (Jun Jin Hwan, Abstract, "PURPOSE: A scout image obtaining method in CT is provided to use a coupling section image, thereby predicting section data after a 3D reconfiguration process." and page 3, [0010]-[0011], “The technical problem that the present invention aims to solve is to provide a method for determining a region of interest in three dimensions that is improved by adding a tomosynthesis function to the scout image acquisition process of Citi. Another technical problem that the present invention aims to solve is to provide a method for predicting cross-sectional data after 3D reconstruction processing in the process of acquiring scout images of a city with added tomosynthesis functions.” and page 4, [0019]-[0020], “The present invention provides the effect of offering a more improved method for determining a region of interest in three dimensions by adding a tomosynthesis function. In addition, the present invention provides the effect of predicting cross-sectional data after 3D reconstruction processing through combined cross-sectional images.”)
9. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method Adler in view of Li to include the disclosure of generating a 3D predicted image of the subject, of Jun Jin Hwan. The motivation for this modification could have been to generate a representative 3D image of the subject’s interior, reflecting the latest information. This could be useful during a medical procedure to understand the current status within the body of the subject.
10. Claim 12 is similar in scope to claim 1 except for a different limitation that Adler in view of Li, and further in view of Jun Jin Hwan discloses: A system for image generation, implemented on a computing device having one or more processors and one or more storage devices, the system comprising:
at least one storage device including a set of instructions; and
at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to direct the system to perform operations including: (Adler, col. 5, lines 39-47, “The CT images 160 and the related CT projection data 170 are input to a computer 150 and, optionally, stored on an external storage medium 154. … The computer 150 includes a central processing unit (CPU) 152 and memory 156.” and col. 5, lines 48-50, “A geometric modeling program 200 stored in memory 156 running on CPU 152 of the computer 150 reads in the CT images 160 and projection data 170 …”)
11. Claim 20, which is similar in scope to claim 1, is thus rejected under the same rationale as described above.
12. Claims 2, 10-11, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Adler et al. (US-6023495-A, hereinafter "Adler") in view of Li et al. (US-2014/0187932-A1, hereinafter "Li"), further in view of Jun Jin Hwan (KR-101226479-B1), and further in view of Xia et al. (US-2022/0130520-A1, hereinafter "Xia").
13. As per claim 2, Adler in view of Li, and further in view of Jun Jin Hwan discloses: The method of claim 1, the generating a 3D predicted image of the subject comprises:
generating the 3D predicted image [[by processing the 3D base image and the one or more 2D scout images using an image generation model, wherein the image generation model is a machine learning model.]] (See rejection for claim 1.)
14. Adler in view of Li, and further in view of Jun Jin Hwan doesn't explicitly disclose but Xia discloses: [[generating the 3D predicted image]] by processing the 3D base image and the one or more 2D scout images using an image generation model, wherein the image generation model is a machine learning model. (Xia, [0042], “In an embodiment, the present disclosure describes an artificial intelligence-based system for automated image quality assessment and protocol optimization. The system may include a computer simulation tool that can simulate CT images, which may be simulated two-dimensional (2D) CT images and/or simulated three-dimensional (3D) CT images, and corresponding dose map(s), from acquired scout scan information and scout scan conditions.”)
15. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Adler in view of Li, and further in view of Jun Jin Hwan to include the disclosure of using a machine learning model to process the 3D base image and one or more 2D scout images, of Xia. The motivation for this modification could have been to use the benefits of machine learning for both training and generating medical images. When fully trained, machine learning often is able to process and generate image data much faster and possibly more accurately than traditional methods.
16. As per claim 10, Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Xia discloses: The method of claim 2, wherein the generating the 3D predicted image by processing the 3D base image and the one or more 2D scout images using the image generation model comprises: (Adler, col. 4, lines 12-15, “In the test case of spinal examination of patients afflicted by idiopathic scoliosis, we can capture the salient features of the spinal deformation by combining CT scan data and the scout data.” and Adler, col. 4, lines 26-37, “Since scout images are produced by the same device (the CT scanner) that produced the CT slices, both types of data (CT slices and scout images) are automatically registered; i.e., their positions in 3D are described in the same coordinate frame. Since the scout data are a form of X-ray projection data, we can apply standard tomographic reconstruction algorithms to create additional 2D cross-sectional slices of the 3D object that has been scanned. These 2D slices are then combined with the original slices produced directly by the CT scanner to provide a 3D volume of data from which the geometry of 3D object is derived.” and Jun Jin Hwan, Abstract, "PURPOSE: A scout image obtaining method in CT is provided to use a coupling section image, thereby predicting section data after a 3D reconfiguration process." and Jun Jin Hwan, page 3, [0010]-[0011], “The technical problem that the present invention aims to solve is to provide a method for determining a region of interest in three dimensions that is improved by adding a tomosynthesis function to the scout image acquisition process of Citi. Another technical problem that the present invention aims to solve is to provide a method for predicting cross-sectional data after 3D reconstruction processing in the process of acquiring scout images of a city with added tomosynthesis functions.” and Jun Jin Hwan, page 4, [0019]-[0020], “The present invention provides the effect of offering a more improved method for determining a region of interest in three dimensions by adding a tomosynthesis function. In addition, the present invention provides the effect of predicting cross-sectional data after 3D reconstruction processing through combined cross-sectional images.”)
generating a preliminary 3D predicted image by processing the 3D base image and the one or more 2D scout images using the image generation model; and (Adler, col. 4, lines 12-15, “In the test case of spinal examination of patients afflicted by idiopathic scoliosis, we can capture the salient features of the spinal deformation by combining CT scan data and the scout data.” and Adler, col. 4, lines 26-37, “Since scout images are produced by the same device (the CT scanner) that produced the CT slices, both types of data (CT slices and scout images) are automatically registered; i.e., their positions in 3D are described in the same coordinate frame. Since the scout data are a form of X-ray projection data, we can apply standard tomographic reconstruction algorithms to create additional 2D cross-sectional slices of the 3D object that has been scanned. These 2D slices are then combined with the original slices produced directly by the CT scanner to provide a 3D volume of data from which the geometry of 3D object is derived.” and Jun Jin Hwan, Abstract, "PURPOSE: A scout image obtaining method in CT is provided to use a coupling section image, thereby predicting section data after a 3D reconfiguration process." and Jun Jin Hwan, page 3, [0010]-[0011], “The technical problem that the present invention aims to solve is to provide a method for determining a region of interest in three dimensions that is improved by adding a tomosynthesis function to the scout image acquisition process of Citi. Another technical problem that the present invention aims to solve is to provide a method for predicting cross-sectional data after 3D reconstruction processing in the process of acquiring scout images of a city with added tomosynthesis functions.” and Jun Jin Hwan, page 4, [0019]-[0020], “The present invention provides the effect of offering a more improved method for determining a region of interest in three dimensions by adding a tomosynthesis function. In addition, the present invention provides the effect of predicting cross-sectional data after 3D reconstruction processing through combined cross-sectional images.”)
generating the 3D predicted image by performing artifact correction on the preliminary 3D predicted image. (Xia, [0063], “For instance, a decrease in rotation time of the gantry during CT imaging decreases motion artifacts and scan time but increases image noise and, in some cases, leads to streaking artifacts. Therefore, with reference to FIG. 2E, a CT scanner with the ability to modulate rotation time within a single scan may improve image quality by optimizing the speed of the gantry.” and [0076], “The image quality assessment at step 328 of sub process 325 includes the generation of values related to, in an example, ‘resolution’, ‘low contrast detectability’, ‘noise magnitude’, ‘noise texture’, and ‘artifact free’. The generated values may be generated for at least one region of the simulated 2D CT image In an embodiment, each output of the image quality assessment at step 328 of sub process 325 may be a PQR reflecting one or more of the individual image quality assessment values.” and [0149], “(9) The apparatus according to (8), wherein the processing circuitry is further configured to generate, based on the evaluating, a subsequent generated simulated image based on the received scout scan data, subsequent scan acquisition parameters, and subsequent image reconstruction parameters, ... the subsequent imaging protocol parameters increasing image quality while reducing radiation exposure.”)
17. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 2 of Adler in view of Li, and further in view of Jun Jin Hwan to include the disclosure of performing artifact correction on the preliminary 3D predicted image, of Xia. The motivation for this modification could have been to provide an improved image that removes any artifacts that might otherwise make observing a subject’s internals difficult to see. For instance, if there is some interference with the image scans, say from metal instrumentation, performing artifact correction would provide a better view of the subject’s internals.
18. As per claim 11, Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Xia discloses: The method of claim 10, wherein the artifact correction is performed using an artifact correction model, which is jointly trained with the image generation model. (Xia, [0042]-[0043], “In an embodiment, the present disclosure describes an artificial intelligence-based system for automated image quality assessment and protocol optimization. The system may include a computer simulation tool that can simulate CT images, which may be simulated two-dimensional (2D) CT images and/or simulated three-dimensional (3D) CT images, and corresponding dose map(s), from acquired scout scan information and scout scan conditions. The system may include an image quality assessment tool, such as a blind image quality assessment tool, to predict and score medical image quality of each of the simulated CT images, without full reference. The predictions and scores may be made for, in an example, at least one region within a slice of a simulated 2D CT image, a single slice of a simulated 2D CT image or simulated 3D CT image, at least one region within multiple slices of a simulated 3D CT image, and/or multiple slices of a simulated 3D CT image.” And [0168], “(28) An apparatus for training a neural network to generate at least one probabilistic quality representation corresponding to a generated simulated image …” and [0083], “During training, the CNN receives training data, or, for instance, a scout scan, as an input and outputs one or more PQRs that are minimized relative to a reference, or ‘true PQRs’. The generated ‘true PQRs’ may be based on ground-truth data or, for instance, physician input regarding image quality.” And [0054], “In an embodiment, for each region, the one or more score values may be several values indicative of specific image quality attributes (e.g., contrast, artifacts, etc.).”; Examiner’s note: Xia discloses a machine learning model trained for image quality to help reduce artifacts in simulated images.)
19. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 10 of Adler in view of Li, and further in view of Jun Jin Hwan to include the disclosure of jointly training the image generation model and artifact correction, of Xia. The motivation for this modification could have been to use the machine learning model to correlate the generation of medical images with correcting medical images. Training them jointly ensures that the model is not only able to recognize quality medical images but can also generate simulated medical images without the artifacts.
20. Claim 13, which is similar in scope to dependent claim 2 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 2.
21. Claims 3-4 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Adler et al. (US-6023495-A, hereinafter "Adler") in view of Li et al. (US-2014/0187932-A1, hereinafter "Li"), further in view of Jun Jin Hwan (KR-101226479-B1), and further in view of Siewerdsen et al. (US-2021/0174502-A1, hereinafter "Siewerdsen").
22. As per claim 3, Adler in view of Li, and further in view of Jun Jin Hwan discloses: The method of claim 1, [[wherein the internal structure of the target subject changes due to at least one of an intervention of an intervention equipment into the target subject or physiological change of a region of interest of the target subject.]] (See rejection for claim 1.)
23. Adler in view of Li, and further in view of Jun Jin Hwan doesn't explicitly disclose but Siewerdsen discloses: [[The method of claim 1,]] wherein the internal structure of the target subject changes due to at least one of an intervention of an intervention equipment into the target subject or physiological change of a region of interest of the target subject. (Siewerdsen, [0004], “An example scenario in intraoperative imaging is the need to precisely visualize the placement of a metal instrument (e.g., an implanted screw) in relation to surrounding anatomy for guidance, navigation, and validation of surgical device placement. A large number and/or high-density of metal objects in the field-of-view (FOV) can severely degrade image quality and confound visualization of nearby anatomy and confirmation of device placement.” and [0130], “An end-to-end neural network is described to localize metal objects from just two scout views without strong prior information of the patient anatomy or metal instruments. Integration of the end-to-end network with the MAA method for non-circular orbits demonstrated strong reduction in metal artifacts in phantom and cadaver studies. Moreover, the method is compatible with established MAR and polyenergetic reconstruction algorithms to further reduce artifacts.”; Examiner’s note: The example scenario describes an operative procedure regarding the internal change of the placement of an implanted screw.)
24. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Adler in view of Li, and further in view of Jun Jin Hwan to include the disclosure of keeping track of when the internal structure of the target subject changes in either a region of interest or by equipment, of Siewerdsen. The motivation for this modification could have been to provide updated images as a medical procedure is happening. A doctor can then properly make decisions about currently visible changes and information about the subject.
25. As per claim 4, Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Siewerdsen discloses: The method of claim 3, the method further comprising:
obtaining reference information relating to at least one of an intervention point of the intervention equipment, a moving route of the intervention equipment, a shape of the intervention equipment, a material of the intervention equipment, or a type of the intervention equipment, wherein the 3D predicted image is generated further based on the reference information. (Siewerdsen, Abstract, “A system and method for metal artifact avoidance in 3D x-ray imaging is provided. The method includes determining a 3D location of metal in an object or volume of interest to be scanned; estimating a source-detector orbit that will reduce the severity of metal artifacts …” and [0130], “An end-to-end neural network is described to localize metal objects from just two scout views without strong prior information of the patient anatomy or metal instruments. Integration of the end-to-end network with the MAA method for non-circular orbits demonstrated strong reduction in metal artifacts in phantom and cadaver studies. Moreover, the method is compatible with established MAR and polyenergetic reconstruction algorithms to further reduce artifacts.”)
26. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 3 of Adler in view of Li, and further in view of Jun Jin Hwan to include the disclosure of obtaining reference information relating to at least one of an intervention point of the intervention equipment, a moving route of the intervention equipment, a shape of the intervention equipment, a material of the intervention equipment, or a type of the intervention equipment, of Siewerdsen. The motivation for this modification could have been to provide updated images as a medical procedure is happening. A doctor can then properly make decisions about currently visible changes and information about the subject.
27. Claim 14, which is similar in scope to dependent claim 3 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 3.
28. Claim 15, which is similar in scope to dependent claim 4 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 4.
29. Claims 5-8 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Adler et al. (US-6023495-A, hereinafter "Adler") in view of Li et al. (US-2014/0187932-A1, hereinafter "Li"), further in view of Jun Jin Hwan (KR-101226479-B1), and further in view of Siewerdsen et al. (US-2021/0174502-A1, hereinafter "Siewerdsen"), and further in view of Xia et al. (US-2022/0130520-A1, hereinafter "Xia").
30. As per claim 5, Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Siewerdsen discloses: The method of claim 3, [[wherein the image generation model is trained using a plurality of training samples, each of which includes:
a ground truth 3D image and one or more sample 2D scout images indicating the internal structure of a sample subject with a sample intervention equipment inside the sample subject, and
a sample 3D base image indicating the internal structure of the sample subject without the sample intervention equipment inside the sample subject.]] (See rejection for claim 3.)
31. Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Siewerdsen doesn't explicitly disclose but Xia discloses: [[The method of claim 3,]] wherein the image generation model is trained using a plurality of training samples, each of which includes: (Xia, [0041]-[0042], “To this end, and according to an embodiment, the present disclosure describes a method for imaging protocol optimization that is body part-, task-, disease-, and otherwise patient-specific while maximizing image quality for diagnosis and minimizing radiation exposure to a patient. The approach is a machine learning-based approach that utilizes neural networks trained according to clinician evaluations of medical images. Such approach provides consistency within and among different scanner types and improves throughput and technology efficiency. In an embodiment, the present disclosure describes an artificial intelligence-based system for automated image quality assessment and protocol optimization. The system may include a computer simulation tool that can simulate CT images, which may be simulated two-dimensional (2D) CT images and/or simulated three-dimensional (3D) CT images, and corresponding dose map(s), from acquired scout scan information and scout scan conditions.”)
a ground truth 3D image and one or more sample 2D scout images indicating the internal structure of a sample subject with a sample intervention equipment inside the sample subject, and (Xia, [0083], “During training, the CNN receives training data, or, for instance, a scout scan, as an input and outputs one or more PQRs that are minimized relative to a reference, or ‘true PQRs’. The generated ‘true PQRs’ may be based on ground-truth data or, for instance, physician input regarding image quality.” and [0095], “Step 781 to step 785 provides a non-limiting example of an optimization method for training the CNN. In step 781 of process 570, an error is calculated (e.g., using a loss function or a cost function) to represent a measure of the difference (e.g., a distance measure) between a matrix, or map, of the ‘true’ generated data (i.e., physician labeling-based ‘true PQR’, ground truth data) and a matrix, or map, of the output data of the CNN as applied in a current iteration of the CNN.” and Siewerdsen, [0004], “An example scenario in intraoperative imaging is the need to precisely visualize the placement of a metal instrument (e.g., an implanted screw) in relation to surrounding anatomy for guidance, navigation, and validation of surgical device placement. A large number and/or high-density of metal objects in the field-of-view (FOV) can severely degrade image quality and confound visualization of nearby anatomy and confirmation of device placement.” and Siewerdsen, [0130], “An end-to-end neural network is described to localize metal objects from just two scout views without strong prior information of the patient anatomy or metal instruments. Integration of the end-to-end network with the MAA method for non-circular orbits demonstrated strong reduction in metal artifacts in phantom and cadaver studies. Moreover, the method is compatible with established MAR and polyenergetic reconstruction algorithms to further reduce artifacts.”)
a sample 3D base image indicating the internal structure of the sample subject without the sample intervention equipment inside the sample subject. (Xia, [0042], “In an embodiment, the present disclosure describes an artificial intelligence-based system for automated image quality assessment and protocol optimization. The system may include a computer simulation tool that can simulate CT images, which may be simulated two-dimensional (2D) CT images and/or simulated three-dimensional (3D) CT images, and corresponding dose map(s), from acquired scout scan information and scout scan conditions.”)
32. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 3 of Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Siewerdsen to include the disclosure of providing ground truth and sample 3D images for training a machine learning model, of Xia. The motivation for this modification could have been to train the machine learning model to recognize and generate images close to ground truth. By doing so, this improves the machine learning model’s accuracy at recognizing and generating 3d medical images. This is especially useful to train the model to recognize equipment inside the sample subject.
33. As per claim 6, Adler in view of Li, further in view of Jun Jin Hwan, further in view of Siewerdsen, and further in view of Xia discloses: The method of claim 5, wherein a training sample of the plurality of training samples is generated by:
generating the sample 3D base image by performing a first 3D scan on the sample subject before the sample intervention equipment is inserted into the sample subject; (Xia, [0048], “At sub process 110 of method 100, the scout scan data acquired at step 105 of method 100 can be used to generate a simulated image. The simulated image may be a simulated 2D image or a simulated 3D image based on an acquired 2D scout scan or an acquired 3D scout scan. Further, the simulated image can be based on an initial set of scan acquisition parameters and image reconstruction parameters. In an embodiment, the simulated image may be based on previous scan data including previous sets of scan acquisition parameters and image reconstruction parameters. For clarity, the generated simulated image is a simulated 2D CT image based on data from an acquired 3D scout scan and description of method 100 is directed to implementation of method 100 in view of at least one region of the simulated 2D CT image ...” and [0047], “Returning to FIG. 1, at step 105 of method 100, scout scan data of a patient can be acquired from a CT scanner at step 106 of method 100. The scout scan data may be data acquired by a 2D scout scan or a 3D scout scan, as introduced above. In an example, the scout scan data may be used to confirm only a region of interest of the patient. To this end, in one instance, the scout scan may be collected at a level of radiation equivalent to ˜10 chest radiographs. The scout scan data can be used, subsequently, at sub process 110 of method 100 to simulate images.”)
generating the ground truth 3D image by performing a second 3D scan on the sample subject after the sample intervention equipment is inserted into the sample subject; and (Xia, [0048], “At sub process 110 of method 100, the scout scan data acquired at step 105 of method 100 can be used to generate a simulated image. The simulated image may be a simulated 2D image or a simulated 3D image based on an acquired 2D scout scan or an acquired 3D scout scan. Further, the simulated image can be based on an initial set of scan acquisition parameters and image reconstruction parameters. In an embodiment, the simulated image may be based on previous scan data including previous sets of scan acquisition parameters and image reconstruction parameters. For clarity, the generated simulated image is a simulated 2D CT image based on data from an acquired 3D scout scan and description of method 100 is directed to implementation of method 100 in view of at least one region of the simulated 2D CT image ...” and Xia, [0047], “Returning to FIG. 1, at step 105 of method 100, scout scan data of a patient can be acquired from a CT scanner at step 106 of method 100. The scout scan data may be data acquired by a 2D scout scan or a 3D scout scan, as introduced above. In an example, the scout scan data may be used to confirm only a region of interest of the patient. To this end, in one instance, the scout scan may be collected at a level of radiation equivalent to ˜10 chest radiographs. The scout scan data can be used, subsequently, at sub process 110 of method 100 to simulate images.” and Xia, [0083], “During training, the CNN receives training data, or, for instance, a scout scan, as an input and outputs one or more PQRs that are minimized relative to a reference, or ‘true PQRs’. The generated ‘true PQRs’ may be based on ground-truth data or, for instance, physician input regarding image quality.” and Siewerdsen, [0004], “An example scenario in intraoperative imaging is the need to precisely visualize the placement of a metal instrument (e.g., an implanted screw) in relation to surrounding anatomy for guidance, navigation, and validation of surgical device placement. A large number and/or high-density of metal objects in the field-of-view (FOV) can severely degrade image quality and confound visualization of nearby anatomy and confirmation of device placement.” and Siewerdsen, [0130], “An end-to-end neural network is described to localize metal objects from just two scout views without strong prior information of the patient anatomy or metal instruments. Integration of the end-to-end network with the MAA method for non-circular orbits demonstrated strong reduction in metal artifacts in phantom and cadaver studies. Moreover, the method is compatible with established MAR and polyenergetic reconstruction algorithms to further reduce artifacts.”; Examiner’s note: Xia discloses that a simulated image may be based on previous scan data, accounting for two or more scans.)
generating the one or more sample 2D scout images based on the ground truth 3D image. (Xia, [0048], “At sub process 110 of method 100, the scout scan data acquired at step 105 of method 100 can be used to generate a simulated image. The simulated image may be a simulated 2D image or a simulated 3D image based on an acquired 2D scout scan or an acquired 3D scout scan. Further, the simulated image can be based on an initial set of scan acquisition parameters and image reconstruction parameters. In an embodiment, the simulated image may be based on previous scan data including previous sets of scan acquisition parameters and image reconstruction parameters. For clarity, the generated simulated image is a simulated 2D CT image based on data from an acquired 3D scout scan and description of method 100 is directed to implementation of method 100 in view of at least one region of the simulated 2D CT image ...” and [0083], “During training, the CNN receives training data, or, for instance, a scout scan, as an input and outputs one or more PQRs that are minimized relative to a reference, or ‘true PQRs’. The generated ‘true PQRs’ may be based on ground-truth data or, for instance, physician input regarding image quality.”)
34. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 5 of Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Siewerdsen to include the disclosure of generating 3D base and ground truth images based on 3D scans and generating a sample 2D scout image based on the ground truth 3D image, of Xia. The motivation for this modification could have been to train the machine learning model to generate images close to ground truth based on multiple scans. Multiple scans can also be correlated with each other to provide more accurate ground truth results for the machine learning model. The multiplate scans will also be useful to train the model to recognize equipment inside the sample subject.
35. As per claim 7, Adler in view of Li, further in view of Jun Jin Hwan, further in view of Siewerdsen, and further in view of Xia discloses: The method of claim 5, wherein a training sample of the plurality of training samples is generated by:
generating the ground truth 3D image by performing a third 3D scan on the sample subject after the sample intervention equipment is inserted into the sample subject; and (Xia, [0048], “At sub process 110 of method 100, the scout scan data acquired at step 105 of method 100 can be used to generate a simulated image. The simulated image may be a simulated 2D image or a simulated 3D image based on an acquired 2D scout scan or an acquired 3D scout scan. Further, the simulated image can be based on an initial set of scan acquisition parameters and image reconstruction parameters. In an embodiment, the simulated image may be based on previous scan data including previous sets of scan acquisition parameters and image reconstruction parameters. For clarity, the generated simulated image is a simulated 2D CT image based on data from an acquired 3D scout scan and description of method 100 is directed to implementation of method 100 in view of at least one region of the simulated 2D CT image ...” and Xia, [0047], “Returning to FIG. 1, at step 105 of method 100, scout scan data of a patient can be acquired from a CT scanner at step 106 of method 100. The scout scan data may be data acquired by a 2D scout scan or a 3D scout scan, as introduced above. In an example, the scout scan data may be used to confirm only a region of interest of the patient. To this end, in one instance, the scout scan may be collected at a level of radiation equivalent to ˜10 chest radiographs. The scout scan data can be used, subsequently, at sub process 110 of method 100 to simulate images.” and Xia, [0083], “During training, the CNN receives training data, or, for instance, a scout scan, as an input and outputs one or more PQRs that are minimized relative to a reference, or ‘true PQRs’. The generated ‘true PQRs’ may be based on ground-truth data or, for instance, physician input regarding image quality.” and Siewerdsen, [0004], “An example scenario in intraoperative imaging is the need to precisely visualize the placement of a metal instrument (e.g., an implanted screw) in relation to surrounding anatomy for guidance, navigation, and validation of surgical device placement. A large number and/or high-density of metal objects in the field-of-view (FOV) can severely degrade image quality and confound visualization of nearby anatomy and confirmation of device placement.” and Siewerdsen, [0130], “An end-to-end neural network is described to localize metal objects from just two scout views without strong prior information of the patient anatomy or metal instruments. Integration of the end-to-end network with the MAA method for non-circular orbits demonstrated strong reduction in metal artifacts in phantom and cadaver studies. Moreover, the method is compatible with established MAR and polyenergetic reconstruction algorithms to further reduce artifacts.”; Examiner’s note: Xia discloses that a simulated image may be based on previous scan data, accounting for two or more scans.)
generating the sample 3D base image and the one or more sample 2D scout images based on the ground truth 3D image. (Xia, [0048], “At sub process 110 of method 100, the scout scan data acquired at step 105 of method 100 can be used to generate a simulated image. The simulated image may be a simulated 2D image or a simulated 3D image based on an acquired 2D scout scan or an acquired 3D scout scan. Further, the simulated image can be based on an initial set of scan acquisition parameters and image reconstruction parameters. In an embodiment, the simulated image may be based on previous scan data including previous sets of scan acquisition parameters and image reconstruction parameters. For clarity, the generated simulated image is a simulated 2D CT image based on data from an acquired 3D scout scan and description of method 100 is directed to implementation of method 100 in view of at least one region of the simulated 2D CT image ...” and [0083], “During training, the CNN receives training data, or, for instance, a scout scan, as an input and outputs one or more PQRs that are minimized relative to a reference, or ‘true PQRs’. The generated ‘true PQRs’ may be based on ground-truth data or, for instance, physician input regarding image quality.”)
36. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 5 of Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Siewerdsen to include the disclosure of generating 3D base and ground truth images based on 3D scans and generating a sample 2D scout image based on the ground truth 3D image, of Xia. The motivation for this modification could have been to train the machine learning model to generate images close to ground truth based on multiple scans. Multiple scans can also be correlated with each other to provide more accurate ground truth results for the machine learning model. The multiplate scans will also be useful to train the model to recognize equipment inside the sample subject.
37. As per claim 8, Adler in view of Li, further in view of Jun Jin Hwan, further in view of Siewerdsen, and further in view of Xia discloses: The method of claim 5, wherein a training sample of the plurality of training samples is generated by:
obtaining a digital model representing the sample subject; (Xia, [0065], “In an embodiment, a bowtie filter may be dynamically-modified in accordance with a 2D field of view map generated from acquired scout scan data. In this way, the patient model can easily be derived therefrom, with beam hardening and extra scatter effects eliminated and patient radiation exposure limited. Additionally, a field of view map may be used to perform optimal reconstructions at the smallest possible field of view that incorporates a complete patient anatomy, or of a complete patient anatomy of a region of interest.” And Adler, col. 9, lines 50-53, “In function block 970, a surface-construction algorithm is applied to the segmented 3D volume 960. This algorithm will produce the required geometric model of the patient's spine in the form of a polyhedral mesh.”)
generating the sample 3D base image by simulating a 3D scan on the digital model; and (Xia, [0043], “The system may include an image quality assessment tool, such as a blind image quality assessment tool, to predict and score medical image quality of each of the simulated CT images, without full reference. The predictions and scores may be made for, in an example, at least one region within a slice of a simulated 2D CT image, a single slice of a simulated 2D CT image or simulated 3D CT image, at least one region within multiple slices of a simulated 3D CT image, and/or multiple slices of a simulated 3D CT image.” and [0048], “At sub process 110 of method 100, the scout scan data acquired at step 105 of method 100 can be used to generate a simulated image. The simulated image may be a simulated 2D image or a simulated 3D image based on an acquired 2D scout scan or an acquired 3D scout scan. Further, the simulated image can be based on an initial set of scan acquisition parameters and image reconstruction parameters. In an embodiment, the simulated image may be based on previous scan data including previous sets of scan acquisition parameters and image reconstruction parameters.”)
generating the ground truth 3D image and the one or more sample 2D scout images based on the sample 3D base image. (Xia, [0048], “At sub process 110 of method 100, the scout scan data acquired at step 105 of method 100 can be used to generate a simulated image. The simulated image may be a simulated 2D image or a simulated 3D image based on an acquired 2D scout scan or an acquired 3D scout scan. Further, the simulated image can be based on an initial set of scan acquisition parameters and image reconstruction parameters. In an embodiment, the simulated image may be based on previous scan data including previous sets of scan acquisition parameters and image reconstruction parameters. For clarity, the generated simulated image is a simulated 2D CT image based on data from an acquired 3D scout scan and description of method 100 is directed to implementation of method 100 in view of at least one region of the simulated 2D CT image ...” and [0083], “During training, the CNN receives training data, or, for instance, a scout scan, as an input and outputs one or more PQRs that are minimized relative to a reference, or ‘true PQRs’. The generated ‘true PQRs’ may be based on ground-truth data or, for instance, physician input regarding image quality.”)
38. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 5 of Adler in view of Li, further in view of Jun Jin Hwan, and further in view of Siewerdsen to include the disclosure of simulating a scan process on a digital model of a sample subject and generating a 3D ground truth image and 2D scout images, of Xia. The motivation for this modification could have been to simulate a scanning process without the need of a subject. This would help to continue to train the machine learning model without needing to perform multiple scans on the subject. The machine learning model could produce its own results without a subject.
39. Claim 16, which is similar in scope to dependent claim 5 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 5.
40. Claim 17, which is similar in scope to dependent claim 6 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 6.
41. Claim 18, which is similar in scope to dependent claim 7 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 7.
42. Claim 19, which is similar in scope to dependent claim 8 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 8.
43. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Adler et al. (US-6023495-A, hereinafter "Adler") in view of Li et al. (US-2014/0187932-A1, hereinafter "Li"), further in view of Jun Jin Hwan (KR-101226479-B1), further in view of Xia et al. (US-2022/0130520-A1, hereinafter "Xia"), and further in view of Buelow et al. (US-2023/0298745-A1, hereinafter “Buelow”).
44. As per claim 9, Adler in view of Li, and further in view of Jun Jin Hwan, and further in view of Xia discloses: The method of claim 2, [[wherein the 3D base image includes a plurality of frames corresponding to a plurality of physiological phases of the target subject, and the generating a 3D predicted image of the subject by processing the 3D base image and the one or more 2D scout images using the image generation model comprises:
determining a target physiological phase of the target subject corresponding to the one or more 2D scout images;
selecting, from the plurality of frames, a target frame corresponding to the target physiological phase; and
generating the 3D predicted image by processing the target frame and the one or more 2D scout images using the image generation model.]] (See rejection for claim 2.)
45. Adler in view of Li, and further in view of Jun Jin Hwan, and further in view of Xia doesn't explicitly disclose but Buelow discloses: The method of claim 2, wherein the 3D base image includes a plurality of frames corresponding to a plurality of physiological phases of the target subject, and the generating a 3D predicted image of the subject by processing the 3D base image and the one or more 2D scout images using the image generation model comprises: (Buelow, [0019]-[0020], “The following discloses systems and methods for extracting the scout scan acquired by a CT scanner in order to check for issues before the clinical images are acquired. A scout scan is an example of a preview image, where a preview image is a “quick” scan that is usually a two-dimensional (2D) scan. Alternatively, the scout scan may be a low dose three-dimensional (3D) scan. ... To provide vendor-agnostic operation, the illustrative system uses a DVI splitter, video camera, or the like to acquire a video feed of the CT scanner controller display, and to detect the scout scan (or other preview image) as a rectangular region with dark boundaries, e.g. using a machine learning (ML) tool or video frame image segmentation. For analyses requiring knowledge of the anatomy being imaged, the anatomy is suitably determined by analysis of the scout image, for example, automated image segmentation and comparison of the segmented regions with a reference image or atlas image.” and Adler, col. 4, lines 12-15, “In the test case of spinal examination of patients afflicted by idiopathic scoliosis, we can capture the salient features of the spinal deformation by combining CT scan data and the scout data.” and Adler, col. 4, lines 26-37, “Since scout images are produced by the same device (the CT scanner) that produced the CT slices, both types of data (CT slices and scout images) are automatically registered; i.e., their positions in 3D are described in the same coordinate frame. Since the scout data are a form of X-ray projection data, we can apply standard tomographic reconstruction algorithms to create additional 2D cross-sectional slices of the 3D object that has been scanned. These 2D slices are then combined with the original slices produced directly by the CT scanner to provide a 3D volume of data from which the geometry of 3D object is derived.” and Jun Jin Hwan, Abstract, "PURPOSE: A scout image obtaining method in CT is provided to use a coupling section image, thereby predicting section data after a 3D reconfiguration process." and Jun Jin Hwan, page 3, [0010]-[0011], “The technical problem that the present invention aims to solve is to provide a method for determining a region of interest in three dimensions that is improved by adding a tomosynthesis function to the scout image acquisition process of Citi. Another technical problem that the present invention aims to solve is to provide a method for predicting cross-sectional data after 3D reconstruction processing in the process of acquiring scout images of a city with added tomosynthesis functions.” and Jun Jin Hwan, page 4, [0019]-[0020], “The present invention provides the effect of offering a more improved method for determining a region of interest in three dimensions by adding a tomosynthesis function. In addition, the present invention provides the effect of predicting cross-sectional data after 3D reconstruction processing through combined cross-sectional images.”)
determining a target physiological phase of the target subject corresponding to the one or more 2D scout images; (Buelow, [0020], “To provide vendor-agnostic operation, the illustrative system uses a DVI splitter, video camera, or the like to acquire a video feed of the CT scanner controller display, and to detect the scout scan (or other preview image) as a rectangular region with dark boundaries, e.g. using a machine learning (ML) tool or video frame image segmentation.” and [0011], “Another advantage resides in providing an automated approach for warning of a disease condition such as a tumor or bony growth automatically detected in a preview image acquired prior to clinical images being acquired in an imaging examination.” and [0038], “The operation 106 can be performed in a variety of manners. In one example, the electronic processing device 20 is programmed to perform the image analysis 38 on the extracted features 44 to identify misplacement of a body part imaged in a FOV by the extracted preview image 12. The misplacement can be due to an improper positioning or orientation of the patient based on how much of the patient is visible in the preview image 12.”)
selecting, from the plurality of frames, a target frame corresponding to the target physiological phase; and (Buelow, [0039], “Given an alert indicating such an inconsistency, the imaging technician can then review the situation to determine whether it is the preview image 12 that is incorrect, or whether one or more potential problems associated with the medical imaging examination is detected.” and [0011], “Another advantage resides in providing an automated approach for warning of a disease condition such as a tumor or bony growth automatically detected in a preview image acquired prior to clinical images being acquired in an imaging examination.” and [0006], “In another aspect, a method for providing real-time checking of one or more potential problems associated with a medical imaging examination includes: receiving a video feed of a GUI displayed on an imaging device controller; extracting a preview image displayed in a preview image viewport of the GUI from the live video feed; performing an image analysis on the extracted preview image to detect one or more image features indicative of one or more potential problems associated with a medical imaging examination performed with the medical imaging device; and output an alert when one or more potential problems associated with a medical imaging examination is detected from the one or more image features.”; Examiner’s note: The alert process is selecting from the plurality of frames associated with a target physiological phase, whether it is a “disease condition” of the patient or an issue with the imaging during an operative procedure.)
generating the 3D predicted image by processing the target frame and the one or more 2D scout images using the image generation model. (Adler, col. 4, lines 12-15, “In the test case of spinal examination of patients afflicted by idiopathic scoliosis, we can capture the salient features of the spinal deformation by combining CT scan data and the scout data.” and Adler, col. 4, lines 26-37, “Since scout images are produced by the same device (the CT scanner) that produced the CT slices, both types of data (CT slices and scout images) are automatically registered; i.e., their positions in 3D are described in the same coordinate frame. Since the scout data are a form of X-ray projection data, we can apply standard tomographic reconstruction algorithms to create additional 2D cross-sectional slices of the 3D object that has been scanned. These 2D slices are then combined with the original slices produced directly by the CT scanner to provide a 3D volume of data from which the geometry of 3D object is derived.” and Jun Jin Hwan, Abstract, "PURPOSE: A scout image obtaining method in CT is provided to use a coupling section image, thereby predicting section data after a 3D reconfiguration process." and Jun Jin Hwan, page 3, [0010]-[0011], “The technical problem that the present invention aims to solve is to provide a method for determining a region of interest in three dimensions that is improved by adding a tomosynthesis function to the scout image acquisition process of Citi. Another technical problem that the present invention aims to solve is to provide a method for predicting cross-sectional data after 3D reconstruction processing in the process of acquiring scout images of a city with added tomosynthesis functions.” and Jun Jin Hwan, page 4, [0019]-[0020], “The present invention provides the effect of offering a more improved method for determining a region of interest in three dimensions by adding a tomosynthesis function. In addition, the present invention provides the effect of predicting cross-sectional data after 3D reconstruction processing through combined cross-sectional images.”)
46. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 2 of Adler in view of Li, and further in view of Jun Jin Hwan, and further in view of Xia to include the disclosure of keeping track of a plurality of frames related to the 3D base image in order to determine and select a physiological phase for generating a 3D predicted image, of Buelow. The motivation for this modification could have been to target specific moments of time during a medical procedure in order to generate the 3D predicted image. This could help a doctor visualize a particular physiological phase of the subject to assist with medical procedure decisions.
Conclusion
47. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW CLOTHIER/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights