DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 4-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a system and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “apply a machine learning model to the plurality of CTA images, wherein the machine learning model is a modality-specific deep learning model trained using machine learning to diagnose plaque erosion using coronary CTA, determine a plaque erosion prediction based on application of the machine learning model to the plurality of CTA images”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim further recites “A system for implementing deep learning to noninvasively diagnose plaque erosion using coronary computed tomography angiography (CTA), the system comprising: one or more electronic processors configured to: receive a plurality of CTA images for a patient having acute coronary syndromes (ACS)”, and “transmit the plaque erosion prediction for display via a display device (2217).”
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. Therefore, the claim is directed to an abstract idea
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a system and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “wherein the plaque erosion prediction includes at least one of a probability of plaque erosion or a probability of plaque rupture”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim does not recite additional elements.
Step 2B, Inventive Concept: Similar to Step 2A Prong 2, the claim does not recite further additional elements.
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a system and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “wherein the machine learning model includes a composite transformer attention component and a momentum distillation component”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim does not recite additional elements.
Step 2B, Inventive Concept: Similar to Step 2A Prong 2, the claim does not recite further additional elements.
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a system and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “wherein the machine learning model includes a spatial transformer and a sequential transformer”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim does not recite additional elements.
Step 2B, Inventive Concept: Similar to Step 2A Prong 2, the claim does not recite further additional elements.
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a system and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “applying a spatial transformer of the machine learning model to each of a plurality of CTA slices included in each of the plurality of CTA images (2250) to generate a corresponding feature vector for each of the plurality of CTA slices; and applying a sequential transformer of the machine learning model to each corresponding feature vector of each of the plurality of CTA slices to output a transformed feature, wherein the machine learning model is configured to determine the plaque erosion prediction based on the transformed feature”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim does not recite additional elements.
Step 2B, Inventive Concept: Similar to Step 2A Prong 2, the claim does not recite further additional elements.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a system and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “determine, with the machine learning model, a slice-level prediction based on the plurality of CTA images; and determine, with the machine learning model, a patient-level prediction based on the plurality of CTA images, wherein the plaque erosion prediction is based on the slice-level prediction and the patient-level prediction.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim does not recite additional elements.
Step 2B, Inventive Concept: Similar to Step 2A Prong 2, the claim does not recite further additional elements.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a system and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “wherein the one or more electronic processors are configured to: access training data including a plurality of training CTA images, wherein at least a portion of the plurality of training CTA images are annotated”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment.
The claim further teaches “train, with the training data, the machine learning model using momentum distillation- enhanced self-supervised pre-training”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim further recites “wherein at least a portion of the plurality of training CTA images are annotated”.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a method and is therefore a process.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “applying, with the one or more electronic processors, a machine learning model to the plurality of medical images, determining, with the one or more electronic processors, a plaque erosion prediction based on application of the machine learning model to the plurality of medical images”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
The claim further teaches “determining, with the one or more electronic processors, a plaque erosion prediction based on application of the machine learning model to the plurality of medical images”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim further recites “A method for implementing deep learning to noninvasively diagnose plaque erosion using coronary computed tomography angiography (CTA), the method comprising: receiving, with one or more electronic processors, a plurality of medical images; transmitting, with the one or more electronic processors, the plaque erosion prediction for display via a display device.”
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. Therefore, the claim is directed to an abstract idea
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a and is therefore a process.
Step 2A, Prong 1, Judicial Exception: the claim does not further recite any judicial exceptions
Step 2A, Prong 2, Integrated into Practical Application: The claim recites “performing, with the one or more electronic processors (2200), momentum distillation- enhanced self-supervised pre-training of the machine learning model prior to receiving the plurality of medical images”
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a method and is therefore a process.
Step 2A, Prong 1, Judicial Exception: the claim does not further recite any judicial exceptions
Step 2A, Prong 2, Integrated into Practical Application: The claim recites “receiving the plurality of medical images includes receiving a plurality of coronary CTA images for a patient that has acute coronary syndromes (ACS).”
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a method and is therefore a process.
Step 2A, Prong 1, Judicial Exception: the claim recites “applying, with the one or more electronic processors, a spatial transformer of the machine learning model to each of a plurality of slices included in each of the plurality of medical images to generate a corresponding feature vector for each of the plurality of slices; and applying, with the one or more electronic processors, a sequential transformer of the machine learning model to each corresponding feature vector of each of the plurality of slices to output a transformed feature”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
The claim further recites “transmitting, with the one or more electronic processors, the plaque erosion prediction for display via a display device.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim recites “wherein determining the plaque erosion prediction includes determining the plaque erosion prediction based on the transformed feature.”
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1: Statutory Category: Yes, the invention recites a medium and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes, the claim recites “applying, to the plurality of CTA images (2250), a modality-specific deep learning model including a spatial transformer and a sequential transformer, wherein applying the modality- specific deep learning model includes: generating, via the spatial transformer, a corresponding feature vector for each of a plurality of CTA slices included in each of the plurality of CTA images (2250)”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
The claim further teaches “determining a plaque erosion prediction based on application of the modality-specific deep learning model to the plurality of CTA images”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: The claim further recites “A non-transitory, computer-readable medium storing instructions that, when executed by one or more electronic processors, perform a set of functions, the set of functions comprising: receiving a plurality of CTA images; transmitting the plaque erosion prediction for display via a display device”
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. Therefore, the claim is directed to an abstract idea
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. Therefore, the claim is directed to an abstract idea
Step 2B, Inventive Concept: The additional elements merely recite insignificant extra solution activities, which do not amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 2 are rejected under 35 U.S.C. 103 as being unpatentable over Buckler et al., (US20220012865A1) in view of Nakajima et al., “Plaque Rupture, Compared With Plaque Erosion, Is Associated With a Higher Level of Pancoronary Inflammation” JACC Cardiovasc Imaging. 2021 Nov 8;15(5):828–839. doi:
Regarding claim 1, Buckler teaches a system for implementing deep learning to noninvasively diagnose plaque erosion using coronary computed tomography angiography (CTA), the system comprising ([0046] coronary computed tomography angiography (CCTA) used in tandem with quantitative analysis software; [0098] using deep learning algorithms for the analysis of Atherosclerosis):
one or more electronic processors configured to:
receive a plurality of CTA images ([0085]the system may provide processing for received CTA images);
apply a machine learning model to the plurality of CTA images ([0135] the images are collected and then used to feed the CNNs; [0085] the deep learning algorithms use the CTA images), wherein the machine learning model is a modality-specific deep learning model trained using machine learning to diagnose plaque erosion using coronary CTA([0085] the deep learning algorithms use the CTA images; claims 17/19);
determine a plaque erosion prediction based on application of the machine learning model to the plurality of CTA images (claim 23 a machine learned classification approach based on known ground truths to process the enriched dataset and determine a predictive outcome related to the pathology; Claim 16 the pathology is related to the vasculature, wherein the machine learned classification approach provides for atherosclerotic plaque erosion) and
transmit the plaque erosion prediction for display via a display device ([0410] the techniques can be displayed on a computer with a display device; [0164] Next pathologies may be determined, based on the biological quantified properties/analytes, and characterized, e.g., by determining phenotype and/or predictive outcomes for the pathologies, and a patient report is generated).
However, Buckler is silent regarding receiving a plurality of CTA images for a patient having acute coronary syndromes.
In the same plaque field of endeavor, Nakajima teaches receiving a plurality of CTA images for a patient having acute coronary syndromes (pg. 3 patients presented with ACS underwent CTA imaging).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to substitute the CTA images of Buckler with the CTA images having ACS as taught by Nakajima, as both inventions relate to imaging plaque erosion, and would yield the predictable result of a deep learning system that uses CTA images of patients having ACS to one of ordinary skill in the art. One of ordinary skill in the art would be able to make such a substitution, and the results of Buckler using CTA images for patients having ACS are reasonably predictable.
Regarding claim 2, modified Buckler teaches the system of claim 1, wherein Buckler further teaches determine, based on the plaque erosion prediction, a recommended course of treatment for the patient ([0059] presence and degree of HRP, and/or erosion better informs treatment for borderline subjects [0082] classifications of the plaque lesion can be used for directing patient treatment); and transmit the recommended course of treatment for display via the display device ([0410] all the techniques described in the spec can be implemented onto a computer with a display, therefore the recommended patient treatment can be displayed).
Claims 3 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of Nakajima as applied to claim 2 above, and further in view of Yamamoto et al., “Clinical and Laboratory Predictors for Plaque Erosion in Patients With Acute Coronary Syndrome” J Am Heart Assoc. 2019 Oct 23;8(21):e012322. doi: 10.1161/JAHA.119.012322
Regarding claim 3, modified Buckler teaches the system of claim 2, but fails to explicitly teaches wherein the recommended course of treatment includes an antithrombotic therapy without stenting when the plaque erosion prediction indicates a first probability of plaque erosion; and wherein the recommended course of treatment includes a percutaneous coronary intervention when the plaque erosion prediction indicates a second probability of plaque erosion, wherein the second probability of plaque erosion is lower than the first probability of plaque erosion.
In the same cardiovascular field of endeavor, Yamamoto teaches wherein the recommended course of treatment includes an antithrombotic therapy without stenting when the plaque erosion prediction indicates a first probability of plaque erosion (pg. 1-2 anti-thrombotic therapy without stenting is an option for patients with ACS, as ACS patients with a high probability of plaque erosion may be managed conservatively with antithrombotic therapy without invasive procedures; pg. 6 probability of NSTE ACS is up to 73% depending on the predictors present); and wherein the recommended course of treatment includes a percutaneous coronary intervention when the plaque erosion prediction indicates a second probability of plaque erosion (pg. 8 percutaneous coronary intervention is used with patients with ST-segment–elevation myocardial infarction (STEMI), and those with STEMI have a plaque erosion rate of 29.8%), wherein the second probability of plaque erosion is lower than the first probability of plaque erosion (page 6 the probability of patients with NSTE ACS with the 5 conditions present is 73% at a rate of 47.9%, and STEMI has a rate of 29.8%, therefore the probability of plaque erosion would be lower than NSTE ACS).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the treatment procedures of Yamamoto, as this would facilitate an early triage of the patients and provide an opportunity for tailored therapy (see Yamamoto pg. 8)
Regarding claim 4, modified Buckler teaches the system of claim 1, but fails to explicitly disclose wherein the plaque erosion prediction includes at least one of a probability of plaque erosion or a probability of plaque rupture.
However in the same cardiovascular field of endeavor, Yamamoto teaches wherein the plaque erosion prediction includes at least one of a probability of plaque erosion or a probability of plaque rupture (pg. 4-6 the predictors of Plaque erosion determines the probability of plaque erosion)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the treatment procedures of Yamamoto, as this would facilitate an early triage of the patients and provide an opportunity for tailored therapy (see Yamamoto pg. 8).
Claims 5 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of Nakajima as applied to claim 1 above, and further in view of Gani et al. (US20240212330A1).
Regarding claim 5, modified Buckler teaches the system of claim 1, but fails to explicitly disclose wherein the machine learning model includes a composite transformer attention component and a momentum distillation component.
In the same medical imaging field of endeavor, Gani teaches wherein the machine learning model (Abstract deep learning system for capturing medical images, a machine learning engine, and display) includes a composite transformer attention component ([0056]-[0059]several transformer blocks with modified self-attention computation are arranged in a first state transformer blocks are arranged in stages 2, 3, and 4 and jointly produce a hierarchical representation, and that the transformer blocks include window based multi head self-attention using regular window and shifted window partitioning configurations) and a momentum distillation component ([0066] the student/teacher view prediction frame work follows the learning paradigm of knowledge self-distillation; [0068]the teacher parameters are updated via exponential moving average of the student weights).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the deep learning system of Gani, as this would encourage better intermediate feature representations which further aids during the fine-tuning stage on the same dataset (see Gani [0069]).
Regarding claim 9, modified Buckler teaches the system of claim 1, wherein Buckler further teaches wherein the one or more electronic processors are configured to: access training data including a plurality of training CTA images wherein at least a portion of the plurality of training CTA images are annotated ([0333] annotations are generated from CTA data and teaches that the annotated CTA contours create input data for a classifier trained on known ground truth [0356] images and raw slice data are collected, providing ground truth erosion labels at respective cross-sections and using the data for network training).
However, the combination of references fails to explicitly disclose train, with the training data, the machine learning model using momentum distillation-enhanced self-supervised pre-training.
In the same medical image analysis field of endeavor, Gani teaches train, with the training data, the machine learning model using momentum distillation-enhanced self-supervised pre-training ([0066] the self supervised training state expressly identifies as “view prediction pre-training” that uses a student/teacher framework; [0068] the teacher parameters are updated through an exponential moving average the student weights).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the deep learning system of Gani, as this would encourage better intermediate feature representations which further aids during the fine-tuning stage on the same dataset (see Gani [0069]).
Claims 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of Nakajima as applied to claim 1 above, and further in view of He et al. (US20230080098A1).
Regarding claim 6, modified Buckler teaches the system of claim 1, but fails to explicitly disclose wherein the machine learning model includes a spatial transformer and a sequential transformer.
In the same medical imaging field of endeavor, He teaches wherein the machine learning model includes a spatial transformer ([0045]-[0046] spatial features are extracted by a first transformer network for each of the images, wherein each feature characterizes spatial information of the corresponding medical image) and a sequential transformer ([0051]-[0052 there’s a second transformer network that receives the fused spatial features and extracts a spatial temporal feature representing changes in the spatial features of the plurality of medical images at different times, including the chronological order and temporal relationship among the medical images; [0092]-[0093] the first and second extraction network are connected within the same image recognition model, and the image recognition model comprises cascaded transformers, and both the first and second networks are each a transformer).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the machine learning procedures of He, as this would allow for more accurate representation of the spatial and timing information of the medical images (see He [0088]).
Regarding claim 7, modified Buckler teaches the system of claim 1, wherein Buckler further teaches wherein the one or more electronic processors are configure dot determine the plaque erosion prediction (claim 23 a machine learned classification approach based on known ground truths to process the enriched dataset and determine a predictive outcome related to the pathology; Claim 16 the pathology is related to the vasculature, wherein the machine learned classification approach provides for atherosclerotic plaque erosion) but fails to explicitly teach a plurality of CTA slices included in each of the plurality of CTA images.
In the same medical image field of endeavor, Nakajima teaches a plurality of CTA slices included in each of the plurality of CTA images (pg. 3 CT acquisition was performed using a 320 Slice CT scanner. Coronary CT angiographic images where acquired with a slice thickness of .5mm).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to substitute the CTA images of Buckler with the CTA images having ACS as taught by Nakajima, as both inventions relate to imaging plaque erosion, and would yield the predictable result of a deep learning system that uses CTA images of patients having ACS to one of ordinary skill in the art. One of ordinary skill in the art would be able to make such a substitution, and the results of Buckler using CTA images for patients having ACS are reasonably predictable.
However the combination of references still fails to explicitly disclose applying a spatial transformer of the machine learning model to each of a plurality of images included in each of the plurality of images to generate a corresponding feature vector for each of the plurality of images; and applying a sequential transformer of the machine learning model to each corresponding feature vector of each of the plurality of images to output a transformed feature, wherein the machine learning model is configured to determine the prediction based on the transformed feature.
In the same medical field of endeavor, He teaches applying a spatial transformer of the machine learning model to each of a plurality of images included in each of the plurality of images to generate a corresponding feature vector for each of the plurality of images ([0093]-[0094] the image recognition model comprises a cascaded first and second extraction networks, each of which is a transformer, and that each first extraction network extracts a spatial feature from one corresponding medical image); and applying a sequential transformer of the machine learning model to each corresponding feature vector of each of the plurality of images to output a transformed feature ([0075] the extracted plurality of spatial features can be stitched so the resulting fusion spatial feature includes spatial features of the medical images); [0051]the second transformer extracts a spatial temporal feature based on the fusion spatial feature; [0052] that spatial temporal features includes the spatial information of each medical image and timing information corresponding to the chronological order of the plurality of medical images) wherein the machine learning model is configured to determine the prediction based on the transformed feature ([0054] a state of the object is recognized based on the spatial-temporal feature, to obtain a recognition result of the object. recognition result of the object is obtained from a state of the object).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the CTA slices and images of modified Buckler with the machine learning procedures of He, as this would allow for more accurate representation of the spatial and timing information of the medical images (see He [0088]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of Nakajima as applied to claim 1 above, and further in view of Kiraly et al., (US20210225511A1).
Regarding claim 8, modified Buckler teaches the method of claim, wherein Buckler further teaches wherein the one or more electronic processors are configured to: determine, with the machine learning model, a slice-level prediction based on the plurality of CTA images ([0356] patient images and raw slice data to form enriched datasets that are provided to trained CNNs for identifying erosion; [0074] individual cross section level analysis have output labels representing phenotype or risk stratification); and determine, with the machine learning model, a patient-level prediction based on the plurality of CTA images ([0074] volume level analysis has output labels representing phenotype or risk stratification; [0067] outcome prediction/risk satisfaction generally occurs at the patient level using the same input data).
However, the combination of references is silent regarding wherein the prediction is based on the slice-level prediction and the patient-level prediction.
In the same medical image analysis field of endeavor, Kiraly teaches a prediction based on the slice level prediction and the patient level prediction ([0017] generating a global probability of a patient having cancer from a CT dataset; [0021] ultimate probability of the patient having cancer using both global patient probabilities and local candidate probabilities).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the prediction combination of Kiraly, as this approach can help improve diagnosis and detection in diagnostic images (see Kiraly [0095]). One of ordinary skill in the art would understand that the combination of modified Buckler’s plaque erosion prediction using Kiraly’s local and global prediction combination arrangement would result in the teaching of the limitation “wherein the plaque erosion prediction is based on the slice-level prediction and the patient-level prediction”.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of He.
Regarding claim 10, Buckler a method for implementing deep learning to noninvasively diagnose plaque erosion using coronary computed tomography angiography (CTA), the method comprising ([0046] coronary computed tomography angiography (CCTA) used in tandem with quantitative analysis software; [0098] using deep learning algorithms for the analysis of Atherosclerosis):
receiving, with one or more electronic processors ([0405]), a plurality of CTA images ([0085] the system may provide processing for received CTA images);
applying, with one or more electronic processors ([0405]), a machine learning model to the plurality of CTA images ([0135] the images are collected and then used to feed the CNNs; [0085] the deep learning algorithms use the CTA images), wherein the machine learning model is a modality-specific deep learning model trained using machine learning to diagnose plaque erosion using coronary CTA([0085] the deep learning algorithms use the CTA images; claims 17/19);
determining, with one or more electronic processors ([0405]), a plaque erosion prediction based on application of the machine learning model to the plurality of CTA images (claim 23 a machine learned classification approach based on known ground truths to process the enriched dataset and determine a predictive outcome related to the pathology; Claim 16 the pathology is related to the vasculature, wherein the machine learned classification approach provides for atherosclerotic plaque erosion) and
transmitting the plaque erosion prediction for display via a display device ([0410] the techniques can be displayed on a computer with a display device; [0164] Next pathologies may be determined, based on the biological quantified properties/analytes, and characterized, e.g., by determining phenotype and/or predictive outcomes for the pathologies, and a patient report is generated).
However, Buckler fails to explicitly teach applying, to the plurality of CTA images, a modality-specific deep learning model including a spatial transformer and a sequential transformer
In the same medical imaging field of endeavor, He teaches applying, to the plurality of CTA images, a modality-specific deep learning model including a spatial transformer ([0045]-[0046] spatial features are extracted by a first transformer network for each of the images, wherein each feature characterizes spatial information of the corresponding medical image) and a sequential transformer ([0051]-[0052 there’s a second transformer network that receives the fused spatial features and extracts a spatial temporal feature representing changes in the spatial features of the plurality of medical images at different times, including the chronological order and temporal relationship among the medical images; [0092]-[0093] the first and second extraction network are connected within the same image recognition model, and the image recognition model comprises cascaded transformers, and both the first and second networks are each a transformer).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the CTA slices and images of Buckler with the machine learning procedures of He, as this would allow for more accurate representation of the spatial and timing information of the medical images (see He [0088]).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of He as applied to claim 10 above, and further in view of Gani and Zhou et al., (US20170200067A1).
Regarding claim 11, modified Buckler teaches the method of claim 10, but fails to explicitly disclose performing, with the one or more electronic processors, momentum distillation- enhanced self-supervised pre-training of the machine learning model.
In the same medical image field of endeavor, Gani teaches performing, with the one or more electronic processors, momentum distillation- enhanced self-supervised pre-training of the machine learning model ([0014] the processing circuitry configured to train a student/teach network through self-supervised view prediction and update the teacher parameters through an exponential moving average of the network parameters; [0066] Additionally, the pre-training follows the learning paradigm of knowledge self-distillation, the exponential moving average teacher update supplies the momentum component of the disclosed self distillation pre training).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the deep learning system of Gani, as this would encourage better intermediate feature representations which further aids during the fine-tuning stage on the same dataset (see Gani [0069]).
However, the combination of references fails to explicitly disclose pre-training prior the receiving the plurality of medical images.
In the same medical image field of endeavor, Zhou teaches pre-training prior the receiving the plurality of medical images (fig. 1 training stage 100 is performed offline to train a deep learning network, and then are used for a medical image analysis task on newly input medical image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Buckler with the pretraining of Zhou, as this would improve image reading outcomes in terms of accuracy, reproducibility, and efficiency (see Zhou [0003]).
Claims 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of He as applied to claim 10 above, and further in view of Nakajima.
Regarding claim 12, modified Buckler teaches the method of claim 10, but is silent regarding receiving a plurality of CTA images for a patient having acute coronary syndromes
In the same plaque field of endeavor, Nakajima teaches receiving a plurality of CTA images for a patient having acute coronary syndromes (pg. 3 patients presented with ACS underwent CTA imaging).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to substitute the CTA images of Buckler with the CTA images having ACS as taught by Nakajima, as both inventions relate to imaging plaque erosion, and would yield the predictable result of a deep learning system that uses CTA images of patients having ACS to one of ordinary skill in the art. One of ordinary skill in the art would be able to make such a substitution, and the results of Buckler using CTA images for patients having ACS are reasonably predictable.
Regarding claim 13, modified Buckler teaches the method of claim 10, wherein Buckler further teaches wherein the one or more electronic processors are configure dot determine the plaque erosion prediction (claim 23 a machine learned classification approach based on known ground truths to process the enriched dataset and determine a predictive outcome related to the pathology; Claim 16 the pathology is related to the vasculature, wherein the machine learned classification approach provides for atherosclerotic plaque erosion) but fails to explicitly teach a plurality of CTA slices included in each of the plurality of CTA images.
In the same medical image field of endeavor, Nakajima teaches a plurality of CTA slices included in each of the plurality of CTA images (pg. 3 CT acquisition was performed using a 320 Slice CT scanner. Coronary CT angiographic images where acquired with a slice thickness of .5mm).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to substitute the CTA images of Buckler with the CTA images having ACS as taught by Nakajima, as both inventions relate to imaging plaque erosion, and would yield the predictable result of a deep learning system that uses CTA images of patients having ACS to one of ordinary skill in the art. One of ordinary skill in the art would be able to make such a substitution, and the results of Buckler using CTA images for patients having ACS are reasonably predictable.
However the combination of references still fails to explicitly disclose applying a spatial transformer of the machine learning model to each of a plurality of images included in each of the plurality of images to generate a corresponding feature vector for each of the plurality of images; and applying a sequential transformer of the machine learning model to each corresponding feature vector of each of the plurality of images to output a transformed feature, wherein the machine learning model is configured to determine the prediction based on the transformed feature.
In the same medical field of endeavor, He teaches applying a spatial transformer of the machine learning model to each of a plurality of images included in each of the plurality of images to generate a corresponding feature vector for each of the plurality of images ([0093]-[0094] the image recognition model comprises a cascaded first and second extraction networks, each of which is a transformer, and that each first extraction network extracts a spatial feature from one corresponding medical image); and applying a sequential transformer of the machine learning model to each corresponding feature vector of each of the plurality of images to output a transformed feature ([0075] the extracted plurality of spatial features can be stitched so the resulting fusion spatial feature includes spatial features of the medical images); [0051]the second transformer extracts a spatial temporal feature based on the fusion spatial feature; [0052] that spatial temporal features includes the spatial information of each medical image and timing information corresponding to the chronological order of the plurality of medical images) wherein the machine learning model is configured to determine the prediction based on the transformed feature ([0054] a state of the object is recognized based on the spatial-temporal feature, to obtain a recognition result of the object. recognition result of the object is obtained from a state of the object).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the CTA slices and images of modified Buckler with the machine learning procedures of He, as this would allow for more accurate representation of the spatial and timing information of the medical images (see He [0088]).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of He.
Regarding claim 14, Buckler teaches a non-transitory, computer-readable medium storing instructions that, when executed by one or more electronic processors, perform a set of functions, the set of functions comprising ([0405]):
receiving a plurality of CTA images ([0085] the system may provide processing for received CTA images);
applying, to the plurality of CTA images, a modality-specific deep learning model ([0135] the images are collected and then used to feed the CNNs; [0085] the deep learning algorithms use the CTA images),
determining a plaque erosion prediction based on application of the modality-specific deep learning model to the plurality of CTA images (claim 23 a machine learned classification approach based on known ground truths to process the enriched dataset and determine a predictive outcome related to the pathology; Claim 16 the pathology is related to the vasculature, wherein the machine learned classification approach provides for atherosclerotic plaque erosion); and
transmitting the plaque erosion prediction for display via a display device ([0410] the techniques can be displayed on a computer with a display device; [0164] Next pathologies may be determined, based on the biological quantified properties/analytes, and characterized, e.g., by determining phenotype and/or predictive outcomes for the pathologies, and a patient report is generated).
However, Buckler fails to explicitly teach applying, to the plurality of CTA images, a modality-specific deep learning model including a spatial transformer and a sequential transformer, wherein applying the modality- specific deep learning model includes: generating, via the spatial transformer, a corresponding feature vector for each of a plurality of CTA slices included in each of the plurality of CTA images; and outputting, via the sequential transformer, a transformed feature based on each corresponding feature vector of each of the plurality of CTA slices.
In the same medical imaging field of endeavor, He teaches applying, to the plurality of CTA images, a modality-specific deep learning model including a spatial transformer ([0045]-[0046] spatial features are extracted by a first transformer network for each of the images, wherein each feature characterizes spatial information of the corresponding medical image) and a sequential transformer ([0051]-[0052 there’s a second transformer network that receives the fused spatial features and extracts a spatial temporal feature representing changes in the spatial features of the plurality of medical images at different times, including the chronological order and temporal relationship among the medical images; [0092]-[0093] the first and second extraction network are connected within the same image recognition model, and the image recognition model comprises cascaded transformers, and both the first and second networks are each a transformer), wherein applying the modality-specific deep learning model includes: generating, via the spatial transformer, a corresponding feature vector for each of a plurality of CTA slices included in each of the plurality of CTA images ([0093]-[0094] the image recognition model comprises a cascaded first and second extraction networks, each of which is a transformer, and that each first extraction network extracts a spatial feature from one corresponding medical image); and outputting, via the sequential transformer, a transformed feature based on each corresponding feature vector of each of the plurality of CTA slices ([0075] the extracted plurality of spatial features can be stitched so the resulting fusion spatial feature includes spatial features of the medical images); [0051]the second transformer extracts a spatial temporal feature based on the fusion spatial feature; [0052] that spatial temporal features includes the spatial information of each medical image and timing information corresponding to the chronological order of the plurality of medical images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the CTA slices and images of Buckler with the machine learning procedures of He, as this would allow for more accurate representation of the spatial and timing information of the medical images (see He [0088]).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Buckler in view of He as applied to claim 14 above, and further in view of Yamamoto
Regarding claim 15, modified Buckler teaches the system of claim 14, wherein Buckler further teaches determining a recommended treatment based on the plaque erosion prediction; and ([0059] presence and degree of HRP, and/or erosion better informs treatment for borderline subjects [0082] classifications of the plaque lesion can be used for directing patient treatment); and outputting the recommended treatment ([0410] all the techniques described in the spec can be implemented onto a computer with a display, therefore the recommended patient treatment can be displayed).
However, the combination of references fails to explicitly disclose wherein, when the plaque erosion prediction indicates a first probability of plaque erosion, the recommended course of treatment includes an antithrombotic therapy without stenting; and wherein the recommended course of treatment includes a percutaneous coronary intervention when the plaque erosion prediction indicates a second probability of plaque erosion, wherein the second probability of plaque erosion is lower than the first probability of plaque erosion.
In the same cardiovascular field of endeavor, Yamamoto teaches wherein, when the plaque erosion prediction indicates a first probability of plaque erosion, the recommended course of treatment includes an antithrombotic therapy without stenting; (pg. 1-2 anti-thrombotic therapy without stenting is an option for patients with ACS, as ACS patients with a high probability of plaque erosion may be managed conservatively with antithrombotic therapy without invasive procedures; pg. 6 probability of NSTE ACS is up to 73% depending on the predictors present); wherein, when the plaque erosion prediction indicates a second probability of plaque erosion, the recommended treatment includes a percutaneous coronary intervention, (pg. 8 percutaneous coronary intervention is used with patients with ST-segment–elevation myocardial infarction (STEMI), and those with STEMI have a plaque erosion rate of 29.8%), wherein the second probability of plaque erosion is lower than the first probability of plaque erosion(page 6 the probability of patients with NSTE ACS with the 5 conditions present is 73% at a rate of 47.9%, and STEMI has a rate of 29.8%, therefore the probability of plaque erosion would be lower than NSTE ACS).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Buckler with the treatment procedures of Yamamoto, as this would facilitate an early triage of the patients and provide an opportunity for tailored therapy (see Yamamoto pg. 8)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL Y FANG whose telephone number is (571)272-0952. The examiner can normally be reached Mon - Friday 9:30 am - 6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at 5712722714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL YIMING FANG/Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights