DETAILED CORRESPONDENCE
This is a non-final office action on merits in response to the arguments and/or amendments filed on 10/16/2025 and the request for continued examination filed on 10/16/2025.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims
Claims 2 and 7 are cancelled. Amendments to claims 1 and 6 are acknowledged and have been carefully considered. Claims 1, 3-6, and 8-10 are pending and considered below.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/14/2025 has been entered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-6, and 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over by Zimmerman et al. (U.S. Patent Publication 2023/0245782 A1), referred to hereinafter as Zimmerman, in view of Shen et al. (TW Shen, TF Laio, Image Processing on ECG Chart for ECG Signal Recovery, 2009, Computers in Cardiology, 36, pages 725−728. (Year: 2009)), referred to hereinafter as Shen, and Yoon et al. (Taeyoung Yoon & Daesung Kang, Bimodal CNN for cardiovascular disease classification by co-training ECG grayscale images and scalograms., 2023, Scientific Reports, 13:2937, pages 1-9 (Year: 2023)), referred to hereinafter as Yoon.
Regarding claim 1, Zimmerman teaches an artificial intelligence-enabled (AI-enabled) electrocardiogram (ECG) algorithm method (Zimmerman, [0333] “Artificial intelligence engines may be implemented, including, by example, a deep neural network using ECG voltage data and a gradient-boosted tree using the baseline QTc with age and sex as additional inputs to both models.”), applicable to identifying patients with ventricular premature contraction (VPC) under an environment during sinus rhythm (Zimmerman [0333] “Many ECG machines create a “portable document format” (PDF) from the voltage-time traces which may then be stored in the medical record. The underlying voltage data may be extracted from these PDFs by first converting the PDF to XML and then parsing the XML file for the underlying data points which make up each of the voltage-time traces. The XML may also be parsed to determine the patient's age, sex, nine continuous numerical measurements output by the ECG machine (QRS duration, QT, QTC, PR interval, ventricular rate, average RR interval and P, Q and T-wave axes) and thirty categorical ECG patterns, including: a normal, left bundle branch block, incomplete left bundle branch block, right bundle branch block, incomplete right bundle branch block, atrial fibrillation, atrial flutter, acute myocardial infarction, left ventricular hypertrophy, premature ventricular contractions”), comprising the following steps:
using an ECG machine having 12 leads to collect a plurality of ECG datasets from the patients, each of the ECG datasets including one-dimensional ECG raw data and an ECG image (Zimmerman [0008] “ECGs can be acquired using a minimum of 2 body surface potential recordings (such that a voltage difference can be calculated from the subtraction of the two electrical potentials). When only one voltage difference is acquired typically for a duration of at least 10 seconds, this is known as a “rhythm strip”. One common ECG is the 12-lead ECG where voltage differences are acquired in 12 different directions (or “leads”) across the surface of the body.”, Zimmerman [0156] “Using all ECGs from a 15-year period, patients were randomly split into a training set (DO dataset: 80% of qualifying studies) and a holdout test set (20%) without overlap of patients between sets. Two versions of the model architecture were compared (as described above): one with ECG voltage versus time traces alone as inputs”, and [0123] “In some embodiments, the trained models database 128 can include a number of trained models that can receive raw ECGs and output AF risk scores. In other embodiments, a digital image of a lead for an ECG may be used.”);
classifying and dividing the ECG datasets into a training set, a validation set, and a test set (Zimmerman, [0153], “For all experiments, data were divided into training, internal validation, and test sets.”);
performing image processing to remove and process background of the ECG image (Zimmerman, [0375] “Before training, a time-series signal processing of ECG data including artifact detection and exclusion may be performed. This includes preprocessing steps such as sampling normalization, voltage trace structure changes, and possible inclusions of noisy data to regularize deep learning models. For example, dead leads and/or spikes in millivolts may be identified (such as over 12 mv).”, and
converting the ECG image into one- dimensional time-series data (Zimmerman [0135] “In some embodiments, a digital image of a raw ECG voltage input data may be used and each lead identified from the digital image and a corresponding voltage (e.g., digital voltage data) may be estimated from analysis of the digital image.”); and
performing at least one of Al and convolutional neural network (CNN) processing based on the one-dimensional time-series data by using an AI-enabled ECG algorithm to establish an evaluation model for identifying VPC patients during normal sinus rhythm (NSR) (Zimmerman [0036] “The trained model further comprises: training a convolutional neural network on a plurality of patients, wherein the plurality of patients include at least patients having a recorded ECG within a diagnosis threshold and patients having a recorded ECG outside a diagnosis threshold; wherein the diagnosis threshold is compared against the time between the date of diagnosis of aortic stenosis and the date of the recorded ECG; and providing the trained convolutional neural network as the trained model. The trained model further comprises: refining the trained neural network using only the plurality of patients having the recorded ECG outside of the diagnosis threshold, wherein the diagnosis threshold is selected from a number of days.”, Zimmerman [0137] “FIG. 4A is an exemplary embodiment of a model 400. Specifically, an architecture of the model 400 is shown. Artificial intelligence models referenced herein, including model 700 and model 724 discussed further below, may be gradient boosting models, random forest models, neural networks (NN)… NNs include conditional random fields, convolutional neural networks,” and Zimmerman, [0202] “Many ECG machines create a “portable document format” (PDF) from the voltage-time traces which may then be stored in the medical record. The underlying voltage data may be extracted from these PDFs by first converting the PDF to XML and then parsing the XML file for the underlying data points which make up each of the voltage-time traces. The XML may also be parsed to determine the patient's age, sex, nine continuous numerical measurements output by the ECG machine (QRS duration, QT, QTC, PR interval, ventricular rate, average RR interval and P, Q and T-wave axes) and thirty categorical ECG patterns, including: a normal, left bundle branch block, incomplete left bundle branch block, right bundle branch block, incomplete right bundle branch block, atrial fibrillation, atrial flutter, acute myocardial infarction, left ventricular hypertrophy, premature ventricular contractions.”, Zimmerman, [0149], “Each ECG was defined as normal or abnormal as follows: 1) normal ECGs were defined as those with pattern labels of “normal ECG” or “within normal limits” and no other abnormalities identified; 2) all other ECGs were considered abnormal. Note that a normal ECG does not imply that the patient was free of heart disease or other medical diagnoses. All the ECG voltage-time traces were preprocessed to ensure that waveforms were centered around the zero baseline, while preserving variance and magnitude features.” and Zimmerman [0214] “Generating predictions from these models may include satisfying an objective to determine the future risk of an adverse clinical outcome, in order to ultimately assist clinicians and patients with earlier treatment and potentially even prevention as a result of the earlier intervention.”);
wherein the ECG image is a 12-lead ECG image and the step of performing image processing and converting the ECG image into the one-dimensional time- series data further comprises the following processing steps (Zimmerman [0111] “Atrial fibrillation (AF) is associated with substantial morbidity, especially when it goes undetected. If new onset AF can be predicted with high accuracy, screening methods could be used to find it early. The present disclosure provides a deep neural network that can predict new onset AF from a resting 12-lead electrocardiogram (ECG). The predicted new onset AF may assist medical practitioners (e.g., a cardiologist) in preventing AF-related adverse outcomes, such as stroke.”, Zimmerman [0348] “Referring to FIG. 29, the method 600 may include one or more steps of data ingestion, QA, or preprocessing 602. In particular, the method may include time-series signal processing of ECG data and artifact detection and exclusion. Ingestion may include, e.g., a plurality of voltage-time traces where a first subset are stored at a first frequency, e.g., 500 Hz, and a second subset are stored at a second, different frequency, e.g., 250 Hz. Such data may be batch loaded due to the exceedingly large volume of clinical data being ingested, and similar batch techniques may be applied to one or both of the training or prediction steps disclosed herein.”, Zimmerman [0135] “In some embodiments, a digital image of a raw ECG voltage input data may be used and each lead identified from the digital image and a corresponding voltage (e.g., digital voltage data) may be estimated from analysis of the digital image.”):
dividing the intensity inversed image into four sub-images according to start and end positions of each of the 12 leads, the four sub-images containing 4 ECG tracings (Zimmerman [0133] “In other embodiments there may be differing time periods for each branch (e.g., the first branch may include 0-2.5 seconds, the second branch may include 2.5-6 seconds, and the third branch may include 6-10 seconds).” and Zimmerman [0138] “In some embodiments, the model 400 can be a deep neural network. In some embodiments, the model 400 can receive the input data shown in FIG. 3. The input data structure to the model 400 can include a first branch 404 including leads I, II, V1, and V5, acquired from time (t)=0 (start of data acquisition) to t=5 seconds (e.g., the first voltage data, the sixth voltage data, the ninth voltage data, and the twelfth voltage data); a second branch 408 including leads V1, V2, V3, II, and V5 from t=5 to t=7.5 seconds (e.g., the second voltage data, the fourth voltage data, the seventh voltage data, the tenth voltage data, and the thirteenth voltage data); and a third branch 412 including leads V4, V5, V6, II, and V1 from t=7.5 to t=10 seconds (e.g., the third voltage data, the fifth voltage data, the eighth voltage data, the eleventh voltage data, and the fourteenth voltage data) as shown in FIG. 3. The arrangement of the branches can be designed to account for concurrent morphology changes throughout the standard clinical acquisition due to arrhythmias and/or premature beats. For example, the model 400 may need to synchronize which voltage information or data is acquired at the same point in time in order to understand the data. Because the ECG leads are not all acquired at the same time, the leads may be aligned to demonstrate to the neural network model which data was collected at the same time. It is noted that not every lead needs to have voltage data spanning the entire time interval. This is an advantage of the model 400, as some ECGs do not include data for all leads over the entire time interval. For example, the model 400 can include ten branches, and can be trained to generate a risk score based in response to receiving voltage data spanning subsequent one second periods from ten different leads. As another example, the model 400 can include four branches, and can be trained to generate a risk score based in response to receiving voltage data spanning subsequent 2.5 second periods from four different leads.”, Zimmerman, [0117] “FIG. 1 is an example 100 of a system 100 for automatically predicting an AF, AS, CA, and/or SP risk score based on ECG data (e.g., data from a resting 12-lead ECG).”), when applying CNN analysis to the 12-lead ECG data)”);
a standard 12-lead ECG sampling rate of 500 Hz (Zimmerman [0122] “The ECG database 120 can include a number of ECGs. In some embodiments, the ECGs can include 12-lead ECGs. Each ECG can include a number of voltage measurements taken at regular intervals (e.g., at a rate of 250 HZ, 500 Hz, 1000 Hz, etc.) over a predetermined time period (e.g., 5 seconds, 10 seconds, 15 seconds, 30 seconds, 60 seconds, etc.) for each lead.”); and
g.normalizing the magnitude of the filtered one-dimensional time-series data of each ECG tracing to a uniform scale (Zimmerman [0375] “Before training, a time-series signal processing of ECG data including artifact detection and exclusion may be performed. This includes preprocessing steps such as sampling normalization, voltage trace structure changes, and possible inclusions of noisy data to regularize deep learning models. For example, dead leads and/or spikes in millivolts may be identified (such as over 12 mv).”); and
wherein the evaluation model is further used by the at least one AI and CNN processing based on the one-dimensional time-series data during NSR to detect a change in the ECG dataset of sinus rhythm when a patient is without VPC episodes so that early treatment is provided to the patient to reduce a risk of heart failure or sudden death of the patient (Zimmerman [0036] “The trained model further comprises: training a convolutional neural network on a plurality of patients, wherein the plurality of patients include at least patients having a recorded ECG within a diagnosis threshold and patients having a recorded ECG outside a diagnosis threshold; wherein the diagnosis threshold is compared against the time between the date of diagnosis of aortic stenosis and the date of the recorded ECG; and providing the trained convolutional neural network as the trained model. The trained model further comprises: refining the trained neural network using only the plurality of patients having the recorded ECG outside of the diagnosis threshold, wherein the diagnosis threshold is selected from a number of days.”, Zimmerman [0137] “FIG. 4A is an exemplary embodiment of a model 400. Specifically, an architecture of the model 400 is shown. Artificial intelligence models referenced herein, including model 700 and model 724 discussed further below, may be gradient boosting models, random forest models, neural networks (NN)… NNs include conditional random fields, convolutional neural networks,” and Zimmerman, [0202] “Many ECG machines create a “portable document format” (PDF) from the voltage-time traces which may then be stored in the medical record. The underlying voltage data may be extracted from these PDFs by first converting the PDF to XML and then parsing the XML file for the underlying data points which make up each of the voltage-time traces. The XML may also be parsed to determine the patient's age, sex, nine continuous numerical measurements output by the ECG machine (QRS duration, QT, QTC, PR interval, ventricular rate, average RR interval and P, Q and T-wave axes) and thirty categorical ECG patterns, including: a normal, left bundle branch block, incomplete left bundle branch block, right bundle branch block, incomplete right bundle branch block, atrial fibrillation, atrial flutter, acute myocardial infarction, left ventricular hypertrophy, premature ventricular contractions.”, Zimmerman, [0149], “Each ECG was defined as normal or abnormal as follows: 1) normal ECGs were defined as those with pattern labels of “normal ECG” or “within normal limits” and no other abnormalities identified; 2) all other ECGs were considered abnormal. Note that a normal ECG does not imply that the patient was free of heart disease or other medical diagnoses. All the ECG voltage-time traces were preprocessed to ensure that waveforms were centered around the zero baseline, while preserving variance and magnitude features.”, Zimmerman [0214] “Generating predictions from these models may include satisfying an objective to determine the future risk of an adverse clinical outcome, in order to ultimately assist clinicians and patients with earlier treatment and potentially even prevention as a result of the earlier intervention.” and Zimmerman [0113] Atrial Fibrillation (AF) is a cardiac rhythm disorder associated with several important adverse health outcomes including stroke and heart failure.”).
Zimmerman fails to explicitly teach
a. removing a red grid background of the leads in the ECG image and converting the ECG image to a gray-scale image;
b. inversing intensities of pixels of the gray-scale image and scaling the intensities of the intensity inversed pixels to form an intensity inversed image with a gray scale of 0 to 255 gray levels;
c. scanning the four sub-images pixel by pixel and recording positions of the pixels where the pixels have a brightest intensity equal to 255 gray levels, the brightest intensity pixels representing the 4 ECG tracings; an X- coordinate and a Y-coordinate of each brightest intensity pixel being respectively assigned as a time stamp and a magnitude of an ECG data point;
d. sequentially grouping ECG data points formed by the brightest intensity pixels to construct a continuous ECG signal for each ECG tracing having 250 ECG data points
e. using an interpolation operation to perform up-sampling and convert the 250 ECG data points into 1250 ECG data points, corresponding to 2.5 seconds of the one-dimensional time-series data
f. filtering the one-dimensional time-series data using a low-pass filter, the low-pass filter having an order of 3 and a cutoff frequency at 15 Hz.
Shen teaches a. removing a red grid background of the leads in the ECG image and converting the ECG image to a gray-scale image (Shen, page 726, “In spatial-oriented process (figure 1), a color image may directly eliminate the red component to remove gridlines or convert to a gray or binary tone image after selecting a suitable threshold with the help of histogram analysis. A suitable threshold selection after histogram analysis can mostly remove the background and gridlines ECG charts.”);
c. scanning the four sub-images pixel by pixel and an X- coordinate and a Y-coordinate of each brightest intensity pixel being respectively assigned as a time stamp and a magnitude of an ECG data point (Shen, page 726, “After filtering, the inverse Fourier transform (IFT) transfers frequency domain to time domain and the result shows in figure 6a.”);
e. using an interpolation operation to perform up-sampling and convert the 250 ECG data points into 1250 ECG data points, corresponding to 2.5 seconds of the one-dimensional time-series data (Shen, page 727-728, “Then, the 2D image was converted to 1D signal by using five interpolation methods (cubic, v5cubic, pchip, spline, linear, and nearest) that rebuild sampling frequency to 500 sps. The example results are plotted in figure 7 & 8.”);
f. filtering the one-dimensional time-series data using a low-pass filter, the low-pass filter having an order of 3 and a cutoff frequency at 15 Hz (Shen, page 727, “In frequency-oriented process (figure 3), 2D FT projects the image from spatial to frequency domain. In the frequency domain, an elliptic-shaped low-pass (LP) filter was applied on the frequency response map to omit the high frequency component as shown in figure 4. The center of frequency response map presents the low frequency component and the outside elliptic-cycle describes the high-frequency component. Figure 5 shows the three dimension plot on the LP filter with crisp threshold on the filter edge. The formula of the LP filter is written as equation (3).”).
Yoon teaches b. inversing intensities of pixels of the gray-scale image and scaling the intensities of the intensity inversed pixels to form an intensity inversed image with a gray scale of 0 to 255 gray levels (Yoon, page 7, “In this study, we converted one-dimensional ECG recordings into two-dimensional grayscale images and scalograms since the proposed bimodal CNN model requires images as inputs. For grayscale images, one-dimensional ECG recordings were plotted as grayscale images with a black background and a white ECG signal.”);
recording positions of the pixels where the pixels have a brightest intensity equal to 255 gray levels, the brightest intensity pixels representing the ECG tracings (Yoon, page 7, “In this study, we converted one-dimensional ECG recordings into two-dimensional grayscale images and scalograms since the proposed bimodal CNN model requires images as inputs. For grayscale images, one-dimensional ECG recordings were plotted as grayscale images with a black background and a white ECG signal.”);
d. sequentially grouping ECG data points formed by the brightest intensity pixels to construct a continuous ECG signal for each ECG tracing having 250 ECG data points (Yoon, page 7, “In this study, we converted one-dimensional ECG recordings into two-dimensional grayscale images and scalograms since the proposed bimodal CNN model requires images as inputs. For grayscale images, one-dimensional ECG recordings were plotted as grayscale images with a black background and a white ECG signal. Then the grayscale images were saved as 300 × 300 pixels.”).
Therefore, it would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the invention to modify the AI-enabled ECG method of Zimmerman by incorporating the ECG image preprocessing and signal recovery techniques taught by Shen, and further incorporating the intensity based ECG image representation techniques taught by Yoon. Zimmerman teaches an AI based framework, including convolutional neural networks, for analyzing ECG data to establish predictive evaluation models from resting 12 lead ECGs, including ECGs recorded during normal rhythm. Shen teaches known and established techniques for recovering accurate ECG waveforms from ECG chart images, including background and gridline removal, grayscale conversion, interpolation to a standard sampling frequency, and filtering to reduce noise. Yoon teaches that representing ECG tracings as high contrast grayscale images, including intensity inversion and normalization, improves extraction and machine learning analysis of ECG signals.
A PHOSITA would have been motivated to apply Shen’s ECG image-to-signal recovery techniques and Yoon’s intensity-based ECG image preprocessing to Zimmerman’s AI-ECG workflow in order to obtain standardized ECG signal inputs for CNN training and inference, thereby improving model performance. The combination applies known preprocessing and signal representation techniques to improve ECG data quality in a closely related AI based ECG analysis system, and would have yielded predictable results. This modification represents the routine use of known methods to improve similar data processing in the same field.
Regarding claim 3, Zimmerman, Shen, and Yoon teach the invention in claim 1, as discussed above, and further teach wherein the ECG datasets comprise 12-lead ECG data (Zimmerman, [0117] “FIG. 1 is an example 100 of a system 100 for automatically predicting an AF, AS, CA, and/or SP risk score based on ECG data (e.g., data from a resting 12-lead ECG).”), when applying CNN analysis to the 12-lead ECG data)”, Zimmerman [0144] “In some embodiments, the model 424 can be a deep neural network. In some embodiments, such as is shown in FIG. 4B, the model 424 can include a single branch 432 that can receive ECG voltage input data 428 generated over a single time interval (e.g., ten seconds). As shown, the model 424 can receive ECG voltage input data 428 generated over a time interval of ten seconds using eight leads. In some embodiments, the ECG voltage input data 428 can include five thousand data points collected over a period of 10 seconds and 8 leads including leads I, II, V1, V2, V3, V4, V5, and V6. The number of data points can vary based on the sampling rate used to sample the leads (e.g., a sampling rate of five hundred Hz will result in five thousand data points over a time period of ten seconds). The ECG voltage input data 428 can be transformed into ECG waveforms.” and Zimmerman [0137] Artificial intelligence models referenced herein, including model 700 and model 724 discussed further below, may be gradient boosting models, random forest models, neural networks (NN), regression models, Naive Bayes models, or machine learning algorithms (MLA). …. NNs include conditional random fields, convolutional neural networks,”), CNN uses CNN kernels to extract all features of the 12-lead ECG data in a two-dimensional (2D) data processing method, and the CNN kernels are activated by a specific function and then identified by neural network analysis (Zimmerman [0264] “Each CNN layer consisted of 16 kernels of size 5. The same network configuration was used to train one model per clinical outcome, resulting in 7 independently trained CNN models (FIG. 25B). Specifically, FIG. 25B displays a block diagram for a composite model that shows the classification pipeline for ECG trace and other EHR data. The output of each neural network (the triangles in FIG. 25B) applied to ECG trace data is concatenated to labs, vitals, and demographics to form a feature vector. The vector is the input to a classification pipeline (min-max scaling, mean imputation, and XGBoost classifier), which outputs a recommendation score for the patient.”).
Therefore, it would be obvious to a PHOSITA before the effective filing date of the invention to implement Zimmerman’s 12 lead ECG AI method together with Shen’s signal recovery steps and Yoon’s grayscale inversion and pixel mapping, because all references address extraction and processing of 12-lead ECG data for CNN analysis. Zimmerman teaches CNN modeling with 12-lead ECG voltage-time traces, Shen teaches interpolation to standardized sampling frequencies and filtering for signal accuracy, and Yoon demonstrates converting ECG waveforms into grayscale images for improved CNN input. A PHOSITA would have been motivated to combine these teachings in order to ensure that 12 lead ECGs are uniformly preprocessed and digitized for accurate AI training, which represents the predictable use of known preprocessing and feature extraction techniques in the same field.
Regarding claim 4, Zimmerman, Shen, and Yoon teach the invention in claim 1, as discussed above, and further teach wherein in performing at least one of AI and CNN processing (Zimmerman [0036] “The trained model further comprises: training a convolutional neural network on a plurality of patients, wherein the plurality of patients include at least patients having a recorded ECG within a diagnosis threshold and patients having a recorded ECG outside a diagnosis threshold; wherein the diagnosis threshold is compared against the time between the date of diagnosis of aortic stenosis and the date of the recorded ECG; and providing the trained convolutional neural network as the trained model. The trained model further comprises: refining the trained neural network using only the plurality of patients having the recorded ECG outside of the diagnosis threshold, wherein the diagnosis threshold is selected from a number of days.”, Zimmerman [0137] “FIG. 4A is an exemplary embodiment of a model 400. Specifically, an architecture of the model 400 is shown. Artificial intelligence models referenced herein, including model 700 and model 724 discussed further below, may be gradient boosting models, random forest models, neural networks (NN)… NNs include conditional random fields, convolutional neural networks,”), the evaluation model to identify VPC patients during NSR for a CNN model is established according to preprocessed ECG two-dimensional data and dimensional features of the ECG datasets (Zimmerman [0149] “FIG. 5A is an exemplary flow 500 of training and testing the model 400 in FIG. 4A, although it will be appreciated that other training and/or testing procedures may be implemented. 2.8 million standard 12-lead ECG traces were extracted from a medical database. All ECGs with known time-to-event or minimum 1-year follow-up were used during model training and a single random ECG was selected for each patient in the holdout set for model evaluation, with results denoted as ‘M0’ in FIG. 5B. FIG. 5B shows a timeline for ECG selection in accordance with FIG. 5A. The traces were acquired between 1984 and June 2019. Additional retraining was performed only the resting 12-lead ECGs: 1) acquired in patients ≥18 years of age, 2) with complete voltage-time traces of 2.5 seconds for 12 leads and 10 seconds for 3 leads (V1, II, V5), and 3) with no significant artifacts. This amounted to 1.6 million ECGs from 431 k patients. The median (interquartile range) follow-up available after each ECG was 4.1 (1.5-8.5) years. Each ECG was defined as normal or abnormal as follows: 1) normal ECGs were defined as those with pattern labels of “normal ECG” or “within normal limits” and no other abnormalities identified; 2) all other ECGs were considered abnormal. Note that a normal ECG does not imply that the patient was free of heart disease or other medical diagnoses. All the ECG voltage-time traces were preprocessed to ensure that waveforms were centered around the zero baseline, while preserving variance and magnitude features.” And Zimmerman [0202] “Many ECG machines create a “portable document format” (PDF) from the voltage-time traces which may then be stored in the medical record. The underlying voltage data may be extracted from these PDFs by first converting the PDF to XML and then parsing the XML file for the underlying data points which make up each of the voltage-time traces. The XML may also be parsed to determine the patient's age, sex, nine continuous numerical measurements output by the ECG machine (QRS duration, QT, QTC, PR interval, ventricular rate, average RR interval and P, Q and T-wave axes) and thirty categorical ECG patterns, including: a normal, left bundle branch block, incomplete left bundle branch block, right bundle branch block, incomplete right bundle branch block, atrial fibrillation, atrial flutter, acute myocardial infarction, left ventricular hypertrophy, premature ventricular contractions, premature atrial contractions.” and Zimmerman [0375] “Before training, a time-series signal processing of ECG data including artifact detection and exclusion may be performed. This includes preprocessing steps such as sampling normalization, voltage trace structure changes, and possible inclusions of noisy data to regularize deep learning models. For example, dead leads and/or spikes in millivolts may be identified (such as over 12 mv).”).
Therefore, it would be obvious to a PHOSITA before the effective filing date of the invention to apply Zimmerman’s convolutional neural network analysis of ECG data in combination with Shen’s preprocessing pipeline and Yoon’s grayscale inversion and pixel-intensity extraction, because each reference teaches complementary steps that enhance ECG feature extraction and CNN training. Zimmerman discloses training CNNs to detect cardiac abnormalities, Shen provides image-to-signal processing (grid removal, interpolation, filtering), and Yoon teaches inversion and pixel based feature mapping for CNN input. A PHOSITA would recognize that integrating these known methods would improve feature quality and model accuracy, yielding no more than the predictable results of established techniques applied in ECG signal analysis.
Regarding claim 5, Zimmerman, Shen, and Yoon teach the invention in claim 4, as discussed above, and further teach wherein the preprocessed ECG two-dimensional data is obtained by using five network computer architectures, including VGG16, ResNetOV2, InceptionV3, InceptionResNetV2, and Xception to use an ImageNet part of the CNN for optimal image recognition (Yoon, page 2, “Table 2 shows the diagnostic performance of a bimodal CNN model with two identical Inception-v3 backbones as described in Fig. 2. The Inception-v3 model is one of the well-known CNN model that scales up networks.”, and Yoon, page 2, “We showed that the proposed bimodal CNN architecture can be applied to other CNN models through ResNet-50 and EfficientNet-B3.”).
Therefore, it would be obvious to a PHOSITA before the effective filing date of the invention to implement Zimmerman’s CNN model for ECG classification together with Shen’s preprocessing techniques and Yoon’s use of established CNN architectures (such as VGG16, ResNet, Inception, Xception), because Yoon demonstrates the effectiveness of applying standard image-recognition networks to ECG image data, while Zimmerman provides the clinical AI context and Shen ensures robust signal reconstruction. A PHOSITA would have been motivated to combine these teachings to leverage widely adopted CNN architectures with preprocessed ECG signals to enhance diagnostic accuracy, representing the predictable application of known deep learning methods in ECG analysis.
Claim 6 is analogous to claim 1, thus claim 6 is similarly analyzed and rejected in a manner consistent with the rejection of claim 1.
Claims 8-10 are analogous to claims 3-5, thus claims 8-10 similarly analyzed and rejected in a manner consistent with the rejection of claims 3-5.
Response to Arguments
Applicant’s arguments and amendments, see Remarks/Amendments submitted 12/01/2025 with respect to the rejection of claims 1, 3-6, and 8-10 have been carefully considered and are addressed below.
Claim Rejections - 35 USC § 103
Applicant’s arguments have been fully considered but are not persuasive. Claims 1, 3-6, and 8-10 remain rejected under 35 U.S.C. §103 over Zimmerman in view of Shen and Yoon because the cited references, when properly combined, teach or suggest the claimed subject matter and provide an rationale with a reasonable expectation of success. Applicant’s remarks ask for verbatim disclosure of the claim’s particular implementation details ( “pixel intensity = 255,” “pixel-by-pixel scanning,” and specific grouping or sample counts), but indicating a different implementation does not overcome obviousness where the prior art teaches comparable techniques for ECG image preprocessing, waveform recovery, and CNN analysis.
For Zimmerman, Applicant states that Zimmerman does not convert an ECG image to one-dimensional time-series data or train a CNN based on the time-series data. This is not persuasive. Zimmerman teaches AI/NN (including CNN) analysis of ECG voltage time traces and describes extracting underlying voltage data from stored ECG representations (PDFs/digital images) by parsing the trace data for use in model training and prediction. Thus, Zimmerman teaches the core concept of deriving digital ECG signal representations (voltage versus time) from stored ECG renderings and using those signal representations as inputs to machine learning models for risk prediction or identification, including in settings where the ECG may be normal and not with an arrhythmia episode.
Applicant’s statements of Shen are also not persuasive. Shen is relied upon for teaching ECG chart and image preprocessing and signal recovery operations (removing grid and background artifacts, converting chart images into recoverable waveform representations, applying interpolation to restore and standardize sampling frequency, and filtering to reduce noise). Applicant’s emphasis that Shen discusses frequency domain processing does not negate that Shen teaches the general problem and solution pairing of recovering an ECG waveform from an ECG chart image using image processing, interpolation, and filtering. A PHOSITA would have understood Shen as providing known, routine options for extracting an ECG signal from an image and preparing it for subsequent analysis; the specific mathematical implementation (frequency-domain versus threshold and pixel-based extraction) is a design choice among known alternatives once the objective of waveform recovery is established.
Applicant’s statements regarding Yoon are also not persuasive. Yoon is not relied upon for the directionality of conversion (1D ->2D versus 2D ->1D) but for teaching high contrast grayscale representations of ECG tracings (white signal on black background via intensity manipulation) that facilitate robust machine interpretation and pixel-intensity-based representation of the ECG trace. Even if Yoon’s primary use case is CNN classification using image inputs, Yoon evidences that intensity inversion and normalization and high-contrast rendering are conventional and useful preprocessing choices to improve extraction and analysis of ECG tracings. Incorporating such known intensity preprocessing into Shen’s ECG image recovery pipeline and Zimmerman’s AI-ECG workflow would have been within the routine skill of a PHOSITA and would predictably improve signal and trace separability and downstream model performance.
Accordingly, it would have been obvious to modify Zimmerman’s AI enabled ECG workflow by applying Shen’s known ECG chart and image signal recovery techniques (grid/background removal, interpolation to standardize sampling, and filtering to reduce noise) and further applying Yoon’s known intensity and contrast preprocessing for ECG traces to yield cleaner digital representations suitable for CNN training and inference. The combination uses known methods to improve analogous ECG input preparation for machine learning, yielding predictable results. Therefore, the §103 rejection of claims 1, 3-6, and 8-10 is maintained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure.
DeMazumder et al. (U.S. Patent Publication 2023/0335289 A1) teaches a computer implemented method that collects ECG signals during sleep or wakefulness, applies AI based dynamic time series analysis to generate a personalized risk score for atrial fibrillation, and AI derived threshold, and administers therapy to prevent or treat AF or related conditions.
Kuck et al. (U.S. Patent Publication 2023/0335289 A1) teaches systems and methods for assessing health risk for a user from a combination of Artificial Intelligence, ECG data available from the user's smart watch or other smart device, and other biometric data and/or medical data provided from the user.
Ullah et al. (Ullah et al., An Automatic Premature Ventricular Contraction Recognition System Based on Imbalanced Dataset and Pre-Trained Residual Network Using Transfer Learning on ECG Signal, 2022, Diagnostics, 13, 87, pages 1-20) teaches a deep learning method using a pretrained ResNet18 with transfer learning that was developed to automatically detect premature ventricular contractions from electrocardiogram data converted into 2D images.
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYRA R LAGOY whose telephone number is (703)756-1773. The examiner can normally be reached Monday - Friday, 8:00 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kambiz Abdi can be reached at (571)272-6702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.L./Examiner, Art Unit 3685
/KAMBIZ ABDI/Supervisory Patent Examiner, Art Unit 3685