DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR
1.17(e), was filed in this application after final rejection. Since this application is eligible for continued
examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the
finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/24/2025 has been entered.
Claims 1 and 12 are amended. No claim has been cancelled, and there are no new claims.
Claims 1–22 are pending for examination.
Response to Arguments
In reference to 35 USC § 101
Applicant’s arguments, filed on 06/24/2025, with respect to the § 101 rejections have been fully considered and are persuasive. Applicant argues, beginning on Pg. 5 in the Remarks, that the newly amended claims “reflect a technically specific solutions for improving machine learning model performance in the context of cardiac signal annotation.” Specifically, applicant argues “Claim 1 of the present application recites a structured process implemented in a specific technological context: an apparatus comprising a processor and memory configured to process cardiac signals, generate automated annotations using a machine learning model, incorporate user corrections, and update both the training data and the model itself in response to these corrections.” Examiner agrees. Examiner notes that while the claims recite several limitations that are abstract ideas (mental concepts including an observation, evaluation, judgment, opinion), the claims as a whole are not directed to an abstract idea. Applicant amended the claims, which recite auto-populating and updating a cardiac signal (“wherein the processor is configured to update the user interface as a function of a user input, wherein the user input comprises at least a user query, wherein the at least a user query comprises across-session state variable configured to autopopulate historical user query data from previous session, and wherein the historical user query data is configured to be used to further update the at least a cardiac signal.”) is not an abstract idea (see MPEP 2106.04(a)(1)). Thus, these limitations must be considered additional elements to the abstract idea. Examiner notes that these additional element integrates the abstract idea into a practical application because the entire claim amounts to a detailed method of generating automated annotations (as opposed to a broad recitation of generating and updating at high levels of generality), and the specific method of generating and updating recited in the elements amounts to an improvement to the functioning of a computer, as set forth by MPEP 2106.05(a)), which states “the claim must include the components or steps of the invention that provide the improvement described in the specification.” Pursuant to this requirement set forth by the MPEP, Examiner points out that the Specification states in at least [0051, 101]: “With continued reference to FIG. 1 , correction 164 may include automatically processing cardiac signal 116. For instance, apparatus 100 may analyze, modify, and/or synthesize a signal representative of cardiac signal 116 in order to improve the cardiac signal 116, for instance by improving transmission, storage efficiency, or signal-to-noise ratio … With continued reference to FIG. 3 , any process of training, retraining, deployment, and/or instantiation of any machine learning model and/or algorithm may be performed and/or repeated after an initial deployment and/or instantiation to correct, refine, and/or improve the machine learning model and/or algorithm.” Thus, the additional elements reflects the improvement set forth and explains what the resulting improvement is.
In reference to 35 USC § 103
Applicant’s arguments filed on 06/24/2025, with respect to the newly amended limitations have been considered but are not persuasive. Applicant argues, beginning on Pg. 9 in the Remarks, that “the above-quoted language does not disclose ‘wherein the processor is configured to update the user interface as a function of a user input, wherein the user input comprises at least a user query, wherein the at least a user query comprises a cross-session state variable configured to autopopulate historical user query data from previous session, and wherein the historical user query data is configured to be used to further update the at least a cardiac signal’.” Examiner agrees. The Final Rejection mailed 03/24/2025 did not include specific rejections for the newly amended limitations. However, examiner respectfully points to the §103 rejections below which now address the newly amended limitations. Examiner notes that Golden teaches all of the newly amended limitations. Without any other specific arguments to the contrary, the §103 rejections are maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–3, 5–8, 10–14, 16–19, and 21–22 are rejected under 35 U.S.C. 103 as being unpatentable over Golden et al., (US 20180259608 A1), hereinafter “Golden”, in view of Gardner et al., (US 9058317 B1), hereinafter “Gardner”, and further in view of Chakravarthy et al., (US 20200352466 A1), hereinafter “Chakravarthy”.
Regarding claim 1, Golden teaches:
an apparatus for correcting machine learning model predictions, the apparatus comprising: a processor (Golden ¶0028: “A machine learning system may be summarized as including at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data; and at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor: receives learning data including a plurality of batches of labeled image sets, each image set including image data representative of an anatomical structure, and each image set including at least one label which identifies the region of a particular part of the anatomical structure depicted in each image of the image set; trains a fully convolutional neural network (CNN) model to segment at least one part of the anatomical structure utilizing the received learning data; and stores the trained CNN model in the at least one nontransitory processor-readable storage medium of the machine learning system”—[emphasis added]); and
a memory communicatively connected to the processor, wherein the memory contains instructions configuring the processor to (Golden ¶0028: “A machine learning system may be summarized as including at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data; and at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor: receives learning data including a plurality of batches of labeled image sets, each image set including image data representative of an anatomical structure, and each image set including at least one label which identifies the region of a particular part of the anatomical structure depicted in each image of the image set; trains a fully convolutional neural network (CNN) model to segment at least one part of the anatomical structure utilizing the received learning data; and stores the trained CNN model in the at least one nontransitory processor-readable storage medium of the machine learning system”—[emphasis added]):
receive at least a cardiac signal having a plurality of segments [from a plurality of electrodes] (Golden ¶0028: “A machine learning system may be summarized as including at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data; and at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor: receives learning data including a plurality of batches of labeled image sets, each image set including image data representative of an anatomical structure, and each image set including at least one label which identifies the region of a particular part of the anatomical structure depicted in each image of the image set; trains a fully convolutional neural network (CNN) model to segment at least one part of the anatomical structure utilizing the received learning data; and stores the trained CNN model in the at least one nontransitory processor-readable storage medium of the machine learning system”; see also Golden Fig. 4, ¶0023: “An example case where both the left ventricle and left atrium are visible in a single slice is shown in images 400a and 400b of FIG. 4. If the clinician fails to refer to the current SAX slice projected on a corresponding LAX view, it may not be obvious that the SAX slice spans both the ventricle and atrium. Further, even if the LAX view is available, it may be difficult to tell on the SAX slice where the valve is located, and therefore, where the segmentation of the ventricle should end, since the ventricle and atrium have similar signal intensities. Segmentation near the base of the heart is therefore one of the major sources of error for ventricular segmentation”—[(emphasis added) wherein each image is a segment of a surgical recording of an anatomical structure (i.e., plurality of cardiac signal segments; e.g., heart surgery)]);
generate, for at least a segment of the plurality of segments, a label representing at least a signal feature (Golden ¶0059: “The at least one processor may, for each of one or more landmarks of the anatomical structure, define a 3D label map based at least in part on the received sets of 3D MRI images and the received plurality of annotations, each 3D label map may encode a likelihood that the landmark is located at a particular location on the 3D label map, wherein the at least one processor may train the CNN model to segment the one or more landmarks utilizing the 3D MRI images and the generated 3D label maps. The images in each of the plurality of sets may represent a heart of a patient at different respective time points of a cardiac cycle, and each annotation may be indicative of a landmark of a heart of a patient depicted in a corresponding image”—[wherein the processor defines (i.e., generates) a 3D label map based on the landmarks of the anatomical structure (i.e., a label representing a signal feature)]);
generate at least an automated annotation for the at least a segment as a function of the label using an annotation machine learning model trained with annotation training data wherein the annotation training data comprises a plurality of exemplary cardiac signals as inputs correlated with a plurality of exemplary annotations as outputs (Golden ¶0059: “The at least one processor may, for each of one or more landmarks of the anatomical structure, define a 3D label map based at least in part on the received sets of 3D MRI images and the received plurality of annotations, each 3D label map may encode a likelihood that the landmark is located at a particular location on the 3D label map, wherein the at least one processor may train the CNN model to segment the one or more landmarks utilizing the 3D MRI images and the generated 3D label maps. The images in each of the plurality of sets may represent a heart of a patient at different respective time points of a cardiac cycle, and each annotation may be indicative of a landmark of a heart of a patient depicted in a corresponding image”; see also Golden ¶0208: “The SAX segmentation model uses both the raw SAX DICOM data as well as the predicted projected lines from the LAX model(s) as inputs in order to make its prediction. The predicted LAX lines serve to guide and bound the SAX predictions, and particularly aid the model near the base of the heart and valve plane, where the segmentations are often ambiguous when viewed on the SAX stack alone”—[(emphasis added) wherein the processor trains the CNN model to segment and label the MRI images with landmarks including annotations indicative of a landmark of the heart, and wherein the segments are annotated with the predicted project lines which are used as inputs in order to make the annotation predictions]);
generate, using a correction module, at least a correction upon detecting an absence of annotations (Golden ¶¶0157–0158: “Unlike previous models which were only concerned with two classes for a cell discrimination task, foreground and background, the SSFP model disclosed herein attempts to distinguish four classes, namely, background, LV Endocardium, LV Epicardium and RV Endocardium. To accomplish this, the network output may include three probability maps, one for each non-background class. During training, ground truth binary masks for each of the three classes are provided to the network, along with the pixel data. The network loss may be determined as the sum of the loss over the three classes. If any of the three ground truth masks are missing for an image (meaning that we have no data, as opposed to the ground truth being an empty mask), that mask may be ignored when calculating the loss. Missing ground truth data is explicitly accounted for during the training process. For example, the network may be trained on an image for which the LV endocardium contour is defined, even if the LV epicardium and RV endocardium contour locations are not known. A more basic architecture that could not account for missing data could only have been trained on a subset (e.g., 20 percent) of training images that have all three types of contours defined. Reducing the training data volume in this way would result in significantly reduced accuracy. Thus, by explicitly modifying the loss function to account for missing data, the full training data volume is used, allowing the network to learn more robust features”—[(emphasis added) wherein the BRI of correction module is any code, instruction, or process executed on a processor, and wherein the system explicitly modifies the loss function to account for missing data]);
receive, using a user interface, at least an input from a user (Golden ¶0011: “The most basic method of creating ventricular contours is to complete the process manually with some sort of polygonal or spline drawing tool, without any automated algorithms or tools. In this case, the user may, for example, create a freehand drawing of the outline of the ventricle, or drop spline control points which are then connected with a smoothed spline contour. After initial creation of the contour, depending on the software's user interface, the user typically has some ability to modify the contour, e.g., by moving, adding or deleting control points or by moving the spline segments”; see also Golden ¶0178: “At the same time that contours (e.g., contours 1002, 1102 and 1202) are displayed to the user, the system calculates and shows ventricle volumes at ED and ES to the user, as well as multiple computed measurements. An example interface 1300 is shown in FIG. 13 which displays multiple computed measurements. In at least some implementations, these measurements include stroke volume (SV) 1302, which is the volume of blood ejected from the ventricle in one cardiac cycle; ejection fraction (EF) 1304, which is the fraction of the blood pool ejected from the ventricle in one cardiac cycle; cardiac output (CO) 1306, which is the average rate at which blood leaves the ventricle, ED mass 1308, which is the mass of the myocardium (i.e., epicardium-endocardium) for the ventricle at end diastole; and ES mass 1310, which is the mass of the myocardium for the ventricle at end systole”; see also Golden ¶0249: “In order to have a more accurate segmentation of the left and right ventricles, it may be advantageous to identify the position and orientation of the valves of the heart. In at least some implementations, within the aforementioned ventricle segmentation interface, a user is able to mark points that lie on the valve plane using the available long axis views. The valve plane is determined from these input points by performing a regression to find the plane that best fits. The normal for the plane is set to point away from the apex of the ventricle. Once the plane has been defined, any portion of the volume that lies on the positive side is subtracted from the total volume for the ventricle. This ensures that nothing outside the valve is included in determining the volume of the ventricle”—[(emphasis added)]);
create at least a user annotation within the at least a cardiac signal using the at least an input (Golden ¶0218: “First, the data handling pipeline is described. This section details the process which is followed to create the database of images with their annotations, along with the specific method used to encode landmark location. Second, the architecture of the machine learning approach is presented. How the network transforms the input 3D image into a prediction of landmark location is presented. Third, how the model is trained to the available data is described. Finally, the inference pipeline is detailed. It is shown how one can apply the neural network to an image never used before to predict the region of all six landmarks”; see also Golden ¶0249: “In order to have a more accurate segmentation of the left and right ventricles, it may be advantageous to identify the position and orientation of the valves of the heart. In at least some implementations, within the aforementioned ventricle segmentation interface, a user is able to mark points that lie on the valve plane using the available long axis views. The valve plane is determined from these input points by performing a regression to find the plane that best fits. The normal for the plane is set to point away from the apex of the ventricle. Once the plane has been defined, any portion of the volume that lies on the positive side is subtracted from the total volume for the ventricle. This ensures that nothing outside the valve is included in determining the volume of the ventricle”—[(emphasis added)]), and
generating, for each incorrect automatic annotation identified based on the validation, the at least a user annotation (Golden ¶0196: “At 2122, the generated splines are forwarded back to the web server after all batches have been processed, where the splines are joined with the inference results from other inference nodes. The web server ensures that the volume is contiguous (i.e., no missing contours in the middle of the volume) by interpolating between neighboring slices if a contour is missing. At 2124, the web server saves the contours in the database, then presents the contours to the user via the web application. If the user edits a spline, the spline's updated version is saved in the database alongside the original, automatically-generated version. In at least some implementations, comparing manually edited contours with their original, automatically-generated versions, may be used to re-train or fine-tune a model only on inference results that required manual correction”; see also Golden ¶0237: “In at least some implementations, the dataset is made of clinical studies uploaded on the web application by previous users. The annotations may be placed by the user on the different images. As explained previously, this dataset is split into a train, validation, and test set”; see also Golden ¶0233: “The training database may be split into a training set used to train the model, a validation set used to quantify the quality of the model, and a test set. The split enforces all the images from a single patient to lie in the same set. This guarantees that the model is not validated with patients used for training. At no point is data from the test set used when training the model. Data from the test set may be used to show examples of landmark localization, but this information is not used for training or for ranking models with respect to one another”—[emphasis added]);
update the at least a cardiac signal and the annotation machine learning model as a function of the at least a correction and the at least a user annotation (Golden ¶0196: “At 2122, the generated splines are forwarded back to the web server after all batches have been processed, where the splines are joined with the inference results from other inference nodes. The web server ensures that the volume is contiguous (i.e., no missing contours in the middle of the volume) by interpolating between neighboring slices if a contour is missing. At 2124, the web server saves the contours in the database, then presents the contours to the user via the web application. If the user edits a spline, the spline's updated version is saved in the database alongside the original, automatically-generated version. In at least some implementations, comparing manually edited contours with their original, automatically-generated versions, may be used to re-train or fine-tune a model only on inference results that required manual correction”—[emphasis added]);
wherein updating the annotation machine learning model comprises:
replacing one or more portions of the cardiac signal by synthesizing the cardiac signal as a function of the correction (Golden “Synthesis of Other Views for Automated Volumes,” ¶¶0204–0208: “Thus, a two-stage ventricle segmentation model may be utilized. In a first stage, the ventricles are segmented in one or more LAX planes. Because of the high spatial resolution of these images, the segmentation can be very precise. A disadvantage is the LAX plane consists of only a single plane instead of a volume. If this LAX segmentation is projected to the SAX stack, the LAX segmentation appears as a line on each of the SAX images. This line may be created precisely if the line is aggregated across segmentations from multiple LAX views (e.g., 2CH, 3CH, 4CH; see the heading ‘Interface for defining valve planes for manual LV/RV volumes’ below). This line may be used to bound the SAX segmentation, which is generated via a different model that operates on the SAX images. The SAX segmentation model uses both the raw SAX DICOM data as well as the predicted projected lines from the LAX model(s) as inputs in order to make its prediction. The predicted LAX lines serve to guide and bound the SAX predictions, and particularly aid the model near the base of the heart and valve plane, where the segmentations are often ambiguous when viewed on the SAX stack alone”—[wherein the system replaces portions of the cardiac signal (e.g., projecting LAX segmentation on the SAX images) which are used to bound the SAX segmentations (i.e., synthesizing the cardiac signal) and input to the model for predictions]);
updating the cardiac signal by replacing each incorrect automatic annotation with the at least a user annotation (Golden ¶0011: “After initial creation of the contour, depending on the software's user interface, the user typically has some ability to modify the contour, e.g., by moving, adding or deleting control points or by moving the spline segments”—[(emphasis added)]); and
adding the updated cardiac signal including the one or more replaced portions of the cardiac signal to the annotation training data in order to increase an accuracy of the annotation machine learning model (Golden Figs. 14, 16–18, 21, “Training Database Creation for 4D Flow Data” ¶¶0180–0191: “Whereas the SSFP DICOM files are acquired and stored in SAX orientation, 4D Flow DICOMs are collected and stored as axial slices. In order to create a SAX multi-planar reconstruction (MPR) of the data, the user may need to place the relevant landmarks for the left and/or right heart. These landmarks are then used to define unique SAX planes for each ventricle as defined by the ventricle apex and valves. FIG. 14 shows a set 1400 of SAX planes (also referred to as a SAX stack) for the LV in which each SAX plane is parallel for a two chamber view 1402, a three chamber view 1404 and a four chamber view 1406 … FIG. 18 shows a process 1800 of creating a training LMDB from clinician annotations. 4D Flow annotations may be stored in a MongoDB 1802. At 1804 and 1806, the system extracts the contours and landmarks, respectively. Contours are stored as a series of (x, y, z) points defining the splines of the contour. Landmarks are stored as a single four-dimensional coordinate (x, y, z, t) for each landmark”—[(emphasis added)]); and
display, using the user interface, the updated at least a cardiac signal to the user (Golden ¶0178: “At the same time that contours (e.g., contours 1002, 1102 and 1202) are displayed to the user, the system calculates and shows ventricle volumes at ED and ES to the user, as well as multiple computed measurements. An example interface 1300 is shown in FIG. 13 which displays multiple computed measurements. In at least some implementations, these measurements include stroke volume (SV) 1302, which is the volume of blood ejected from the ventricle in one cardiac cycle; ejection fraction (EF) 1304, which is the fraction of the blood pool ejected from the ventricle in one cardiac cycle; cardiac output (CO) 1306, which is the average rate at which blood leaves the ventricle, ED mass 1308, which is the mass of the myocardium (i.e., epicardium-endocardium) for the ventricle at end diastole; and ES mass 1310, which is the mass of the myocardium for the ventricle at end systole”; see also Golden ¶0196: “At 2122, the generated splines are forwarded back to the web server after all batches have been processed, where the splines are joined with the inference results from other inference nodes. The web server ensures that the volume is contiguous (i.e., no missing contours in the middle of the volume) by interpolating between neighboring slices if a contour is missing. At 2124, the web server saves the contours in the database, then presents the contours to the user via the web application. If the user edits a spline, the spline's updated version is saved in the database alongside the original, automatically-generated version. In at least some implementations, comparing manually edited contours with their original, automatically-generated versions, may be used to re-train or fine-tune a model only on inference results that required manual correction”—[emphasis added]).
wherein the processor is configured to update the user interface as a function of a user input (Golden ¶0196: “At 2122, the generated splines are forwarded back to the web server after all batches have been processed, where the splines are joined with the inference results from other inference nodes. The web server ensures that the volume is contiguous (i.e., no missing contours in the middle of the volume) by interpolating between neighboring slices if a contour is missing. At 2124, the web server saves the contours in the database, then presents the contours to the user via the web application. If the user edits a spline, the spline's updated version is saved in the database alongside the original, automatically-generated version. In at least some implementations, comparing manually edited contours with their original, automatically-generated versions, may be used to re-train or fine-tune a model only on inference results that required manual correction”; see also Golden ¶0168: “The inference service is responsible for loading a model, generating contours, and displaying them for the user. After inference is invoked at 902, at 904 images are sent to an inference server. At 906, the production model or network that is used by the inference service is loaded onto the inference server. The network may have been previously selected from the corpus of models trained during hyperparameter search. The network may be chosen based on a tradeoff between accuracy, memory usage and speed of execution. The user may alternatively be given a choice between a “fast” or “accurate” model via a user preference option”—[(emphasis added)]),
wherein the user input comprises at least a user query (Golden ¶0168: “The inference service is responsible for loading a model, generating contours, and displaying them for the user. After inference is invoked at 902, at 904 images are sent to an inference server. At 906, the production model or network that is used by the inference service is loaded onto the inference server. The network may have been previously selected from the corpus of models trained during hyperparameter search. The network may be chosen based on a tradeoff between accuracy, memory usage and speed of execution. The user may alternatively be given a choice between a “fast” or “accurate” model via a user preference option”; see also Golden ¶0224: “Note that alternative strategies may also be used to define the standard deviation (arbitrary value, parameter search) and may lead to comparable results. FIG. 29 shows this transition from a landmark position, identified with a cross 2902 in a view 2904, to a Gaussian 2906 in a view 2908 evaluated on the image for the 2D case”—[(emphasis added)]),
wherein the at least a user query comprises across-session state variable configured to autopopulate historical user query data from previous session (Golden ¶¶0167–0168: “At 902, after a user has loaded a study in the web application, the user may invoke the inference service (e.g., by clicking a “generate missing contours” icon), which automatically generates any missing (not yet created) contours. Such contours may include LV Endo, LV Epi, or RV Endo, for example. In at least some implementations, inference may be invoked automatically when the study is either loaded by the user in the application or when the study is first uploaded by the user to a server. If inference is performed at upload time, the predictions may be stored in a nontransitory processor-readable storage medium at that time but not displayed until the user opens the study … The network may have been previously selected from the corpus of models trained during hyperparameter search”; see also Golden ¶0231: “Third, the parameters selected after the hyperparameter search can differ from the DeepVentricle parameters, and are specifically selected to solve the problem at hand. Additionally, the standard deviation used to define the label maps, discussed above, may be considered as a hyperparameter”; see also Golden Fig. 13, ¶0178: “At the same time that contours (e.g., contours 1002, 1102 and 1202) are displayed to the user, the system calculates and shows ventricle volumes at ED and ES to the user, as well as multiple computed measurements. An example interface 1300 is shown in FIG. 13 which displays multiple computed measurements”—[(emphasis added) wherein the BRI of across-session state variable is any variable recording data entered on remote device during a previous session (see present disclosure [0046])]), and
wherein the historical user query data is configured to be used to further update the at least a cardiac signal (Golden “Synthesis of Other Views for Automated Volumes,” ¶¶0204–0208: “Thus, a two-stage ventricle segmentation model may be utilized. In a first stage, the ventricles are segmented in one or more LAX planes … The SAX segmentation model uses both the raw SAX DICOM data as well as the predicted projected lines from the LAX model(s) as inputs in order to make its prediction. The predicted LAX lines serve to guide and bound the SAX predictions, and particularly aid the model near the base of the heart and valve plane, where the segmentations are often ambiguous when viewed on the SAX stack alone”; see also Golden ¶0024: “In the 4D Flow workflow of a cardiac imaging application, the user may be required to define the regions of different landmarks in the heart in order to see different cardiac views (e.g., 2CH, 3CH, 4CH, SAX) and segment the ventricles”; see also Golden ¶0168: “The inference service is responsible for loading a model, generating contours, and displaying them for the user. After inference is invoked at 902, at 904 images are sent to an inference server. At 906, the production model or network that is used by the inference service is loaded onto the inference server. The network may have been previously selected from the corpus of models trained during hyperparameter search. The network may be chosen based on a tradeoff between accuracy, memory usage and speed of execution. The user may alternatively be given a choice between a “fast” or “accurate” model via a user preference option”—[(emphasis added) wherein the system replaces portions of the cardiac signal (i.e., updating the cardiac signal) which uses the SAX model capturing both the raw SAX DICOM data as well as the predicted projected lines (i.e., historical user query data) provided by the LAX model configured with user hyperparameter search data (i.e., query) required to define the regions of the cardiac views]).
Golden does not appear to explicitly teach:
[receive at least a cardiac signal having a plurality of segments] from a plurality of electrodes;
removing at least one exemplary input and correlated exemplary output of the annotation training data as a function of the user annotation; and
wherein creating the at least a user annotation comprises: validating the at least an automated annotation.
However, Gardner teaches:
wherein creating the at least a user annotation comprises: validating the at least an automated annotation (Gardner Col. 5, lines 52–60: “The routine 200 proceeds from operation 208 to operation 210, where the identified inaccurate annotations are corrected. Operations 208 and 210 may include inspecting and/or validating annotations generated according to the predictive annotations. Inspection may be performed in response to receiving a user selection of one or more types of annotations to review. Following operation 210, at operation 212 one or more new annotations may be created, i.e. annotations that were not generated by predictive annotation”; see also Gardner Col. 7, lines 32–52: “Now referring specifically to buttons within the illustrated group 302, "Inspection" enables the inspection mode and "Creation" enables the creation mode with respect to the selected text; "All" enables the inspection and updating of all annotations to the text; "Sentence" enables inspection and updating of sentence annotations when the Inspection mode is active and creation of sentence annotations when the Creation mode is active; "Phrase" enables the inspection and updating of phrase annotations in the Inspection mode and creation of phrase annotations in the Creation mode; "Token" enables the inspection and updating of token annotations in the Inspection mode and the creation of token annotations in the Creation mode; "Coreference" enables the inspection and updating of coreference annotations in the Inspection mode and creation of coreference annotations in the Creation mode; "Assertion" enables the inspection and updating of assertion annotations in the Inspection mode and the creation of assertion annotations in the Creation mode; and "Note" enables the inspection and updating of note annotations in the Inspection mode and the creation of note annotations in the Creation mode. Within the illustrated group of buttons 304 and 306, "Save" enables the saving of open files; "Predict Annotations" enables the predictions of annotations to a selected file; "Validate DAF" enables the validations of the annotations in a selected file; and "Toggle LTR/RTL" enables toggling text alignment in the selected file”—[emphasis added]);
removing at least one exemplary input and correlated exemplary output of the annotation training data as a function of the user annotation (Gardner Figs. 2, 3, Col. 5, line 11 – Col. 6, line 19: “From operation 204, the routine 200 proceeds to operation 206, where predictive annotations to the sequence of characters are generated based at least in part on the identified data features. Generating the predictive annotations may include predictively labeling factual assertions, parts of speech, syntactic roles, token boundaries, and/or categories associated with the sequence of characters. Predictive labeling may be based on resource data that includes lexicons identifying predefined associations between particular sequences of characters and labels. From operation 206, the routine 200 proceeds to operation 208, where inaccurate annotations generated according to the predictive annotations are identified”; see also Gardner Col. 7, line 32 – Col. 8, line 35: “Now referring specifically to buttons within the illustrated group 302, “Inspection” enables the inspection mode and “Creation” enables the creation mode with respect to the selected text; “All” enables the inspection and updating of all annotations to the text”—[wherein the system identifies the inaccurate annotations, based on the input and output of the generative prediction model, and replaces them with the corrected user annotations (e.g., created via the user interface)]);
The system of Golden, the teachings of Gardner, and the instant application are analogous art because they pertain to training machine learning models with annotated segmented data.
It would be obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system of Golden with the teachings of Gardner to provide for validating that the system automated annotation is correct. One would be motivated to do so to correct incorrect annotations (Gardner Col. 5, lines 52–60: “The routine 200 proceeds from operation 208 to operation 210, where the identified inaccurate annotations are corrected. Operations 208 and 210 may include inspecting and/or validating annotations generated according to the predictive annotations. Inspection may be performed in response to receiving a user selection of one or more types of annotations to review. Following operation 210, at operation 212 one or more new annotations may be created, i.e. annotations that were not generated by predictive annotation”).
Golden in view of Gardner does not appear to explicitly teach:
[receive at least a cardiac signal having a plurality of segments] from a plurality of electrodes;
However, Chakravarthy teaches:
[receive at least a cardiac signal having a plurality of segments] from a plurality of electrodes (Chakravarthy ¶0047: “Sensing circuitry 52 and communication circuitry 54 may be selectively coupled to electrodes 16A, 16B via switching circuitry 60 as controlled by processing circuitry 50. Sensing circuitry 52 may monitor signals from electrodes 16A, 16B in order to monitor electrical activity of a heart of patient 4 of FIG. 1 and produce cardiac electrogram data for patient 4. In some examples, processing circuitry 50 may perform feature delineation of the sensed cardiac electrogram data to detect an episode of cardiac arrhythmia of patient 4. In some examples, processing circuitry 50 transmits, via communication circuitry 54, the cardiac electrogram data for patient 4 to an external device, such as external device 12 of FIG. 1. For example, IMD 10 sends digitized cardiac electrogram data to network 25 for processing by machine learning system 150 of FIG. 1. In some examples, IMD 10 transmits one or more segments of the cardiac electrogram data in response to detecting, via feature delineation, an episode of arrhythmia”—[(emphasis added)]).
The system of Golden, the teachings of Chakravarthy, and the instant application are analogous art because they pertain to training machine learning models with annotated segmented data.
It would be obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system of Golden in view of Gardner with the teachings of Chakravarthy to provide for the cardiac signals to be received data from electrodes. One would be motivated to do so to monitor and produce sensed electrogram cardiac data for a patient (Chakravarthy ¶0047: “Sensing circuitry 52 and communication circuitry 54 may be selectively coupled to electrodes 16A, 16B via switching circuitry 60 as controlled by processing circuitry 50. Sensing circuitry 52 may monitor signals from electrodes 16A, 16B in order to monitor electrical activity of a heart of patient 4 of FIG. 1 and produce cardiac electrogram data for patient 4”).
Regarding claim 2, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 1.
Golden teaches:
wherein generating the label comprises: receiving the plurality of training data comprising the plurality of exemplary cardiac signals as inputs correlated with the plurality of exemplary labels as outputs (Golden ¶0028: “A machine learning system may be summarized as including at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data; and at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor: receives learning data including a plurality of batches of labeled image sets, each image set including image data representative of an anatomical structure, and each image set including at least one label which identifies the region of a particular part of the anatomical structure depicted in each image of the image set; trains a fully convolutional neural network (CNN) model to segment at least one part of the anatomical structure utilizing the received learning data; and stores the trained CNN model in the at least one nontransitory processor-readable storage medium of the machine learning system”; see also Golden ¶0226: “FIG. 30 shows a process 3000 for a preprocessing pipeline. At 3006 and 3008, the 3D MRIs 3002 and label maps 3004, respectively, are resized to a predefined size $n_x \times n_y \times n_z$ such that all of the MRIs can be fed to the same neural network. At 3010, the intensity of the MM pixels are clipped between the 1st and 99th percentile. This means that the pixel intensity will saturate at the value of the intensity corresponding to the 1st and 99th percentile. This removes outlier pixel intensities that may be caused by artifacts. At 3012, the intensities are then scaled to lie between 0 and 1. At 3014, the intensity histogram is then normalized using contrast limited adaptive histogram equalization to maximize contrast in the image and minimize intra-image intensity differences (as may be caused by, for example, magnetic field inhomogeneities). Finally, at 3016 the image is centered to have zero mean. Other strategies may be used for the normalization of the image intensity, such as normalizing the variance of the input to one, and may yield similar results. This pipeline results in preprocessed images 3018 and labels 3020 which can be fed to the network”—[(emphasis added) wherein the received learning data includes MRI’s and label maps (i.e., exemplary cardiac signals as inputs correlated with a plurality of exemplary labels as outputs)]);
training a labeling machine learning model as a function of the plurality of training data (Golden ¶0225: “At 2812, once the 3D volumes have been defined for both the MM and the label map, the images are prepocessed. Generally, the goal is to normalize the images size and appearance for future training”; see also Golden ¶0228: “Returning to FIG. 28, at 2814 an upload ID is defined to be the key that identifies the pair (MRI, label map), which is stored in a training LMDB database at 2816. Finally, at 2818 the pair (MRI, label map) is written to the LMDB”; see also Golden Fig. 8, ¶0232: “The following discussion describes how the deep neural network can be trained using the LMDB database of 3D MM and label map pairs. The overall objective is to tune the parameters of the network such that the network is able to predict the position of the heart landmarks on previously unseen images. A flowchart of the training process is shown in FIG. 8 and described above”); and
generating the label using the labeling machine learning model (Golden ¶0231: “The network used for landmark detection differs from the DeepVentricle implementation discussed above in three main ways. First, the architecture is three dimensional: the network processes a 3D MM in a single pass, producing a 3D label map for every landmark. Second, the network predicts 6 classes, one for each landmark. Third, the parameters selected after the hyperparameter search can differ from the DeepVentricle parameters, and are specifically selected to solve the problem at hand. Additionally, the standard deviation used to define the label maps, discussed above, may be considered as a hyperparameter. The output of the network is a 3D map which encodes where the landmark is positioned. High values of the map may correspond to likely landmark position, and low values may correspond to unlikely landmark position”—[emphasis added]).
Regarding claim 3, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 1.
Golden teaches:
wherein: the at least a cardiac signal comprises time series data, wherein the plurality of segments comprises a plurality of time series segments (Golden ¶0219: “For the presented machine learning approach, a database of 4D Flow data is used, which includes three dimensional (3D) magnetic resonance images (MM) of the heart, stored as series of two dimensional (2D) DICOM images. Typically, around 20 3D volumetric images are acquired throughout a single cardiac cycle, each corresponding to one snapshot of the heartbeat. The initial database thus corresponds to the 3D images of different patients at different time steps. Each 3D MRI presents a number of landmark annotations, from zero landmark to six landmarks, placed by the user of the web application. The landmark annotations, if present, are stored as vectors of coordinates (x, y, z, t) indicating the position (x, y, z) of the landmark in the 3D MRI corresponding to the time point t”; see also Golden ¶0244: “In at least some implementations, the automatic location of cardiac landmarks may be achieved by directly predicting the coordinates (x, y, z) of the different landmarks. For that, a different network architecture may be used. This alternative network may be composed of a contracting path, followed with several fully connected layers, with a length-three vector of (x, y, z) coordinates as the output for each landmark. This is a regression, rather than a segmentation network. Note that, in the regression network, unlike in the segmentation network, there is no expanding path in the network. Other architectures may also be used with the same output format. In at least some implementations, time may also be included in the output as a fourth dimension if 4D data (x, y, z, time) is given as input”—[wherein the data includes 3D images of different patients at different time steps and 4D segment data including time]); and
displaying the at least a cardiac signal comprises displaying at least a time series segment of the plurality of time series segments (Golden ¶¶0178–0179: “At the same time that contours (e.g., contours 1002, 1102 and 1202) are displayed to the user, the system calculates and shows ventricle volumes at ED and ES to the user, as well as multiple computed measurements. An example interface 1300 is shown in FIG. 13 which displays multiple computed measurements. In at least some implementations, these measurements include stroke volume (SV) 1302, which is the volume of blood ejected from the ventricle in one cardiac cycle; ejection fraction (EF) 1304, which is the fraction of the blood pool ejected from the ventricle in one cardiac cycle; cardiac output (CO) 1306, which is the average rate at which blood leaves the ventricle, ED mass 1308, which is the mass of the myocardium (i.e., epicardium-endocardium) for the ventricle at end diastole; and ES mass 1310, which is the mass of the myocardium for the ventricle at end systole. For 4D Flow data, the same DeepVentricle architecture, hyperparameter search methodology, and training database as described above for SSFP data may be used. Training a 4D Flow model may be the same as in the SSFP operation discussed above, but the creation of an LMDB and inference may be different for the 4D Flow implementation”; see also Golden ¶0241: “In at least some implementations, the 3D images acquired are 4D Flow sequences. This means that the phase of the signal is also acquired, and may be used to quantify the velocity of the blood flow in the heart and arteries, as shown in the image 3400 of FIG. 34 which shows four different views. This information can be useful to locate the different landmarks of the heart. In this case, the previously described model may be augmented to include flow information”—[wherein the system displays the 4D segmented time series data]).
Regarding claim 5, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 1.
Golden teaches:
wherein generating the label comprises: displaying the at least a cardiac signal with a first visual indicator (Golden ¶0182: “Whereas the SSFP DICOM files are acquired and stored in SAX orientation, 4D Flow DICOMs are collected and stored as axial slices. In order to create a SAX multi-planar reconstruction (MPR) of the data, the user may need to place the relevant landmarks for the left and/or right heart. These landmarks are then used to define unique SAX planes for each ventricle as defined by the ventricle apex and valves. FIG. 14 shows a set 1400 of SAX planes (also referred to as a SAX stack) for the LV in which each SAX plane is parallel for a two chamber view 1402, a three chamber view 1404 and a four chamber view 1406”—[wherein the relevant landmarks are placed (i.e., a first visual indicator) and displayed in a SAX plane]);
displaying at least a labeled segment in at least a second visual indicator, wherein the at least a second visual indicator is different from the first visual indicator (Golden ¶0180: “Whereas the SSFP DICOM files are acquired and stored in SAX orientation, 4D Flow DICOMs are collected and stored as axial slices. In order to create a SAX multi-planar reconstruction (MPR) of the data, the user may need to place the relevant landmarks for the left and/or right heart. These landmarks are then used to define unique SAX planes for each ventricle as defined by the ventricle apex and valves. FIG. 14 shows a set 1400 of SAX planes (also referred to as a SAX stack) for the LV in which each SAX plane is parallel for a two chamber view 1402, a three chamber view 1404 and a four chamber view 1406”—[wherein more than one relevant landmarks are placed to define unique SAX planes for each ventricle as defined by the ventricle apex and valves (i.e., a second visual indicator) and displayed in a SAX plane]); and
overlaying the at least a labeled segment within the at least a cardiac signal (Golden Fig. 14, ¶0180: “Whereas the SSFP DICOM files are acquired and stored in SAX orientation, 4D Flow DICOMs are collected and stored as axial slices. In order to create a SAX multi-planar reconstruction (MPR) of the data, the user may need to place the relevant landmarks for the left and/or right heart. These landmarks are then used to define unique SAX planes for each ventricle as defined by the ventricle apex and valves. FIG. 14 shows a set 1400 of SAX planes (also referred to as a SAX stack) for the LV in which each SAX plane is parallel for a two chamber view 1402, a three chamber view 1404 and a four chamber view 1406”—[wherein more than one relevant landmarks are placed to define unique SAX planes for each ventricle as defined by the ventricle apex and valves and displayed in a SAX plane overlay (e.g., see Fig. 14)]).
Regarding claim 6, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 5.
Golden teaches:
wherein: creating the at least a user annotation comprises modifying the at least a cardiac signal (Golden ¶0196: “At 2122, the generated splines are forwarded back to the web server after all batches have been processed, where the splines are joined with the inference results from other inference nodes. The web server ensures that the volume is contiguous (i.e., no missing contours in the middle of the volume) by interpolating between neighboring slices if a contour is missing. At 2124, the web server saves the contours in the database, then presents the contours to the user via the web application. If the user edits a spline, the spline's updated version is saved in the database alongside the original, automatically-generated version. In at least some implementations, comparing manually edited contours with their original, automatically-generated versions, may be used to re-train or fine-tune a model only on inference results that required manual correction”—[(emphasis added) wherein the spline is updated based on the user annotation]); and
displaying the at least a cardiac signal comprises displaying the modified at least a cardiac signal (Golden ¶0011: “The most basic method of creating ventricular contours is to complete the process manually with some sort of polygonal or spline drawing tool, without any automated algorithms or tools. In this case, the user may, for example, create a freehand drawing of the outline of the ventricle, or drop spline control points which are then connected with a smoothed spline contour. After initial creation of the contour, depending on the software's user interface, the user typically has some ability to modify the contour, e.g., by moving, adding or deleting control points or by moving the spline segments”; see also Golden ¶0175: “At 918, these splines are stored in a database and displayed to the user in the web application. If the user modifies a spline, the database may be updated with the modified spline”—[wherein the user modified spline is displayed]).
Regarding claim 7, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 5.
Gardner teaches:
wherein: the user interface comprises at least an annotation window (Gardner Fig 4., Col. 8, lines 47–65: “The exemplary implementation of FIG. 4 relates to an area of special interest when annotating fictional works, namely the treatment of objects or animals as main characters. In many fictional works where objects or animals are characters in the story line, they act and communicate as a person would in a non-fictional work. This treatment is exemplified with regard to the "White Rabbit" of the selected text 412. As shown by the drop-down windows and specific menus 406, 408, and 410, the annotated "Category" of the selected phrase is "Person" because the white rabbit interacts with and communicates with the primary character in the story as a person would. By implementing various processes for annotation described in accordance with one or more embodiments above, DAFs having the annotated text can be created. The DAFs may then be used to train models for performing specific NLP functions, following an iterative process to improve annotation prediction accuracy, which may thereby reduce the need for human interaction in generating accurate training data”); and
creating the at least a user annotation comprises creating at least an annotation segment within the at least an annotation window (Gardner Col. 7, line 53 – Col. 8, line 35: “Now referring specifically to buttons within the illustrated group 302, "Inspection" enables the inspection mode and "Creation" enables the creation mode with respect to the selected text; "All" enables the inspection and updating of all annotations to the text; "Sentence" enables inspection and updating of sentence annotations when the Inspection mode is active and creation of sentence annotations when the Creation mode is active; "Phrase" enables the inspection and updating of phrase annotations in the Inspection mode and creation of phrase annotations in the Creation mode; "Token" enables the inspection and updating of token annotations in the Inspection mode and the creation of token annotations in the Creation mode; "Coreference" enables the inspection and updating of coreference annotations in the Inspection mode and creation of coreference annotations in the Creation mode; "Assertion" enables the inspection and updating of assertion annotations in the Inspection mode and the creation of assertion annotations in the Creation mode; and "Note" enables the inspection and updating of note annotations in the Inspection mode and the creation of note annotations in the Creation mode. Within the illustrated group of buttons 304 and 306, "Save" enables the saving of open files; "Predict Annotations" enables the predictions of annotations to a selected file; "Validate DAF" enables the validations of the annotations in a selected file; and "Toggle LTR/RTL" enables toggling text alignment in the selected file”).
The same motivation used to combine Golden with Gardner in claim 1 is equally applicable to claim 7.
Regarding claim 8, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 7.
Golden teaches:
wherein creating the at least an annotation segment comprises: displaying the at least an annotation segment in at least a third visual indicator (Golden ¶0180: “Whereas the SSFP DICOM files are acquired and stored in SAX orientation, 4D Flow DICOMs are collected and stored as axial slices. In order to create a SAX multi-planar reconstruction (MPR) of the data, the user may need to place the relevant landmarks for the left and/or right heart. These landmarks are then used to define unique SAX planes for each ventricle as defined by the ventricle apex and valves. FIG. 14 shows a set 1400 of SAX planes (also referred to as a SAX stack) for the LV in which each SAX plane is parallel for a two chamber view 1402, a three chamber view 1404 and a four chamber view 1406”—[wherein more than one relevant landmarks are placed to define unique SAX planes for each ventricle as defined by the ventricle apex and valves (i.e., a third visual indicator) and displayed in a SAX plane]); and
overlaying the at least an annotation segment within the at least a cardiac signal (Golden Fig. 14, ¶0180: “Whereas the SSFP DICOM files are acquired and stored in SAX orientation, 4D Flow DICOMs are collected and stored as axial slices. In order to create a SAX multi-planar reconstruction (MPR) of the data, the user may need to place the relevant landmarks for the left and/or right heart. These landmarks are then used to define unique SAX planes for each ventricle as defined by the ventricle apex and valves. FIG. 14 shows a set 1400 of SAX planes (also referred to as a SAX stack) for the LV in which each SAX plane is parallel for a two chamber view 1402, a three chamber view 1404 and a four chamber view 1406”—[wherein more than one relevant landmarks are placed to define unique SAX planes for each ventricle as defined by the ventricle apex and valves and displayed in a SAX plane overlay (e.g., see Fig. 14)]).
Regarding claim 10, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 7.
Gardner teaches:
wherein the processor is further configured to adjust a length of the at least an annotation window as a function of the input (Gardner Col. 4, lines 14–25: “A user interface executing on the user computer 104 may be configured to function as a management user interface for, in response to receiving user input from a user that include performing specific machine learning management functions such as managing annotations performed by managed computers coupled via a network. Annotation functions may be managed by monitoring the progress of annotations made to text data by the managed computers. Exemplary interface configurations and corresponding functions will be described in further detail with reference to embodiments shown in FIGS. 3 and 4”; see also Gardner Figs. 3 and 4, Col. 7, lines 32–52: “FIGS. 3 and 4 are screen diagrams 300 and 400 of a user interface for machine learning management, according to one embodiment. In particular, an annotation interface configuration is shown. The user interface has numerous input controls configured for interaction with a computer user and, in response to receiving an action by a user, causing one or more computers to perform specific functions for machine learning management. Navigating between an "Inspection" and "Creation" mode, and between text files may be performed by selecting the appropriate mode and text file to inspect or annotate. Accordingly, to navigate to the Inspection mode and a Chapter 1 text file, a user would select the "Inspection" button and then the "ch0 1 .txt" tab, and to navigate to the Creation mode and the Chapter 1 text file, the user would select the "Creation" button and then the "ch0 1 .txt" tab. Similarly, to navigate to the Inspection mode and Chapter 2 text file, a user would select the "Inspection" button and the "ch02.txt" tab, and to navigate to the Creation mode and Chapter 2 text file, a user would select the "Creation" button and "ch02.txt" tab”; see also Gardner Col. 8, lines 47–65: “The exemplary implementation of FIG. 4 relates to an area of special interest when annotating fictional works, namely the treatment of objects or animals as main characters. In many fictional works where objects or animals are characters in the story line, they act and communicate as a person would in a non-fictional work. This treatment is exemplified with regard to the "White Rabbit" of the selected text 412. As shown by the drop-down windows and specific menus 406, 408, and 410, the annotated "Category" of the selected phrase is "Person" because the white rabbit interacts with and communicates with the primary character in the story as a person would. By implementing various processes for annotation described in accordance with one or more embodiments above, DAFs having the annotated text can be created. The DAFs may then be used to train models for performing specific NLP functions, following an iterative process to improve annotation prediction accuracy, which may thereby reduce the need for human interaction in generating accurate training data”—[(emphasis added) wherein the user selects the creation mode which allows the user to annotate in the window that changes in length based on the amount of annotation (see Fig. 4)]).
The same motivation used to combine Golden with Gardner in claim 1 is equally applicable to claim 10.
Regarding claim 11, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 1.
Golden teaches:
wherein creating the at least a user annotation comprises modifying the label (Golden ¶0240: “When a cardiac MRI is uploaded to the web application, the user can select a “Views” button under a “Cardiac” section. This opens a new panel on the right of the image with a “Locate Landmarks” button. Selecting this button automatically locates the six landmarks on every 3D image at every time point. A list of the located landmark is visible on the right panel. Selecting the landmark name brings the focus of the image to the predicted landmark location, and the user is allowed to make any modifications deemed necessary. Once the user is satisfied, the user can select a “standard views” button which creates the standard 2, 3, 4 chamber and SAX views of the heart”).
Regarding claims 12–14, 16–19, and 21–22, although varying in scope, the limitations of claims 12–14, 16–19, and 21–22 are substantially the same as the limitations of claims 1–3, 5–8, and 10–11, respectively. Thus, claims 12–14, 16–19, and 21–22 are rejected using the same reasoning and analysis as claims 1–3, 5–8, and 10–11 above, respectively.
Claims 4, 9, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Golden in view of Gardner and Chakravarthy, and further in view of Pedalty et al., (US 11617533 B2), hereinafter “Pedalty”.
Regarding claim 4, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 3.
Pedalty teaches:
wherein the time series data comprises at least an intracardiac electrogram (IEGM) signal (Pedalty Col. 15, lines 34–46: “Computing device 400 may use class activation mapping to identify regions of an input time series, e.g., of cardiac EGM data, that constitute the reason for the time series being given a particular classification by the machine learning model of machine learning system 450. A class activation map for a given classification may be a univariate time series where each element (e.g., at each timestamp at the sampling frequency of the input time series) may be a weighted sum or other value derived from the outputs of an intermediate layer of a neural network or other machine learning model. The intermediate layer may be a global average pooling layer and/or last layer prior to the output layer neurons for each classification”—[(emphasis added) wherein EGM data is time series data comprises at least an intracardiac electrogram (IEGM) signal]).
The system of Golden, the teachings of Pedalty, and the instant application are analogous art because they pertain to machine learning techniques involving cardiac electrograms and annotations.
It would be obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system of Golden in view of Gardner and Chakravarthy with the teachings of Pedalty to provide time series data in the form of intracardiac electrogram signals. One would be motivated to do so to allow physicians to interact with the system to detect medical issues in the heart (Pedalty Col. 5, line 62 – Col. 6, line 12: “In some examples, computing system 24 includes at least one of a handheld computing device, computer workstation, server or other networked computing device, smartphone, tablet, or external programmer that includes a user interface for presenting information to and receiving input from a user. In some examples, computing system 24 may include one or more devices that implement a machine learning system 150, such as neural network, a deep learning system, or other type of predictive analytics system. A user, such as a physician, technician, surgeon, electro-physiologist, or other clinician, may interact with computing system 24 to retrieve physiological or diagnostic information from IMD 10. A user may also interact with computing system 24 to program IMD 10, e.g., select values for operational parameters of the IMD. Computing system 24 may include a processor configured to evaluate EGM and/or other sensed signals transmitted from IMD 10 to computing system 24”; see also Pedalty Col. 4, lines 45–58: “Machine learning systems, such as deep learning and artificial intelligence (AI), that perform arrythmia detection provide a flexible platform to develop algorithms with different objectives. For example, machine learning systems may be used to detect atrial fibrillation (AF) segments, detect the presence of AF, or detect other types of cardiac arrythmia. Further, such machine learning systems may be implemented without expert design and feature engineering that may be required by other techniques, such as feature delineation. However, the conclusions drawn by such machine learning systems, as well as the data drawn upon to make such conclusions, may be difficult to explain, hindering evaluation of the performance of the machine learning system”).
Regarding claim 9, Golden in view of Gardner and Chakravarthy teaches all the limitations of claim 7.
Pedalty teaches:
wherein the first visual indicator, the at least a second visual indicator, and at least the third visual indicator are color coded (Pedalty Col. 17, lines 33–50: “In response to determining that the level of confidence is greater than the predetermined threshold (e.g., “YES” block of 510), computing system 24 outputs the cardiac electrogram data for review by a clinician. In some examples, computing system 24 outputs a portion of the cardiac electrogram data, a first indication that the episode of arrythmia has occurred in patient 4, and a second indication of the level of certainty in the determination that the episode of arrythmia has occurred in patient 4 (512). In some examples, computing system 24 selects a visualization method according to the level of confidence by machine learning system 150 that the episode of arrythmia has occurred. For example, computing system 24 may apply color coding to indicate results (e.g., “green” for low confidence than an episode of arrythmia has occurred, “yellow” for medium confidence than an episode of arrythmia has occurred, or “red” for a high confidence than an episode of arrythmia has occurred)”; see also Pedalty Figs. 10–11, Col. 25, lines 20–36: “FIG. 11C depicts display 1103 presenting advanced cardiac information. For example, display 1103 includes cardiac electrogram data 1110 and further depicts, for a plurality of segments of cardiac electrogram data 1110, a likelihood that an episode of cardiac arrythmia of one or more types has occurred. For example, display 1103 colors segments for which computing system 24 has determined a high likelihood that atrial fibrillation has occurred in patient 4 in green (not present in FIG. 11C), segments for which computing system 24 has made an uncertain determination of whether atrial fibrillation has occurred in patient 4 in yellow (not present in FIG. 11C), and segments for which computing system 24 has determined a low likelihood that atrial fibrillation has occurred in patient 4 in red (1116). Furthermore, display 1103 depicts an overall likelihood 1130 that atrial fibrillation has occurred in patient 4 over time in blue”—[(emphasis added)]).
The same motivation used to combine Golden in view of Gardner and Chakravarthy with Pedalty in claim 4 is equally applicable to claim 9.
Regarding claims 15 and 20, although varying in scope, the limitations of claims 15 and 20 are substantially the same as the limitations of claims 4 and 9, respectively. Thus, claims 15 and 20 are rejected using the same reasoning and analysis as claims 4 and 9 above, respectively.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS SHINE whose telephone number is (571)272-2512. The examiner can normally be reached M-F, 11am – 7pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached on (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.B.S./Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights