DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 2-4, 10 & 13-15 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2-4 & 13-15
The Claims of 2-4 & 13-15 list image views with acronyms and parentheses. The acronyms need to be listed out at least once within the claims. The parentheses are confusing because it is unclear of the relationship between the acronyms inside and outside of the acronym. For example, “PSAX (MV cross-section)” is not understood if MV cross-section is a part of the limitation or not. Additionally it is not understood what the acronym IVC stands for. For examination purposes, the Examiner is interpreting the claim as best understood.
Claim 10
Claim 10 recites DICOM in parentheses. It is not understood if DICOM is a claim limitation or not. For examination purposes, the Examiner is interpreting DICOM as required by the claim. Clarification is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 5, 7, 11-12, 16 & 18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Syeda-Mahmood et al. (U.S. Patent Application 2020/0185084 A1).
Claim 1: Syeda-Mahmood teaches –
A method for providing information regarding an ultrasound image view [determining abnormality using a deep learning formulation to learn automatically the relevant features for the normal/abnormal discrimination] (Para 0018), which is implemented by a processor [in a data processing system comprising at least one processor and at least one memory] (Para 0005), comprising:
receiving [receive a medical image] (Para 0039) an ultrasound image of an object [echocardiography, i.e. the ultrasound study of the heart] (Para 0016); and
classifying respective image views on the basis of the received ultrasound image, by using an image view classification model trained to classify a plurality of image views by using the ultrasound image as an input [processing of a medical image (e.g., echocardiogram) via the trained classifier comprising the trained deep learning neural network, recognition of the viewpoint, mode, and cardiac cycle phase, e.g., end-systole or end-diastole, is performed using the deep learning neural network] (Para 0022) and output respective image views [medical image viewer application of the cognitive system 190 may be automatically controlled by the cognitive system 190 to output the medical images of patients] (Para 0054),
wherein the respective image views indicate the ultrasound image views in a plurality of different ultrasound modes [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039) [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039).
Claim 5/1: Syeda-Mahmood teaches wherein the plurality of ultrasound modes includes at least one of an M-mode, a Doppler mode, and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039) and a non-classified mode defined as an ultrasound image view which is different from the M-mode, the Doppler mode, and the B-mode [different modes (e.g., A-mode, where a single transducer scans a line through the body with the echoes plotted as a function of depth, or B-mode which displays the acoustic impedance of a two-dimensional cross-section of tissue)] (Para 0016).
Examiner’s Note: Where A-mode is different from B-mode, Doppler mode and B-mode.
Claim 7/1: Syeda-Mahmood teaches further comprising after the classifying respective image views, verifying the respective image views [The user of the client computing device 210 may…override incorrect classifications, and any of a plethora of other operations that may be performed through human-computer interaction based on the human's viewing of the medical images via the cognitive system 200] (Para 0063).
Claim 11/1: Syeda-Mahmood teaches wherein the image view classification model has output nodes corresponding to the number of the plurality of image views according to the plurality of ultrasound modes [Based on how well the deep learning neural network performs its segmentation and classification operations, the deep learning neural network's operational parameters, e.g., weights associated with nodes of the deep learning neural network and the like, may be adjusted so as to minimize a loss function associated with the deep learning neural network until convergence is reached, e.g., a level of improvement in the operation of the deep learning neural network between epochs is equal to or less than a threshold level of improvement] (Para 0020).
Claim 12: Syeda-Mahmood teaches –
A device [cognitive system] and/or [normality classifier] (Figure 2, Element 100 & 200) for providing information regarding an ultrasound image view, comprising:
a communication unit [communication interfaces] (Para 0061) configured to receive [receive a medical image] (Para 0039) an ultrasound image of an object [echocardiography, i.e. the ultrasound study of the heart] (Para 0016); and
a processor [in a data processing system comprising at least one processor and at least one memory] (Para 0005) operably connected to the communication unit [The cognitive system 200 is implemented on one or more computing devices 204A-D (comprising one or more processors and one or more memories, and potentially any other computing device elements generally known in the art including buses, storage devices, communication interfaces, and the like) connected to the computer network 202] (Para 0061),
wherein the processor is configured to classify respective image views on the basis of the received ultrasound image, by using an image view classification model trained to classify a plurality of image views by using the ultrasound image as an input [processing of a medical image (e.g., echocardiogram) via the trained classifier comprising the trained deep learning neural network, recognition of the viewpoint, mode, and cardiac cycle phase, e.g., end-systole or end-diastole, is performed using the deep learning neural network] (Para 0022) and output respective image views [medical image viewer application of the cognitive system 190 may be automatically controlled by the cognitive system 190 to output the medical images of patients] (Para 0054), and
the respective image views indicate the ultrasound image views in a plurality of different ultrasound modes [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039) [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039).
Claim 16/12: Syeda-Mahmood teaches wherein the plurality of ultrasound modes includes at least one of an M-mode, a Doppler mode, and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039) and a non-classified mode defined as an ultrasound image view which is different from the M-mode, the Doppler mode, and the B-mode [different modes (e.g., A-mode, where a single transducer scans a line through the body with the echoes plotted as a function of depth, or B-mode which displays the acoustic impedance of a two-dimensional cross-section of tissue)] (Para 0016).
Examiner’s Note: Where A-mode is different from B-mode, Doppler mode and B-mode.
Claim 18/12: Syeda-Mahmood teaches wherein the processor is further configured to verify the respective image views [The user of the client computing device 210 may…override incorrect classifications, and any of a plethora of other operations that may be performed through human-computer interaction based on the human's viewing of the medical images via the cognitive system 200] (Para 0063).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 2-4, 6, 13-15 & 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Syeda-Mahmood et al. (U.S. Patent Application 2020/0185084 A1) and further in view of Hare, II et al. (U.S. Patent Application 2020/0226757 A1).
Claim 2/1: Syeda-Mahmood teaches –
wherein the plurality of ultrasound modes includes an M-mode and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039),
the ultrasound image is an echocardiographic image [echocardiography images] (Para 0039), and
the classifying respective image views includes determining
at least one M-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039) and
at least one B-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
for the echocardiographic image, using the image view classification model [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of an M-mode [The technician has the option of adding to these 2D echo images a waveform captured from various possible modalities including: continuous wave Doppler, m-mode, pulsed wave Doppler and pulsed wave tissue Doppler] (Para 0040) (PLAX/PSAX) [PLAX, PSAX] (Para 0080) through LV (Figure 6A; the anatomical picture of a PLAX view with the right ventricle at the top and the left ventricle to the bottom of the right ventricle on the left side) and Hare teaches the specific views of among A2C (Figure 6B) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim 3/1: Syeda-Mahmood teaches –
wherein the plurality of ultrasound modes includes a Doppler mode and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039),
the ultrasound image is an echocardiographic image [echocardiography images] (Para 0039), and
the classifying respective image views includes determining
at least one Doppler mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
at least one B-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
for the echocardiographic image, using the image view classification model [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of a Doppler septal annulus PW TDI (A4C) [A4C + PWTDI (SEPTAL) – A4C Plus Pulse Wave Tissue Doppler on the Septal side] (Figure 6F), and Hare teaches the specific views of among A2C (Figure 6B) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim 4/1: Syeda-Mahmood teaches –
wherein the plurality of ultrasound modes includes a Doppler mode, an M-mode, and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039) and
the ultrasound image is an echocardiographic image [echocardiography images] (Para 0039), and
the classifying respective image views includes determining
at least one Doppler mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
at least one M-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
at least one B-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
for the echocardiographic image, using the image view classification model [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of a Doppler septal annulus PW TDI (A4C) [A4C + PWTDI (SEPTAL) – A4C Plus Pulse Wave Tissue Doppler on the Septal side] (Figure 6F).
Hare II teaches the specific views of an M-mode [The technician has the option of adding to these 2D echo images a waveform captured from various possible modalities including: continuous wave Doppler, m-mode, pulsed wave Doppler and pulsed wave tissue Doppler] (Para 0040) (PLAX/PSAX) [PLAX, PSAX] (Para 0080) through LV (Figure 6A; the anatomical picture of a PLAX view with the right ventricle at the top and the left ventricle to the bottom of the right ventricle on the left side) and Hare teaches the specific views of among A2C (Figure 6B).
Hare teaches the specific views of among A2C (Figure 6B) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim 6/1: Syeda-Mahmood teaches wherein the ultrasound image is an echocardiographic image [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram] (Para 0039), and the classifying respective image views includes: classifying 4-chamber, 2-chamber, long axis image views [(parasternal long axis or 2, 3, 4, 5-chamber view, etc.)] (Para 0039) for the echocardiographic image, using the image view classification model [medical image classification is performed] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of base, mid, and apex (Para 0062, 0079-0080, 0089 & 0097 and Figure 12A) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim 13/12: Syeda-Mahmood teaches –
wherein the plurality of ultrasound modes includes an M-mode and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039),
the ultrasound image is an echocardiographic image [echocardiography images] (Para 0039), and
the classifying respective image views includes determining
at least one M-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039) and
at least one B-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
for the echocardiographic image, using the image view classification model [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of an M-mode [The technician has the option of adding to these 2D echo images a waveform captured from various possible modalities including: continuous wave Doppler, m-mode, pulsed wave Doppler and pulsed wave tissue Doppler] (Para 0040) (PLAX/PSAX) [PLAX, PSAX] (Para 0080) through LV (Figure 6A; the anatomical picture of a PLAX view with the right ventricle at the top and the left ventricle to the bottom of the right ventricle on the left side) and Hare teaches the specific views of among A2C (Figure 6B) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim 14/12: Syeda-Mahmood teaches –
wherein the plurality of ultrasound modes includes a Doppler mode and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039),
the ultrasound image is an echocardiographic image [echocardiography images] (Para 0039), and
the classifying respective image views includes determining
at least one Doppler mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
at least one B-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
for the echocardiographic image, using the image view classification model [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of a Doppler septal annulus PW TDI (A4C) [A4C + PWTDI (SEPTAL) – A4C Plus Pulse Wave Tissue Doppler on the Septal side] (Figure 6F), and Hare teaches the specific views of among A2C (Figure 6B) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim 15/12: Syeda-Mahmood teaches –
wherein the plurality of ultrasound modes includes a Doppler mode, an M-mode, and a B-mode [B-Mode, Doppler, M-Mode, etc] (Para 0039) and
the ultrasound image is an echocardiographic image [echocardiography images] (Para 0039), and
the classifying respective image views includes determining
at least one Doppler mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
at least one M-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
at least one B-mode image view [echocardiography images may be acquired with different modes (B-Mode, Doppler, M-Mode, etc.) and at different viewpoints…the mode recognition component 110 analyzes the medical image to classify the medical image into different modes] (Para 0039),
for the echocardiographic image, using the image view classification model [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram, acquired at a specific cardiac phase (e.g., end-systole or end diastole)] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of a Doppler septal annulus PW TDI (A4C) [A4C + PWTDI (SEPTAL) – A4C Plus Pulse Wave Tissue Doppler on the Septal side] (Figure 6F).
Hare II teaches the specific views of an M-mode [The technician has the option of adding to these 2D echo images a waveform captured from various possible modalities including: continuous wave Doppler, m-mode, pulsed wave Doppler and pulsed wave tissue Doppler] (Para 0040) (PLAX/PSAX) [PLAX, PSAX] (Para 0080) through LV (Figure 6A; the anatomical picture of a PLAX view with the right ventricle at the top and the left ventricle to the bottom of the right ventricle on the left side) and Hare teaches the specific views of among A2C (Figure 6B).
Hare teaches the specific views of among A2C (Figure 6B) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim 17/12: Syeda-Mahmood teaches wherein the ultrasound image is an echocardiographic image [medical image classification is performed on specific viewpoints of a medical image, e.g., a B-Mode sonogram image or echocardiogram] (Para 0039), and the classifying respective image views includes: classifying 4-chamber, 2-chamber, long axis image views [(parasternal long axis or 2, 3, 4, 5-chamber view, etc.)] (Para 0039) for the echocardiographic image, using the image view classification model [medical image classification is performed] (Para 0039).
Syeda-Mahmood teaches various views [parasternal long axis or 2, 3, 4, 5-chamber view, etc.] (Para 0039) in general but fails to claim the specific views as claimed. However, Hare II teaches the specific views of base, mid, and apex (Para 0062, 0079-0080, 0089 & 0097 and Figure 12A) in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to expand the generics views as taught by Syeda-Mahmood to includes the specific views as taught by Hare II in order to be capable of structuring the automated measurements and labelled views across multiple sources of data, to enable training and validation of disease prediction algorithms across multiple remote patient cohorts (Para 0008).
Hare teaches training the classifier with more specific and diverse data set which will improve the accuracy of the classifier and result in more reliable results.
Claim(s) 8-10 & 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Syeda-Mahmood et al. (U.S. Patent Application 2020/0185084 A1) and further in view of Koktava et al. (U.S. Patent Application 2016/0092748 A1).
Claim 10/7/1: Syeda-Mahmood teaches wherein the ultrasound image [echocardiography, i.e. the ultrasound study of the heart] (Para 0016) is an image of digital imaging and communications in medicine (DICOM) format displaying metadata including tagging information regarding the ultrasound mode of the ultrasound image [The medical image data may have associated metadata generated by the equipment and/or computing systems associated with the equipment, to provide further identifiers of characteristics of the medical image, e.g., DICOM tags, metadata specifying mode, viewpoint, or the like] (Para 0066)
Seyda-Mahmood teaches verifying the respective image views and after the verifying, correcting a classification result of the respective image views [The user of the client computing device 210 may…override incorrect classifications] (Para 0063). Seyda-Mahmood fails to teach verifying the respective views on the basis of the metadata when the classified image view is different from the tagging information. However, Koktava teaches verifying the respective views on the basis of the metadata [process of FIG. 3 may be used to validate or populate other forms of metadata, for example metadata associated with different types of file other than DICOM, attributes that are not standard DICOM attributes, or additional files that are associated with image data sets 40. Metadata may include tags, labels, records, files or any other suitable form of data that is additional to the image data and may be associated with the image data] (Para 0117) and after verifying, further includes correcting a classification result when the image metadata is different from the tagging information [the user is required to view an image derived from the image data set 40 and to select one input…if the DICOM data elements are correct and a different input…if the DICOM data elements are incorrect. Such embodiments may allow for the user to override the results of the consistency test in cases where the metadata unit 26 has incorrectly determined that the consistency test has been failed] (Para 0085) in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Examiner’s Note: Claim 10 was addressed first because the rejection of Claim 10 assists in understanding the rejection on non-dependent Claims 8-9.
Claim 8/7/1: Seyda-Mahmood fails to teach verifying metadata in determining a still or moving image. However, Koktava teaches wherein the verifying the respective image views includes: determining whether the ultrasound image is a still cut image or a moving image, on the basis of the number of frames of the respective image views [process of FIG. 3 may be used to validate or populate other forms of metadata, for example metadata associated with different types of file other than DICOM, attributes that are not standard DICOM attributes, or additional files that are associated with image data sets 40. Metadata may include tags, labels, records, files or any other suitable form of data that is additional to the image data and may be associated with the image data] (Para 0117) in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Koktava teaches identifying images based on a trained classifier (Para 0035). Koktava teaches validation of images using the metadata (Figure 3, Element 58). Koktava teaches using other metadata, which would obviously include media types (still and moving image). Although Koktava fails to specifically state media type, DICOM is a standard with a finite set of solutions. The Examiner is applying KSR from MPEP § 2143(I)(E); "Obvious To Try" – Choosing From a Finite Number of Identified, Predictable Solutions, With a Reasonable Expectation of Success for the combination of Seyda-Mahmood and Koktava as described below:
(1) a finding that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem;
The Examiner finds that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem. Specifically, Koktava acknowledges the need for validating classifier accuracy in order to avoid errors in diagnosing patients (Para 0009). Koktava teaches the solution to the problem of validating classifier data by verifying DICOM metadata (Para 0117-0118).
(2) a finding that there had been a finite number of identified, predictable potential solutions to the recognized need or problem;
The Examiner finds that there had been a finite number of identified, predictable potential solutions to the recognized need or problem. DICOM has a limited number of solutions within the standard. Media type (image or video) being one of the solutions within the DICOM standard (see attached1).
(3) a finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success; and
The Examiner finds that one of ordinary skill in the could have pursued the known potential solutions of media type with a reasonable expectation of success. The Examiner contends that one of ordinary skill in the art could have pursued validation of the classification not just in media type but other metadata types as well.
(4) whatever additional findings based on the Graham factual inquiries may be necessary, in view of the facts of the case under consideration, to explain a conclusion of obviousness.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency media type checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Claim 9/7/1: Seyda-Mahmood fails to teach determining whether the respective image views are color images or monochromic images. However, Koktava teaches determining whether the respective images are color images or monochromic images [process of FIG. 3 may be used to validate or populate other forms of metadata, for example metadata associated with different types of file other than DICOM, attributes that are not standard DICOM attributes, or additional files that are associated with image data sets 40. Metadata may include tags, labels, records, files or any other suitable form of data that is additional to the image data and may be associated with the image data] (Para 0117) in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Koktava teaches identifying images based on a trained classifier (Para 0035). Koktava teaches validation of images using the metadata (Figure 3, Element 58). Koktava teaches using other metadata, which would obviously include media types (still and moving image). Although Koktava fails to specifically state media type, DICOM is a standard with a finite set of solutions. The Examiner is applying KSR from MPEP § 2143(I)(E); "Obvious To Try" – Choosing From a Finite Number of Identified, Predictable Solutions, With a Reasonable Expectation of Success for the combination of Seyda-Mahmood and Koktava as described below:
(1) a finding that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem;
The Examiner finds that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem. Specifically, Koktava acknowledges the need for validating classifier accuracy in order to avoid errors in diagnosing patients (Para 0009). Koktava teaches the solution to the problem of validating classifier data by verifying DICOM metadata (Para 0117-0118).
(2) a finding that there had been a finite number of identified, predictable potential solutions to the recognized need or problem;
The Examiner finds that there had been a finite number of identified, predictable potential solutions to the recognized need or problem. DICOM has a limited number of solutions within the standard. Image Pixel Module (color or monochrome) being one of the solutions within the DICOM standard (see attached2).
(3) a finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success; and
The Examiner finds that one of ordinary skill in the could have pursued the known potential solutions of media type with a reasonable expectation of success. The Examiner contends that one of ordinary skill in the art could have pursued validation of the classification not just in Image Pixel Module but other metadata types as well.
(4) whatever additional findings based on the Graham factual inquiries may be necessary, in view of the facts of the case under consideration, to explain a conclusion of obviousness.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency image pixel module checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Claim 19/18/12: Seyda-Mahmood fails to teach wherein the processor is further configured to determine whether the ultrasound image is a still cut image or a moving image, on the basis of the number of frames of the respective image views. However, Koktava teaches wherein the processor is further configured to determine whether the ultrasound image [ultrasound data] (Para 0141) is a still cut image or a moving image, on the basis of the number of frames of the respective image views [process of FIG. 3 may be used to validate or populate other forms of metadata, for example metadata associated with different types of file other than DICOM, attributes that are not standard DICOM attributes, or additional files that are associated with image data sets 40. Metadata may include tags, labels, records, files or any other suitable form of data that is additional to the image data and may be associated with the image data] (Para 0117) in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Koktava teaches identifying images based on a trained classifier (Para 0035). Koktava teaches validation of images using the metadata (Figure 3, Element 58). Koktava teaches using other metadata, which would obviously include media types (still and moving image). Although Koktava fails to specifically state media type, DICOM is a standard with a finite set of solutions. The Examiner is applying KSR from MPEP § 2143(I)(E); "Obvious To Try" – Choosing From a Finite Number of Identified, Predictable Solutions, With a Reasonable Expectation of Success for the combination of Seyda-Mahmood and Koktava as described below:
(1) a finding that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem;
The Examiner finds that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem. Specifically, Koktava acknowledges the need for validating classifier accuracy in order to avoid errors in diagnosing patients (Para 0009). Koktava teaches the solution to the problem of validating classifier data by verifying DICOM metadata (Para 0117-0118).
(2) a finding that there had been a finite number of identified, predictable potential solutions to the recognized need or problem;
The Examiner finds that there had been a finite number of identified, predictable potential solutions to the recognized need or problem. DICOM has a limited number of solutions within the standard. Media type (image or video) being one of the solutions within the DICOM standard (see attached).
(3) a finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success; and
The Examiner finds that one of ordinary skill in the could have pursued the known potential solutions of media type with a reasonable expectation of success. The Examiner contends that one of ordinary skill in the art could have pursued validation of the classification not just in media type but other metadata types as well.
(4) whatever additional findings based on the Graham factual inquiries may be necessary, in view of the facts of the case under consideration, to explain a conclusion of obviousness.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency media type checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Claim 20/18/12: Seyda-Mahmood fails to teach wherein the processor is further configured to determine whether the respective image views are color images or monochromic images. However, Koktava teaches wherein the processor is further configured to determine whether the respective image views are color images or monochromic images [process of FIG. 3 may be used to validate or populate other forms of metadata, for example metadata associated with different types of file other than DICOM, attributes that are not standard DICOM attributes, or additional files that are associated with image data sets 40. Metadata may include tags, labels, records, files or any other suitable form of data that is additional to the image data and may be associated with the image data] (Para 0117) in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Koktava teaches identifying images based on a trained classifier (Para 0035). Koktava teaches validation of images using the metadata (Figure 3, Element 58). Koktava teaches using other metadata, which would obviously include media types (still and moving image). Although Koktava fails to specifically state media type, DICOM is a standard with a finite set of solutions. The Examiner is applying KSR from MPEP § 2143(I)(E); "Obvious To Try" – Choosing From a Finite Number of Identified, Predictable Solutions, With a Reasonable Expectation of Success for the combination of Seyda-Mahmood and Koktava as described below:
(1) a finding that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem;
The Examiner finds that at the relevant time, there had been a recognized problem or need in the art, which may include a design need or market pressure to solve a problem. Specifically, Koktava acknowledges the need for validating classifier accuracy in order to avoid errors in diagnosing patients (Para 0009). Koktava teaches the solution to the problem of validating classifier data by verifying DICOM metadata (Para 0117-0118).
(2) a finding that there had been a finite number of identified, predictable potential solutions to the recognized need or problem;
The Examiner finds that there had been a finite number of identified, predictable potential solutions to the recognized need or problem. DICOM has a limited number of solutions within the standard. Image Pixel Module (color or monochrome) being one of the solutions within the DICOM standard (see attached3).
(3) a finding that one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success; and
The Examiner finds that one of ordinary skill in the could have pursued the known potential solutions of media type with a reasonable expectation of success. The Examiner contends that one of ordinary skill in the art could have pursued validation of the classification not just in Image Pixel Module but other metadata types as well.
(4) whatever additional findings based on the Graham factual inquiries may be necessary, in view of the facts of the case under consideration, to explain a conclusion of obviousness.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the correcting method of Seyda-Mahmood to include the metadata consistency image pixel module checks as taught by Koktava in order to avoid potential issues that incorrect DICOM data or other metadata may lead to errors in diagnosis (Para 0009).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Krishnan et al. (U.S. Patent Application 2005/0020903 A1) – Krishnan teaches a method involves obtaining an information from an image data of a patient`s heart. Another information is obtained from non-image data records of the patient. A condition of the heart is accessed using the information. An automated wall motion analysis is performed to obtain information related to regional myocardial function of the heart, using image features. A confidence of a probability of diagnosis is determined.
Aase et al. (U.S. Patent Application 2019/0076127 A1) – Aase teaches a system and method for automatically selecting ultrasound image loops from a continuously captured stress echocardiogram is provided. The method may include continuously capturing ultrasound image data of a heart. The method may include separating the continuously captured ultrasound image data into image loops. Each of the image loops may have a predetermined number of heart cycles. The method may include automatically assigning an image view type from to at least a portion of the image loops. The method may include automatically assigning an image characteristic metric to each of the image loops having the assigned image view type. The method may include automatically presenting, at a display system, an image loop for each of the image view types based on the image characteristic metric.
Beymer et al. (U.S. Patent Application 2013/0011033 A1) – Beymer teaches method involves receiving a spatial and temporal model for a known transducer viewpoint from sample learning data. A spatial and temporal model fit is evaluated using a combined fit of a motion model, and appearance of heart-cycle variations is received using heart-cycle variations for isolated features using spatial and temporal models from a sample learning set. A matching model is determined using a matching algorithm for recognizing cardiac echo view from the appearance of the heart cycle variations.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HELENE C BOR whose telephone number is (571)272-2947. The examiner can normally be reached Mon - Fri 10:30 - 6:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached at (571) 272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Helene Bor/Examiner, Art Unit 3797
/JOSEPH M SANTOS RODRIGUEZ/Primary Examiner, Art Unit 3797
1 8.7.3 DICOM Media Type Sets https://dicom.nema.org/medical/dicom/current/output/chtml/part18/sect_8.7.3.html
2 C.7.6.3 Image Pixel Module - https://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.3.html
3 C.7.6.3 Image Pixel Module - https://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.3.html
Prosecution Insights