DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 is objected to because of the following informalities: the limitation implementing two or more LSTM network layers with a dropout should instead read implementing two or more Long Short-Term Memory (LSTM) network layers with a dropout. Appropriate correction is required.
Claim Rejections - 35 USC § 112: Indefiniteness
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 8 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 8, the claim recites the limitation wherein the spectral features comprises at least one of selected from a chroma, a mel-spectrogram. There is insufficient antecedent basis for this limitation in the claim because the term “the spectral feature” lacks antecedent basis. For the purposes of examination, the spectral features are interpreted as the MFCC features.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, in step 1 of the 101 analysis set forth in MPEP 2106, the claim recites A method of constructing long and short-range dependency network learning model. The claim recites a method. A method is one of the four statutory categories of invention.
In Step 2A, Prong 1 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following limitations recite a process that, under broadest reasonable interpretation, covers a mental process or mathematical concept but for the recitation of generic computer components:
preprocessing the plurality of audio files; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with pen and paper like selecting which audio files to use, which is either a mental process of observation/evaluation/judgement (MPEP 2106)).
restructuring extracted MFCCs into an input layer of the long and short-range dependency network learning model; (i.e., the broadest reasonable interpretation includes a step of observation, evaluation, and judgement and could be performed mentally or with pen and paper like determining the compatible data format the MFCC needs to be for the model, which is either a mental process of observation/evaluation/judgement (MPEP 2106)).
and employing a softmax activation function to a final output layer. (i.e., the broadest reasonable interpretation includes mathematical calculation like performing a softmax calculation, a mathematical calculation is considered a mathematical concept (MPEP 2106)).
If the claim limitations, under their broadest reasonable interpretation, covers activities classified under Mental processes: concepts performed in the human mind (including observation, evaluation, judgement, or opinion) (see MPEP 2106.04(a)(2), subsection (III)) or Mathematical concepts: mathematical relationships, mathematical formulas or equations, or mathematical calculations (see MPEP 2106.04(a)(2), subsection (I)). Accordingly, the claim recites an abstract idea.
In Step 2A, Prong 2 of the 101 analysis, set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
the method comprising: obtaining heart sound data comprising a plurality of audio files; (i.e., the broadest reasonable interpretation of receiving a data instance is mere data gathering, which is an insignificant extra solution activity (MPEP 2106.05(g))).
extracting Mel-Frequency Cepstral Coefficients (MFCCs) from the preprocessed plurality of audio files; (i.e., the broadest reasonable interpretation of receiving a data instance is mere data gathering, which is an insignificant extra solution activity (MPEP 2106.05(g))).
implementing two or more LSTM network layers with a dropout; (i.e., the generic computer components recited in this limitation merely add the words “apply it”, or an equivalent, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f))).
Since the claim does not contain any other additional elements, that amount to integration into a practical application, the claim is directed to an abstract idea.
In Step 2B of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception:
Regarding limitation(s) (IV) and (V), under the broadest reasonable interpretation, recite steps of mere data gathering, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018)).
Examiner uses Berkheimer: Option 2, a citation to one or more of the court decisions discussed in MPEP 2106.05(d)(II) as noting well-understood, routine, and conventional nature of the additional elements:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
Further, limitation (VI), under the broadest reasonable interpretation, merely recite steps that apply a generic LSTM with dropout, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Considering additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 2, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 2 recites wherein the preprocessing comprises at least one of: sampling the plurality of audio files at a first frequency range; segmenting the plurality of audio files into compressed frames; encoding the segmented audio files into a numerical format. Under the broadest reasonable interpretation, the limitations recite selecting audio files based on their frequency which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 2 does not solve the deficiencies of claim 1.
Regarding claim 3, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 3 recites wherein the preprocessing further comprises arranging a consistent duration for each audio file from the plurality of audio files. Under the broadest reasonable interpretation, the limitations recite ensuring that audio files have a consistent duration which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 3 does not solve the deficiencies of claim 1.
Regarding claim 4, it is dependent upon claim 2 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 4 recites wherein the first frequency range is from 20,000 up to 250,000 Hz. Under the broadest reasonable interpretation, the limitations recite selecting audio files based on their frequency range from 20khz to 250khz which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 4 does not solve the deficiencies of claim 2.
Regarding claim 5, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 5 recites wherein the heart sound data is obtained from a database. Under the broadest reasonable interpretation, the limitations recite steps of mere data gathering, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity (MPEP 2106.05(g)). Therefore, claim 5 does not solve the deficiencies of claim 1.
Regarding claim 6, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 6 recites wherein the heart sound data is obtained in real-time. Under the broadest reasonable interpretation, the limitations recite steps of mere data gathering, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity (MPEP 2106.05(g)). Therefore, claim 6 does not solve the deficiencies of claim 1.
Regarding claim 7, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 7 recites wherein MFCCs extraction comprises at least one of the following: computing 25 MFCCs; calculating a mean of MFCCs over a time axis; computing spectral features of MFCCs. Under the broadest reasonable interpretation, the limitations recite calculating a mean of MFCCs which is interpreted as using a mathematical calculation. A mathematical calculation is interpreted as a mathematical concept. Therefore, claim 7 does not solve the deficiencies of claim 1.
Regarding claim 8, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 8 recites wherein the spectral features comprises at least one of selected from a chroma, a mel-spectrogram. Under the broadest reasonable interpretation, the limitations recite steps of mere data gathering, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity (MPEP 2106.05(g)). Therefore, claim 8 does not solve the deficiencies of claim 1.
Regarding claim 9, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 9 recites wherein the long and short-range dependency network learning model is a transformer model. Under the broadest reasonable interpretation, the limitations merely recite steps that apply a generic transformer model, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Therefore, claim 9 does not solve the deficiencies of claim 1.
Regarding claim 10, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 10 recites wherein the long and short-range dependency network learning model is a Long Short-Term Memory (LSTM) network. Under the broadest reasonable interpretation, the limitations merely recite steps that apply a generic transformer model, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Therefore, claim 10 does not solve the deficiencies of claim 1.
Regarding claim 11, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 11 recites wherein the two or more LSTM network layers are further implemented with a recurrent dropout. Under the broadest reasonable interpretation, the limitations merely recite steps that apply a generic recurrent dropout, which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Therefore, claim 11 does not solve the deficiencies of claim 1.
Regarding claim 12, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 12 recites further comprising: wrapping the two or more LSTM network layers into two or more bidirectional LSTM network layers; and applying plurality of dense layers with Rectified Linear Unit (ReLU) activation to bidirectional LSTM network layers. Under the broadest reasonable interpretation, the limitations merely recite steps that apply a generic Bidirectional LSTM, dense layers, and ReLU which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Therefore, claim 12 does not solve the deficiencies of claim 1.
Regarding claim 13, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 13 recites further comprising optimizing the constructed long and short-range dependency network learning model with an Adam optimizer. Under the broadest reasonable interpretation, the limitations merely recite steps that apply a generic Adam optimizer which represents merely adding the words “apply it”, or an equivalent, which are not indicative of an inventive concept (MPEP 2106.05(f)). Therefore, claim 13 does not solve the deficiencies of claim 1.
Regarding claim 14, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 14 recites further comprising calculating a loss using categorical cross-entropy. Under the broadest reasonable interpretation, the limitations recite calculating a categorical cross-entropy loss which is interpreted as using a mathematical calculation. A mathematical calculation is interpreted as a mathematical concept. Therefore, claim 14 does not solve the deficiencies of claim 1.
Regarding claim 15, it is dependent upon claim 1 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 15 recites The method of classifying heart rate sound data, the method comprising: capturing from a patient heart rate sounds to be classified; inputting the captured heart rate sounds to the input layer of the constructed long and short-range dependency network learning model;. Under the broadest reasonable interpretation, the limitations recite steps of mere data gathering, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity (MPEP 2106.05(g)). Claim 15 also recites and using output of the final output layer of the constructed long and short-range dependency network learning model as an indication of classification of the heart rate sounds;. Under the broadest reasonable interpretation, the limitations recite steps of mere data outputting, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data outputting as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity (MPEP 2106.05(g)). Therefore, claim 15 does not solve the deficiencies of claim 1.
Regarding claim 16, it is dependent upon claim 15 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 16 recites wherein the heart sound data is classified to at least one category selected from: normal, murmur, extrasystole, and artifact. Under the broadest reasonable interpretation, the limitations recite determining whether a sample is a certain class which is a step of observation, evaluation, and judgement which can be performed mentally or with pen and paper. The steps of observation, evaluation, and judgement are mental processes. Therefore, claim 16 does not solve the deficiencies of claim 15.
Regarding claim 17, it is dependent upon claim 15 and fails to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. For example, claim 17 recites wherein the heart rate sounds are captured from the patient in real-time. Under the broadest reasonable interpretation, the limitations recite steps of mere data gathering, which has been recognized by the courts as being well-understood, routine, and conventional functions. Specifically, the courts have recognized computer functions directed to mere data gathering as well-understood, routine, and conventional functions when they are claimed in a merely generic manner or as insignificant extra-solution activity (MPEP 2106.05(g)). Therefore, claim 17 does not solve the deficiencies of claim 15.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-7, and 10-17 are rejected under 35 U.S.C. 103 as being unpatentable over Padhy, et al., Non-Patent Literature “Heart Sound Classification Based on MFCC Feature Extraction and Long-Short Term Neural Networks” (“Padhy”) in view of Moon, et al., Non-Patent Literature “RNNDROP: A novel dropout for RNNS in ASR” (“Moon”) and further in view of Megalmani, et al., Non-Patent Literature “Unsegmented Heart Sound Classification Using Hybrid CNN-LSTM Neural Networks” (“Megalmani”).
Regarding claim 1, Padhy discloses:
A method of constructing long and short-range dependency network learning model, (Padhy, pg. 1 col. 2, “The LSTM network (Long-Short Term Memory) [7], an evolved iteration of DRNNs, is designed to adeptly handle sequential data by effectively capturing extensive relationships over extended ranges [A method of constructing long and short-range dependency network learning model,].”).
the method comprising: obtaining heart sound data comprising a plurality of audio files; (Padhy, pg. 1 col. 2, “Precisely, PCG captures the complete set of cardiac sounds throughout one cycle of the heart’s activity [the method comprising: obtaining heart sound data comprising a plurality of audio files;].”).
preprocessing the plurality of audio files; (Padhy, pg. 3 col. 1 and Figure 1, “During training, the initial cardiac sound signal undergoes preprocessing to eliminate artifacts of low-frequency, high frequency interference and baseline wandering [preprocessing the plurality of audio files;].”).
extracting Mel-Frequency Cepstral Coefficients (MFCCs) from the preprocessed plurality of audio files; restructuring extracted MFCCs into an input layer of the long and short-range dependency network learning model; (Padhy, pg. 3 col. 1 and Figure 1, “The training database is built by extracting standard MFCC features from the sound signals [extracting Mel-Frequency Cepstral Coefficients (MFCCs) from the preprocessed plurality of audio files;]. These labeled features based on MFCC serve as reference samples for training and learning the LSTM classifier [restructuring extracted MFCCs into an input layer of the long and short-range dependency network learning model;].”).
While Padhy teaches a system that classifies heart sounds using MFCCs and an LTSM, Padhy does not explicitly teach:
implementing two or more LSTM network layers with a dropout;
and employing a softmax activation function to a final output layer.
Moon teaches implementing two or more LSTM network layers with a dropout; (Moon, abstract, “Our experiments show that rnnDrop is a better regularization method than others including weight noise injection. Namely, when deep bidirectional long short-term memory (LSTM) RNNs were trained with rnnDrop as acoustic models for phoneme and speech recognition [LSTM network layers with a dropout;]”, and Moon, pg. 68 col. 1, “The DBLSTM networks we used had 5 layers, each of which contains 250 LSTM memory blocks [implementing two or more LTSM network layers]”).
Padhy and Moon are both in the same field of endeavor (i.e. LTSM). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy and Moon to teach the above limitation(s). The motivation for doing so is that using dropout improves an LTSM’s performance by avoiding co-adaption and forcing the LTSM to learn temporal dependencies (cf. Moon, pg. 65 col. 2, “The proposed method is a better way of applying dropout to RNNs, since it can learn temporal dependencies avoiding co-adaptation, which leads to better performances.”).
While Padhy in view of Moon teaches a system that classifies heart sounds using MFCCs and an LTSM with dropout, the combination does not explicitly teach:
and employing a softmax activation function to a final output layer.
Megalmani teaches and employing a softmax activation function to a final output layer. (Megalmani, pg. 715 col. 1, “The last fully connected layer utilizes a softmax activation to obtain class probabilities [and employing a softmax activation function to a final output layer.].”).
Padhy, in view of Moon, and Megalmani are both in the same field of endeavor (i.e. heart sound classification). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, in view of Moon, and Megalmani to teach the above limitation(s). The motivation for doing so is that using a softmax activation at the last layer allows a model to output class probabilities for classification.
Regarding claim 2, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Padhy further teaches wherein the preprocessing comprises at least one of: sampling the plurality of audio files at a first frequency range; segmenting the plurality of audio files into compressed frames; encoding the segmented audio files into a numerical format. (Padhy, pg. 3 col. 2, “The fundamental step of the classification algorithm is the preprocessing of the heart sound signals. This involves applying a fifth-order Butterworth bandpass filter with a passband ranging from 25 to 400 Hz [wherein the preprocessing comprises at least one of: sampling the plurality of audio files at a first frequency range;].”).
Regarding claim 3, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Megalmani further teaches wherein the preprocessing further comprises arranging a consistent duration for each audio file from the plurality of audio files. (Megalmani, pg. 715 col. 2, “The dataset consists of audio files with varying durations, with set b having recordings only up to 8 seconds long. To maintain a fixed duration of audio files to train and evaluate all the models, we considered audio files greater than 10s [wherein the preprocessing further comprises arranging a consistent duration for each audio file from the plurality of audio files.].”).
Padhy, in view of Moon, and Megalmani are all in the same field of endeavor (i.e. heart sound classification). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, in view of Moon, and Megalmani to teach the above limitation(s). The motivation for doing so is that using a consistent duration limits the number of external factors when evaluating a model (cf. Megalmani, pg. 715 col. 2, “The dataset consists of audio files with varying durations, with set b having recordings only up to 8 seconds long. To maintain a fixed duration of audio files to train and evaluate all the models, we considered audio files greater than 10s”).
Regarding claim 5, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Padhy further teaches wherein the heart sound data is obtained from a database. (Padhy, pg. 3 col. 2, “This database encompasses nine distinct subdatabases contributed by different research teams. It includes 3,240 real sound recordings of the human heart that were collected from both patients having various cadiac diseases and healthy people [wherein the heart sound data is obtained from a database.].”).
Regarding claim 6, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Padhy further teaches wherein the heart sound data is obtained in real-time. (Padhy, pg. 3 col. 2, “This database encompasses nine distinct subdatabases contributed by different research teams. It includes 3,240 real sound recordings of the human heart that were collected from both patients having various cadiac diseases and healthy people [wherein the heart sound data is obtained in real-time.].”).
Regarding claim 7, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Padhy further teaches wherein MFCCs extraction comprises at least one of the following: computing 25 MFCCs; calculating a mean of MFCCs over a time axis; computing spectral features of MFCCs. (Padhy, pg. 2 col. 1, “Employing Mel-frequency cepstrum analysis on the cardiac sound signal yields a potent feature for detecting cardiac disorders [computing spectral features of MFCCs.].”).
Regarding claim 10, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Padhy further teaches wherein the long and short-range dependency network learning model is a Long Short-Term Memory (LSTM) network. (Padhy, abstract, “This study suggests an enhanced Mel-Frequency Cep strum Coefficient (MFCC) feature-based technique for classifying heart sounds and a Long-Short Term Memory neural network (LSTM) [wherein the long and short-range dependency network learning model is a Long Short-Term Memory (LSTM) network.].”).
Regarding claim 11, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Moon further teaches wherein the two or more LSTM network layers are further implemented with a recurrent dropout. (Moon, abstract, “Our experiments show that rnnDrop is a better regularization method than others including weight noise injection. Namely, when deep bidirectional long short-term memory (LSTM) RNNs were trained with rnnDrop as acoustic models for phoneme and speech recognition [wherein the two or more LSTM network layers are further implemented with a recurrent dropout.]”).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the present application to combine the teachings of Moon with the teachings of Padhy and Megalmani for the same reasons disclosed in claim 1.
Regarding claim 12, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Moon further teaches further comprising: wrapping the two or more LSTM network layers into two or more bidirectional LSTM network layers; (Moon, pg. 66 col. 2, “One can stack multiple LSTM layers to make the network structure deep, and combine two separate LSTM networks that run in forward and backward directions to implement bidirectional architecture [further comprising: wrapping the two or more LSTM network layers into two or more bidirectional LSTM network layers;].”).
Padhy, Megalmani, and Moon are all in the same field of endeavor (i.e. LTSM). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, Megalmani, and Moon to teach the above limitation(s). The motivation for doing so is that using a bidirectional LSTM improves a LSTM’s ability to interpret longer term dependencies (cf. Moon, pg. 65 col. 1-2, “deep bidirectional long short-term memory (DBLSTM) networks have been drawing much attention because of their ability to model long-term dependencies and shown state-of-the-art performances in several applications”).
Moon teaches the bidirectional LSTM network layers as shown above and Megalmani further teaches and applying plurality of dense layers with Rectified Linear Unit (ReLU) activation to bidirectional LSTM network layers. (Megalmani, pg. 715 see Table I, Table I shows that the LSTM layers are connected to dense layers with ReLU activation functions (i.e. and applying plurality of dense layers with Rectified Linear Unit (ReLU) activation to bidirectional LSTM network layers.)).
Padhy, Moon, and Megalmani are all in the same field of endeavor (i.e. LTSM). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, Moon, and Megalmani to teach the above limitation(s). The motivation for doing so is that using dense layers with ReLU activations allows a model to learn non-linear relationships.
Regarding claim 13, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Megalmani further teaches further comprising optimizing the constructed long and short-range dependency network learning model with an Adam optimizer. (Megalmani pg. 716 col. 1 and see Table I, “The objective function of both the networks are optimized with the Adam optimizer [further comprising optimizing the constructed long and short-range dependency network learning model with an Adam optimizer.]”).
Padhy, Moon, and Megalmani are all in the same field of endeavor (i.e. LTSM). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, Moon, and Megalmani to teach the above limitation(s). The motivation for doing so is that using an Adam optimizer improves the training speed.
Regarding claim 14, Padhy in view of Moon and Megalmani teaches the method according to claim 1. Megalmani further teaches further comprising calculating a loss using categorical cross-entropy. (Megalmani pg. 716 col. 1 and see Table I, “The categorical cross entropy function is used to train both the models [further comprising calculating a loss using categorical cross-entropy.].”).
Padhy, in view of Moon, and Megalmani are all in the same field of endeavor (i.e. heart sound classification). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, Moon, and Megalmani to teach the above limitation(s). The motivation for doing so is that using categorical cross-entropy allows learning on multiple classes.
Regarding claim 15, Padhy in view of Moon and Megalmani teaches wherein the long and short-range dependency network learning model is constructed according to claim 1. Padhy further teaches:
The method of classifying heart rate sound data, (Padhy, abstract, “The classification of heart sounds is of utmost importance in promptly identifying cardiovascular disorders [The method of classifying heart rate sound data,]”).
the method comprising: capturing from a patient heart rate sounds to be classified; (Padhy, pg. 1 col. 2, “Precisely, PCG captures the complete set of cardiac sounds throughout one cycle of the heart’s activity [the method comprising: capturing from a patient heart rate sounds to be classified;].”).
inputting the captured heart rate sounds to the input layer of the constructed long and short-range dependency network learning model; (Padhy, pg. 1 col. 2, “This feature gives LSTM an edge in effectively understanding extended sequences that involve significant time gaps [8]. Hence, the LSTM is highly appropriate for effectively segmenting heart sounds [inputting the captured heart rate sounds to the input layer of the constructed long and short-range dependency network learning model;].”).
and using output of the final output layer of the constructed long and short-range dependency network learning model as an indication of classification of the heart rate sounds; (Padhy, abstract and Figure 1, “This study suggests an enhanced Mel-Frequency Cep strum Coefficient (MFCC) feature-based technique for classifying heart sounds and a Long-Short Term Memory neural network (LSTM). The neural network receives MFCC-based features to perform feature learning, followed by the classification task [and using output of the final output layer of the constructed long and short-range dependency network learning model as an indication of classification of the heart rate sounds;].”).
Regarding claim 16, Padhy in view of Moon and Megalmani teaches the method according to claim 15. Padhy further teaches wherein the heart sound data is classified to at least one category selected from: normal, murmur, extrasystole, and artifact. (Padhy, pg. 3 see Figure 3, Figure 3 shows that the heard sound samples are classified into 5 different classes that include normal, murmur, extrasystole, and artifact (i.e. wherein the heart sound data is classified to at least one category selected from: normal, murmur, extrasystole, and artifact.)).
Regarding claim 17, Padhy in view of Moon and Megalmani teaches the method according to claim 15. Padhy further teaches wherein the heart rate sounds are captured from the patient in real-time. (Padhy, pg. 3 col. 2, “This database encompasses nine distinct subdatabases contributed by different research teams. It includes 3,240 real sound recordings of the human heart that were collected from both patients having various cadiac diseases and healthy people [wherein the heart rate sounds are captured from the patient in real-time.].”).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Padhy, et al., Non-Patent Literature “Heart Sound Classification Based on MFCC Feature Extraction and Long-Short Term Neural Networks” (“Padhy”) in view of Moon, et al., Non-Patent Literature “RNNDROP: A novel dropout for RNNS in ASR” (“Moon”) and further in view of Megalmani, et al., Non-Patent Literature “Unsegmented Heart Sound Classification Using Hybrid CNN-LSTM Neural Networks” (“Megalmani”) and Bashar, et al., Non-Patent Literature “Heart Abnormality Classification Using Phonocardiogram (PCG) Signals” (“Bashar”).
Regarding claim 4, Padhy in view of Moon and Megalmani teaches the method according to claim 2. While the combination teaches the first frequency range, the combination does not explicitly teach wherein the first frequency range is from 20,000 up to 250,000 Hz.
Bashar teaches wherein the first frequency range is from 20,000 up to 250,000 Hz. (Bashar, pg. 337 col. 1, “Original dataset consists of four classes with 26 normal sound, 30 murmur sound, 13 Extra heart sound, and 40 artifacts. In this study, only normal and murmur signals have been considered. These signals were collected through mobile phone with sampling frequency of 44.1 KHz [wherein the first frequency range is from 20,000 up to 250,000 Hz].”).
Padhy, in view of Moon and Megalmani, and Bashar are all in the same field of endeavor (i.e. heart sound classification). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, in view of Moon and Megalmani, and Bashar to teach the above limitation(s). The motivation for doing so is that sampling at a higher frequency provides more information to the sounds captured.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Padhy, et al., Non-Patent Literature “Heart Sound Classification Based on MFCC Feature Extraction and Long-Short Term Neural Networks” (“Padhy”) in view of Moon, et al., Non-Patent Literature “RNNDROP: A novel dropout for RNNS in ASR” (“Moon”) and further in view of Megalmani, et al., Non-Patent Literature “Unsegmented Heart Sound Classification Using Hybrid CNN-LSTM Neural Networks” (“Megalmani”) and Nguyen, et al., Non-Patent Literature “Heart Sound Classification Using Deep Learning Techniques Based on Log-mel Spectrogram” (“Nguyen”).
Regarding claim 8, Padhy in view of Moon and Megalmani teaches the method according to claim 1. While the combination teaches using MFCC features, the combination does not explicitly teach wherein the spectral features comprises at least one of selected from a chroma, a mel-spectrogram.
Nguyen teaches wherein the spectral features comprises at least one of selected from a chroma, a mel-spectrogram. (Nguyen, abstract, “In this study, two models for classifying heart rate sounds are proposed to classify heart sound by deep learning techniques based on the log-mel spectrogram of heart sound signals [wherein the spectral features comprises at least one of selected from a chroma, a mel-spectrogram.].”).
Padhy, in view of Moon and Megalmani, and Nguyen are all in the same field of endeavor (i.e. heart sound classification). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, in view of Moon and Megalmani, and Nguyen to teach the above limitation(s). The motivation for doing so is that using mel-spectrogram features imporves the accuracy of heart sound classification (cf. Nguyen, pg. 358, “Two deep learning models, LSTM and CNN, were proposed to classify heart sounds into five classes: Normal, Aortic stenosis, Mitral regurgitation, Mitral stenosis, and Mitral valve prolapse based on the extracted features as a log-mel spectrogram. The results show that both the proposed models have high performance, and the CNN model has higher accuracy which achieved a higher accuracy of 0.9967.”).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Padhy, et al., Non-Patent Literature “Heart Sound Classification Based on MFCC Feature Extraction and Long-Short Term Neural Networks” (“Padhy”) in view of Moon, et al., Non-Patent Literature “RNNDROP: A novel dropout for RNNS in ASR” (“Moon”) and further in view of Megalmani, et al., Non-Patent Literature “Unsegmented Heart Sound Classification Using Hybrid CNN-LSTM Neural Networks” (“Megalmani”) and Yang, et al., Non-Patent Literature “Assisting Heart Valve Diseases Diagnosis via Transformer-Based Classification of Heart Sound Signals” (“Yang”).
Regarding claim 9, Padhy in view of Moon and Megalmani teaches the method according to claim 1. While the combination teaches an LSTM for heart sound classification, the combination does not explicitly teach wherein the long and short-range dependency network learning model is a transformer model.
Yang teaches wherein the long and short-range dependency network learning model is a transformer model. (Yang, abstract, “We utilized a Transformer model for the multi-classification of heart sound signals [wherein the long and short-range dependency network learning model is a transformer model.].”).
Padhy, in view of Moon and Megalmani, and Yang are all in the same field of endeavor (i.e. heart sound classification). It would have been obvious for a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Padhy, in view of Moon and Megalmani, and Yang to teach the above limitation(s). The motivation for doing so is that using a transformer model improves the long-range dependencies between input values (cf. Yang, pg. 14, “one of the main advantages of using self-attention Transformer for HS classification is the improved performance. Self-attention mechanisms allow the model to capture long-range dependencies in the input data, which can be particularly useful for analyzing the sequential data like HSs. On the other hand, another advantage of using self-attention Transformer for HS classification is the efficiency when it processes long sequences of data.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kingma, et al., “ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION” discloses an optimization function named Adam that improves the training speed of gradient-based machine learning models.
Agarap, “Deep Learning using Rectified Linear Units” discloses the use of rectified linear units as activation functions in neural networks.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS S WU whose telephone number is (571)270-0939. The examiner can normally be reached Monday - Friday 8:00 am - 4:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at 571-431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.S.W./Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights