DETAILED ACTION
This communication is in response to the Amendments and Arguments filed on 7/5/2025.
Claims 1, 3, 6-8, 10-11, 14-15 and 19-22 are pending and have been examined.
All previous objections / rejections not mentioned in this Office Action have been withdrawn by the examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Applicant’s arguments filed on 7/5/2025 have been fully considered.
Regarding the Applicant’s arguments for the rejections under 35 U.S.C. § 103, applicant has amended independent claims 1, 8, 11, and 21 as well as dependent claims. The specific written limitations of claim 1, 8, 11, and 21 are different, but contain limitations that are in alignment. Applicant asserts that prior art reference Fernandes or Bocklet does not specifically teach “taking the mean value for each time band as well as running mean normalization”. Examiner respectfully disagrees. During patent examination, pending claims must be “given their broadest reasonable interpretation consistent with the specification.” MPEP 2111. The claim language, broadly interpreted, is taught by the prior art references. First, prior art reference Bocklet teaches normalization through the use of mean and standard deviation for each time band. Bocklet P0012-P0014. Examiner does not interpret mean normalization with the formula described in applicants’ response since the as filed disclosure does not incorporate the formula to define mean normalization. Second, prior art reference Fernandes teaches the use of dimensional reduction and normalization in the process of emotion recognition. Fernandes teaches the reduction into a single dimension for each audio sample by taking the mean across each mel coefficients. Fernandes Section IV. Dimensionality Reduction. The claim limitation “taking the mean value for each time band” can be broadly defined and is interpreted as taking the mean value across each mel coefficients.
Furthermore, Applicant asserts that prior art references does not teach RNN and layer specific limitation in claim 7, 10, and 22 that include “layers to transform data into useful numbers” and “layers to reduce data overfitting”. Examiner respectfully disagrees. First, prior art reference Sinha teaches the use of RNN to implement the machine learning model. Sinha P0030-P0031. Second, the layer specific limitation is broad and the limitation “layers to transform data into useful numbers” can be defined as any transformation of numbers inside a neural network. The limitation “layers to reduce data overfitting” can also be defined as any regularization technique in a neural network. The prior art references teach the claim limitation and are applied in the 103 rejections.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 7, 8, 11, 14, 21, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Sinha (U.S. PG Pub No. 20220084543) in view of “Speech Emotion Recognition using Mel Frequency Cepstral Coefficient and SVM Classifier” by Fernandes et al., hereinafter Fernandes, in further view of Bocklet et al. (U.S. PG Pub No. 20180322863), hereinafter Bocklet.
Regarding claim 1 Sinha teaches:
A method of analyzing spoken audio input for emotional content, comprising (P0019, Present disclosure relates to systems and methods for real-time emotion detection from human speech.)
capturing a time series of the spoken audio input and defining time bands in the audio input, (P0020, System which comprises of an audio receiver. The audio receiver could be a microphone … that can capture audio signals.; P0025, A signal sampled at 44.1 kHz, would have 44,100 samples per second of time which implies that a window of 1,024 samples represents 23 ms of the time domain signal.)
pre-processing the time series of the spoken audio inputs to generate a multi-dimensional Mel Spectrogram, or Mel-Frequency Cepstral Coefficient (MFCC) matrix, and (P0026, The spectrogram is generated by using a moving window over the full duration of the signal. Each window of samples is transformed using the FFT to generate frequency components and their relative power. Power is represented on a decibel (dB) logarithmic scale.; P0028, The Mel spectrogram is a spectrogram representation where frequency is on a Mel scale. Once the power spectrum has been generated for a window of samples, a set of Mel filters is applied to gather the spectral energy in each of the Mel scale frequency bands. Each Mel filter is typically a triangular filter with a value of 1 at the center frequency and decreasing linearly to 0 till it reaches the center frequency on each adjacent side. Typically, a set of 40 such filters are used to extract spectral energy in 40 Mel frequency bins. A Discrete Cosine Transform (DCT) is then applied to the output of the Mel filter bank to remove spurious side-effects of the Mel filters. The final outputs are called Mel Frequency Cepstral Coefficients (MFCC).)
feeding the single dimension matrix output from the preprocessing into a trained neural network, trained with data processed with the same pre-processing steps, which provides an output that identifies at least one emotion in the audio input, and (P0030, In 615, features are extracted from the audio set. … Step 620 selects a ML model that is likely to perform well with the classification task. This may be as simple as … a deep learning neural networks. In 625, the model is trained to classify emotion based on extracted features. A portion of the data set is used for training and the rest is used to validate the model.)
presenting the emotion visually on a display screen in the form of a written description or graphic depiction. (P0021, Users can configure their preferences based on what emotions they would like to detect (e.g., happy, angry, fearful, etc.), and they can also configure how notifications are sent (e.g., on a user's screen or as a response to an API call from an external system).; P0020, The processing system analyzes the received audio signal to detect and classify emotions, and based on user preferences, send an appropriate notification to the user or another system. In an example embodiment, the processing system could be an application running on the mobile device or a program running in a natural language processing system. The notification system communicates with the user (e.g., through a device notification) or another system (e.g., via an Application Programming Interface).; Fig. 12, Example of application written notification.)
Sinha does not specifically teach:
reducing the multi-dimensionality of the Mel Spectrogram or MFCC matrix to a single-dimension matrix output by taking the mean value for each time band as well as running mean normalization across the entire data set to define the single-dimension matrix output,
feeding the single dimension matrix output from the preprocessing inti a trained neural network, trained with data processed with the same pre-processing steps, which provides an output that identifies at least one emotion in the audio input, and
Fernandes, however, teaches:
reducing the multi-dimensionality of the Mel Spectrogram or MFCC matrix to a single-dimension matrix output by taking the mean value for each time band as well as running mean normalization across the entire data set to define the single-dimension matrix output, (Abstract, Section IV, A supervised learning approach for Speech Emotion Recognition using Mel-Frequency Cepstral Coefficient (MFCC) feature extraction. The CC matrix is reduced in dimension from two-dimensional 12xF matrix to a 1 x 60 single dimension output.; Section IV, the 12XF matrix (two-dimensional matrix) is reduced to a 1 x 60 matrix (single dimension) by extracting mean, median, standard deviation, kurtosis and skewness across each mel coefficients.)
feeding the single dimension matrix output from the preprocessing inti a trained neural network, trained with data processed with the same pre-processing steps, which provides an output that identifies at least one emotion in the audio input, and (Section IV Dimensionality Reduction, Mean is extracted across each mel coefficients into a single dimension matrix. The single dimension matrix is used to train a classifier model.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to reduce the dimension of the matrix and perform normalization. It would have been obvious to combine the references because using dimensionality reduction and normalization increases the efficiency of prediction accuracy for emotion recognition. (Fernandes, Abstract)
Sinha in view of Fernandes does not specifically teach:
reducing the multi-dimensionality of the Mel Spectrogram or MFCC matrix to a single-dimension matrix output by taking the mean value for each time band as well as running mean normalization across the entire data set to define the single-dimension matrix output,
Bocklet, however, teaches:
reducing the multi-dimensionality of the Mel Spectrogram or MFCC matrix to a single-dimension matrix output by taking the mean value for each time band as well as running mean normalization across the entire data set to define the single-dimension matrix output, (P0004, Analyze the speech for Mel-Frequency Cepstral Coefficients (MFCCs). In order to improve the accuracy and reliability of the MFCC, Cepstral Mean Subtraction (CMS) is used in combination with Cepstral Variance Normalization (CVN).; P0013, The features are normalized by taking the feature for each frame, subtracting the mean and diving by the variance.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize mean normalization where mean is subtracted from the feature. It would have been obvious to combine the references because mean normalization improves the accuracy and reliability of the MFCC where the process leads to the removal of stationary noise. (Bocklet P0004)
Regarding claim 3 Sinha in view of Fernandes and further view of Bocklet teach claim 1.
Sinha further teaches:
wherein the audio input comprises an analog input or a digital input configured to define a pre-defined number of frequency values. (P0025, The DFT is typically computed over a short window of samples using an efficient Fast Fourier Transform (FFT) algorithm. A typical window could have 1,024 samples. A signal sampled at 44.1 kHz, would have 44,100 samples per second of time which implies that a window of 1,024 samples represents 23 ms of the time domain signal. Overlapping smooth windows are used to remove spurious frequencies that can arise due to sudden truncation of the signal at the end of the window.)
Regarding claim 7 Sinha in view of Fernandes and further view of Bocklet teach claim 1.
Sinha further teaches:
wherein the artificial neural network comprises a recurrent neural network (RNN) that includes layers for performing one or more of the following steps: transforming data into useful numbers, and reducing data overfitting. (P0031, LSTM networks are Recurrent Neural Networks (RNN) that use inputs over a period that may be related to map them to outputs. P0032, At the heart of an LSTM network is an LSTM cell as shown in FIG. 7. It has an input gate, it, an output gate, ot, and a forget gate, ft. The subscript t indicates a time step. At any time step t, the cell processes the input vector, xt, and computes various activation vectors as illustrated by the system of equations. Matrices W and U are weights and biases that are learned during training, while a refers to the standard sigmoid activation function.)
Regarding claim 8 Sinha teaches:
A system for analyzing emotions in speech, comprising a verbal input receiving device for receiving a spoken input signal, (P0019, Present disclosure relates to systems and methods for real-time emotion detection from human speech.; P0020, System which comprises of an audio receiver. The audio receiver could be a microphone … that can capture audio signals.)
a processor and memory configured with machine readable code to define a pre-processing stage wherein the pre-processing stage is configured to generate a multi-dimensional Mel Spectrogram or Mel-Frequency Cepstral Coefficient (MFCC) matrix from time bands defined in the spoken input signal, and (P0038, Utilize one or more generic or specialized processors.; P0039, Non-transitory computer-readable medium having instructions stored thereon for programming a computer. … Software can include instructions executable by a processor.; P0026, The spectrogram is generated by using a moving window over the full duration of the signal. Each window of samples is transformed using the FFT to generate frequency components and their relative power. Power is represented on a decibel (dB) logarithmic scale.; P0028, The Mel spectrogram is a spectrogram representation where frequency is on a Mel scale. Once the power spectrum has been generated for a window of samples, a set of Mel filters is applied to gather the spectral energy in each of the Mel scale frequency bands. Each Mel filter is typically a triangular filter with a value of 1 at the center frequency and decreasing linearly to 0 till it reaches the center frequency on each adjacent side. Typically, a set of 40 such filters are used to extract spectral energy in 40 Mel frequency bins. A Discrete Cosine Transform (DCT) is then applied to the output of the Mel filter bank to remove spurious side-effects of the Mel filters. The final outputs are called Mel Frequency Cepstral Coefficients (MFCC).)
the system further comprising an emotion model in the form of a trained multi-layered neural network architecture arranged to receive as its input, the single dimensional output from the pre-processing stage, wherein the neural network is configured either as a convolutional neural network or as a recurrent neural network, and provides as its output, data defining one [or] more emotions in the speech. (P0030, In 615, features are extracted from the audio set. … Step 620 selects a ML model that is likely to perform well with the classification task. This may be as simple as … a deep learning neural networks. In 625, the model is trained to classify emotion based on extracted features. A portion of the data set is used for training and the rest is used to validate the model.; P0031, LSTM networks are Recurrent Neural Networks (RNN) that use inputs over a period that may be related to map them to outputs.; At the heart of an LSTM network is an LSTM cell as shown in FIG. 7. It has an input gate, it, an output gate, ot, and a forget gate, ft.)
Sinha does not specifically teach:
wherein the preprocessing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single dimensional output,
the system further comprising an emotion model in the form of a trained multi-layered neural network architecture arranged to receive as its input, the single dimensional output from the pre-processing stage, wherein the neural network is configured either as a convolutional neural network or as a recurrent neural network, and provides as its output, data defining one [or] more emotions in the speech.
Fernandes, however, teaches:
wherein the preprocessing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single dimensional output, (Abstract, Section IV, A supervised learning approach for Speech Emotion Recognition using Mel-Frequency Cepstral Coefficient (MFCC) feature extraction. The CC matrix is reduced in dimension from two-dimensional 12xF matrix to a 1 x 60 single dimension output.; Section IV, the 12XF matrix (two-dimensional matrix) is reduced to a 1 x 60 matrix (single dimension) by extracting mean, median, standard deviation, kurtosis and skewness across each mel coefficients.)
the system further comprising an emotion model in the form of a trained multi-layered neural network architecture arranged to receive as its input, the single dimensional output from the pre-processing stage, wherein the neural network is configured either as a convolutional neural network or as a recurrent neural network, and provides as its output, data defining one [or] more emotions in the speech. (Section IV Dimensionality Reduction, Mean is extracted across each mel coefficients into a single dimension matrix. The single dimension matrix is used to train a classifier model.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to reduce the dimension of the matrix and perform normalization. It would have been obvious to combine the references because using dimensionality reduction and normalization increases the efficiency of prediction accuracy for emotion recognition. (Fernandes, Abstract)
Sinha in view of Fernandes does not specifically teach:
wherein the preprocessing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single dimensional output,
Bocklet, however, teaches:
wherein the preprocessing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single dimensional output, (P0004, Analyze the speech for Mel-Frequency Cepstral Coefficients (MFCCs). In order to improve the accuracy and reliability of the MFCC, Cepstral Mean Subtraction (CMS) is used in combination with Cepstral Variance Normalization (CVN).; P0013, The features are normalized by taking the feature for each frame, subtracting the mean and diving by the variance.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize mean normalization where mean is subtracted from the feature. It would have been obvious to combine the references because mean normalization improves the accuracy and reliability of the MFCC where the process leads to the removal of stationary noise. (Bocklet P0004)
Regarding claim 11 Sinha teaches:
A system for analyzing spoken audio input from one or more participants, for emotional content, comprising a pre-processing stage configured to generate a multi-dimensional Mel Spectrogram or Mel-Frequency Cepstral Coefficient (MFCC) matrix from time bands defined in the spoken audio input (P0019, Present disclosure relates to systems and methods for real-time emotion detection from human speech.; P0020, System which comprises of an audio receiver. The audio receiver could be a microphone … that can capture audio signals.; P0026, The spectrogram is generated by using a moving window over the full duration of the signal. Each window of samples is transformed using the FFT to generate frequency components and their relative power. Power is represented on a decibel (dB) logarithmic scale.)
a trained convolutional neural network (CNN) or a trained recurrent trained neural network (RNN) arranged to receive as its input the single-dimensional output from the pre-processing stage, and configured to identify at its output at least one emotion in the audio input, (P0030, In 615, features are extracted from the audio set. … Step 620 selects a ML model that is likely to perform well with the classification task. This may be as simple as … a deep learning neural networks. In 625, the model is trained to classify emotion based on extracted features. A portion of the data set is used for training and the rest is used to validate the model.; P0031, LSTM networks are Recurrent Neural Networks (RNN) that use inputs over a period that may be related to map them to outputs.)
the system further comprising one or more display screens for representing the at least one emotion from the output of the CNN or RNN in visual form to one or more participants by means of one or more of written description, and graphic representation. (P0021, Users can configure their preferences based on what emotions they would like to detect (e.g., happy, angry, fearful, etc.), and they can also configure how notifications are sent (e.g., on a user's screen or as a response to an API call from an external system).; Fig. 12, Example of application written notification.; Fig. 8, Output emotion classification from neural network when threshold is met.)
Sinha does not specifically teach:
wherein the pre-processing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output,
a trained convolutional neural network (CNN) or a trained recurrent trained neural network (RNN) arranged to receive as its input the single-dimensional output from the pre-processing stage, and configured to identify at its output at least one emotion in the audio input,
Fernandes, however, teaches:
wherein the pre-processing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output, (Abstract, Section IV, A supervised learning approach for Speech Emotion Recognition using Mel-Frequency Cepstral Coefficient (MFCC) feature extraction. The CC matrix is reduced in dimension from two-dimensional 12xF matrix to a 1 x 60 single dimension output.; Section IV, the 12XF matrix (two-dimensional matrix) is reduced to a 1 x 60 matrix (single dimension) by extracting mean, median, standard deviation, kurtosis and skewness across each mel coefficients.)
a trained convolutional neural network (CNN) or a trained recurrent trained neural network (RNN) arranged to receive as its input the single-dimensional output from the pre-processing stage, and configured to identify at its output at least one emotion in the audio input, (Section IV Dimensionality Reduction, Mean is extracted across each mel coefficients into a single dimension matrix. The single dimension matrix is used to train a classifier model.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to reduce the dimension of the matrix and perform normalization. It would have been obvious to combine the references because using dimensionality reduction and normalization increases the efficiency of prediction accuracy for emotion recognition. (Fernandes, Abstract)
Sinha in view of Fernandes does not specifically teach:
wherein the pre-processing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output,
Bocklet, however, teaches:
wherein the pre-processing stage takes the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output, (P0004, Analyze the speech for Mel-Frequency Cepstral Coefficients (MFCCs). In order to improve the accuracy and reliability of the MFCC, Cepstral Mean Subtraction (CMS) is used in combination with Cepstral Variance Normalization (CVN).; P0013, The features are normalized by taking the feature for each frame, subtracting the mean and diving by the variance.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize mean normalization where mean is subtracted from the feature. It would have been obvious to combine the references because mean normalization improves the accuracy and reliability of the MFCC where the process leads to the removal of stationary noise. (Bocklet P0004)
Regarding claim 14 Sinha in view of Fernandes and further view of Bocklet teach claim 11.
Sinha further teaches:
wherein the participants are speakers taking part in a telephone or online conversation. (P0019, Systems and methods for real-time emotion detection from human speech. They can also be embedded in interactive voice-based systems such as call centers.)
Regarding claim 21 Sinha teaches:
A method of analyzing spoken audio input obtained from one or more speaking participants for emotional content, comprising (P0019, Present disclosure relates to systems and methods for real-time emotion detection from human speech.)
capturing time bands of the spoken audio input, (P0020, System which comprises of an audio receiver. The audio receiver could be a microphone … that can capture audio signals.; P0025, A signal sampled at 44.1 kHz, would have 44,100 samples per second of time which implies that a window of 1,024 samples represents 23 ms of the time domain signal.)
processing the time bands of the spoken audio input by transforming the time bands into a multi-dimensional Mel Spectrogram or Mel- Frequency Cepstral Coefficient (MFCC) matrix, and (P0026, The spectrogram is generated by using a moving window over the full duration of the signal. Each window of samples is transformed using the FFT to generate frequency components and their relative power. Power is represented on a decibel (dB) logarithmic scale.; P0028, The Mel spectrogram is a spectrogram representation where frequency is on a Mel scale. Once the power spectrum has been generated for a window of samples, a set of Mel filters is applied to gather the spectral energy in each of the Mel scale frequency bands. Each Mel filter is typically a triangular filter with a value of 1 at the center frequency and decreasing linearly to 0 till it reaches the center frequency on each adjacent side. Typically, a set of 40 such filters are used to extract spectral energy in 40 Mel frequency bins. A Discrete Cosine Transform (DCT) is then applied to the output of the Mel filter bank to remove spurious side-effects of the Mel filters. The final outputs are called Mel Frequency Cepstral Coefficients (MFCC).)
feeding the single-dimensional output into a trained neural network, trained with data processed with the same pre-processing steps, to identify at least one emotion in the audio input, wherein the trained neural network comprises either a convolutional neural network (CNN) or a recurrent neural network (RNN), wherein the output from the CNN or RNN defines one or more emotions in the spoken audio input, and (P0030, In 615, features are extracted from the audio set. … Step 620 selects a ML model that is likely to perform well with the classification task. This may be as simple as … a deep learning neural networks. In 625, the model is trained to classify emotion based on extracted features. A portion of the data set is used for training and the rest is used to validate the model.; P0031, LSTM networks are Recurrent Neural Networks (RNN) that use inputs over a period that may be related to map them to outputs.)
presenting the one or more emotion visually on one or more display screens for representing the at least one emotion in visual form to one or more participants by means of one or more of written description, and graphic representation or by presenting the emotions of one or more of the participants by way of verbal feedback to one or more of the participants. (P0021, Users can configure their preferences based on what emotions they would like to detect (e.g., happy, angry, fearful, etc.), and they can also configure how notifications are sent (e.g., on a user's screen or as a response to an API call from an external system).; Fig. 12, Example of application written notification.)
Sinha does not specifically teach:
reducing the multi-dimensional matrix from the Mel Spectrogram or MFCC matrix to a single-dimensional matrix output, by taking the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output, the method further comprising,
feeding the single-dimensional output into a trained neural network, trained with data processed with the same pre-processing steps, to identify at least one emotion in the audio input, wherein the trained neural network comprises either a convolutional neural network (CNN) or a recurrent neural network (RNN), wherein the output from the CNN or RNN defines one or more emotions in the spoken audio input, and
Fernandes, however, teaches:
reducing the multi-dimensional matrix from the Mel Spectrogram or MFCC matrix to a single-dimensional matrix output, by taking the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output, the method further comprising, (Section IV, The 12XF matrix (two-dimensional matrix) is reduced to a 1X60 matrix (single dimension) by extracting mean, median, standard deviation, kurtosis and skewness (other dimensionality reduction).)
feeding the single-dimensional output into a trained neural network, trained with data processed with the same pre-processing steps, to identify at least one emotion in the audio input, wherein the trained neural network comprises either a convolutional neural network (CNN) or a recurrent neural network (RNN), wherein the output from the CNN or RNN defines one or more emotions in the spoken audio input, and (Section IV Dimensionality Reduction, Mean is extracted across each mel coefficients into a single dimension matrix. The single dimension matrix is used to train a classifier model.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to reduce the dimension of the matrix and perform normalization. It would have been obvious to combine the references because using dimensionality reduction and normalization increases the efficiency of prediction accuracy for emotion recognition. (Fernandes, Abstract)
Sinha in view of Fernandes does not specifically teach:
reducing the multi-dimensional matrix from the Mel Spectrogram or MFCC matrix to a single-dimensional matrix output, by taking the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output, the method further comprising,
Bocklet, however, teaches:
reducing the multi-dimensional matrix from the Mel Spectrogram or MFCC matrix to a single-dimensional matrix output, by taking the mean value for each time band as well as running mean normalization across the entire data set to define a single-dimensional output, the method further comprising, (P0004, Analyze the speech for Mel-Frequency Cepstral Coefficients (MFCCs). In order to improve the accuracy and reliability of the MFCC, Cepstral Mean Subtraction (CMS) is used in combination with Cepstral Variance Normalization (CVN).; P0013, The features are normalized by taking the feature for each frame, subtracting the mean and diving by the variance.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize mean normalization where mean is subtracted from the feature. It would have been obvious to combine the references because mean normalization improves the accuracy and reliability of the MFCC where the process leads to the removal of stationary noise. (Bocklet P0004)
Regarding claim 22 Sinha in view of Fernandes and further view of Bocklet teach claim 21.
Sinha further teaches:
wherein the case of a Recurrent Neural Network architecture (RNN), the RNN includes on or more of: layers to transform data into useful numbers, and layers to reduce data overfitting. (P0031, LSTM networks are Recurrent Neural Networks (RNN) that use inputs over a period that may be related to map them to outputs. P0032, At the heart of an LSTM network is an LSTM cell as shown in FIG. 7. It has an input gate, it, an output gate, ot, and a forget gate, ft. The subscript t indicates a time step. At any time step t, the cell processes the input vector, xt, and computes various activation vectors as illustrated by the system of equations. Matrices W and U are weights and biases that are learned during training, while a refers to the standard sigmoid activation function.)
Claim 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Sinha in view Fernandes, in view of Bocklet, and further view of Petrushin (U.S. PG Pub No. 20020194002).
Regarding claim 6 Sinha in view of Fernandes and in further view of Bocklet teach claim 1.
Sinha in view of Fernandes and further view of Bocklet does not specifically teach:
further comprising representing the emotional content of each speaker in auditory form to a user.
Petrushin, however, teaches:
further comprising representing the emotional content of each speaker in auditory form to a user. (P0002, Analysis of speech and more particularly to detecting emotion using statistics and neural networks.; P0064, The output may take the form of a signal or message on a computer, a printed message from a printer, a video display or output device connected to a computer, an audible signal or tone output from an audio output device, or even an alarm. The output may also be routed to predetermined locations based on the emotional content of the message. Routings may include a voice-mail system, an e-mail system or destination, a call center, a customer service center, a manager, or even emergency response personnel.; P0068, The voice mail center, distributing messages and corresponding to the output operation, is an additional tool that helps operators and supervisors to output, hear, or visualize the emotional content of voice messages.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to represent emotional content in auditory form. It would have been obvious to combine the references because notification of emotion in auditory form may alert businesses to persons who may be attempting to cheat or defraud them. (Petrushin P0032)
Regarding claim 15 Sinha in view of Fernandes and further view of Bocklet teach claim 11.
Sinha in view of Fernandes and further view of Bocklet does not specifically teach:
further comprising an audio output for representing the at least one emotion in auditory form to a participant.
Petrushin, however, teaches:
further comprising an audio output for representing the at least one emotion in auditory form to a participant. (P0002, Analysis of speech and more particularly to detecting emotion using statistics and neural networks.; P0064, The output may take the form of a signal or message on a computer, a printed message from a printer, a video display or output device connected to a computer, an audible signal or tone output from an audio output device, or even an alarm. The output may also be routed to predetermined locations based on the emotional content of the message. Routings may include a voice-mail system, an e-mail system or destination, a call center, a customer service center, a manager, or even emergency response personnel.; P0068, The voice mail center, distributing messages and corresponding to the output operation, is an additional tool that helps operators and supervisors to output, hear, or visualize the emotional content of voice messages.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to represent emotional content in auditory form. It would have been obvious to combine the references because notification of emotion in auditory form may alert businesses to persons who may be attempting to cheat or defraud them. (Petrushin P0032)
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Sinha in view Fernandes, in view of Bocklet, and further view of Bui et al. (U.S. PG Pub No. 20220076693), hereinafter Bui.
Regarding claim 10 Sinha in view of Fernandes and further view of Bocklet teach claim 8.
Sinha in view of Fernandes and further view of Bocklet does not specifically teach:
wherein the neural network architecture comprises a recurrent neural network with layers to transform data into useful numbers, and layers to reduce data overfitting.
Bui, however, teaches:
wherein the neural network architecture comprises a recurrent neural network with layers to transform data into useful numbers, and layers to reduce data overfitting. (P0055, The speech emotion recognition system can receive at the audio bi-directional recurrent encoder, as the input vector, extracted features in vector form. … The speech emotion recognition system may apply one or more pre-processing acts in generating the input vector. For example, the speech emotion recognition system may apply first/second order derivatives, audio segment frame size/rate adjustment (e.g., relative to the Hamming window), concatenation of values, minimization of the cross-entropy loss function using the Adam optimizer, regularization (e.g., via the dropout method), etc.; P0082, The speech emotion prediction generator passes the hidden feature vector through a softmax function to determine the probability distribution. Based on the probabilities in the probability distribution, the speech emotion prediction generator can select an emotion from the plurality of candidate emotions.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize a recurrent neural network with dropout and softmax function. It would have been obvious to combine the references because recurrent neural network with droput and softmax function are known neural network setup technique in machine learning to yield a predictable result of speech classification. (Bui P0018)
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sinha in view Fernandes, in view of Bocklet, and further view of Gainsboro et al. (U.S. PG Pub No. 20160217807), hereinafter Gainsboro.
Regarding claim 19 Sinha in view of Fernandes and further view of Bocklet teach claim 11.
Sinha in view of Fernandes and further view of Bocklet does not specifically teach:
wherein the system is part of an online conference call network and the participants are connected to the network by means of user access devices.
Gainsboro, however, teaches:
wherein the system is part of an online conference call network and the participants are connected to the network by means of user access devices. (P0041, It is a further object of the present invention to improve the speech-to-text conversion of electronically monitored teleconferences and classrooms and the like through segmenting speech by individual, and to improve the searchability of databases of recorded and/or text-converted conversations by storing prosody information which is correlated with audio and/or text information, enabling searching audio and/or text conversation databases for emotional criteria as well as word or phrase criteria.; Abstract, Automated prosody measurement algorithms are used in conjunction with speaker segmentation to extract emotional content of the speech of participants within a particular conversation, and speaker interactions and emotions are displayed in graphical form.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, for system to be part of an online conference call network. It would have been obvious to combine the references because a search for emotional criteria allows persons reviewing recorded conversations (such as classroom sessions, teleconferences, meetings and the like) to more rapidly find the portions of the recording that may be of most relevance or interest. (Gainsboro P0041)
Regarding claim 20 Sinha in view of Fernandes, in view of Bocklet, and further view of Gainsboro teach claim 19.
Sinha further teaches:
wherein the user access devices include one or more of cell phone, tablet, laptop, or desktop computer. (P0019, The systems and methods may use mobile devices (e.g. iPhone, Android device, tablets, smart watches, etc.))
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL WONSUK CHUNG whose telephone number is (571)272-1345. The examiner can normally be reached Monday - Friday (7am-4pm)[PT].
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PIERRE-LOUIS DESIR can be reached at (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL W CHUNG/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659