DETAILED ACTION
This office action is in response to Applicant’s Request for Continued Examination (RCE), received on 03/16/2025. Claims 1, 6-7, 11-12 have been amended. Claims 16-20 have been added. Claims 1-20 are pending and have been considered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/16/2026 has been entered.
Response to Arguments
Applicant’s arguments, see pg. 10, filed 03/05/2026, with respect to claims 7 and 12 have been fully considered and are persuasive. The objections of claims 7 and 12 have been withdrawn.
Applicant's arguments filed 03/05/2026, see pg. 11, have been fully considered but they are not persuasive.
Applicant’s representative asserts, “In contrast to the invention recited in claim 1, DiMambro does not disclose creating a first activation sequence from the captured audio data and a second activation sequence from the record audio data, the first and second activation sequences having a different number of vectors.
Instead, DiMambro describes that in practice, a user can present a spoken utterance corresponding to a pass phrase that was used during voice enrollment; that is when the user registered their biometric voice print with a voice authorization server. For example, during enrollment, a user pronounces the same pass phrase three times. A feature matrix is calculated for each recording of the pass phrase (see para. [0050]). During verification, the user speaks the same spoken utterance corresponding to the pass phrase, and a biometric voice print is generated. The biometric voice print is compared against previously stored voice prints for identifying a match. During the verification process, a feature matrix is also calculated from the spoken phrase using the voice authentication algorithm as used in enrollment (see para. [0051]).
Moreover, there is no disclosure or suggestion in DiMambro to create feature matrices during enrollment and verification that have a different number of vectors.
In other words, DiMambro does not disclose creating a first activation sequence from the captured audio data and a second activation sequence from the record audio data, the first and second activation sequences having a different number of vectors, as recited in claim 1.”
In response, the examiner respectfully disagrees with Applicant’s assertion that DiMambro does not disclose feature matrices during enrollment and verification that have a different number of vectors. Specifically, the examiner would like to refer to [0050]-[0051] of DiMambro, also cited by Applicant. [0050] describes the enrollment process, wherein “three feature matrices are used to create the biometric voice print…the feature matrices define the features of the voice.” Creating an enrollment voiceprint based upon three matrices (which the examiner asserts to be equivalent to a 2D and/or 1D vector) indicates at least three feature matrices comprised of at least three vectors used for enrollment. [0051] describes the verification process, wherein “the user speaks the same spoken utterance corresponding to the pass phrase, and a biometric voice print is generated. The biometric voice print is compared against previously stored voice prints for identifying a match. During the verification process, a feature matrix is also calculated from the spoken phrase using the voice authentication algorithm 800 as used in enrollment. This feature matrix is compared against one or more reference matrices store in a voiceprint database”. This indicates one feature matrix comprised of at least one vector used for verification. Considering there is one verification voiceprint (represented by one feature matrix) being compared to enrollment voiceprints (represented by at least three feature matrices), this indicates at least a 3:1 vector ratio between enrollment and verification; therefore, the examiner respectfully asserts that DiMambro does disclose creation of feature matrices during enrollment and verification that have a different number of vectors. Further, with regard to activation sequences, the examiner asserts that the matrices of DiMambro track to activation sequences as currently claimed. In view of the previous analysis of the feature matrices of DiMambro, the examiner respectfully asserts that they qualify to be “first and second activation sequences having a different number of vectors”.
Applicant’s arguments, see pgs. 11-12, filed 03/05/2026, with respect to the rejection(s) of claim(s) 1 under 35 U.S.C. 103 (with respect to the “aligning…” step against Tuo) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of DiMambro. Specifically, the examiner will be referring to new disclosure from [0051] relating to how the voiceprint verification is performed. See updated rejections below.
Applicant’s arguments with respect to claim(s) 6, 11, and 16-20, see pgs. 13-14, have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 6-8, 11-13, 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Di Mambro et al. (US-20130018657-A1), hereinafter Di Mambro, in view of Tuo et al. (US-20220270611-A1), hereinafter Tuo.
Regarding claim 1, Di Mambro discloses: a method for authentication of users (Abstract, A method (700) and system (900) for authenticating a user), comprising the steps of:
capturing, by an electronic device ([0023] one or more mobile devices 102), audio data of a pass phrase spoken by a user ([0029] When the user speaks the pass phrase into the mobile device 102 [Indicating capture by the mobile device]);
obtaining record audio data of the pass phrase spoken by the user ([0050] a user can present a spoken utterance corresponding to a pass phrase that was used during voice enrollment… [0051] During verification, the user speaks the same spoken utterance corresponding to the pass phrase, and a biometric voice print is generated [Comparisons of voiceprints indicates a required retrieval/obtaining of the enrollment, i.e. record, voiceprint for comparison]); and,
creating a first activation sequence from the captured audio data ([0051] During the verification process, a feature matrix is also calculated from the spoken phrase, [0053] normalize the feature matrix over the one or more vocalized frames [Wherein a feature matrix represents features of the speaker’s voice, see [0050], in view of [0079] of the instant app which defines activation sequences as processed feature matrices indicating normalization is a processing step to feature matrices resulting in activation sequences. Further, the figures representing activation sequences in the instant app (Figs. 6, 7) have a y-axis of normalized values indicating this is the same processing as that performed in Di Mambro]) and a second activation sequence from the record audio data ([0050] during enrollment, a user pronounces the same pass phrase three times. A feature matrix is calculated for each recording of the pass phrase [In view of the feature matrix normalization previously disclosed of Di Mambro, indicating this same normalization could be applied to the enrollment feature matrix of Di Mambro without a change in functionality. Further, in view of the biometric voiceprint comparison ([0042]) of Di Mambro, indicating both matrices need to be normalized for an appropriate comparison]), the first and second activation sequences having a different number of vectors ([0050] three feature matrices are used to create the biometric voice print, [0053] calculate a feature matrix from the one or more feature vectors, [Generating a feature matrix, i.e. activation sequence, comprised of feature vectors indicates the activation sequence to be including vectors. Further, considering the previously cited three enrollment feature matrices, wherein each of the three enrollment matrices are used to create the biometric voiceprint to be compared to the verification voiceprint, indicates at least a 3:1 ratio of vectors from enrollment to verification represented through each time the pass phrase is spoken]); and,
aligning the first activation sequence and the second activation sequence to have a same number of vectors ([0051] The biometric voice print is compared against previously stored voice prints for identifying a match…using the voice authentication algorithm 800 as used in enrollment. This feature matrix is compared against one or more reference matrices store in a voiceprint database, [Comparing matrices together, wherein multiple feature matrices are gathered in enrollment, indicates one enrollment matrix being compared to one verification matrix at a time, i.e. an alignment of 1:1 in terms of matrices comprising the same spoken pass phrase, indicating the same amount of vectors within the matrices to represent the same amount of speech/feature data for an appropriate comparison. Considering the same matrix is used to represent each voiceprint, see Table 1, this indicates a matrix-to-matrix comparison will be aligned in terms of vectors representing the matrices]).
Di Mambro does not disclose:
stacking the aligned first and second activation sequences to create a tensor;
calculating a first weighted activation sequence and a second weighted activation sequence using the tensor;
calculating an embedding for the first weighted activation sequence and an embedding for the second weighted activation sequence;
calculating a similarity score between the calculated embeddings;
in response to determining the similarity score satisfies a threshold score, simultaneously successfully authenticating the user and the pass phrase spoken by the user.
Tuo discloses:
stacking the aligned first and second activation sequences to create a tensor ([0060] A tensor may be generated having dimensions based on the number of speakers N, the number of utterances M for each speaker, and a number of embedding dimensions, [Turning a plurality of speaker utterances M into a dimension related to a speaker N based on a number of embeddings P indicates the tensor comprising N*M*P dimensions to be stacked with regard to utterances, i.e. activation sequences, and/or the number of embeddings associated with each utterance in the N/M/P dimension]);
calculating a first weighted activation sequence and a second weighted activation sequence using the tensor ([0060] This tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances [In view of the definition of weighted activation sequences in the instant app, [0071] which only discloses generation of weighted sequences through the tensor, indicating a tensor generating a similarity matrix represents first and second weighted, i.e. in terms of similarity, activation sequences within the similarity matrix in view of the activation sequences of Di Mambro]);
calculating an embedding for the first weighted activation sequence and an embedding for the second weighted activation sequence ([0072] determine a distance between the embedding vector(s) associated with the received recording of the user utterance and predefined embedding vector(s) [Using vector embeddings, in view of the weighted activation sequence vectors, indicating embeddings are calculated for the first, i.e. received, and second, i.e. predefined, weighted activation sequences in view of the similarity matrix of Tuo which would be comprising embedding vectors]);
calculating a similarity score between the calculated embeddings ([0072] calculates, for each respective user of a plurality of users, a similarity score between embedding vectors associated with the respective user and the generated plurality of embedding vectors for the user utterance [Wherein embedding vectors associated with the respective user represent second, i.e. record, audio and embedding associated with a user utterance represent first, i.e. live, audio]);
in response to determining the similarity score satisfies a threshold score ([0030] compare the similarity scores generated for each of the plurality of users with a threshold similarity score), simultaneously successfully authenticating the user and the pass phrase spoken by the user ([0030] Scores that are below the threshold similarity score may be deemed to be associated with users that are highly unlikely to be the user from whom the incoming voice recording was received. Thus, the users associated with these scores, may be removed from the plurality of users [Removing user with scores below a threshold similarity indicates an awareness that certain scores also exceed the threshold, i.e. so the system knows not to discard these samples. In view of the pass-phrase used for authentication of Di Mambro]).
Di Mambro and Tuo are considered analogous art within user speech authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro to incorporate the teachings of Tuo, because of the novel way to compare received voice utterances to a plurality of utterances from a plurality of speakers and selecting a final speaker based on similarity scores, improving user identification and verification methodologies (Tuo, [0003]).
Regarding claim 2, Di Mambro in view of Tuo discloses: the method according to claim 1.
Di Mambro further discloses:
said step of creating a first activation sequence and a second activation sequence comprising the steps of:
extracting features from the captured audio data and the record audio data ([0049] features extracted during the LPC/LSP analysis can be included in a feature matrix [In view of the previously disclosed first and record audio data of Di Mambro, indicating this operation could be applied to both signals]);
creating a first feature matrix using the features extracted from the captured audio data and a second feature matrix using the features extracted from the record audio data ([0050] A feature matrix is calculated for each recording of the pass phrase [In view of the previously disclosed enrollment, i.e. record, and verification, i.e. captured, audio of Di Mambro indicating this operation is applied to both signals (see [0051] which discloses generation of a feature matrix for verification audio)]); and,
creating the first activation sequence from the first feature matrix and the second activation sequence from the second feature matrix ([0053] normalize the feature matrix over the one or more vocalized frames [In view of the enrollment and verification feature matrices of Di Mambro, normalization tracks to a method of generating activation sequences, see [0079] of the instant app]).
Regarding claim 3, Di Mambro in view of Tuo discloses: the method according to claim 1.
Tuo further discloses:
said aligning step of comprising comparing the first activation sequence against the second activation sequence to determine a path that represents a best match between the first and second activation sequences ([0060] tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances selected from the data set, and a predictive vector may be generated from the generated similarity matrix. In some aspects, the predictive vector may include, for each entry in the similarity matrix, an indication of a speaker predicted to be associated with each of the utterances. The speaker predicted to be associated with each of the utterances may be identified by calculating a loss based on a cross-entropy between the predictive vector and a ground truth vector identifying a speaker associated with each of the utterances [generation of a similarity matrix indicates a comparison, in view of the first and second activation sequences of Di Mambro, wherein the similarity is determined between a generated, i.e. first, vector and a ground truth, i.e. second/record, vector indicating a path that represents a best match, i.e. highest predicted similarity, between the activation sequences]).
Regarding claim 6, Di Mambro discloses: an electronic device ([0023] one or more mobile devices 102) for authentication of users (Abstract, A method (700) and system (900) for authenticating a user) comprising:
a processor ([0052] voice processor 144); and,
a memory configured to store data ([Fig. 1, Database 140], [A database indicates memory for storage of data]), said electronic device being associated with a network ([Fig. 1, Internet 120]) and said memory being in communication with said processor ([Fig. 3, connection between mobile device 102, indicating a processor, and database server 130]) and having instructions stored thereon which, when read and executed by said processor, cause said electronic device to:
capture audio data of a pass phrase spoken by a user ([0029] When the user speaks the pass phrase into the mobile device 102 [Indicating capture by the mobile device]);
obtain record audio data of the pass phrase spoken by the user ([0050] a user can present a spoken utterance corresponding to a pass phrase that was used during voice enrollment… [0051] During verification, the user speaks the same spoken utterance corresponding to the pass phrase, and a biometric voice print is generated [Comparisons of voiceprints indicates a required retrieval/obtaining of the enrollment, i.e. record, voiceprint for comparison]); and,
create a first activation sequence from the captured audio data ([0051] During the verification process, a feature matrix is also calculated from the spoken phrase, [0053] normalize the feature matrix over the one or more vocalized frames [Wherein a feature matrix represents features of the speaker’s voice, see [0050], in view of [0079] of the instant app which defines activation sequences as processed feature matrices indicating normalization is a processing step to feature matrices resulting in activation sequences. Further, the figures representing activation sequences in the instant app (Figs. 6, 7) have a y-axis of normalized values indicating this is the same processing as that performed in Di Mambro]) and a second activation sequence from the record audio data ([0050] during enrollment, a user pronounces the same pass phrase three times. A feature matrix is calculated for each recording of the pass phrase [In view of the feature matrix normalization previously disclosed of Di Mambro, indicating this same normalization could be applied to the enrollment feature matrix of Di Mambro without a change in functionality. Further, in view of the biometric voiceprint comparison ([0042]) of Di Mambro, indicating both matrices need to be normalized for an appropriate comparison]), the first and second activation sequences having a different number of vectors ([0050] three feature matrices are used to create the biometric voice print, [0053] calculate a feature matrix from the one or more feature vectors, [Generating a feature matrix, i.e. activation sequence, comprised of feature vectors indicates the activation sequence to be including vectors. Further, considering the previously cited three enrollment feature matrices, wherein each of the three enrollment matrices are used to create the biometric voiceprint to be compared to the verification voiceprint, indicates at least a 3:1 ratio of vectors from enrollment to verification represented through each time the pass phrase is spoken]); and,
align the first activation sequence and the second activation sequence to have a same number of vectors ([0051] The biometric voice print is compared against previously stored voice prints for identifying a match…using the voice authentication algorithm 800 as used in enrollment. This feature matrix is compared against one or more reference matrices store in a voiceprint database, [Comparing matrices together, wherein multiple feature matrices are gathered in enrollment, indicates one enrollment matrix being compared to one verification matrix at a time, i.e. an alignment of 1:1 in terms of matrices comprising the same spoken pass phrase, indicating the same amount of vectors within the matrices to represent the same amount of speech/feature data for an appropriate comparison. Considering the same matrix is used to represent each voiceprint, see Table 1, this indicates a matrix-to-matrix comparison will be aligned in terms of vectors representing the matrices]).
Di Mambro does not disclose:
stack the aligned first and second activation sequences to create a tensor;
calculate a first weighted activation sequence and a second weighted activation sequence using the tensor;
calculate an embedding for the first weighted activation sequence and an embedding for the second weighted activation sequence;
calculate a similarity score between the calculated embeddings;
in response to determining the similarity score satisfies a threshold score, simultaneously successfully authenticate the user and the pass phrase spoken by the user.
Tuo discloses:
stack the aligned first and second activation sequences to create a tensor ([0060] A tensor may be generated having dimensions based on the number of speakers N, the number of utterances M for each speaker, and a number of embedding dimensions, [Turning a plurality of speaker utterances M into a dimension related to a speaker N based on a number of embeddings P indicates the tensor comprising N*M*P dimensions to be stacked with regard to utterances, i.e. activation sequences, and/or the number of embeddings associated with each utterance in the N/M/P dimension]);
calculate a first weighted activation sequence and a second weighted activation sequence using the tensor ([0060] This tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances [In view of the definition of weighted activation sequences in the instant app, [0071] which only discloses generation of weighted sequences through the tensor, indicating a tensor generating a similarity matrix represents first and second weighted, i.e. in terms of similarity, activation sequences within the similarity matrix in view of the activation sequences of Di Mambro]);
calculate an embedding for the first weighted activation sequence and an embedding for the second weighted activation sequence ([0072] determine a distance between the embedding vector(s) associated with the received recording of the user utterance and predefined embedding vector(s) [Using vector embeddings, in view of the weighted activation sequence vectors, indicating embeddings are calculated for the first, i.e. received, and second, i.e. predefined, weighted activation sequences in view of the similarity matrix of Tuo which would be comprising embedding vectors]);
calculate a similarity score between the calculated embeddings ([0072] calculates, for each respective user of a plurality of users, a similarity score between embedding vectors associated with the respective user and the generated plurality of embedding vectors for the user utterance [Wherein embedding vectors associated with the respective user represent second, i.e. record, audio and embedding associated with a user utterance represent first, i.e. live, audio]);
in response to determining the similarity score satisfies a threshold score ([0030] compare the similarity scores generated for each of the plurality of users with a threshold similarity score), simultaneously successfully authenticate the user and the pass phrase spoken by the user ([0030] Scores that are below the threshold similarity score may be deemed to be associated with users that are highly unlikely to be the user from whom the incoming voice recording was received. Thus, the users associated with these scores, may be removed from the plurality of users [Removing user with scores below a threshold similarity indicates an awareness that certain scores also exceed the threshold, i.e. so the system knows not to discard these samples. In view of the pass-phrase used for authentication of Di Mambro]).
Di Mambro and Tuo are considered analogous art within user speech authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro to incorporate the teachings of Tuo, because of the novel way to compare received voice utterances to a plurality of utterances from a plurality of speakers and selecting a final speaker based on similarity scores, improving user identification and verification methodologies (Tuo, [0003]).
Regarding claim 7, Di Mambro in view of Tuo discloses: the electronic device according to claim 6.
Di Mambro further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said electronic device ([In view of the previously disclosed electronic device of Di Mambro]) to:
extract features from the captured audio data and the record audio data ([0049] features extracted during the LPC/LSP analysis can be included in a feature matrix [In view of the previously disclosed first and record audio data of Di Mambro, indicating this operation could be applied to both signals]);
create a first feature matrix using the features extracted from the captured audio data and a second feature matrix using the features extracted from the record audio data ([0050] A feature matrix is calculated for each recording of the pass phrase [In view of the previously disclosed enrollment, i.e. record, and verification, i.e. captured, audio of Di Mambro indicating this operation is applied to both signals (see [0051] which discloses generation of a feature matrix for verification audio)]); and,
create the first activation sequence from the first feature matrix and the second activation sequence from the second feature matrix ([0053] normalize the feature matrix over the one or more vocalized frames [In view of the enrollment and verification feature matrices of Di Mambro, normalization tracks to a method of generating activation sequences, see [0079] of the instant app]).
Regarding claim 8, Di Mambro in view of Tuo discloses: the electronic device according to claim 6.
Tuo further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said electronic device ([In view of the previously disclosed electronic device of Di Mambro]) to:
compare the first activation sequence against the second activation sequence to determine a path that represents a best match between the first and second activation sequences ([0060] tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances selected from the data set, and a predictive vector may be generated from the generated similarity matrix. In some aspects, the predictive vector may include, for each entry in the similarity matrix, an indication of a speaker predicted to be associated with each of the utterances. The speaker predicted to be associated with each of the utterances may be identified by calculating a loss based on a cross-entropy between the predictive vector and a ground truth vector identifying a speaker associated with each of the utterances [generation of a similarity matrix indicates a comparison, in view of the first and second activation sequences of Di Mambro, wherein the similarity is determined between a generated, i.e. first, vector and a ground truth, i.e. second/record, vector indicating a path that represents a best match, i.e. highest predicted similarity, between the activation sequences]); and
create a first sub activation sequence and a second sub activation sequence wherein the first sub activation sequence includes a subset of vectors along the path from the first activation sequence ([0030] Scores that are below the threshold similarity score may be deemed to be associated with users that are highly unlikely to be the user from whom the incoming voice recording was received. Thus, the users associated with these scores, may be removed from the plurality of users… [0072] comparison metrics that may be used to determine a distance between the embedding vector(s) associated with the received recording of the user utterance and predefined embedding vector(s) [Removing users from a plurality of users indicates reducing the user dimension of a tensor, as defined in Tuo, further indicating a sub activation sequence as compared to the original activation sequence of Di Mambro, wherein the sub activation sequence is determined based on a distance, i.e. a path representing similarity/matching]), and
the second sub activation sequence includes a subset of vectors along the path from the second activation sequence ([Similar to the operation being performed in the above step, this operation could be applied to second activation sequences to generate a vector subset without a change in functionality, i.e. the comparison between sequences is a two-directional calculation, indicating the operation of determining the subset of vectors for a first activation sequence defining a minimum distance to a second activation sequence will be the same distance calculation as that of the second activation sequence back to the first.]).
Regarding claim 11, Di Mambro discloses: a non-transitory computer-readable recording medium ([0072] Portions of the present method and system may also be embedded in a computer program product) in an electronic device for authenticating users (Abstract, A method (700) and system (900) for authenticating a user [Wherein the system is comprising a smartphone 102, indicating the embedding is in the non-transitory storage of the mobile device, as traditionally included in smartphones]), the non-transitory computer-readable recording medium storing instructions which when executed by a hardware processor ([0053] voice processor 144) cause the non-transitory recording medium to perform steps comprising:
capturing audio data of a pass phrase spoken by a user ([0029] When the user speaks the pass phrase into the mobile device 102 [Indicating capture by the mobile device]);
obtaining record audio data of the pass phrase spoken by the user ([0050] a user can present a spoken utterance corresponding to a pass phrase that was used during voice enrollment… [0051] During verification, the user speaks the same spoken utterance corresponding to the pass phrase, and a biometric voice print is generated [Comparisons of voiceprints indicates a required retrieval/obtaining of the enrollment, i.e. record, voiceprint for comparison]); and,
creating a first activation sequence from the captured audio data ([0051] During the verification process, a feature matrix is also calculated from the spoken phrase, [0053] normalize the feature matrix over the one or more vocalized frames [Wherein a feature matrix represents features of the speaker’s voice, see [0050], in view of [0079] of the instant app which defines activation sequences as processed feature matrices indicating normalization is a processing step to feature matrices resulting in activation sequences. Further, the figures representing activation sequences in the instant app (Figs. 6, 7) have a y-axis of normalized values indicating this is the same processing as that performed in Di Mambro]) and a second activation sequence from the record audio data ([0050] during enrollment, a user pronounces the same pass phrase three times. A feature matrix is calculated for each recording of the pass phrase [In view of the feature matrix normalization previously disclosed of Di Mambro, indicating this same normalization could be applied to the enrollment feature matrix of Di Mambro without a change in functionality. Further, in view of the biometric voiceprint comparison ([0042]) of Di Mambro, indicating both matrices need to be normalized for an appropriate comparison]), the first and second activation sequences having a different number of vectors ([0050] three feature matrices are used to create the biometric voice print, [0053] calculate a feature matrix from the one or more feature vectors, [Generating a feature matrix, i.e. activation sequence, comprised of feature vectors indicates the activation sequence to be including vectors. Further, considering the previously cited three enrollment feature matrices, wherein each of the three enrollment matrices are used to create the biometric voiceprint to be compared to the verification voiceprint, indicates at least a 3:1 ratio of vectors from enrollment to verification represented through each time the pass phrase is spoken]); and,
aligning the first activation sequence and the second activation sequence to have a same number of vectors ([0051] The biometric voice print is compared against previously stored voice prints for identifying a match…using the voice authentication algorithm 800 as used in enrollment. This feature matrix is compared against one or more reference matrices store in a voiceprint database, [Comparing matrices together, wherein multiple feature matrices are gathered in enrollment, indicates one enrollment matrix being compared to one verification matrix at a time, i.e. an alignment of 1:1 in terms of matrices comprising the same spoken pass phrase, indicating the same amount of vectors within the matrices to represent the same amount of speech/feature data for an appropriate comparison. Considering the same matrix is used to represent each voiceprint, see Table 1, this indicates a matrix-to-matrix comparison will be aligned in terms of vectors representing the matrices]).
Di Mambro does not disclose:
stacking the aligned first and second activation sequences to create a tensor;
calculating a first weighted activation sequence and a second weighted activation sequence using the tensor;
calculating an embedding for the first weighted activation sequence and an embedding for the second weighted activation sequence;
calculating a similarity score between the calculated embeddings;
in response to determining the similarity score satisfies a threshold score, simultaneously successfully authenticating the user and the pass phrase spoken by the user.
Tuo discloses:
stacking the aligned first and second activation sequences to create a tensor ([0060] A tensor may be generated having dimensions based on the number of speakers N, the number of utterances M for each speaker, and a number of embedding dimensions, [Turning a plurality of speaker utterances M into a dimension related to a speaker N based on a number of embeddings P indicates the tensor comprising N*M*P dimensions to be stacked with regard to utterances, i.e. activation sequences, and/or the number of embeddings associated with each utterance in the N/M/P dimension]);
calculating a first weighted activation sequence and a second weighted activation sequence using the tensor ([0060] This tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances [In view of the definition of weighted activation sequences in the instant app, [0071] which only discloses generation of weighted sequences through the tensor, indicating a tensor generating a similarity matrix represents first and second weighted, i.e. in terms of similarity, activation sequences within the similarity matrix in view of the activation sequences of Di Mambro]);
calculating an embedding for the first weighted activation sequence and an embedding for the second weighted activation sequence ([0072] determine a distance between the embedding vector(s) associated with the received recording of the user utterance and predefined embedding vector(s) [Using vector embeddings, in view of the weighted activation sequence vectors, indicating embeddings are calculated for the first, i.e. received, and second, i.e. predefined, weighted activation sequences in view of the similarity matrix of Tuo which would be comprising embedding vectors]);
calculating a similarity score between the calculated embeddings ([0072] calculates, for each respective user of a plurality of users, a similarity score between embedding vectors associated with the respective user and the generated plurality of embedding vectors for the user utterance [Wherein embedding vectors associated with the respective user represent second, i.e. record, audio and embedding associated with a user utterance represent first, i.e. live, audio]);
in response to determining the similarity score satisfies a threshold score ([0030] compare the similarity scores generated for each of the plurality of users with a threshold similarity score), simultaneously successfully authenticating the user and the pass phrase spoken by the user ([0030] Scores that are below the threshold similarity score may be deemed to be associated with users that are highly unlikely to be the user from whom the incoming voice recording was received. Thus, the users associated with these scores, may be removed from the plurality of users [Removing user with scores below a threshold similarity indicates an awareness that certain scores also exceed the threshold, i.e. so the system knows not to discard these samples. In view of the pass-phrase used for authentication of Di Mambro]).
Di Mambro and Tuo are considered analogous art within user speech authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro to incorporate the teachings of Tuo, because of the novel way to compare received voice utterances to a plurality of utterances from a plurality of speakers and selecting a final speaker based on similarity scores, improving user identification and verification methodologies (Tuo, [0003]).
Regarding claim 12, Di Mambro in view of Tuo discloses: the non-transitory computer-readable medium according to claim 11.
Di Mambro further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said non-transitory computer-readable recording medium ([In view of the previously disclosed non-transitory computer-readable medium of Di Mambro]) to perform steps comprising:
extracting features from the captured audio data and the record audio data ([0049] features extracted during the LPC/LSP analysis can be included in a feature matrix [In view of the previously disclosed first and record audio data of Di Mambro, indicating this operation could be applied to both signals]);
creating a first feature matrix using the features extracted from the captured audio data and a second feature matrix using the features extracted from the record audio data ([0050] A feature matrix is calculated for each recording of the pass phrase [In view of the previously disclosed enrollment, i.e. record, and verification, i.e. captured, audio of Di Mambro indicating this operation is applied to both signals (see [0051] which discloses generation of a feature matrix for verification audio)]); and,
creating the first activation sequence from the first feature matrix and the second activation sequence from the second feature matrix ([0053] normalize the feature matrix over the one or more vocalized frames [In view of the enrollment and verification feature matrices of Di Mambro, normalization tracks to a method of generating activation sequences, see [0079] of the instant app]).
Regarding claim 13, Di Mambro in view of Tuo discloses: the non-transitory computer-readable storage medium according to claim 11.
Tuo further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said non-transitory computer-readable recording medium ([In view of the previously disclosed non-transitory computer-readable medium of Di Mambro]) to perform steps comprising:
comparing the first activation sequence against the second activation sequence to determine a path that represents a best match between the first and second activation sequences ([0060] tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances selected from the data set, and a predictive vector may be generated from the generated similarity matrix. In some aspects, the predictive vector may include, for each entry in the similarity matrix, an indication of a speaker predicted to be associated with each of the utterances. The speaker predicted to be associated with each of the utterances may be identified by calculating a loss based on a cross-entropy between the predictive vector and a ground truth vector identifying a speaker associated with each of the utterances [generation of a similarity matrix indicates a comparison, in view of the first and second activation sequences of Di Mambro, wherein the similarity is determined between a generated, i.e. first, vector and a ground truth, i.e. second/record, vector indicating a path that represents a best match, i.e. highest predicted similarity, between the activation sequences]); and
creating a first sub activation sequence and a second sub activation sequence wherein the first sub activation sequence includes a subset of vectors along the path from the first activation sequence ([0030] Scores that are below the threshold similarity score may be deemed to be associated with users that are highly unlikely to be the user from whom the incoming voice recording was received. Thus, the users associated with these scores, may be removed from the plurality of users… [0072] comparison metrics that may be used to determine a distance between the embedding vector(s) associated with the received recording of the user utterance and predefined embedding vector(s) [Removing users from a plurality of users indicates reducing the user dimension of a tensor, as defined in Tuo, further indicating a sub activation sequence as compared to the original activation sequence of Di Mambro, wherein the sub activation sequence is determined based on a distance, i.e. a path representing similarity/matching]), and
the second sub activation sequence includes a subset of vectors along the path from the second activation sequence ([Similar to the operation being performed in the above step, this operation could be applied to second activation sequences to generate a vector subset without a change in functionality, i.e. the comparison between sequences is a two-directional calculation, indicating the operation of determining the subset of vectors for a first activation sequence defining a minimum distance to a second activation sequence will be the same distance calculation as that of the second activation sequence back to the first.]).
Regarding claim 16, Di Mambro in view of Tuo discloses: the method according to claim 1.
Di Mambro further discloses:
said aligning step comprising creating a first activation sub-sequence from the first activation sequence and a second activation sub-sequence from the second activation sequence, the first and second activation sub-sequences having a same number of vectors ([0050] during enrollment, a user pronounces the same pass phrase three times. A feature matrix is calculated for each recording of the pass phrase. The feature matrix is a matrix of numeric values that represent features of the speaker's voice. In this case, three feature matrices are used to create the biometric voice print, [0051] The biometric voice print is compared against previously stored voice prints for identifying a match…a feature matrix is also calculated from the spoken phrase…using the voice authentication algorithm 800 as used in enrollment. This feature matrix is compared against one or more reference matrices store in a voiceprint database, [Creation of a biometric voice print for comparison based on a feature matrix indicates the voiceprint to be a sub-sequence, i.e. one matrix, from the activation sequences, i.e. one or more matrices, wherein a comparison of one feature matrix to a reference matrix which are containing the same spoken information, i.e. voiced information pass phrase ([0063]), will necessarily have the same amount of vectors to represent the same amount of speech, i.e. the pass phrase, as the vector(s) comprising the matrix are gathered on a segment-level ([0063])]).
Regarding claim 17, Di Mambro in view of Tuo discloses: the method according to claim 16.
Tuo further discloses:
said stacking step comprising stacking the aligned first and second activation sub-sequences to create the tensor ([0060] A tensor may be generated having dimensions based on the number of speakers N, the number of utterances M for each speaker, and a number of embedding dimensions, [A tensor with an utterance and speaker dimension indicates each utterance, i.e. turned into voiceprint via Di Mambro, by the same speaker, i.e. considering the enrollment and verification audio of Di Mambro, to be stacked sub-sequences corresponding to each utterance by one speaker]).
Regarding claim 18, Di Mambro in view of Tuo discloses: the electronic device according to claim 6.
Di Mambro further discloses:
wherein the instructions when read and executed by said processor, cause said aligning step comprising creating a first activation sub-sequence from the first activation sequence and a second activation sub-sequence from the second activation sequence, the first and second activation sub-sequences having a same number of vectors ([0050] during enrollment, a user pronounces the same pass phrase three times. A feature matrix is calculated for each recording of the pass phrase. The feature matrix is a matrix of numeric values that represent features of the speaker's voice. In this case, three feature matrices are used to create the biometric voice print, [0051] The biometric voice print is compared against previously stored voice prints for identifying a match…a feature matrix is also calculated from the spoken phrase…using the voice authentication algorithm 800 as used in enrollment. This feature matrix is compared against one or more reference matrices store in a voiceprint database, [Creation of a biometric voice print for comparison based on a feature matrix indicates the voiceprint to be a sub-sequence, i.e. one matrix, from the activation sequences, i.e. plurality of matrices, wherein a comparison of one feature matrix to a reference matrix which are containing the same spoken information, i.e. voiced information pass phrase ([0063]), will necessarily have the same amount of vectors to represent the same amount of speech, i.e. the pass phrase, as the vector comprising the matrix are gathered on a segment-level ([0063])]).
Regarding claim 19, Di Mambro in view of Tuo discloses: the electronic device according to claim 18.
Tuo further discloses:
wherein the instructions when read and executed by said processor, cause said stacking step comprising stacking the aligned first and second activation sub-sequences to create the tensor ([0060] A tensor may be generated having dimensions based on the number of speakers N, the number of utterances M for each speaker, and a number of embedding dimensions, [A tensor with an utterance and speaker dimension indicates each utterance, i.e. turned into voiceprint via Di Mambro, by the same speaker, i.e. considering the enrollment and verification audio of Di Mambro, to be stacked sub-sequences corresponding to each utterance by one speaker]).
Regarding claim 20, Di Mambro in view of Tuo discloses: the non-transitory computer-readable medium according to claim 11.
Di Mambro further discloses:
wherein the instructions when read and executed by said processor, cause said non-transitory computer-readable medium to perform the step comprising creating a first activation sub-sequence from the first activation sequence and a second activation sub-sequence from the second activation sequence, the first and second activation sub-sequences having a same number of vectors ([0050] during enrollment, a user pronounces the same pass phrase three times. A feature matrix is calculated for each recording of the pass phrase. The feature matrix is a matrix of numeric values that represent features of the speaker's voice. In this case, three feature matrices are used to create the biometric voice print, [0051] The biometric voice print is compared against previously stored voice prints for identifying a match…a feature matrix is also calculated from the spoken phrase…using the voice authentication algorithm 800 as used in enrollment. This feature matrix is compared against one or more reference matrices store in a voiceprint database, [Creation of a biometric voice print for comparison based on a feature matrix indicates the voiceprint to be a sub-sequence, i.e. one matrix, from the activation sequences, i.e. plurality of matrices, wherein a comparison of one feature matrix to a reference matrix which are containing the same spoken information, i.e. voiced information pass phrase ([0063]), will necessarily have the same amount of vectors to represent the same amount of speech, i.e. the pass phrase, as the vector comprising the matrix are gathered on a segment-level ([0063])]).
Claim(s) 4-5, 9-10, 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Di Mambro in view of Tuo, further in view of Gimenez et al. (US-20150088509-A1), hereinafter Gimenez.
Regarding claim 4, Di Mambro in view of Tuo discloses: the method according to claim 1.
Tuo further discloses:
said step of calculating an embedding for the first weighted activation sequence and an embedding for the second weighted activation sequence comprising the steps of:
calculating a unique weight vector from the tensor ([0039] This tensor may be transformed into a similarity matrix with dimensions N*M*N, which may allow for a similarity score to be generated for each pair of voice recordings and for each speaker [A similarity matrix, comprised of vectors for each audio input, indicates the matrix is comprising at least one weight, i.e. similarity, vector]);
calculating a first weighted activation sequence from the first activation sequence using the unique weight vector and a second weighted activation sequence from the second activation sequence using the unique weight vector ([0039] A predictive vector may be generated from the similarity matrix. The resulting predictive vector may have a size of N*M, and each entry in the predictive vector may include an indication of a speaker predicted to be associated with each of the N*M recordings [In view of the activation sequences, i.e. normalized feature vectors, of Di Mambro, determining a predictive vector based on the similarity matrix indicates the predictive vector is a first weighted activation function dependent upon the unique weight vector (contained within the similarity matrix). Further, this operation could be applied to the second activation sequence of Di Mambro without a change in functionality]).
Di Mambro in view of Tuo does not disclose:
calculating a mean and a covariance of the first and second weighted activation sequences;
concatenating the mean and the covariance of the first weighted activation sequence and of the second weighted activation sequence;
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence;
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence.
Gimenez discloses:
calculating a mean ([0036] average feature vectors of audio data may e.g. be calculated by calculating the mean) and a covariance ([Fig. 1D, [0019] The Gaussian classifier may be a full-covariance Gaussian classifier, e.g. a Gaussian classifier in which each Gaussian is described including a full covariance, [Determining a covariance classifier, wherein that classifier is used to compare audio data features, see Fig. 1D, in view of the record and obtained audio of Di Mambro]) of the first and second weighted activation sequences ([In view of the previously disclosed first and second weighted activation sequence of Di Mambro in view of Tuo which could be used as the audio data file and adaptation audio data file of Gimenez, respectively]);
concatenating the mean and the covariance of the first weighted activation sequence and of the second weighted activation sequence ([0101] a mean vector and a covariance matrix [Means in vector format and a covariance in matrix format indicates a required concatenation of means and covariances to develop the structure, in view of the first and second weighted activation sequences of Di Mambro in view of Tuo]);
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence ([In view of the previous concatenation of mean and covariance for the first weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation to be performed. As such, the first concatenation should result in embeddings as defined in this step]);
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence ([In view of the previous concatenation of mean and covariance for the second weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation is to be performed. As such, the first concatenation should result in embeddings as defined in this step.]).
Di Mambro, Tuo, and Gimenez are considered analogous art within audio signal authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro in view of Tuo to incorporate the teachings of Gimenez, because of the novel way to compare audio (Gimenez, [0004]).
Regarding claim 5, Di Mambro in view of Tuo discloses: the method according to claim 1.
Tuo further discloses:
said step of calculating an embedding comprising the steps of:
calculating a first weight vector from the tensor and the first activation sequence ([0060] This tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances [In view of the definition of weighted activation sequences in the instant app, [0071], which only discloses generation of weighted sequences through the tensor, indicating a tensor generating a similarity matrix represents at least a first weight vector, i.e. in view of the record and obtained audio sequences and associated activation sequence of Di Mambro indicating at least two vectors which could be weighted based on a similarity determination]);
calculating a second weight vector from the tensor and the second activation sequence ([In view of the second activation sequence of Di Mambro, applying the method disclosed in Tuo (above element) to a second activation sequence does not change the functionality of the weight, i.e. similarity, determination, indicating a second weight vector calculation in view of the multiple input signals/activation sequences of Di Mambro]);
calculating a first weighted activation sequence from the first activation sequence and the first weight vector ([0060] a predictive vector may be generated from the generated similarity matrix [Generating a predictive vector from a similarity matrix comprised of vectors, i.e. weighted activation sequences, wherein the similarity matrix is determined based on a comparison of activation sequences, in view of the activation sequences of Di Mambro, indicates the predictive vector is calculated to represent a first weighted activation sequence]);
calculating a second weighted activation sequence from the second activation sequence and the second weight vector ([In view of the second activation sequence of Di Mambro and the similarity matrix containing at least a second weight vector of Tuo, performing the same predictive vector calculation as applied to the first activation sequence (see above claim element) could be applied using the second activation sequence and weight vector without a change in functionality]).
Di Mambro in view of Tuo does not disclose:
calculating a mean and a covariance of the first and second weighted activation sequences;
concatenating the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence;
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence;
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence.
Gimenez discloses:
calculating a mean ([0036] average feature vectors of audio data may e.g. be calculated by calculating the mean) and a covariance ([Fig. 1D, [0019] The Gaussian classifier may be a full-covariance Gaussian classifier, e.g. a Gaussian classifier in which each Gaussian is described including a full covariance, [Determining a covariance classifier, wherein that classifier is used to compare audio data features, see Fig. 1D, in view of the record and obtained audio of Di Mambro]) of the first and second weighted activation sequences ([In view of the previously disclosed first and second weighted activation sequence of Di Mambro in view of Tuo which could be used as the audio data file and adaptation audio data file respectively]);
concatenating the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence ([0101] a mean vector and a covariance matrix [Means in vector format and a covariance in matrix format indicates a required concatenation of means and covariances to develop the structure, in view of the first and second weighted activation sequences of Di Mambro in view of Tuo]);
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence ([In view of the previous concatenation of mean and covariance for the first weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation to be performed. As such, the first concatenation should result in embeddings as defined in this step]);
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence ([In view of the previous concatenation of mean and covariance for the second weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation is to be performed. As such, the first concatenation should result in embeddings as defined in this step.]).
Di Mambro, Tuo, and Gimenez are considered analogous art within audio signal authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro in view of Tuo to incorporate the teachings of Gimenez, because of the novel way to compare audio signals based on mean and covariance parameters among the signals, improving a speaker recognition system's ability to detect adversarial speaker attacks (Gimenez, [0004]).
Regarding claim 9, Di Mambro in view of Tuo discloses: the electronic device according to claim 6.
Tuo further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said electronic device ([In view of the previously disclosed electronic device of Di Mambro]) to:
calculate a unique weight vector from the tensor ([0039] This tensor may be transformed into a similarity matrix with dimensions N*M*N, which may allow for a similarity score to be generated for each pair of voice recordings and for each speaker [A similarity matrix, comprised of vectors for each audio input, indicates the matrix is comprising at least one weight, i.e. similarity, vector]);
calculate a first weighted activation sequence from the first activation sequence using the unique weight vector and a second weighted activation sequence from the second activation sequence using the unique weight vector ([0039] A predictive vector may be generated from the similarity matrix. The resulting predictive vector may have a size of N*M, and each entry in the predictive vector may include an indication of a speaker predicted to be associated with each of the N*M recordings [In view of the activation sequences, i.e. normalized feature vectors, of Di Mambro, determining a predictive vector based on the similarity matrix indicates the predictive vector is a first weighted activation function dependent upon the unique weight vector (contained within the similarity matrix). Further, this operation could be applied to the second activation sequence of Di Mambro without a change in functionality]).
Di Mambro in view of Tuo does not disclose:
calculate a mean and a covariance of the first and second weighted activation sequences;
concatenate the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence;
concatenate the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence;
concatenate the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence.
Gimenez discloses:
calculate a mean ([0036] average feature vectors of audio data may e.g. be calculated by calculating the mean) and covariance ([Fig. 1D, [0019] The Gaussian classifier may be a full-covariance Gaussian classifier, e.g. a Gaussian classifier in which each Gaussian is described including a full covariance, [Determining a covariance classifier, wherein that classifier is used to compare audio data features, see Fig. 1D, in view of the record and obtained audio of Di Mambro]) of the first and second weighted activation sequences ([In view of the previously disclosed first and second weighted activation sequence of Di Mambro in view of Tuo which could be used as the audio data file and adaptation audio data file of Gimenez, respectively]);
concatenate the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence ([0101] a mean vector and a covariance matrix [Means in vector format and a covariance in matrix format indicates a required concatenation of means and covariances to develop the structure, in view of the first and second weighted activation sequences of Di Mambro in view of Tuo]);
concatenate the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence ([In view of the previous concatenation of mean and covariance for the first weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation to be performed. As such, the first concatenation should result in embeddings as defined in this step]);
concatenate the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence ([In view of the previous concatenation of mean and covariance for the second weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation is to be performed. As such, the first concatenation should result in embeddings as defined in this step.]).
Di Mambro, Tuo, and Gimenez are considered analogous art within audio signal authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro in view of Tuo to incorporate the teachings of Gimenez, because of the novel way to compare audio signals based on mean and covariance parameters among the signals, improving a speaker recognition system's ability to detect adversarial speaker attacks (Gimenez, [0004]).
Regarding claim 10, Di Mambro in view of Tuo discloses: the electronic device according to claim 6.
Tuo further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said electronic device ([In view of the previously disclosed electronic device of Di Mambro]) to:
calculate a first weight vector from the tensor and the first activation sequence ([0060] This tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances [In view of the definition of weighted activation sequences in the instant app, [0071], which only discloses generation of weighted sequences through the tensor, indicating a tensor generating a similarity matrix represents at least a first weight vector, i.e. in view of the record and obtained audio sequences and associated activation sequence of Di Mambro indicating at least two vectors which could be weighted based on a similarity determination]);
calculate a second weight vector from the tensor and the second activation sequence ([In view of the second activation sequence of Di Mambro, applying the method disclosed in Tuo (above element) to a second activation sequence does not change the functionality of the weight, i.e. similarity, determination, indicating a second weight vector calculation in view of the multiple input signals/activation sequences of Di Mambro]);
calculate a first weighted activation sequence from the first activation sequence and the first weight vector ([0060] a predictive vector may be generated from the generated similarity matrix [Generating a predictive vector from a similarity matrix comprised of vectors, i.e. weighted activation sequences, wherein the similarity matrix is determined based on a comparison of activation sequences, in view of the activation sequences of Di Mambro, indicates the predictive vector is calculated to represent a first weighted activation sequence]);
calculate a second weighted activation sequence from the second activation sequence and the second weight vector ([In view of the second activation sequence of Di Mambro and the similarity matrix containing at least a second weight vector of Tuo, performing the same predictive vector calculation as applied to the first activation sequence (see above claim element) could be applied using the second activation sequence and weight vector without a change in functionality]).
Di Mambro in view of Tuo does not disclose:
calculate a mean and a covariance of the first and second weighted activation sequences;
concatenate the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence;
concatenate the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence;
concatenate the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence.
Gimenez discloses:
calculate a mean ([0036] average feature vectors of audio data may e.g. be calculated by calculating the mean) and covariance ([Fig. 1D, [0019] The Gaussian classifier may be a full-covariance Gaussian classifier, e.g. a Gaussian classifier in which each Gaussian is described including a full covariance, [Determining a covariance classifier, wherein that classifier is used to compare audio data features, see Fig. 1D, in view of the record and obtained audio of Di Mambro]) of the first and second weighted activation sequences ([In view of the previously disclosed first and second weighted activation sequence of Di Mambro in view of Tuo which could be used as the audio data file and adaptation audio data file respectively]);
concatenate the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence ([0101] a mean vector and a covariance matrix [Means in vector format and a covariance in matrix format indicates a required concatenation of means and covariances to develop the structure, in view of the first and second weighted activation sequences of Di Mambro in view of Tuo]);
concatenate the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence ([In view of the previous concatenation of mean and covariance for the first weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation to be performed. As such, the first concatenation should result in embeddings as defined in this step]);
concatenate the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence ([In view of the previous concatenation of mean and covariance for the second weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation is to be performed. As such, the first concatenation should result in embeddings as defined in this step.]).
Di Mambro, Tuo, and Gimenez are considered analogous art within audio signal authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro in view of Tuo to incorporate the teachings of Gimenez, because of the novel way to compare audio signals based on mean and covariance parameters among the signals, improving a speaker recognition system's ability to detect adversarial speaker attacks (Gimenez, [0004]).
Regarding claim 14, Di Mambro in view of Tuo discloses: the non-transitory computer-readable storage medium according to claim 11.
Tuo further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said non-transitory computer-readable recording medium ([In view of the previously disclosed non-transitory computer-readable medium of Di Mambro]) to perform steps comprising:
calculating a unique weight vector from the tensor ([0039] This tensor may be transformed into a similarity matrix with dimensions N*M*N, which may allow for a similarity score to be generated for each pair of voice recordings and for each speaker [A similarity matrix, comprised of vectors for each audio input, indicates the matrix is comprising at least one weight, i.e. similarity, vector]);
calculating a first weighted activation sequence from the first activation sequence using the unique weight vector and a second weighted activation sequence from the second activation sequence using the unique weight vector ([0039] A predictive vector may be generated from the similarity matrix. The resulting predictive vector may have a size of N*M, and each entry in the predictive vector may include an indication of a speaker predicted to be associated with each of the N*M recordings [In view of the activation sequences, i.e. normalized feature vectors, of Di Mambro, determining a predictive vector based on the similarity matrix indicates the predictive vector is a first weighted activation function dependent upon the unique weight vector (contained within the similarity matrix). Further, this operation could be applied to the second activation sequence of Di Mambro without a change in functionality]).
Di Mambro in view of Tuo does not disclose:
calculating a mean and a covariance of the first and second weighted activation sequences;
concatenating the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence;
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence;
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence.
Gimenez discloses:
calculating a mean ([0036] average feature vectors of audio data may e.g. be calculated by calculating the mean) and a covariance ([Fig. 1D, [0019] The Gaussian classifier may be a full-covariance Gaussian classifier, e.g. a Gaussian classifier in which each Gaussian is described including a full covariance, [Determining a covariance classifier, wherein that classifier is used to compare audio data features, see Fig. 1D, in view of the record and obtained audio of Di Mambro]) of the first and second weighted activation sequences ([In view of the previously disclosed first and second weighted activation sequence of Di Mambro in view of Tuo which could be used as the audio data file and adaptation audio data file of Gimenez, respectively]);
concatenating the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence ([0101] a mean vector and a covariance matrix [Means in vector format and a covariance in matrix format indicates a required concatenation of means and covariances to develop the structure, in view of the first and second weighted activation sequences of Di Mambro in view of Tuo]);
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence ([In view of the previous concatenation of mean and covariance for the first weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation to be performed. As such, the first concatenation should result in embeddings as defined in this step]);
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence ([In view of the previous concatenation of mean and covariance for the second weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation is to be performed. As such, the first concatenation should result in embeddings as defined in this step.]).
Di Mambro, Tuo, and Gimenez are considered analogous art within audio signal authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro in view of Tuo to incorporate the teachings of Gimenez, because of the novel way to compare audio signals based on mean and covariance parameters among the signals, improving a speaker recognition system's ability to detect adversarial speaker attacks (Gimenez, [0004]).
Regarding claim 15, Di Mambro in view of Tuo discloses: the non-transitory computer-readable storage medium according to claim 11.
Tuo further discloses:
wherein the instructions when read and executed by said processor ([In view of the previously disclosed processor of Di Mambro]), cause said non-transitory computer-readable recording medium ([In view of the previously disclosed non-transitory computer-readable medium of Di Mambro]) to perform steps comprising:
calculating a first weight vector from the tensor and the first activation sequence ([0060] This tensor may be used to generate a similarity matrix based on a calculated similarity between each pair of utterances [In view of the definition of weighted activation sequences in the instant app, [0071], which only discloses generation of weighted sequences through the tensor, indicating a tensor generating a similarity matrix represents at least a first weight vector, i.e. in view of the record and obtained audio sequences and associated activation sequence of Di Mambro indicating at least two vectors which could be weighted based on a similarity determination]);
calculating a second weight vector from the tensor and the second activation sequence ([In view of the second activation sequence of Di Mambro, applying the method disclosed in Tuo (above element) to a second activation sequence does not change the functionality of the weight, i.e. similarity, determination, indicating a second weight vector calculation in view of the multiple input signals/activation sequences of Di Mambro]);
calculating a first weighted activation sequence from the first activation sequence and the first weight vector ([0060] a predictive vector may be generated from the generated similarity matrix [Generating a predictive vector from a similarity matrix comprised of vectors, i.e. weighted activation sequences, wherein the similarity matrix is determined based on a comparison of activation sequences, in view of the activation sequences of Di Mambro, indicates the predictive vector is calculated to represent a first weighted activation sequence]);
calculating a second weighted activation sequence from the second activation sequence and the second weight vector ([In view of the second activation sequence of Di Mambro and the similarity matrix containing at least a second weight vector of Tuo, performing the same predictive vector calculation as applied to the first activation sequence (see above claim element) could be applied using the second activation sequence and weight vector without a change in functionality]).
Di Mambro in view of Tuo does not disclose:
calculating a mean and a covariance of the first and second weighted activation sequences;
concatenating the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence;
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence;
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence.
Gimenez discloses:
calculating a mean ([0036] average feature vectors of audio data may e.g. be calculated by calculating the mean) and a covariance ([Fig. 1D, [0019] The Gaussian classifier may be a full-covariance Gaussian classifier, e.g. a Gaussian classifier in which each Gaussian is described including a full covariance, [Determining a covariance classifier, wherein that classifier is used to compare audio data features, see Fig. 1D, in view of the record and obtained audio of Di Mambro]) of the first and second weighted activation sequences ([In view of the previously disclosed first and second weighted activation sequence of Di Mambro in view of Tuo which could be used as the audio data file and adaptation audio data file respectively]);
concatenating the mean and covariance of the first weighted activation sequence and of the second weighted activation sequence ([0101] a mean vector and a covariance matrix [Means in vector format and a covariance in matrix format indicates a required concatenation of means and covariances to develop the structure, in view of the first and second weighted activation sequences of Di Mambro in view of Tuo]);
concatenating the concatenated mean and covariance of the first weighted activation sequence to calculate the embedding for the first weighted activation sequence ([In view of the previous concatenation of mean and covariance for the first weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation to be performed. As such, the first concatenation should result in embeddings as defined in this step]);
concatenating the concatenated mean and covariance of the second weighted activation sequence to calculate the embedding for the second weighted activation sequence ([In view of the previous concatenation of mean and covariance for the second weighted activation sequence, it is unclear to the examiner what a further concatenation is achieving. There appears to be concatenation of two previously concatenated elements, indicating no operation is to be performed. As such, the first concatenation should result in embeddings as defined in this step.]).
Di Mambro, Tuo, and Gimenez are considered analogous art within audio signal authentication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Di Mambro in view of Tuo to incorporate the teachings of Gimenez, because of the novel way to compare audio signals based on mean and covariance parameters among the signals, improving a speaker recognition system's ability to detect adversarial speaker attacks (Gimenez, [0004]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Weinstein et al. (US-20150154002-A1) discloses “Characteristics of a speaker are estimated using speech processing and machine learning. The characteristics of the speaker are used to automatically customize a user interface of a client device for the speaker.” (abstract). Specifically, see the flow of Fig. 3 in view of the supervectors of Weinstein. See entire document.
Khitrov et al. (US-20190325880-A1) discloses “A computer-implemented method for verifying identity of a speaker is proposed. A low dimensional p-vector based on a speech of the speaker is extracted from the generated high dimensional speaker model and is then compared with the stored specific speaker's p-vector obtained previously during the enrollment process. The resulting biometric score is then used to determine whether to verify the speaker, or not.” (abstract). See disclosure of comparison of p-vectors as disclosed in Fig. 2. See entire document.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THEODORE JOHN WITHEY whose telephone number is (703)756-1754. The examiner can normally be reached Monday - Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at (571) 272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THEODORE WITHEY/Examiner, Art Unit 2655 /ANDREW C FLANDERS/Supervisory Patent Examiner, Art Unit 2655
Prosecution Insights