DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Roh U.S. PAP 2020/0168230 A1 in view of Kopuri U.S. Patent No. 11,893,999 B1.
Regarding claim 1 Roh teaches an electronic device (method and apparatus for processing voice data of a speech received from a speaker are provided, see abstract), comprising:
a microphone (electronic device 100 may receive the speech from the speaker via a microphone, see par. [0133]);
at least one processor including processing circuitry (controller 120 may include a processor 122, see par. [0093]);
and memory including one or more storage media storing instructions, wherein the instructions, when executed by the at least one processor individually or collectively ( a data training module 123, and a memory 124. The controller 120 may be realized as a computer-readable recording medium including software, hardware, or the combination of software and hardware, see par. [0093]), cause the electronic device to:
obtain first vectors respectively corresponding to the plurality of frames, determine a speaker corresponding to each of the first vectors by using groups in which the first vectors are respectively included, which are obtained by grouping the first vectors, and second vectors stored in the memory, wherein the grouping of the first vectors, and the second vectors are performed by the at least one processor of the electronic device (extracting a speaker feature vector from the voice data of the speech received from a speaker, generating a speaker feature map by positioning the extracted speaker feature vector at a specific position on a multi-dimensional vector space, forming a plurality of clusters indicating features of voices of a plurality of speakers by grouping at least one speaker feature vector positioned on the speaker feature map, and classifying the plurality of speakers according to the plurality of formed clusters, see par. [0009]),
store, within the memory, information which is determined by using a speaker respectively corresponding to the first vectors, the information indicating a speaker of at least one time section of the audio data ( extracting a speaker feature vector from voice data of the received speech and mapping the extracted speaker feature vector to a cluster from among the plurality of clusters on the speaker feature map, and registering the identification information of the user to the mapped cluster from among the plurality of clusters, see par. [0011]),
However Roh does not teach while obtaining audio data by using the microphone, obtain a plurality of frames by dividing the audio data, and based on a total number of the first vectors and the second vectors greater than a preset number, delete at least one vector among the first vectors and the second vectors from the memory to adjust the total number to be lower than or equal to the preset number.
In the same field of endeavor Kopuri teaches Audio data may also be associated with a number of speech frames of the audio data, as illustrated in FIG. 4. If a user recognition feature vector is generated from audio data that does not include many frames, the user recognition feature vector may not be an accurate representation of the voice of the user that provided the user input. As such, the server(s) 120 may be configured to use a user recognition feature vector to generate an anonymous voice profile feature vector only if the user recognition feature vector was generated from audio data satisfying a condition (e.g., including at least a threshold amount of frames, see col. 8 lines 23-39) (obtaining audio data by using the microphone, obtain a plurality of frames by dividing the audio data). When the clustering component 1104 generates an anonymous voice profile feature vector, the clustering component 1104 may cause the audio data, used to generate the user recognition feature vectors from which the anonymous voice profile feature vector was generated, to be deleted from the audio storage 375. This ensures the audio storage 375 only includes audio data that has not been used to generate an anonymous voice profile feature vector, see col. 26 lines 17-24 (delete at least one vector among the first vectors and the second vectors from the memory to adjust the total number to be lower than or equal to the preset number).
It would have been obvious to combine the Roh invention with the teachings of Kopuri for the benefit of ensuring the audio storage only includes audio data that has not been used to generate an anonymous voice profile feature vector, see col. 26 lines 17-24.
Regarding claim 2 Kopuri teaches the electronic device of claim 1, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: determine whether to adjust the preset number by using a duration required for grouping the preset number of vectors (The storage may be updated with a new batch of user recognition feature vectors on a periodic basis (e.g., every 30 minutes, once any hours, etc.), see col. 6, lines 60-67).
Regarding claim 3 Kopuri teaches the electronic device of claim 2, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: in response to the duration longer than a preset duration, decrease the preset number (the filter component 1102 determines the top scoring confidence value satisfies a condition (e.g., the threshold confidence), the filter component 1102 may delete the audio data (and associated user recognition confidence data 795) from the audio storage 375, see col. 24 lines 14-46).
Regarding claim 4 Kopuri teaches the electronic device of claim 1, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: based on the total number of the first vectors and the second vectors greater than the preset number, determine whether to store each of the first vectors and the second vectors by using distribution of the first vectors and the second vectors within a vector space ( The user recognition component 395 may send a signal to the anonymous voice profile feature vector storage 385 and/or the explicit voice profile feature vector storage 365, with the signal requesting only feature vectors 1005 associated with the device identifier, the group profile identifier, and/or the user profile identifiers represented in the metadata. This limits the universe of possible feature vectors 1005 the user recognition component 395 should consider at runtime and thus decreases the amount of time to perform user recognition by decreasing the amount of feature vectors 1005 needed to be processed, see col. 19 lines 1-30).
Regarding claim 5 Kopuri teaches the electronic device of claim 4, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: in a state that a plurality of speakers with respect to the plurality of frames are determined, determine whether to store each of the first vectors and the second vectors by using similarities between the first vectors and the second vectors which are determined by using the groups within the vector space respectively corresponding to the plurality of speakers (The system may generate an implicit voice profile from user inputs having similar characteristics. After an implicit voice profile is generated, the system may receive a user input having speech characteristics similar to that of the implicit voice profile, see abstract).
Regarding claim 6 Kopuri teaches the electronic device of claim 4, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: in a state that a plurality of speakers with respect to the plurality of frames are determined, determine whether to store each of the first vectors, and the second vectors, by using distances between centroid vectors of the groups within the vector space respectively corresponding to the plurality of speakers and the first vectors and the second vectors (DBSCAN has two parameters, ε and minPts. ε dictates when two user recognition feature vectors are considered close enough to be grouped into a single cluster. Items with at least minPts neighbors within a distance of c may be defined as “core points.”, see col. 25, lines 3-10).
Regarding claim 7 Kopuri teaches the electronic device of claim 1, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: in response to detecting a voice section indicating that voice is recorded from the audio data, obtain the plurality of frames by dividing the voice section (Audio data may also be associated with a number of speech frames of the audio data, as illustrated in FIG. 4).
Regarding claim 8 Kopuri teaches the electronic device of claim 7, wherein lengths of the plurality of frames are identical to each other, and wherein the frames are at least partially overlapped to each other in a time domain (each frame may include twenty-five (25) milliseconds (m/s) of audio data, with an overlap of the next frame of 10 ms of data, see col. 15 lines 20-48).
Regarding claim 9 Roh teaches the electronic device of claim 1, further comprising: a display, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: in response to an input indicating to cease obtaining of the audio data, display on the display a screen associated with the information ( a display, and the controller may further be configured to recognize the speaker of the received speech based on the predicted relationship among the speakers and display a personalized service synchronized to the recognized speaker on the display, see par. [0031]).
Regarding claim 10 Kopuri teaches the electronic device of claim 1, wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the electronic device to: while obtaining the audio data, compare the preset number to a total number of the first vectors and the second vectors to maintain a number of vectors, which are stored in the memory and associated with the audio data, as the preset number (processing may be performed over time with respect to various user inputs received by a device 110a or group of devices 110a. As such, the server(s) 120 may accumulate and store various user recognition feature vectors representing user inputs that could not be attributed to any already stored voice profile (either anonymous or explicit). The storage may be updated with a new batch of user recognition feature vectors on a periodic basis (e.g., every 30 minutes, once any hours, etc.), see col. 6 lines 60-67).
Regarding claim 11 Roh teaches a method of an electronic device including a microphone (method and apparatus for processing voice data of a speech received from a speaker are provided, see abstract), the method comprising:
determining a speaker corresponding to each of the first vectors by using groups in which the first vectors are respectively included, which are obtained by grouping the first vectors, and second vectors stored in memory of the electronic device, wherein the grouping of the first vectors, and the second vectors is performed by at least one processor of the electronic device (extracting a speaker feature vector from the voice data of the speech received from a speaker, generating a speaker feature map by positioning the extracted speaker feature vector at a specific position on a multi-dimensional vector space, forming a plurality of clusters indicating features of voices of a plurality of speakers by grouping at least one speaker feature vector positioned on the speaker feature map, and classifying the plurality of speakers according to the plurality of formed clusters, see par. [0009]);
storing, within the memory, information that is determined by using a speaker respectively corresponding to the first vectors, the information indicating a speaker of at least one time section of the audio data( extracting a speaker feature vector from voice data of the received speech and mapping the extracted speaker feature vector to a cluster from among the plurality of clusters on the speaker feature map, and registering the identification information of the user to the mapped cluster from among the plurality of clusters, see par. [0011]).
However Roh does not teach while obtaining audio data by using the microphone, obtaining a plurality of frames by dividing the audio data; obtaining first vectors respectively corresponding to the plurality of frames; and based on a total number of the first vectors and the second vectors greater than a preset number, deleting at least one vector among the first vectors and the second vectors from the memory to adjust the total number to be lower than or equal to the preset number.
In the same field of endeavor Kopuri teaches Audio data may also be associated with a number of speech frames of the audio data, as illustrated in FIG. 4. If a user recognition feature vector is generated from audio data that does not include many frames, the user recognition feature vector may not be an accurate representation of the voice of the user that provided the user input. As such, the server(s) 120 may be configured to use a user recognition feature vector to generate an anonymous voice profile feature vector only if the user recognition feature vector was generated from audio data satisfying a condition (e.g., including at least a threshold amount of frames, see col. 8 lines 23-39) (obtaining audio data by using the microphone, obtain a plurality of frames by dividing the audio data). When the clustering component 1104 generates an anonymous voice profile feature vector, the clustering component 1104 may cause the audio data, used to generate the user recognition feature vectors from which the anonymous voice profile feature vector was generated, to be deleted from the audio storage 375. This ensures the audio storage 375 only includes audio data that has not been used to generate an anonymous voice profile feature vector, see col. 26 lines 17-24 (delete at least one vector among the first vectors and the second vectors from the memory to adjust the total number to be lower than or equal to the preset number).
It would have been obvious to combine the Roh invention with the teachings of Kopuri for the benefit of ensuring the audio storage only includes audio data that has not been used to generate an anonymous voice profile feature vector, see col. 26 lines 17-24.
Regarding claim 12 Kopuri teaches the method of claim 11, further comprising: determining whether to adjust the preset number by using a duration required to grouping the preset number of vectors (The storage may be updated with a new batch of user recognition feature vectors on a periodic basis (e.g., every 30 minutes, once any hours, etc.), see col. 6, lines 60-67).
Regarding claim 13 Kopuri teaches the method of claim 12, wherein the determining whether to adjust the preset number comprising: in response to the duration longer than a preset duration, decreasing the preset number (the filter component 1102 determines the top scoring confidence value satisfies a condition (e.g., the threshold confidence), the filter component 1102 may delete the audio data (and associated user recognition confidence data 795) from the audio storage 375, see col. 24 lines 14-46).
Regarding claim 14 Kopuri teaches the method of claim 11, wherein the storing the preset number of the vectors comprising: based on the total number of the first vectors and the second vectors greater than the preset number, determining whether to store each of the first vectors and the second vectors by using distribution of the first vectors and the second vectors within a vector space ( The user recognition component 395 may send a signal to the anonymous voice profile feature vector storage 385 and/or the explicit voice profile feature vector storage 365, with the signal requesting only feature vectors 1005 associated with the device identifier, the group profile identifier, and/or the user profile identifiers represented in the metadata. This limits the universe of possible feature vectors 1005 the user recognition component 395 should consider at runtime and thus decreases the amount of time to perform user recognition by decreasing the amount of feature vectors 1005 needed to be processed, see col. 19 lines 1-30).
Regarding claim 15 Kopuri teaches the method of claim 14, wherein the storing the preset number of the vectors comprising: in a state that a plurality of speakers with respect to the plurality of frames are determined, determine whether to store each of the first vectors and the second vectors by using similarities between the first vectors and the second vectors which are determined by using the groups within the vector space respectively corresponding to the plurality of speakers (The system may generate an implicit voice profile from user inputs having similar characteristics. After an implicit voice profile is generated, the system may receive a user input having speech characteristics similar to that of the implicit voice profile, see abstract)
Regarding claim 16 Kopuri teaches the method of claim 14, wherein the storing the preset number of the vectors comprising: in a state that a plurality of speakers with respect to the plurality of frames are determined, determining whether to store each of the first vectors, and the second vectors, by using distances between centroid vectors of the groups within the vector space respectively corresponding to the plurality of speakers and the first vectors and the second vectors (DBSCAN has two parameters, ε and minPts. ε dictates when two user recognition feature vectors are considered close enough to be grouped into a single cluster. Items with at least minPts neighbors within a distance of c may be defined as “core points.”, see col. 25, lines 3-10).
Regarding claim 17 Kopuri teaches the method of claim 11, further comprising: in response to detecting a voice section indicating that voice is recorded from the audio data, obtaining the plurality of frames by dividing the voice section (Audio data may also be associated with a number of speech frames of the audio data, as illustrated in FIG. 4).
Regarding claim 18 Kopuri teaches the method of claim 17, wherein lengths of the plurality of frames are identical to each other, and wherein the plurality of frames are at least partially overlapped to each other in a time domain (each frame may include twenty-five (25) milliseconds (m/s) of audio data, with an overlap of the next frame of 10 ms of data, see col. 15 lines 20-48).
Regarding claim 19 Roh teaches the method of claim 11, further comprising: in response to an input indicating to cease obtainment of the audio data, displaying on a display of the electronic device a screen associated with the information ( a display, and the controller may further be configured to recognize the speaker of the received speech based on the predicted relationship among the speakers and display a personalized service synchronized to the recognized speaker on the display, see par. [0031])..
Regarding claim 20 Roh teaches one or more non-transitory computer-readable storage media storing one or more computer programs including computer-executable instructions that, when executed by one or more processors of an electronic device, individually or collectively ( a computer program product including a non-transitory computer-readable recording medium is provided. The computer program product includes instructions to control an electronic device to receive a speech from a speaker, extract a speaker feature vector from the voice data of the speech that is received, generate a speaker feature map by positioning the extracted speaker feature vector at a specific position on a multi-dimensional vector space, see par. [0034]), cause the electronic device to perform operations:
determining a speaker corresponding to each of the first vectors by using groups in which the first vectors are respectively included, which are obtained by grouping the first vectors, and second vectors stored in memory of the electronic device, wherein the grouping of the first vectors, and the second vectors is performed by at least one processor of the electronic device(extracting a speaker feature vector from the voice data of the speech received from a speaker, generating a speaker feature map by positioning the extracted speaker feature vector at a specific position on a multi-dimensional vector space, forming a plurality of clusters indicating features of voices of a plurality of speakers by grouping at least one speaker feature vector positioned on the speaker feature map, and classifying the plurality of speakers according to the plurality of formed clusters, see par. [0009]);
storing, within the memory, information that is determined by using a speaker respectively corresponding to the first vectors, the information indicating a speaker of at least one time section of the audio data( extracting a speaker feature vector from voice data of the received speech and mapping the extracted speaker feature vector to a cluster from among the plurality of clusters on the speaker feature map, and registering the identification information of the user to the mapped cluster from among the plurality of clusters, see par. [0011]).
However Roh does not teach while obtaining audio data by using a microphone of the electronic device, obtaining a plurality of frames by dividing the audio data; obtaining first vectors respectively corresponding to the plurality of frames; and based on a total number of the first vectors and the second vectors greater than a preset number, deleting at least one vector among the first vectors and the second vectors from the memory to adjust the total number to be lower than or equal to the preset number.
In the same field of endeavor Kopuri teaches Audio data may also be associated with a number of speech frames of the audio data, as illustrated in FIG. 4. If a user recognition feature vector is generated from audio data that does not include many frames, the user recognition feature vector may not be an accurate representation of the voice of the user that provided the user input. As such, the server(s) 120 may be configured to use a user recognition feature vector to generate an anonymous voice profile feature vector only if the user recognition feature vector was generated from audio data satisfying a condition (e.g., including at least a threshold amount of frames, see col. 8 lines 23-39) (obtaining audio data by using the microphone, obtain a plurality of frames by dividing the audio data). When the clustering component 1104 generates an anonymous voice profile feature vector, the clustering component 1104 may cause the audio data, used to generate the user recognition feature vectors from which the anonymous voice profile feature vector was generated, to be deleted from the audio storage 375. This ensures the audio storage 375 only includes audio data that has not been used to generate an anonymous voice profile feature vector, see col. 26 lines 17-24 (delete at least one vector among the first vectors and the second vectors from the memory to adjust the total number to be lower than or equal to the preset number).
It would have been obvious to combine the Roh invention with the teachings of Kopuri for the benefit of ensuring the audio storage only includes audio data that has not been used to generate an anonymous voice profile feature vector, see col. 26 lines 17-24.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ma ‘471 teaches a voice data processing method performed by a voice interaction device. The method includes acquiring historical voice data, acquiring historical voice feature vectors corresponding to the historical voice data, and performing clustering on the historical voice feature vectors to obtain a voice feature cluster, see par. [0007].
Sidi ‘887 teaches a method of diarization of audio data, audio data is segmented into a plurality of utterances. Each utterance is represented as an utterance model representative of a plurality of feature vectors. The utterance models are clustered, see abstract.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Ortiz-Sanchez whose telephone number is (571)270-3711. The examiner can normally be reached Monday- Friday 9AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ORTIZ-SANCHEZ/Primary Examiner, Art Unit 2656
Prosecution Insights