DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This communication is responsive to the applicant’s amendment dated 2/27/2026. The applicant amended claims 1, 2, and 9. Next, the applicant has cancelled claims 14 and 22. Lastly, the applicant has added new claims 25 and 26.
Response to Arguments
Applicant’s arguments, see Remarks (pg. 9, line 28 – pg. 10, line 5), filed 2/27/2026, with respect to the title of the application have been fully considered and are persuasive. The objection of specification has been withdrawn.
Applicant’s arguments, see Remarks (pg. 10, line 6 – pg. 12, line 6), filed 2/27/2026, with respect to claims 2-9, 13-15, 17-22, and 24 have been fully considered and are persuasive. The 35 U.S.C. 101 rejection of claims 2-9, 13-15, 17-22, and 24 has been withdrawn.
Applicant's arguments with respect to the prior art rejection filed 2/27/2026 have been fully considered but they are not persuasive.
First the applicant argues that Gorin involves a first segment that occurs first in time while in the present application multiple speech segments compromise multiple first segments with different lengths than the second segment. Additionally, the applicant argues that there is no second segment in Gorin. The examiner respectfully disagrees. FIG. 6 of Gorin shows a graphical comparison of the number of segments versus duration. The examiner interprets longer segments as the multiple first segment and the shorter ones as the second segment. Next, the applicant argues that Gorin discloses all obtained segments are clustered whereas the application claims that only the first segments are clustered and the second are assigned to a class to obtain role separation. The examiner respectfully disagrees. Gorin discusses clustering [Column 6, line 62 – Column 8, line 13; FIG. 4] and states that clustering is performed on group of segments based off distance measurements [Column 7, line 33- Column 7, line 44]. Here the examiner interprets the distance to be associated with the size of the segments and therefore, interprets that the clustering to happen among the first (longer) segments. Additionally, Gorin discusses labelling segments which tracks speakers in a telephone conversation ([Column 4, line 6 – Column 4, line 12]; [Column 6, line 58-61]; [Column 8, line 51-54]) which examiner interprets as analogous as classification based off roles in a conversation. Therefore, the 35 U.S.C. 102 rejection is maintained.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 7, and 9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gorin et al. US 7295970 B1 (hereinafter Gorin).
Regarding independent claim 1, Gorin teaches a speech processing method, comprising: obtaining single-channel speech corresponding to multiple participating roles collected by a conference system ([Column 5, line 1-4] “If the input speech sample 202 included more than two speakers, then the speaker segmentation process 250 would generate additional speaker segmentations”; [Claim 16] “wherein the speech data is one of a telephone conversation between two or more speakers; and archived recorded broadcast news program; and a recorded meeting between multiple speakers”);
segmenting the single-channel speech according to role change point information in the single-channel speech to obtain multiple speech segments, wherein the role change point information is used to indicate a position where a speaking role changes in the single-channel speech, the multiple speech segments comprise multiple first segments and at least one second segment, and a length of any first segment is greater than a length of any second segment (FIG. 2, 202, 250, 206, 208; [Column 4, line 65 – Column 5, line 1] “The output of the speaker segmentation process 250 is two distinct segmentations, speaker segmentation 206 and speaker segmentation 208, which correspond to two speakers”; FIG. 6, [Column 9, line 52-60] “Many of these short duration segments are less than 1 second. The median segment duration is 2.74 seconds while the average segment duration is 3.75 seconds”; FIG 6; [Column 6, line 62 – Column 8, line 13; FIG. 4]; [Column 7, line 33- Column 7, line 44]; [Column 4, line 6 – Column 4, line 12]; [Column 6, line 58-61]; [Column 8, line 51-54]);
performing clustering on the multiple first segments, and assigning the at least one second segment to a class obtained after the clustering, to obtain a role separation result of the single- channel speech; and (FIG. 4, [Column 6, line 63-66] “The segments obtained by scanning the input speech sample with the windowed GLR function, as described above, are clustered to associate groups of segments with different speakers.”; [Column 7, line 14-16] “Thus all segments grouped with that first segment should be labeled as spoken by the representative”, examiner interprets label as the class);
outputting speaking text corresponding to each participating role according to the role separation result and text information corresponding to the single-channel speech ([Column 2, line 4-12] “Unsupervised segmentation of multi-speaker speech data has applications in, … speech models in order to improve the quality of ASR transcriptions, tracking speaker-specific segments in telephone conversations to aid in surveillance applications”, 112 & 114).
Regarding independent claim 2, Gorin teaches a speech processing method, comprising:
obtaining to-be-processed speech ([Column 5, line 1-4] “If the input speech sample 202 included more than two speakers, then the speaker segmentation process 250 would generate additional speaker segmentations”; [Claim 16] “wherein the speech data is one of a telephone conversation between two or more speakers; and archived recorded broadcast news program; and a recorded meeting between multiple speakers”);
segmenting the to-be-processed speech according to role change point information in the to-be- processed speech to obtain multiple speech segments, wherein the role change point information is used to indicate a position where a speaking role changes in the to-be-processed speech, the multiple speech segments comprise multiple first segments and at least one second segment, and a length of any first segment is greater than a length of any second segment (FIG. 2, 202, 250, 206, 208; [Column 4, line 65 – Column 5, line 1] “The output of the speaker segmentation process 250 is two distinct segmentations, speaker segmentation 206 and speaker segmentation 208, which correspond to two speakers”; FIG. 6, [Column 9, line 52-60] “Many of these short duration segments are less than 1 second. The median segment duration is 2.74 seconds while the average segment duration is 3.75 seconds”);
performing clustering on the multiple first segments, and assigning the at least one second segment to a class obtained after the clustering, to obtain a role separation result of the to-be- processed speech; and (FIG. 4, [Column 6, line 63-66] “The segments obtained by scanning the input speech sample with the windowed GLR function, as described above, are clustered to associate groups of segments with different speakers.”; [Column 7, line 14-16] “Thus all segments grouped with that first segment should be labeled as spoken by the representative”, examiner interprets label as the class).
outputting speaking text corresponding to each role according to the role separation result of the to-be-processed speech ([Column 2, line 4-12] “Unsupervised segmentation of multi-speaker speech data has applications in, … speech models in order to improve the quality of ASR transcriptions, tracking speaker-specific segments in telephone conversations to aid in surveillance applications”, 112 & 114).
Regarding claim 7 Gorin teaches wherein performing the clustering on the multiple first segments comprises:
traversing 2 to a preset class quantity, performing clustering on the multiple first segments by a supervised clustering algorithm under traversed class quantities to obtain clustering results corresponding to the class quantities ([Column 7, line 17-21] “The input to the clustering procedure is a table of pairwise distances between each segment and every other segment. The following procedure is used to generate such a table. Each segment is modeled by a low-order (typically 2- or 4-component) GMM.”);
determining, according to the clustering results corresponding to different class quantities, a quantity of roles and a clustering result corresponding to the to-be-processed speech ([Column 7, line 36-38] “At each iteration, the clustering procedure merges two groups to form a new group such that the merger produces the smallest increase in distance”).
Regarding independent claim 9, Gorin teaches a speech processing method, comprising:
obtaining to-be-processed speech ([Column 5, line 1-4] “If the input speech sample 202 included more than two speakers, then the speaker segmentation process 250 would generate additional speaker segmentations”; [Claim 16] “wherein the speech data is one of a telephone conversation between two or more speakers; and archived recorded broadcast news program; and a recorded meeting between multiple speakers”);
segmenting the to-be-processed speech to obtain multiple speech segments, wherein the multiple speech segments comprise multiple first segments and at least one second segment with lower credibility than the first segments (FIG. 2, 202, 250, 206, 208; [Column 4, line 65 – Column 5, line 1] “The output of the speaker segmentation process 250 is two distinct segmentations, speaker segmentation 206 and speaker segmentation 208, which correspond to two speakers”; FIG. 6, [Column 9, line 52-60] “Many of these short duration segments are less than 1 second. The median segment duration is 2.74 seconds while the average segment duration is 3.75 seconds”);
performing clustering on the multiple first segments, and assigning the at least one second segment to a class obtained after the clustering, to obtain a role separation result of the to-be- processed speech; and (FIG. 4, [Column 6, line 63-66] “The segments obtained by scanning the input speech sample with the windowed GLR function, as described above, are clustered to associate groups of segments with different speakers.”; [Column 7, line 14-16] “Thus all segments grouped with that first segment should be labeled as spoken by the representative”, examiner interprets label as the class);
outputting speaking text corresponding to each role according to the role separation result of the to-be-processed speech ([Column 2, line 4-12] “Unsupervised segmentation of multi-speaker speech data has applications in, … speech models in order to improve the quality of ASR transcriptions, tracking speaker-specific segments in telephone conversations to aid in surveillance applications”, 112 & 114);
wherein credibility of a speech segment is used to characterize credibility of a clustering result obtained by clustering based on the speech segment ([Column 12, line 11-15] “The ability to segment speakers accurately increases for longer segment durations and the segments do not need to be located with great precision, so that a detectability criterion of pdet=0.5 is adequate”; Table 1 & 2; FIG. 8, [Column 11, line 50-54] “These effects can be seen more clearly by examining clustering performance in terms of the measurements previously discussed. FIG. 8 shows clustering performance for the same two values of lrthr examined in Table 1 as a function of segmentation/modeling iteration”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-5, 13, 15, 17-19, 21, and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Gorin as shown above in claim 1, in further view of Fujita et al. US 20220254352 A1 (hereinafter Fujita).
Regarding claim 3, Gorin teaches all of the limitations of claim 2, upon which claim 3 depends.
Gorin fails to teach wherein segmenting the to-be-processed speech according to the role change point information in the to-be-processed speech to obtain the multiple speech segments comprises: determining at least one valid speech segment in the to-be-processed speech through voice activity endpoint detection ([0014] “using the diarization of the audio input (and/or portions of the audio input), the audio analysis platform may more accurately determine individual voice activity relative to previously used complex processes to separate speech from the audio input, recognize individual speaker identity based on characteristics (e.g., pitch, tone, frequency, wavelength, and/or the like) of the speech, and/or the like”);
However, Fujita teaches wherein segmenting the to-be-processed speech according to the role change point information in the to-be-processed speech to obtain the multiple speech segments comprises: determining at least one valid speech segment in the to-be-processed speech through voice activity endpoint detection ([0014] “using the diarization of the audio input (and/or portions of the audio input), the audio analysis platform may more accurately determine individual voice activity relative to previously used complex processes to separate speech from the audio input, recognize individual speaker identity based on characteristics (e.g., pitch, tone, frequency, wavelength, and/or the like) of the speech, and/or the like”);
performing role change point detection on the valid speech segment, and segmenting the at least one valid speech segment into the multiple speech segments according to obtained role change point information (FIG. 1, 140, [0028] “the audio analysis platform may generate an annotation (e.g., a voice activity label) that identifies which of the speakers (e.g., which of Speaker 1 to Speaker N) is currently speaking during a particular portion of the audio”);
wherein each speech segment is speech corresponding to a single role ([0028] “Using the DNN, as described herein, the audio analysis platform may determine and provide a diarization output for that portion of the audio input that indicates that Speaker 1 and Speaker 3 are actively speaking during that portion of the audio input”).
Gorin in view of Fujita are considered to be analogous to the claimed invention because both are the same field of multi-speaker speech segmentation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the techniques unsupervised segmentation of telephone conversations by speaker of Gorin with the technique of voice activity detection taught by Fujita in order to process, using a neural network, a portion of the audio input to determine voice activity of the plurality of speakers during the portion of the audio input (see Fujita [Abstract]).
Regarding claim 4, Gorin in view of Fujita teaches all of the limitations of claim 3, upon which claim 4 depends.
Additionally, Gorin teaches wherein performing the role change point detection on the valid speech segment comprises: determining at least one speech window corresponding to the valid speech segment based on a preset window length and/or a sliding duration, and extracting a feature of the speech window ([Column 4, line 49-52] “In order to resolve short segments, the data window used to detect acoustic changes and mark the segments must also be short to avoid including more than one speaker change in the window”);
determining the role change point information according to a similarity between features of adjacent speech windows ([Column 5, line 65 – Column 6, line 3] “To determine the location of boundaries between speaker segments, the GLR function is calculated over successive overlapping windows throughout the data sample”).
Regarding claim 5, Gorin in view of Fujita teaches all of the limitations of claim 4, upon which claim 5 depends.
Additionally, Gorin teaches wherein determining the at least one speech window corresponding to the valid speech segment based on the preset window length and/or the sliding duration, and extracting the feature of the speech window comprises: performing parallelization processing on respective valid speech segments by using multiple threads, and for each valid speech segment, determining at least one speech window corresponding to the valid speech segment based on the preset window length and/or the sliding duration, and extracting the feature of the speech window ([Column 6, line 5-14] “or the GLR to perform well, the window should be long enough to obtain stable statistics yet short enough to avoid containing more than one speaker segment change”);
correspondingly, segmenting the at least one valid speech segment into the multiple speech segments according to the obtained role change point information comprises: splicing features obtained after the parallelization processing in chronological order, and segmenting the at least one valid speech segment into the multiple speech segments in combination with the role change point information (FIG. 3, [Column 8, line 15-38] “the segmentation modeling and resegmentation 256 is iterated three times in order to obtain stable segmentations. The segmentations are stable when the difference between segmentations from one iteration to the next iteration is below a specified threshold”).
Regarding claim 13, 23, and 24 Gorin teaches all of the limitations of claim 2, 1, and 9 upon which claim 13, 23, and 24 depends.
Gorin fails to teach a speech processing device, comprising: at least one processor; and
a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the speech processing device.
However, Fujita teaches a speech processing device, comprising:
at least one processor (FIG. 5, 520); and
a memory communicatively connected with the at least one processor (530);
wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the speech processing device ([0066] “Device 500 may perform one or more processes described herein. Device 500 may perform these processes based on processor 520 executing software instructions stored by a non-transitory computer-readable medium, such as memory 530 and/or storage component 540.”).
Gorin in view of Fujita are considered to be analogous to the claimed invention because both are the same field of multi-speaker speech segmentation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the techniques unsupervised segmentation of telephone conversations by speaker of Gorin with the technique using general purpose computer elements taught by Fujita in order to process, using a neural network, a portion of the audio input to determine voice activity of the plurality of speakers during the portion of the audio input (see Fujita [Abstract]).
Regarding claim 15, Gorin teaches all of the limitations of claim 2, upon which claim 15 depends.
Gorin fails to teach a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and when a processor executes the computer execution instructions
However, Fujita teaches a non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and when a processor executes the computer execution instructions ([0066] “Device 500 may perform one or more processes described herein. Device 500 may perform these processes based on processor 520 executing software instructions stored by a non-transitory computer-readable medium, such as memory 530 and/or storage component 540. As used herein, the term “computer-readable medium” refers to a non-transitory memory device”)
Gorin in view of Fujita are considered to be analogous to the claimed invention because both are the same field of multi-speaker speech segmentation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the techniques unsupervised segmentation of telephone conversations by speaker of Gorin with the technique using general purpose computer elements taught by Fujita in order to process, using a neural network, a portion of the audio input to determine voice activity of the plurality of speakers during the portion of the audio input (see Fujita [Abstract]).
Regarding claim 17, Gorin in view of Fujita teaches all of the limitations of claim 13, upon which claim 17 depends.
Additionally, Fujita teaches wherein the instructions are executed by the at least one processor to cause the speech processing device to execute the following operations: determining at least one valid speech segment in the to-be-processed speech through voice activity endpoint detection ([0014] “using the diarization of the audio input (and/or portions of the audio input), the audio analysis platform may more accurately determine individual voice activity relative to previously used complex processes to separate speech from the audio input, recognize individual speaker identity based on characteristics (e.g., pitch, tone, frequency, wavelength, and/or the like) of the speech, and/or the like.”);
performing role change point detection on the valid speech segment, and segmenting the at least one valid speech segment into the multiple speech segments according to obtained role change point information (FIG. 1, 140, [0028] “the audio analysis platform may generate an annotation (e.g., a voice activity label) that identifies which of the speakers (e.g., which of Speaker 1 to Speaker N) is currently speaking during a particular portion of the audio.”);
wherein each speech segment is speech corresponding to a single role ([0028] “Using the DNN, as described herein, the audio analysis platform may determine and provide a diarization output for that portion of the audio input that indicates that Speaker 1 and Speaker 3 are actively speaking during that portion of the audio input.”).
Regarding claim 18, Gorin in view of Fujita teaches all of the limitations of claim 17, upon which claim 18 depends.
Additionally, Gorin teaches wherein the instructions are executed by the at least one processor to cause the speech processing device to further execute the following operations: determining at least one speech window corresponding to the valid speech segment based on a preset window length and/or a sliding duration, and extracting a feature of the speech window ([Column 4, line 49-52] “In order to resolve short segments, the data window used to detect acoustic changes and mark the segments must also be short to avoid including more than one speaker change in the window”);
determining the role change point information according to a similarity between features of adjacent speech windows ([Column 5, line 65 – Column 6, line 3] “To determine the location of boundaries between speaker segments, the GLR function is calculated over successive overlapping windows throughout the data sample”).
Regarding claim 19, Gorin in view of Fujita teaches all of the limitations of claim 18, upon which claim 19 depends.
Additionally, Gorin teaches wherein the instructions are executed by the at least one processor to cause the speech processing device to further execute the following operations: performing parallelization processing on respective valid speech segments by using multiple threads, and for each valid speech segment, determining at least one speech window corresponding to the valid speech segment based on the preset window length and/or the sliding duration, and extracting the feature of the speech window ([Column 6, line 5-14] “or the GLR to perform well, the window should be long enough to obtain stable statistics yet short enough to avoid containing more than one speaker segment change”);
splicing features obtained after the parallelization processing in chronological order, and segmenting the at least one valid speech segment into the multiple speech segments in combination with the role change point information (FIG. 3, [Column 8, line 15-38] “the segmentation modeling and resegmentation 256 is iterated three times in order to obtain stable segmentations. The segmentations are stable when the difference between segmentations from one iteration to the next iteration is below a specified threshold”).
Regarding claim 21, Gorin in view of Fujita teaches all of the limitations of claim 13, upon which claim 21 depends.
Additionally, Gorin teaches wherein the instructions are executed by the at least one processor to cause the speech processing device to further execute the following operations: traversing 2 to a preset class quantity, performing clustering on the multiple first segments by a supervised clustering algorithm under traversed class quantities to obtain clustering results corresponding to the class quantities ([Column 7, line 17-21] “The input to the clustering procedure is a table of pairwise distances between each segment and every other segment. The following procedure is used to generate such a table. Each segment is modeled by a low-order (typically 2- or 4-component) GMM.”);
determining, according to the clustering results corresponding to different class quantities, a quantity of roles and a clustering result corresponding to the to-be-processed speech ([Column 7, line 36-38] “At each iteration, the clustering procedure merges two groups to form a new group such that the merger produces the smallest increase in distance”).
Claims 6 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gorin in view of Fujita, as shown above in claim 3, in further view of Ziv et al. US 20140142944 A1 (hereinafter Ziv).
Regarding claim 6, Gorin in view of Fujita teaches all of the limitations of claim 4, upon which claim 6 depends.
Gorin in view of Fujita fails to teach wherein performing the clustering on the multiple first segments, and assigning the at least one second segment to the class obtained after the clustering comprises: calculating, for each first segment, a mean of a feature of at least one speech window corresponding to the first segment to obtain a feature corresponding to the first segment, and performing clustering on the multiple first segments according to features corresponding to the multiple first segments; calculating, for each second segment, a mean of a feature of at least one speech window corresponding to the second segment to obtain a feature corresponding to the second segment, and assigning the at least one second segment to the class obtained after clustering according to a feature corresponding to at least one second segment.
However, Ziv teaches wherein performing the clustering on the multiple first segments, and assigning the at least one second segment to the class obtained after the clustering comprises: calculating, for each first segment, a mean of a feature of at least one speech window corresponding to the first segment to obtain a feature corresponding to the first segment, and performing clustering on the multiple first segments according to features corresponding to the multiple first segments ([0026] “In the VAD at 304 an audio frame may be identified as speech or non-speech based upon a plurality of characteristics or probabilities exemplarily based upon mean energy, band energy, peakiness, or residual energy”);
calculating, for each second segment, a mean of a feature of at least one speech window corresponding to the second segment to obtain a feature corresponding to the second segment, and assigning the at least one second segment to the class obtained after clustering according to a feature corresponding to at least one second segment (FIG. 2, 204, [0030] “. With the additional context of both enhanced identification of speaker segments and clustering and labeling of the speaker in the audio data, an automated transcription 122 can be outpu”).
Gorin in view of Fujita in view of Ziv are considered to be analogous to the claimed invention because both are the same field of multi-speaker speech segmentation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the speech segmentation techniques of Gorin in view of Fujita with the technique of calculating a mean of a feature in a speech window taught by Ziv in order to improve diarization using acoustic labeling (see Ziv [0002]).
Regarding claim 20, Gorin in view of Fujita teaches all of the limitations of claim 18, upon which claim 20 depends.
Gorin in view of Fujita fails to teach wherein the instructions are executed by the at least one processor to cause the speech processing device to further execute the following operations: calculating, for each first segment, a mean of a feature of at least one speech window corresponding to the first segment to obtain a feature corresponding to the first segment, and performing clustering on the multiple first segments according to features corresponding to the multiple first segments; calculating, for each second segment, a mean of a feature of at least one speech window corresponding to the second segment to obtain a feature corresponding to the second segment, and assigning the at least one second segment to the class obtained after clustering according to a feature corresponding to at least one second segment.
However, Ziv teaches wherein the instructions are executed by the at least one processor to cause the speech processing device to further execute the following operations: calculating, for each first segment, a mean of a feature of at least one speech window corresponding to the first segment to obtain a feature corresponding to the first segment, and performing clustering on the multiple first segments according to features corresponding to the multiple first segments ([0026] “In the VAD at 304 an audio frame may be identified as speech or non-speech based upon a plurality of characteristics or probabilities exemplarily based upon mean energy, band energy, peakiness, or residual energy”);
calculating, for each second segment, a mean of a feature of at least one speech window corresponding to the second segment to obtain a feature corresponding to the second segment, and assigning the at least one second segment to the class obtained after clustering according to a feature corresponding to at least one second segment (FIG. 2, 204, [0030] “. With the additional context of both enhanced identification of speaker segments and clustering and labeling of the speaker in the audio data, an automated transcription 122 can be outpu”)
Gorin in view of Fujita in view of Ziv are considered to be analogous to the claimed invention because both are the same field of multi-speaker speech segmentation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the speech segmentation techniques of Gorin in view of Fujita with the technique of calculating a mean of a feature in a speech window taught by Ziv in order to improve diarization using acoustic labeling (see Ziv [0002]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Gorin in view of Li et al. US 20220199099 A1 (hereinafter Li).
Regarding claim 8 Gorin teaches all of the limitations of claim 7, upon which claim 8 depends.
Gorin fails to teach wherein determining, according to the clustering results corresponding to different class quantities, the quantity of roles and the clustering result corresponding to the to-be-processed speech comprises: setting a current class quantity to the preset class quantity, and repeating the following steps until a final clustering result is obtained: calculating an inter-class distance and an intra-class distance of a clustering result under the current class quantity; if the inter-class distance and the intra-class distance meet a requirement, the quantity of roles corresponding to the to-be-processed speech is the current class quantity, and the final clustering result is the clustering result under the current class quantity; if the inter-class distance and the intra-class distance do not meet the requirement, the current class quantity is reduced by one.
However, Li teaches wherein determining, according to the clustering results corresponding to different class quantities, the quantity of roles and the clustering result corresponding to the to-be-processed speech comprises:
setting a current class quantity to the preset class quantity, and repeating the following steps until a final clustering result is obtained ([0015] “It can be learned that, first clustering is performed first by using the spatial characteristic matrix to determine specific positions at which a speaker speaks in the current scenario, to obtain an estimated quantity of speakers; and then second clustering is performed by using the preset audio feature, to split or combine the initial clusters obtained through first clustering, to obtain an actual quantity of speakers in the current scenario”):
calculating an inter-class distance and an intra-class distance of a clustering result under the current class quantity ([0019] “the obtaining, based on the speaker quantity and the speaker identity corresponding to the N channels of observed signals, output audio including a speaker label includes: determining K distances, where the K distances are distances between the spatial characteristic matrix”);
if the inter-class distance and the intra-class distance meet a requirement, the quantity of roles corresponding to the to-be-processed speech is the current class quantity, and the final clustering result is the clustering result under the current class quantity ([0109] “determining, based on the K distances, a speaker quantity corresponding to each first audio frame group, specifically including: determining L distances greater than a distance threshold in the H distances, and using L as the speaker quantity corresponding to the first audio frame group; then determining a time window corresponding to the first audio frame group, and marking a speaker quantity corresponding to an audio frame of the output audio in the time window as L; and finally sequentially determining speaker quantities”);
if the inter-class distance and the intra-class distance do not meet the requirement, the current class quantity is reduced by one ([0104] “If the source signal corresponding to each initial cluster is an audio signal corresponding to one speaker, after a plurality of clustering iterations are performed, the several pieces of sample data correspond to one target clustering center”).
Gorin in view of Li are considered to be analogous to the claimed invention because both are the same field of audio signal processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the techniques unsupervised segmentation of telephone conversations by speaker of Gorin with the technique of determining class qualities taught by Li in order to improve an audio processing method and a related product (see Li [0002]).
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Gorin in view of Shin et al. US 20220093103 A1 (hereinafter Shin).
Regarding claim 25, Gorin teaches all of the limitations of claim 2, upon which claim 25 depends.
Gorin fails to teach wherein the method is applied to a court trial assistance system, the obtaining to-be-processed speech comprises: obtaining the to-be-processed speech that is outputted by multiple roles and collected in a court trial site, and the method further comprises: generating a court trial record according to the role separation result of the to-be-processed speech and the speaking text corresponding to the to-be-processed speech.
However, Shin teaches wherein the method is applied to a court trial assistance system, the obtaining to-be-processed speech comprises: obtaining the to-be-processed speech that is outputted by multiple roles and collected in a court trial site, and the method further comprises: generating a court trial record according to the role separation result of the to-be-processed speech and the speaking text corresponding to the to-be-processed speech ([0073] “The text transcript creation part 310 may separate each speaker's utterances and automatically record them, in the case of an audio file that is recorded in a situation, such as a meeting, an interview, a business deal, and a trial, where a number of speakers make utterances in random order”).
Gorin in view of Shin are considered to be analogous to the claimed invention because both are the same field of audio signal processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the techniques unsupervised segmentation of telephone conversations by speaker of Gorin with the technique of employing speech separation technology in a trial setting taught by Shin in order to improve a technology for managing a text transcript of an audio recording of speech (see Shin [0002]).
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Gorin in view of Fu et al. US 11100943 B1 (hereinafter Fu).
Regarding claim 26, Gorin teaches all of the limitations of claim 9, upon which claim 26 depends.
Gorin fails to teach wherein the method is applied to an education assistance system, the obtaining to-be-processed speech comprises: obtaining the to-be-processed speech that is outputted by multiple roles and collected by the education assistance system
However, Fu teaches wherein the method is applied to an education assistance system, the obtaining to-be-processed speech comprises: obtaining the to-be-processed speech that is outputted by multiple roles and collected by the education assistance system ([Column 21, line 33-36] “Some examples of the present invention can help students take lecture notes. Certain examples of the present invention can help deaf students to learn, thus improving their educational experience”)
Gorin in view of Fu are considered to be analogous to the claimed invention because both are the same field of audio signal processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the techniques unsupervised segmentation of telephone conversations by speaker of Gorin with the technique of processing and presenting conversations to improve the educational experience taught by Fu in order to improve a system for processing and presenting a conversation includes a sensor, a processor, and a presenter (see Fu [Abstract]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wang et al. (US 20200152207 A1) teaches Techniques for training and/or utilizing an end-to-end speaker diarization model. In various implementations, the model is a recurrent neural network (RNN) model, such as an RNN model that includes at least one memory layer, such as a long short-term memory (LSTM) layer. Audio features of audio data can be applied as input to an end-to-end speaker diarization model trained according to implementations disclosed herein, and the model utilized to process the audio features to generate, as direct output over the model, speaker diarization results. Further, the end-to-end speaker diarization model can be a sequence-to-sequence model, where the sequence can have variable length. Accordingly, the model can be utilized to generate speaker diarization results for any of various length audio segments.
Arslan et al. (US 10026405 B2) teaches a speaker diarization process for determining which speaker is speaking at what time during the course of a conversation. The entire process can be most easily described in five main parts: Segmentation where speech/non-speech decisions are made; frame feature extraction where useful information is obtained from the frames; segment modeling where the information from the frame feature extraction is combined with segment start and end time information to create segment specific features; speaker decisions when the segments are clustered to create speaker models; and corrections where frame level corrections are applied to the information extracted.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZEESHAN SHAIKH whose telephone number is (703)756-1730. The examiner can normally be reached Monday-Friday 7:30AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at (571) 272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZEESHAN MAHMOOD SHAIKH/Examiner, Art Unit 2658
/RICHEMOND DORVIL/Supervisory Patent Examiner, Art Unit 2658
Prosecution Insights