DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that use the word “means” or “step” but are nonetheless not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph because the claim limitation(s) recite(s) sufficient structure, materials, or acts to entirely perform the recited function. Such claim limitation(s) is/are:
a voice processing module
a feature extraction module
an overlapping voice detection module
an embedding extraction training module
a classification training module
a sample voice acquisition module
Claim 9
Claim 9
Claim 9
Claims 10, 11
Claims 10, 12
Claims 13-16
Because this/these claim limitation(s) is/are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are not being interpreted to cover only the corresponding structure, material, or acts described in the specification as performing the claimed function, and equivalents thereof.
If applicant intends to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to remove the structure, materials, or acts that performs the claimed function; or (2) present a sufficient showing that the claim limitation(s) does/do not recite sufficient structure, materials, or acts to perform the claimed function.
Allowable Subject Matter
Claims 2-8 and 10-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (CN #112992155 A) in view of Grangier et al. (US #2022/0375492).
Regarding Claim 1, Zhang discloses a multi-speaker overlapping voice detection method (title, abstract, ¶0031: Fig. 1, step S2-2: Perform overlapping sliding window sampling on the augmented data to obtain sampled data, ¶0032: when sampling the enhanced data, the sampling frequency is 16kHz, the frame length is 25ms, and the speech overlap is 15ms), comprising:
obtaining a voice to be detected (Zhang ¶0038 discloses the speech activity detection model is a speech activity detection model based on the U-Net neural network structure, which can be trained end-to-end with very little data) and removing silence from the voice to be detected (Zhang ¶0037 discloses step S1-2: Use the pre-trained speech activity detection model [VAD] to perform speech segmentation on the pre-processed short speech to obtain the segmentation result, and extract speech vector features from the segmentation result after filtering out noise and silence. ¶0038 discloses applying speech activity detection models to semantic activity detection can segment short-term speech in noisy, reverberant backgrounds, thereby effectively removing noise and silence);
extracting a feature of the voice to be detected after silence removal to obtain a voice feature of the voice to be detected (Zhang ¶0079 discloses since the pre-trained speech activity detection model is used to perform speech segmentation on the pre-processed short speech to obtain the segmentation result, and the speech vector features are extracted from the segmentation result after filtering out noise and silence, the noise, echo and silence interference in the speech segment can be greatly removed, and the foundation is laid for improving the speaker recognition accuracy of the residual neural network model);
inputting the voice feature into an overlapping voice detection model (Zhang ¶0012 discloses step S2-1, performing speech enhancement on the audio to be tested to obtain enhanced data; step S2-2, performing overlapping sliding window sampling on the enhanced data to obtain sampled data).
Zhang may not explicitly disclose inputting the voice feature into an overlapping voice detection model to obtain an overlapping speaker number corresponding to the voice to be detected output by the overlapping voice detection model, wherein the overlapping speaker number represents a number of speakers speaking simultaneously in the voice to be detected; wherein the overlapping voice detection model is obtained by a supervised training based on a voice feature of a sample voice and a corresponding label of the overlapping speaker number, and the overlapping voice detection model extracts an embedding of the voice feature and classifies the overlapping speaker number to obtain the overlapping speaker number of the voice to be detected based on the overlapping speaker number classified by an extracted speaker embedding.
However, Grangier (title, abstract, Figs. 1-7) teaches inputting the voice feature into an overlapping voice detection model (Grangier ¶0006 discloses during each of the plurality of iterations for each temporal embedding in the sequence of temporal embeddings, determining the probability that the corresponding temporal embedding includes the presence of voice activity by the single new speaker includes determining a probability distribution of possible event types for the corresponding temporal embedding. The possible event types include the presence of voice activity by the single new speaker, a presence of voice activity for a single previous speaker for which another respective speaker embedding was previously selected during a previous iteration, a presence of overlapped speech, and a presence of silence) to obtain an overlapping speaker number corresponding to the voice to be detected output by the overlapping voice detection model (Grangier ¶0006 discloses the multi-class linear classifier can be trained on a corpus of training audio signals that are each encoded into a sequence of training temporal embeddings. Here, each training temporal embedding includes a respective speaker label), wherein the overlapping speaker number represents a number of speakers speaking simultaneously in the voice to be detected (Grangier ¶0028 discloses as such, the diarization results 280 can provide time-stamped speaker labels based on the per-speaker voice activity indicators 262 predicted at each time step that not only identify who is speaking at a given time, but also identify when speaker changes [e.g., speaker turns] occur between adjacent time steps);
wherein the overlapping voice detection model is obtained by a supervised training based on a voice feature of a sample voice and a corresponding label of the overlapping speaker number (Grangier ¶0043 discloses referring to Fig. 3, a schematic view 300 illustrates an example training process 301 and inference 304 for the DIVE system 200. In some implementations, the training process 301 jointly trains the temporal encoder 210, the iterative speaker selector 230 that includes the multi-class linear classifier with the fully-connected network, and the VAD 260 of the DIVE system 200 on fully-labeled training data 302 that includes a corpus of training audio signals x* each including utterances 120 spoken by multiple different speakers 10. The training audio signals x* can include long speech sequences representing speech for several minutes. In some examples, the training process 301 samples W fixed-length windows per training audio signal x*, encodes the training audio signal x* using the temporal encoder 210, and concatenates W fixed-length windows along the temporal axis. By concatenating the W fixed-length windows, the fully-labeled training data 302 increases speaker diversity and speaker turns for each training audio signal x* and keeps while keeping the memory usage low. That is, the training audio signal x* can represent same speakers over windows far apart during the long speech sequences. Some training audio signals x* can include portions where utterances 120 spoken by two or more different speakers 10 overlap), and the overlapping voice detection model extracts an embedding of the voice feature and classifies the overlapping speaker number to obtain the overlapping speaker number of the voice to be detected based on the overlapping speaker number classified by an extracted speaker embedding (Grangier ¶0043 discloses each training audio signal x* is encoded by the temporal encoder 210 into a sequence of training temporal embeddings 220 that are each assigned a respective speaker label 350 indicating an active speaker or silence. ¶0044 discloses the speaker labels 350 can be represented as a sequence of training speaker labels ŷ, where entry ŷ, in the sequence represents the speaker label 350 assigned to training temporal embedding 220 at time step t. In the example shown, the training process 301 provides the sequence of training temporal embeddings 220T encoded by the temporal encoder 210 and assigned speaker labels 350 for training the iterative speaker selector 230 during each of the plurality of i iterations, and subsequently, the VAD 260 based on the speaker embeddings 240 selected by the iterative speaker selector 230 during the plurality of iterations. ¶0045 discloses as the temporal encoder 210, the iterative speaker selector 230, and the VAD 260 are trained jointly, the VAD 260 is also trained on the corpus of training audio signals x *, where each training audio signal x* is encoded into a sequence of training temporal embeddings each including a corresponding voice activity indicator [i.e., a speaker label ŷ] indicating which voice is present/active in the corresponding training temporal embedding. The training process can train the VAD 260 on the following VAD loss, as in equation 6).
Zhang and Grangier are analogous art as they pertain to multi-speaker overlap voice detection. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify speech activity detection (as taught by Zhang) to provide time-stamped speaker labels based on the per-speaker voice activity indicators predicted at each time step that not only identify who is speaking at a given time, but also identify when speaker changes occur between adjacent time steps (as taught by Grangier, ¶0028) to include possible event types, such as the presence of voice activity by the single new speaker, a presence of voice activity for a single previous speaker for which another respective speaker embedding was previously selected during a previous iteration, a presence of overlapped speech, and a presence of silence (Grangier, ¶0006).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YOGESHKUMAR G PATEL whose telephone number is (571)272-3957. The examiner can normally be reached 7:30 AM-4 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at (571) 272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YOGESHKUMAR PATEL/Primary Examiner, Art Unit 2691