DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-17 are pending and have been examined.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-17 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1-17 of U.S. Patent No. 11978456. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the issued patent anticipate the claims of the instant application. Please see below for mapping in the table, where the bold limitations indicate the corresponding limitations between the issued patent and the instant application. With respect to the dependent claims, each of the claims map to a corresponding dependent claim of the issued patent or are found within the scope of the independent claim.
With respect to each of the independent and dependent claims the mapping is as follows:
[Instant Application : Issued Patent]
[Claim 1 : Claim 1], [Claim 2 : Claim 2], [Claim 3 : Claim 4], [Claim 4 : Claim 5], [Claim 5 : Claim 6], [Claim 7 : Claim 8], [Claim 9 : Claim 10], [Claim 10 : Claim 11], [Claim 11 : Claim 14], [Claim 12 : Claim 15], [Claim 13 : Claim 16], [Claim 14 : Claim 17], [Claim 15 : Claim 18], [Claim 16 : Claim 19], [Claim 17 : Claim 20]
Instant Application: 18957271
Issued Patent: US 11978456
A user device for using visual information in a video stream of a first recorded teleconference among a plurality of participants to diarize speech comprising: one or more processors; and non-transitory computer-readable memory operatively connected to the one or more processors, the non-transitory computer-readable memory including machine readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of: (a) sending the first recorded teleconference among the plurality of participants conducted over a network to a teleconferencing system, the first recorded teleconference among the plurality of participants conducted over the network comprising the following components:
(1) an audio component including utterances of respective participants that spoke during the first recorded teleconference, the audio component being parsed into a plurality of speech segments in which one or more participants were speaking during the first recorded teleconference, each respective speech segment being associated with a respective time segment including a start timestamp indicating a first time in the first recorded teleconference when the respective speech segment begins, and a stop timestamp associated with a second time in the first recorded teleconference when the respective speech segment ends;
(2) a video component including a video feed as to respective participants that spoke during the first recorded teleconference;
(3) teleconference metadata associated with the first recorded teleconference and including a first plurality of timestamp information and respective speaker identification information associated with each respective timestamp information, each respective speech segment being tagged with the respective speaker identification information based on the teleconference metadata associated with the respective time segment; and
(4) transcription data associated with the first recorded teleconference, wherein said transcription data is indexed by timestamp;
wherein the transcription data is indexed in accordance with respective speech segments and the respective speaker identification information to generate a segmented transcription data set for the first recorded teleconference,
(b) receiving a diarized first recorded teleconference from the teleconferencing system, the diarized first recorded teleconference comprising labeled speech segments and identified respective speaker information associated with the labeled speech segments,
the identified respective speaker information associated with respective speech segments being identified using a neural network with at least a portion of the video feed corresponding in time to at least a portion of a segmented transcription data set determined according to the indexing as an input, and source indication information for each respective speech segment is provided as an output and using a training set including visual content tagged with prior source indication information, the portion of the video feed includes a first artificial visual representation not including a face generated by telephone conferencing software in the visual content associated with a first participant that spoke during a first speech segment of the first recorded teleconference, and the portion of the video feed does not include any artificial visual representation associated with a second participant that did not speak during the first speech segment of the first recorded teleconference, and the source indication information is based at least on presence of the first artificial visual representation; the labeled speech segments are labeled based on the identified respective speaker information associated with the respective speech segment and the identified respective speaker information is based on the source indication information.
A method for using visual information in a video stream of a first recorded teleconference among a plurality of participants to diarize speech, the method comprising: (a) obtaining, by a computer system, components of the first recorded teleconference among the plurality of participants conducted over a network, wherein the respective components include:
(i) an audio component including utterances of respective participants that spoke during the first recorded teleconference;
(b) parsing, by the computer system, the audio component into a plurality of speech segments in which one or more participants were speaking during the first recorded teleconference, wherein each respective speech segment is associated with a respective time segment including a start timestamp indicating a first time in the first recorded teleconference when the respective speech segment begins, and a stop timestamp associated with a second time in the first recorded teleconference when the respective speech segment ends;
(ii) a video component including a video feed as to respective participants that spoke during the first recorded teleconference;
(iii) teleconference metadata associated with the first recorded teleconference and including a first plurality of timestamp information and respective speaker identification information associated with each respective timestamp information; and
(c) tagging, by the computer system, each respective speech segment with the respective speaker identification information based on the teleconference metadata associated with the respective time segment; and
(iv) transcription data associated with the first recorded teleconference, wherein said transcription data is indexed by timestamps;
(i) indexing the transcription data in accordance with respective speech segments and the respective speaker identification information to generate a segmented transcription data set for the first recorded teleconference;
(d) diarizing, by the computer system, the first recorded teleconference in a process comprising:
(ii) identifying respective speaker information associated with respective speech segments using a neural network with at least a portion of the video feed corresponding in time to at least a portion of the segmented transcription data set determined according to the indexing as an input, and providing source indication information for each respective speech segment as an output and using a training set including visual content tagged with prior source indication information, wherein the portion of the video feed includes a first artificial visual representation not including a face generated by telephone conferencing software in the visual content associated with a first participant that spoke during a first speech segment of the first recorded teleconference, and the portion of the video feed does not include any artificial visual representation associated with a second participant that did not speak during the first speech segment of the recorded teleconference, and the source indication information is based at least on presence of the first artificial visual representation; and (iii) labeling each respective speech segment based on the identified respective speaker information associated with the respective speech segment; wherein the identified respective speaker information is based on the source indication information.
Allowable Subject Matter
Claims 1-17 would be allowable upon a filing of the Terminal Disclaimer to address the nonstatutory double patenting rejection as presented above.
The following is a statement of reasons for the indication of allowable subject matter:
The closest prior art Bradley et al. (U.S. PG Pub No. 20220115019) teaches a method and system to perform speaker diarization utilizing video data to identify and label different speakers. Bradley teaches the process of obtaining recorded teleconference (audio, video, metadata, transcription data), parsing the audio into speech segments, tagging the speech segments with speaker identification information, and diarizing the recorded teleconference. Bradley further teaches the process of diarization by indexing transcription data, identifying respective speaker information using neural network from video feed, and labeling respective speech segment with speaker information. Bradley, however, does not teach the use of a neural network training set where the training set includes portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content.
Peeler et al. (U.S. PG Pub No. 20220189489) teaches the identification of the active speaker and display of a visual representation of the active speaker. Peeler, however, does not teach the use of a neural network training set where the training set includes portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content.
Moorsom et al. (U.S. PG Pub No. 20230162733) teaches the analysis of audio and video data and teaches the use of a database of speaker identification information to identify speakers. Moorsom, however, does not teach the use of a neural network training set where the training set includes portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content.
Bakis (U.S. Patent No. 6219639) teaches identification of individuals who is speaking by utilizing visual data that shows lip movement. Bakis, however, does not teach the use of a neural network training set where the training set includes portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content.
Karakotsios (U.S. Patent No. 9479736) teaches the use of visual content to analyze lip movement when producing speech . Karakotsios, however, does not teach the use of a neural network training set where the training set includes portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content.
Pell et al. (U.S. PG Pub No. 20180176508) teaches the use of optical character recognition on the video data for analysis. Pell, however, does not teach the use of a neural network training set where the training set includes portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content.
Reshef et al. (U.S. PG Pub No. 20190318743) teaches analysis of the speaker diarization result. Reshef, however, does not teach the use of a neural network training set where the training set includes portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content.
None of the cited references either alone or in combination thereof teaches or makes obvious the series of limitations. More specifically the limitations of training the neural network where the training set includes “portion of video feed that includes artificial visual representation not including a face generated by telephone conferencing software in the visual content”.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL WONSUK CHUNG whose telephone number is (571)272-1345. The examiner can normally be reached Monday - Friday (7am-4pm)[PT].
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PIERRE-LOUIS DESIR can be reached at (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL W CHUNG/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights