Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Applicant’s amendments filed on 8/17/26 have been entered. In view of the amendments, the rejections for claims are maintained as modified in response to the amendments and provided in the response below.
With this Final Office Action, claims 1-20 stand rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 11-12, 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Thomson (US 20200175961 A1) in further view of Pakhomov (US 6535849 B1) and Fukomochi (US 5289375 A).
With respect to claim 1 Thomson teaches A method comprising:
obtaining a first transcript of words uttered in an audio file by –
forwarding a first copy of the audio file to a first application programming interface that is associated with a first transcription service (Thomson¶[0105] In some embodiments, each of the first and second devices 104 and 106 may be configured to obtain audio during a communication session…Furthermore, the term “audio” may be used generically to include audio in any format, such as a digital format, ¶[0333] Additionally or alternatively, the ASR systems 1320 may be part of different API services, such as services provided by different vendors, ¶[0161] For example, in some embodiments, providing the transcriptions by the transcription system 108 may be described as a transcription service)),
and receiving the first transcript from the first transcription service (Thomson ¶[0335] FIG. 13 also illustrates a fuser 1324. In some embodiments, the fuser 1324 may be configured to merge the transcriptions generated by the ASR systems 1320 to create a fused transcription. In some embodiments, the fused transcription may include an accuracy that is improved with respect to the accuracy of the individual transcriptions combined to generate the fused transcription. Additionally or alternatively, the fuser 1324 may generate multiple transcriptions.);;
obtaining a second transcript of the words uttered in the audio file by –
forwarding a second copy of the audio file to a second application programming interface that is associated with a second transcription service ([0333] Additionally or alternatively, the ASR systems 1320 may be part of different API services, such as services provided by different vendors), and
receiving the second transcript from the second transcription service (Thomson ¶[0335] FIG. 13 also illustrates a fuser 1324. In some embodiments, the fuser 1324 may be configured to merge the transcriptions generated by the ASR systems 1320 to create a fused transcription. In some embodiments, the fused transcription may include an accuracy that is improved with respect to the accuracy of the individual transcriptions combined to generate the fused transcription. Additionally or alternatively, the fuser 1324 may generate multiple transcriptions);
[[producing, based on an analysis of the first and second transcripts, a tuple for each of the words uttered in the audio file, so as to create a series of tuples that are populated into a data structure in temporal order ((97) The syntax analyzing section 12 creates a syntax-structure tree based on the information about a part of speech for each word stored in the buffer C [data structure]. Herein, since the verb "makes" matches to the sentence pattern 3, the grammatical rule (m) shown in Table 10 is applied for creating the syntax-structure tree as shown in FIG. 11.)
examining the data structure to identify a conflicting translation between the first and second transcripts ]]
wherein the conflicting translation corresponds to an instance in which the first transcription service had a first interpretation of a given word and the second transcription service had a second interpretation of the given word (Thomson¶[0357] An example of the process 1400 is now provided. In this example, a speaker may say “OK, let's meet at four.” During the transcription generation process 1402, three different ASR systems (e.g., ASR systems 1320 of FIG. 13) may each generate one of the below hypotheses: [0358] 1. OK, let's meet more, ¶[0359] 2. OK, says meet at 4:00. [0360] 3. OK, ha let's meet at far, 0384] In these and other embodiments, the voting process 1408 may multiply the count of each token by the probability. Thus, the token count for “four” may be 0.05, the token count for “more” may be 0.01, and the token count for “far” may be 0.02. The token “four” may have the highest probability and may be selected so that the fused hypotheses forms “ok let us meet at four.”)
identifying an appropriate translation for the given word from among the first and second transcripts based on an analysis of grammar or sentence structure, of the first interpretation and one or more surrounding words and of the second interpretation and the one or more surrounding words [[wherein the analysis is performed using part-of-speech information defined within the tuple for the given word and in the tuples for the one or more surrounding words, and wherein the one or more surrounding words are obtained from tuples that neighbor, within the data structure, the tuple corresponding to the given word ]] (Thomson¶[0357] An example of the process 1400 is now provided. In this example, a speaker may say “OK, let's meet at four.” During the transcription generation process 1402, three different ASR systems (e.g., ASR systems 1320 of FIG. 13) may each generate one of the below hypotheses: [0358] 1. OK, let's meet more, ¶[0359] 2. OK, says meet at 4:00. [0360] 3. OK, ha let's meet at far, 0384] In these and other embodiments, the voting process 1408 may multiply the count of each token by the probability. Thus, the token count for “four” may be 0.05, the token count for “more” may be 0.01, and the token count for “far” may be 0.02. The token “four” may have the highest probability and may be selected so that the fused hypotheses forms “ok let us meet at four.”); and
posting, to an interface, a third transcript that is derived from the first and second transcripts and includes the appropriate translation (Thomson ¶[0095] In some embodiments, systems and methods in this disclosure may be configured to combine or fuse multiple transcriptions into a single transcription that is provided to a device for display to a user).
Thomson does not explicitly disclose however Pakhomov teaches producing, based on an analysis of the first and second transcripts, a tuple for each of the words uttered in the audio file, so as to create a series of tuples that are populated into a data structure in temporal order (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
examining the data structure to identify a conflicting translation between the first and second transcripts (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson to include data structures of Pakhomov in order to getter better speech recognition (Col2ll6-16, Pakhomov).
None of Thomson and Pakhomov explicitly disclose however Fukomochi teaches wherein the analysis is performed using part-of-speech information defined within the tuple for the given word and in the tuples for the one or more surrounding words, and wherein the one or more surrounding words are obtained from tuples that neighbor, within the data structure, the tuple corresponding to the given word (Col19ll34-40 The syntax analyzing section 12 creates a syntax-structure tree based on the information about a part of speech for each word stored in the buffer C. Herein, since the verb "makes" matches to the sentence pattern 3, the grammatical rule (m) shown in Table 10 is applied for creating the syntax-structure tree as shown in FIG. 11.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov to include analysis of Fukumochi in order to improve computation efficiency and contextual accuracy of translation.
With respect to claim 2, Pakhomov further teaches wherein each tuple includes a field in which it is indicated whether interpretations of a corresponding word across the first and second transcripts are identical (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
With respect to claim 3, Pakhomov further teaches wherein said identifying comprises: establishing, based on an analysis of the data structure, that in a given tuple corresponding to the given word, the field indicates that the first and second interpretations are not (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
With respect to claim 4, Thomson teaches wherein the third transcript is displayed such that the appropriate translation is visually distinguishable from other words that are identically translated across the first and second transcripts (Thomson ¶[0159] Differences determined by the fuser 124 may be determined to be errors in the third transcription. Corrections of the errors may be provided to the first device 104 for correcting the third transcription being presented by the first device 104. Corrections may be marked in the presentation by the first device 104 in any manner of suitable methods including, but not limited to, highlighting, changing the font, or changing the brightness of the text that is replaced.)
With respect to claim 11 Thomson teaches A non-transitory medium with instructions stored thereon that, when executed by a processor of a computing device, cause the computing device to perform operations comprising:
forwarding separate copies of an audio file to each of multiple transcription services, from which multiple transcripts are received in return (Thomson¶[0105] In some embodiments, each of the first and second devices 104 and 106 may be configured to obtain audio during a communication session…Furthermore, the term “audio” may be used generically to include audio in any format, such as a digital format, ¶[0333] Additionally or alternatively, the ASR systems 1320 may be part of different API services, such as services provided by different vendors, ¶[0161] For example, in some embodiments, providing the transcriptions by the transcription system 108 may be described as a transcription service, ¶[0335] FIG. 13 also illustrates a fuser 1324. In some embodiments, the fuser 1324 may be configured to merge the transcriptions generated by the ASR systems 1320 to create a fused transcription. In some embodiments, the fused transcription may include an accuracy that is improved with respect to the accuracy of the individual transcriptions combined to generate the fused transcription. Additionally or alternatively, the fuser 1324 may generate multiple transcriptions,¶[0333] Additionally or alternatively, the ASR systems 1320 may be part of different API services, such as services provided by different vendors);
identifying an appropriate translation for the given word from among the first and second interpretations based on an analysis of grammar or sentence structure, of the first interpretation and one or more surrounding words and of the second interpretation and one or more surrounding words (Thomson¶[0357] An example of the process 1400 is now provided. In this example, a speaker may say “OK, let's meet at four.” During the transcription generation process 1402, three different ASR systems (e.g., ASR systems 1320 of FIG. 13) may each generate one of the below hypotheses: [0358] 1. OK, let's meet more, ¶[0359] 2. OK, says meet at 4:00. [0360] 3. OK, ha let's meet at far, 0384] In these and other embodiments, the voting process 1408 may multiply the count of each token by the probability. Thus, the token count for “four” may be 0.05, the token count for “more” may be 0.01, and the token count for “far” may be 0.02. The token “four” may have the highest probability and may be selected so that the fused hypotheses forms “ok let us meet at four.”); and
posting, to an interface, a third transcript that is derived from the first and second transcripts and includes the appropriate translation (Thomson ¶[0095] In some embodiments, systems and methods in this disclosure may be configured to combine or fuse multiple transcriptions into a single transcription that is provided to a device for display to a user).
Thomson does not explicitly disclose however Pakhomov teaches populating a data structure such that for each word in the audio file, a separate entry is populated with interpretations of that word across the multiple transcriptions (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
detecting, based on an analysis of the data structure, a given word for which a first transcript of the multiple transcripts has a first interpretation and a second transcript of the multiple transcripts has a second interpretation different than the first interpretation ((Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson to include data structures of Pakhomov in order to getter better speech recognition (Col2ll6-16, Pakhomov).
With respect to claim 16, Thomson teaches identifying an appropriate translation for the given word from among the first and second interpretations based on an analysis of grammar or sentence structure, of the first interpretation and one or more surrounding words and of the second interpretation and one or more surrounding words (Thomson¶[0357] An example of the process 1400 is now provided. In this example, a speaker may say “OK, let's meet at four.” During the transcription generation process 1402, three different ASR systems (e.g., ASR systems 1320 of FIG. 13) may each generate one of the below hypotheses: [0358] 1. OK, let's meet more, ¶[0359] 2. OK, says meet at 4:00. [0360] 3. OK, ha let's meet at far, 0384] In these and other embodiments, the voting process 1408 may multiply the count of each token by the probability. Thus, the token count for “four” may be 0.05, the token count for “more” may be 0.01, and the token count for “far” may be 0.02. The token “four” may have the highest probability and may be selected so that the fused hypotheses forms “ok let us meet at four.”); and
posting, to an interface, a third transcript that is derived from the first and second transcripts and includes the appropriate translation (Thomson ¶[0095] In some embodiments, systems and methods in this disclosure may be configured to combine or fuse multiple transcriptions into a single transcription that is provided to a device for display to a user).
Thomson does not explicitly disclose however Pakhomov teaches obtaining multiple transcripts of words uttered in an audio file (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
producing, based on an analysis of the multiple transcripts, a tuple for each of the words uttered in the audio file, so as to create a series of tuples that are arranged in temporal order (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
wherein each tuple in the series of tuples includes a field in which it is indicated whether interpretations of a corresponding word across the multiple transcripts are identical (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
detecting, based on an analysis of the series of tuples, a given word for which a first transcript of the multiple transcripts has a first interpretation and a second transcript of the multiple transcripts has a second interpretation different than the first interpretation (Pakhomov ¶ Col10ll8-22 FIGS. 7 and 8 illustrate one method in which the integrator 14 may function, with FIG. 8 depicting a flow diagram and FIG. 7 showing one example of the integration process. In the embodiment of FIG. 8, the partial transcript 20 and the hypothesized output 40 are input to a dynamic programing algorithm 80 that aligns 82 the hypothesized output 40 and the partial transcript 20 word for word (or by "line"). An example of the result is shown in FIG. 7, with 29 lines of words aligned. FIG. 7 contains six columns for the aligned transcripts 82: a line number 101 [each line is mapped to a data structure and col 102-104 for each row= tuple], a column for the word from the partial transcript portion 102, a column for the corresponding aligned word from the hypothesized output portion 103, a label 104 [interpretation of corresponding word field; ‘match’=identical, other labels indicate non-identical] for each line number, the output 105 to the semi-literal transcript 45 that may be used for acoustic modeling, and the output 106 to the semi-literal transcript 45 that may be used for language modeling).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson to include data structures of Pakhomov in order to getter better speech recognition (Col2ll6-16, Pakhomov).
With respect to claim 17 Thomson teaches wherein said obtaining comprises: applying, to the audio file, a first speech-to-text model that outputs the first transcript, and applying, to the audio file, a second speech-to-text model that outputs the second transcript (Thomson ¶[0129] The second ASR system 120b and the third ASR system 120c may be speaker-independent. In some embodiments, the second ASR system 120b and the third ASR system 120c may include analogous or the same modules that may be trained using similar or the same speech patterns and/or methods. Alternatively or additionally, the second ASR system 120b and the third ASR system 120c may include different modules that may be trained using some or all different speech patterns. Additionally or alternatively, two or more ASR systems 120 may use substantially the same software or may have software modules in common, but use different ASR models.).
With respect to claim 18 Thomson teaches wherein the first and second speech-to-text models are built differently and/or trained differently (Thomson ¶[0129] The second ASR system 120b and the third ASR system 120c may be speaker-independent. In some embodiments, the second ASR system 120b and the third ASR system 120c may include analogous or the same modules that may be trained using similar or the same speech patterns and/or methods. Alternatively or additionally, the second ASR system 120b and the third ASR system 120c may include different modules that may be trained using some or all different speech patterns. Additionally or alternatively, two or more ASR systems 120 may use substantially the same software or may have software modules in common, but use different ASR models.).
With respect to claim 19, Thomson teaches wherein said obtaining comprises: forwarding separate copies of the audio file to a first transcription service, from which the first transcript is received in return, and a second transcription service, from which the second transcript is received in return (Thomson ¶ [0142] The first ASR system 120a may generate a first transcription based on the revoiced audio. The second ASR system 120b may generate a second transcription based on the revoiced audio. The third ASR system 120c may generate a third transcription based on the regular audio.).
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Thomson, Pakhomov and Fukumochi in further view of Estelle (US 20120123765 A1).
With respect to claims 5 Thomson, Pakhomov and Fukumochi do not explicitly disclose however Estelle teaches wherein the third transcript is displayed such that an alternative translation is positioned adjacent to the appropriate translation, and wherein the alternative translation is whichever of the first and second interpretations is not identified as the appropriate translation (Estelle ¶[0054] FIG. 4C illustrates an example translation interface 480 where alternative translations for a sentence in the translated text are presented. In FIG. 4C, a user has selected a portion of translated text. In the illustrated example, in response to the user selection, the user device has presented alternative translations 484 and 486 for the entire translated text 482 and highlighted the entire source text 488. In some implementations, in the case of source text that contains more than one sentence, the segmentation data can indicate that each sentence of the translated text constitutes a separate segment of the translated text. In this case, in response to a user selection of a portion of a sentence in the translated text, the user device can display alternative translations for only the selected sentence.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov to include display of Estelle in order to in order to provide improved the recognition accuracy.
With respect to claims 6, Thomson, Pakhomov and Fukumochi do not explicitly disclose however Estelle teaches wherein each tuple includes a pair of fields in which interpretations of a corresponding word across the first and second transcripts are populated in a predetermined order (Estelle [0052] FIG. 4A illustrates an example translation interface 400 where alternative translations for a word in the translated text are presented to a user. In FIG. 4A, a user has selected a portion of translated text that is included in the segment 402. In response to the user selection, the user device has presented alternative translations 404, 406, 408, and 410 [404, 406=fields] of the segment 402 according to the above description. The user device has also highlighted a corresponding portion 404 of the source text.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov in view of analysis of Fukumochi to include display of Estelle in order to in order to provide improved the recognition accuracy
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Thomson, Pakhomov and Fukumochi in further view of Szentes (US 10854190 B1).
With respect to claims 7, Thomson, Pakhomov and Fukumochi do not explicitly disclose however Szentes teaches indicating, on the interface, a type of issue responsible for the nonidentical interpretation of the given word (Szentes ¶Claim 1. assigning a differentiator label to each word of the first transcription based on a comparison of the word of the first transcription with the corresponding word of the baseline transcription, wherein the differentiator label indicates whether each word in the first transcription is a type of correct, inserted, deleted, or substituted, wherein each type of the differentiator label is used at least once in the first transcription, and wherein vertically aligning each word of the first transcription comprises: in response to the first transcription including an inserted word, inserting a first row into the baseline transcription in the row that the inserted word was added to the first transcription such that each subsequent word in the first transcription continues to be aligned with the corresponding word of the baseline transcription, and generating the word error rate based on determination of whether each word, as vertically aligned in the first transcription, has a differentiator label corresponding to correct, inserted, deleted, or substituted; determine a speaker error rate by indicating a number of times the first transcription correctly indicates a speaker change and a speaker identification; and generate a graphical display of performance, wherein the graphical display is a heat map and includes an indication of the word error rate plotted in a x-axis and a length difference of the first transcription compared to the baseline transcription in a y-axis.).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov in view of analysis of Fukumochi to include issues of Szentes in order to in order to provide improved the recognition accuracy.
With respect to claim 8, Thomson teaches wherein the type of issue is misinterpretation of a non-speech utterance, substitution of an acronym, mispronunciation of an acronym, or misuse of an acronym (Thomson ¶ ([0112] The non-speech sounds may include, background noise, vocalizations such as laughter, filler words such as “um,” and speaker identifiers such as “new speaker,” among others.)
Claims 9-10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Thomson, Pakhomov and Fukumochi in further view of Vozila (US 20190272902 A1) and Han (US 11138970 B1).
With respect to claim 9, Thomson, Pakhomov and Fukumochi do not explicitly disclose however Vozila teaches receiving input that is indicative of a selection of a portion of the third transcript that includes the given word (Vozila ¶[0098] The use of dedicated hardware (e.g., a peripheral device) may be utilized by ACD process 10 to improve editing efficiency for medical transcriptionists/physicians, etc. to help navigate and browse (e.g., sentence by sentence) through the draft medical report, the associated conversation transcript excerpt, and/or associated audio file, annotated (e.g., highlighted, bolded, etc.) with its corresponding audio cued up for easy playback.);
identifying a portion of the audio file that corresponds to the selected portion of the third transcript (Vozila ¶[0098] The use of dedicated hardware (e.g., a peripheral device) may be utilized by ACD process 10 to improve editing efficiency for medical transcriptionists/physicians, etc. to help navigate and browse (e.g., sentence by sentence) through the draft medical report, the associated conversation transcript excerpt, and/or associated audio file, annotated (e.g., highlighted, bolded, etc.) with its corresponding audio cued up for easy playback.);
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov in view of analysis of Fukumochi to include selection of Vozilla in order to improve editing efficiency for transcriptionists ([0098], Vozila).
None of Thomson, Pakhomov and Vozilla explicitly disclose however Han teaches forwarding the portion of the audio file to a third application programming interface that is associated with a third transcription service (Han ¶Col 1, ll The system sends the modified audio recording and the extracted audio clips to separate transcription services.) Examiner Note: Thomson teaches API for sending to transcription services
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov in view of analysis of Fukumochi to include forwarding of in order to create a complete transcription of an audio recording from separately transcribed words, thus enabling a company to protect the privacy (Col 1 ll 32-35 Han).
With respect to claim 10 Hans further teaches further comprising: receiving, from the third transcription service, a fourth transcript for the portion of the audio file; and replacing the selected portion of the third transcript with the fourth transcript (Hans ¶Col 1 ll 66 to Col 2 ll 9) When the system receives the transcribed modified audio recording [master transcript] and the transcribed audio clips back from the separate transcription services, the system combines the two by aligning the word-level timestamps of the transcribed audio clips with the word-level timestamps of the transcribed modified audio recording and inserting the transcriptions of the audio clips into the appropriate locations of the transcription of the modified audio recording so that the combination creates a complete transcription of the original audio recording while enabling the user to protect the privacy of any confidential information during the transcription process.)
With respect to claim 20, Thomson and Pakhomov do not explicitly disclose however Vozila teaches receiving input that is indicative of a selection of a portion of the third transcript that includes the given word (Vozila : [0098] The use of dedicated hardware (e.g., a peripheral device) may be utilized by ACD process 10 to improve editing efficiency for medical transcriptionists/physicians, etc. to help navigate and browse (e.g., sentence by sentence) through the draft medical report, the associated conversation transcript excerpt, and/or associated audio file, annotated (e.g., highlighted, bolded, etc.) with its corresponding audio cued up for easy playback.;
identifying a portion of the audio file that corresponds to the selected portion of the third transcript (Vozila : [0098] The use of dedicated hardware (e.g., a peripheral device) may be utilized by ACD process 10 to improve editing efficiency for medical transcriptionists/physicians, etc. to help navigate and browse (e.g., sentence by sentence) through the draft medical report, the associated conversation transcript excerpt, and/or associated audio file, annotated (e.g., highlighted, bolded, etc.) with its corresponding audio cued up for easy playback.; ;
None of Thomson, Pakhomov and Vozilla explicitly disclose however Han teaches applying, to the portion of the audio file, a speech-to-text model that outputs a fourth transcript for the portion of the audio file (Col 1, ll The system sends the modified audio recording and the extracted audio clips to separate transcription services. The transcription services may be third-party transcription services (i.e., an outside company hired to handle the transcription services) or in-house transcription services (i.e., the company itself performing any transcription services) or a combination of the two, so long as the transcription services assigned to transcribe the confidential information contained in the extracted audio clips have an appropriate level of security that is compliant with any applicable privacy laws.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov in view of analysis of Fukumochi to include forwarding of in order to create a complete transcription of an audio recording from separately transcribed words, thus enabling a company to protect the privacy (Col 1 ll 32-35, Han).
Claims 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Thomson, Pakhomov and Fukumochi in further view of Nelson (US 20190108834 A1).
With respect to claim 13, Thomson, Pakhomov and Fukumochi do not explicitly disclose however Nelson teaches wherein the operations further comprise: receiving input that is indicative of a selection of the multiple transcription services (Nelson: [0274] The selected translation/transcription services are selected in a manner to provide the most accurate results for particular text or audio data, and the translation/transcription services that are selected may be different for each set of text and audio data. [0275] FIG. 12 depicts example data that may be included in selection data 1152. In the examples depicted in FIG. 12, the labels “S1”, “S2”, and “S3” refer to three different translation/transcription services, such as translation/transcription services 1170, 1180, 1190, but data may be provided for any number of translation/transcription services [the input selection is shown in table]).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov in view of analysis of Fukumochi to include service selection of Nelson in order to reduce the amount of computational resources and/or time required to perform the translation/transcription ([0291], Nelson).
With respect to claim 14 Nelson further teaches wherein the operations further comprise: initiating, in response to said receiving, connections with multiple application programming interfaces, each of which is associated with a different one of the multiple transcription services (Nelson: [0267] Translation/transcription services 1130 include services with the capability to translate text data from one language to one or more other languages, transcribe audio data into text, or both. Translation/transcription services 1130 may be implemented, for example, as Web applications or other processes on servers or other networking elements and translation/transcription services 1130 may support one or more application programming interfaces (APIs).).
Claims 15 are rejected under 35 U.S.C. 103 as being unpatentable over Thomson, Pakhomov and Fukumochi in further view of Allibhai (US 20190341052 A1).
With respect to claim 15 Thomson, Pakhomov and Fukumochi do not explicitly disclose however Allibhai teaches wherein said identifying comprises applying, to at least the first and second interpretations, a computer-implemented model that is trained to identify the appropriate translation (Allibhai: ¶[0086] When two or more expert deep neural networks are selected and used, they may be used in parallel or sequentially and the transcription results of the two or more expert deep neural networks can be normalized, compared, and/or combined to select a final transcription on a word-by-word basis, sentence-by-sentence basis, paragraph-by-paragraph basis, or part-by-part basis. If just a single expert deep neural network is used, then its output is used as the final transcription.)
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the invention to modify transcript of Thomson in view of data structures of Pakhomov in view of analysis of Fukumochi to include modeling of Allibhai in order to produce the most accurate transcription s ([0084], Allibhai);
. Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ATHAR N PASHA whose telephone number is (408)918-7675. The examiner can normally be reached Monday-Thursday Alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached on (571)272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
//ATHAR N PASHA/ Primary Examiner, Art Unit 2657