DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed August 7th, 2026, has been entered. Claims 1, 2, 4, 5, 9, 10, 12, 13, and 17-20 have been amended. Claims 1-20 are pending and have been examined. Applicant’s amendments to the specification and the claims have overcome all objections previously set forth.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-3, 5-11, and 13-19 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-11, and 13-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (CN Pat. No. 112133309 A hereinafter Chen), in view of Shir, Maayan (US Pat. Pub. No. 2021/0233535 A1 hereinafter Shir).
Regarding claim 1, Chen discloses a method for text processing, comprising: acquiring a first text and a second text which are to be compared, wherein the first text is a text obtained after text conversion processing is carried out on a dubbed audio corresponding to the second text by an artificial intelligence model (Chen, Page 11, Paragraph 2: "obtaining the audio and the first text to be matched"; "performing speech recognition to each audio segment in the audio segment set to obtain each segment sentence"); segmenting the first text to obtain a plurality of first text segments; segmenting the second text to obtain a plurality of initial second text segments (Chen, Page 11, Paragraph 2: "dividing the first text to obtain the first sentence set; dividing the audio to obtain the audio segment set"; "combining each segment sentence to obtain the second text; obtaining the character sequence corresponding to the second text;"); sequentially determining, for each first text segment in the first text, a matched second text segment that is subjected to an enhancement processing and matched with the first text segment, wherein the enhancement processing refers to movement processing carried out on an endpoint position of the initial second text segment as a processing target in the second text (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and determining, based on a plurality of matched second text segments respectively matched with the plurality of first text segments in the first text, a text matching result of the first text and the second text, wherein the text matching result is used for positioning to a corresponding first text segment, based on a second text segment corresponding to a current dubbed content, in a process of playing the dubbed audio and synchronously displaying the first text (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched;"; Page 3, Paragraph 2: "matching the character sequence of the sentence with the character sequence extracted from the text obtained from the speech recognition; realizing the accurate determination of the synchronous relationship between the audio segment and the sentence; then during the process of playing the audio of the book, according to the established synchronous relationship of the audio segment and the statement; displaying the sentence corresponding to the currently read audio segment by highlight and so on display mode"). However, Chen fails to expressly recite wherein each of the plurality of first text segments respectively corresponds to at least one complete initial second text segment of the plurality of initial second text segments, and wherein a content length of a first text segment is greater than a content length of an initial second text segment.
Shir teaches wherein each of the plurality of first text segments respectively corresponds to at least one complete initial second text segment of the plurality of initial second text segments, and wherein a content length of a first text segment is greater than a content length of an initial second text segment (Shir, [0090]: “Referring back to FIG. 3, at 306, at least one target keyword of the identified non-aligned text fragment of the manually generated transcription is mapped to multiple phonemes, for example, by being fed into an acoustic and/or language model, as described herein.”; [0092]: “Referring back to FIG. 9, the target keywords “fannie mae” of the identified non-aligned text fragment 908 of the manually generated transcription 902 is mapped to multiple phonemes.”; Here, each keyword is being aligned with one or more phonemes, and each phoneme is seen as a complete text segment that is shorter than a keyword.).
Chen and Shir are analogous arts because they each belong to the same field of audio and text processing systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synchronization method of Chen to incorporate the teachings of Li to align text segments to one or more shorter second text segments. This more detailed approach ensures the system properly handles unusual words or transcription errors (Shir, [0091]). As such, the system is able to provide better results for the user.
Regarding claim 2, the rejection of claim 1 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the sequentially determining, for each first text segment in the first text, the matched second text segment that is subjected to the enhancement processing and matched with the first text segment comprises: sequentially determining, according to a logical sequence of each first text segment in the first text, the matched second text segment that is subjected to the enhancement processing and matched with the first text segment according to steps as follows (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"): determining an initial second text segment to be matched with the first text segment (Chen, Page 12: "step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence;"); carrying out the enhancement processing on the initial second text segment to obtain an enhanced second text segment corresponding to the initial second text segment (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and using, in response to a similarity between the enhanced second text segment and the first text segment being greater than a first set threshold, the enhanced second text segment as the matched second text segment matched with the first text segment (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.), and repeatedly executing, in response to the similarity between the enhanced second text segment and the first text segment being smaller than or equal to the first set threshold, a step of determining the initial second text segment to be matched with the first text segment, until the matched second text segment that is subjected to the enhancement processing and matched with the first text segment is determined (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2.").
Regarding claim 3, the rejection of claim 2 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the carrying out the enhancement processing on the initial second text segment to obtain an enhanced second text segment corresponding to the initial second text segment comprises: carrying out a plurality of first movement processing on a first endpoint position of the initial second text segment in the second text to obtain a first enhanced text segment, wherein the first enhanced text segment is an enhanced text segment with a highest similarity with the first text segment, which is obtained after the plurality of first movement processing is carried out on the first endpoint position (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); carrying out a plurality of second movement processing on a second endpoint position of the first enhanced text segment in the second text to obtain a second enhanced text segment, wherein the second enhanced text segment is an enhanced text segment with a highest similarity with the first text segment, which is obtained after the plurality of second movement processing is carried out on the second enhanced text segment (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2." ; Here, the repeated movement of the preset window is seen as second movement processing.); and using, in response to the similarity between the second enhanced text segment and the first text segment being greater than or equal to a second set threshold, the second enhanced text segment as the enhanced second text segment (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.).
Regarding claim 5, the rejection of claim 3 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses after the determining the matched second text segment that is subjected to the enhancement processing and matched with the first text segment, further comprising: determining a third text segment failed in matching in the first text (Chen, Page 12: "step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6"); executing, for each third text segment, a step of determining an initial second text segment to be matched (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); using, in response to a similarity between the obtained enhanced second text segment and the third text segment being greater than a third set threshold, the enhanced second text segment as a matched second text segment matched with the third text segment (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and repeatedly executing, in response to the similarity between the enhanced second text segment and the third text segment being smaller than or equal to the third set threshold, a step of determining the initial second text segment to be matched, until the matched second text segment that is subjected to the enhancement processing and matched with the third text segment is determined (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.). However, Chen fails to expressly recite wherein the third set threshold is smaller than the second set threshold.
Shir further teaches wherein the third set threshold is smaller than the second set threshold (Shir, [0030]: "In a further implementation form of the third aspect, the requirement is a range having an upper threshold value of the phoneme distance denoting identical words and a lower threshold value of the phoneme distance denoting similar but difference words.").
Chen and Shir are analogous arts because they each belong to the same field of audio and text processing systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synchronization method of Chen to incorporate the teachings of Shir to use a varying similarity threshold. A varying threshold allows the system to differentiate between text segments that exactly match and ones that are only similar (Shir, [0030]). This helps the system still find matches even when there are discrepancies in the data.
Regarding claim 6, the rejection of claim 2 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the determining an initial second text segment to be matched comprises: determining a matched text in the second text; and using a text segment positioned behind an end of the matched text as the initial second text segment to be matched (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched;").
Regarding claim 7, the rejection of claim 2 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein whether the similarity between the enhanced second text segment and the first text segment is greater than the first set threshold is determined by: determining a first pinyin unit corresponding to each word in the enhanced second text segment and a second pinyin unit corresponding to each word in the first text segment (Chen, Page 11-12: "In one alternative embodiment, the executable instructions further cause the processor to perform the following operations: according to the sequence of each audio segment in the audio, combining the segment sentence of each audio segment to obtain the second text; obtaining the pinyin initial of all characters contained in the second text as the character sequence corresponding to the second text, and recording the starting character and ending character corresponding to each audio segment in the character sequence corresponding to the second text; obtaining the Pinyin initial of all characters contained in the first sentence as the first character sequence."); and determining, according to a similarity between each first pinyin unit and each second pinyin unit, whether the similarity between the enhanced second text segment and the first text segment is greater than the first set threshold (Chen, Page 12: "step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6").
Regarding claim 8, the rejection of claim 1 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses after the determining a text matching result of the first text and the second text, further comprising: playing, in response to a playing triggering operation for the second text, the dubbed audio corresponding to the second text; determining, according to the second text segment corresponding to the current dubbed content and the text matching result, a first text segment corresponding to the second text segment; and synchronously displaying positioning to the determined first text segment of the first text in the process of playing the dubbed audio (Chen, Page 3, Paragraph 2: "matching the character sequence of the sentence with the character sequence extracted from the text obtained from the speech recognition; realizing the accurate determination of the synchronous relationship between the audio segment and the sentence; then during the process of playing the audio of the book, according to the established synchronous relationship of the audio segment and the statement; displaying the sentence corresponding to the currently read audio segment by highlight and so on display mode").
Regarding claim 9, Chen discloses a computer device, comprising: at least one processor and at least one storage, wherein the at least one storage stores machine-readable instructions executable by the at least one processor; the at least one processor communicates with the at least one storage upon running of the computer device, and the machine-readable instructions, upon being executed by the at least one processor implements a method for text processing (Chen, Page 11: "The third embodiment of the present invention provides a non-volatile computer storage medium, the storage medium stores at least one executable instruction, the executable instruction can perform the synchronization method of audio and text in any of the above method embodiments. The executable instructions may specifically be used to cause the processor to perform the following operations"), and the method comprises: acquiring a first text and a second text which are to be compared, wherein the first text is a text obtained after text conversion processing is carried out on a dubbed audio corresponding to the second text by an artificial intelligence model (Chen, Page 11, Paragraph 2: "obtaining the audio and the first text to be matched"; "performing speech recognition to each audio segment in the audio segment set to obtain each segment sentence"); segmenting the first text to obtain a plurality of first text segments; segmenting the second text to obtain a plurality of initial second text segments (Chen, Page 11, Paragraph 2: "dividing the first text to obtain the first sentence set; dividing the audio to obtain the audio segment set"; "combining each segment sentence to obtain the second text; obtaining the character sequence corresponding to the second text;"); sequentially determining, for each first text segment in the first text, a matched second text segment that is subjected to an enhancement processing and matched with the first text segment, wherein the enhancement processing refers to movement processing carried out on an endpoint position of the initial second text segment as a processing target in the second text (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and determining, based on a plurality of matched second text segments respectively matched with the plurality of first text segments in the first text, a text matching result of the first text and the second text, wherein the text matching result is used for positioning to a corresponding first text segment, based on a second text segment corresponding to a current dubbed content, in a process of playing the dubbed audio and synchronously displaying the first text (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched;"; Page 3, Paragraph 2: "matching the character sequence of the sentence with the character sequence extracted from the text obtained from the speech recognition; realizing the accurate determination of the synchronous relationship between the audio segment and the sentence; then during the process of playing the audio of the book, according to the established synchronous relationship of the audio segment and the statement; displaying the sentence corresponding to the currently read audio segment by highlight and so on display mode"). However, Chen fails to expressly recite wherein each of the plurality of first text segments respectively corresponds to at least one complete initial second text segment of the plurality of initial second text segments, and wherein a content length of a first text segment is greater than a content length of an initial second text segment.
Shir teaches wherein each of the plurality of first text segments respectively corresponds to at least one complete initial second text segment of the plurality of initial second text segments, and wherein a content length of a first text segment is greater than a content length of an initial second text segment (Shir, [0090]: “Referring back to FIG. 3, at 306, at least one target keyword of the identified non-aligned text fragment of the manually generated transcription is mapped to multiple phonemes, for example, by being fed into an acoustic and/or language model, as described herein.”; [0092]: “Referring back to FIG. 9, the target keywords “fannie mae” of the identified non-aligned text fragment 908 of the manually generated transcription 902 is mapped to multiple phonemes.”; Here, each keyword is being aligned with one or more phonemes, and each phoneme is seen as a complete text segment that is shorter than a keyword.).
Chen and Shir are analogous arts because they each belong to the same field of audio and text processing systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synchronization method of Chen to incorporate the teachings of Li to align text segments to one or more shorter second text segments. This more detailed approach ensures the system properly handles unusual words or transcription errors (Shir, [0091]). As such, the system is able to provide better results for the user.
Regarding claim 10, the rejection of claim 9 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the sequentially determining, for each first text segment in the first text, the matched second text segment that is subjected to the enhancement processing and matched with the first text segment comprises: sequentially determining, according to a logical sequence of each first text segment in the first text, the matched second text segment that is subjected to the enhancement processing and matched with the first text segment according to steps as follows (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"): determining an initial second text segment to be matched with the first text segment (Chen, Page 12: "step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence;"); carrying out the enhancement processing on the initial second text segment to obtain an enhanced second text segment corresponding to the initial second text segment (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and using, in response to a similarity between the enhanced second text segment and the first text segment being greater than a first set threshold, the enhanced second text segment as the matched second text segment matched with the first text segment (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.), and repeatedly executing, in response to the similarity between the enhanced second text segment and the first text segment being smaller than or equal to the first set threshold, a step of determining the initial second text segment to be matched with the first text segment, until the matched second text segment that is subjected to the enhancement processing and matched with the first text segment is determined (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2.").
Regarding claim 11, the rejection of claim 10 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the carrying out the enhancement processing on the initial second text segment to obtain an enhanced second text segment corresponding to the initial second text segment comprises: carrying out a plurality of first movement processing on a first endpoint position of the initial second text segment in the second text to obtain a first enhanced text segment, wherein the first enhanced text segment is an enhanced text segment with a highest similarity with the first text segment, which is obtained after the plurality of first movement processing is carried out on the first endpoint position (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); carrying out a plurality of second movement processing on a second endpoint position of the first enhanced text segment in the second text to obtain a second enhanced text segment, wherein the second enhanced text segment is an enhanced text segment with a highest similarity with the first text segment, which is obtained after the plurality of second movement processing is carried out on the second enhanced text segment (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2." ; Here, the repeated movement of the preset window is seen as second movement processing.); and using, in response to the similarity between the second enhanced text segment and the first text segment being greater than or equal to a second set threshold, the second enhanced text segment as the enhanced second text segment (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.).
Regarding claim 13, the rejection of claim 11 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses after the determining the matched second text segment that is subjected to the enhancement processing and matched with the first text segment, further comprising: determining a third text segment failed in matching in the first text (Chen, Page 12: "step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6"); executing, for each third text segment, a step of determining an initial second text segment to be matched (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); using, in response to a similarity between the obtained enhanced second text segment and the third text segment being greater than a third set threshold, the enhanced second text segment as a matched second text segment matched with the third text segment (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and repeatedly executing, in response to the similarity between the enhanced second text segment and the third text segment being smaller than or equal to the third set threshold, a step of determining the initial second text segment to be matched, until the matched second text segment that is subjected to the enhancement processing and matched with the third text segment is determined (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.). However, Chen fails to expressly recite wherein the third set threshold is smaller than the second set threshold.
Shir further teaches wherein the third set threshold is smaller than the second set threshold (Shir, [0030]: "In a further implementation form of the third aspect, the requirement is a range having an upper threshold value of the phoneme distance denoting identical words and a lower threshold value of the phoneme distance denoting similar but difference words.").
Chen and Shir are analogous arts because they each belong to the same field of audio and text processing systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synchronization method of Chen to incorporate the teachings of Shir to use a varying similarity threshold. A varying threshold allows the system to differentiate between text segments that exactly match and ones that are only similar (Shir, [0030]). This helps the system still find matches even when there are discrepancies in the data.
Regarding claim 14, the rejection of claim 10 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the determining an initial second text segment to be matched comprises: determining a matched text in the second text; and using a text segment positioned behind an end of the matched text as the initial second text segment to be matched (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched;").
Regarding claim 15, the rejection of claim 10 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein whether the similarity between the enhanced second text segment and the first text segment is greater than the first set threshold is determined by: determining a first pinyin unit corresponding to each word in the enhanced second text segment and a second pinyin unit corresponding to each word in the first text segment (Chen, Page 11-12: "In one alternative embodiment, the executable instructions further cause the processor to perform the following operations: according to the sequence of each audio segment in the audio, combining the segment sentence of each audio segment to obtain the second text; obtaining the pinyin initial of all characters contained in the second text as the character sequence corresponding to the second text, and recording the starting character and ending character corresponding to each audio segment in the character sequence corresponding to the second text; obtaining the Pinyin initial of all characters contained in the first sentence as the first character sequence."); and determining, according to a similarity between each first pinyin unit and each second pinyin unit, whether the similarity between the enhanced second text segment and the first text segment is greater than the first set threshold (Chen, Page 12: "step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6").
Regarding claim 16, the rejection of claim 9 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses after the determining a text matching result of the first text and the second text, further comprising: playing, in response to a playing triggering operation for the second text, the dubbed audio corresponding to the second text; determining, according to the second text segment corresponding to the current dubbed content and the text matching result, a first text segment corresponding to the second text segment; and synchronously displaying positioning to the determined first text segment of the first text in the process of playing the dubbed audio (Chen, Page 3, Paragraph 2: "matching the character sequence of the sentence with the character sequence extracted from the text obtained from the speech recognition; realizing the accurate determination of the synchronous relationship between the audio segment and the sentence; then during the process of playing the audio of the book, according to the established synchronous relationship of the audio segment and the statement; displaying the sentence corresponding to the currently read audio segment by highlight and so on display mode").
Regarding claim 17, Chen discloses a non-transitory computer-readable storage medium storing computer programs, wherein the computer programs, upon being run by at least one processor, implement a method for text processing (Chen, Page 11: "The third embodiment of the present invention provides a non-volatile computer storage medium, the storage medium stores at least one executable instruction, the executable instruction can perform the synchronization method of audio and text in any of the above method embodiments. The executable instructions may specifically be used to cause the processor to perform the following operations"), and the method comprises: acquiring a first text and a second text which are to be compared, wherein the first text is a text obtained after text conversion processing is carried out on a dubbed audio corresponding to the second text by an artificial intelligence model (Chen, Page 11, Paragraph 2: "obtaining the audio and the first text to be matched"; "performing speech recognition to each audio segment in the audio segment set to obtain each segment sentence"); segmenting the first text to obtain a plurality of first text segments; segmenting the second text to obtain a plurality of initial second text segments (Chen, Page 11, Paragraph 2: "dividing the first text to obtain the first sentence set; dividing the audio to obtain the audio segment set"; "combining each segment sentence to obtain the second text; obtaining the character sequence corresponding to the second text;"); sequentially determining, for each first text segment in the first text, a matched second text segment that is subjected to an enhancement processing and matched with the first text segment, wherein the enhancement processing refers to movement processing carried out on an endpoint position of the initial second text segment as a processing target in the second text (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and determining, based on a plurality of matched second text segments respectively matched with the plurality of first text segments in the first text, a text matching result of the first text and the second text, wherein the text matching result is used for positioning to a corresponding first text segment, based on a second text segment corresponding to a current dubbed content, in a process of playing the dubbed audio and synchronously displaying the first text (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched;"; Page 3, Paragraph 2: "matching the character sequence of the sentence with the character sequence extracted from the text obtained from the speech recognition; realizing the accurate determination of the synchronous relationship between the audio segment and the sentence; then during the process of playing the audio of the book, according to the established synchronous relationship of the audio segment and the statement; displaying the sentence corresponding to the currently read audio segment by highlight and so on display mode"). However, Chen fails to expressly recite wherein each of the plurality of first text segments respectively corresponds to at least one complete initial second text segment of the plurality of initial second text segments, and wherein a content length of a first text segment is greater than a content length of an initial second text segment.
Shir teaches wherein each of the plurality of first text segments respectively corresponds to at least one complete initial second text segment of the plurality of initial second text segments, and wherein a content length of a first text segment is greater than a content length of an initial second text segment (Shir, [0090]: “Referring back to FIG. 3, at 306, at least one target keyword of the identified non-aligned text fragment of the manually generated transcription is mapped to multiple phonemes, for example, by being fed into an acoustic and/or language model, as described herein.”; [0092]: “Referring back to FIG. 9, the target keywords “fannie mae” of the identified non-aligned text fragment 908 of the manually generated transcription 902 is mapped to multiple phonemes.”; Here, each keyword is being aligned with one or more phonemes, and each phoneme is seen as a complete text segment that is shorter than a keyword.).
Chen and Shir are analogous arts because they each belong to the same field of audio and text processing systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synchronization method of Chen to incorporate the teachings of Li to align text segments to one or more shorter second text segments. This more detailed approach ensures the system properly handles unusual words or transcription errors (Shir, [0091]). As such, the system is able to provide better results for the user.
Regarding claim 18, the rejection of claim 17 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the sequentially determining for each first text segment in the first text, the matched second text segment that is subjected to the enhancement processing and matched with the first text segment comprises: sequentially determining, according to a logical sequence of each first text segment in the first text, the matched second text segment that is subjected to the enhancement processing and matched with the first text segment according to steps as follows (Chen, Page 12: "step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"): determining an initial second text segment to be matched with the first text segment (Chen, Page 12: "step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence;"); carrying out the enhancement processing on the initial second text segment to obtain an enhanced second text segment corresponding to the initial second text segment (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); and using, in response to a similarity between the enhanced second text segment and the first text segment being greater than a first set threshold, the enhanced second text segment as the matched second text segment matched with the first text segment (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.), and repeatedly executing, in response to the similarity between the enhanced second text segment and the first text segment being smaller than or equal to the first set threshold, a step of determining the initial second text segment to be matched with the first text segment, until the matched second text segment that is subjected to the enhancement processing and matched with the first text segment is determined (Chen, Page 12: "step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2.").
Regarding claim 19, the rejection of claim 18 is incorporated. Chen, in view of Shir, discloses all of the elements of the current invention as stated above. Chen further discloses wherein the carrying out the enhancement processing on the initial second text segment to obtain an enhanced second text segment corresponding to the initial second text segment comprises: carrying out a plurality of first movement processing on a first endpoint position of the initial second text segment in the second text to obtain a first enhanced text segment, wherein the first enhanced text segment is an enhanced text segment with a highest similarity with the first text segment, which is obtained after the plurality of first movement processing is carried out on the first endpoint position (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2."); carrying out a plurality of second movement processing on a second endpoint position of the first enhanced text segment in the second text to obtain a second enhanced text segment, wherein the second enhanced text segment is an enhanced text segment with a highest similarity with the first text segment, which is obtained after the plurality of second movement processing is carried out on the second enhanced text segment (Chen, Page 12: "step 1, extracting the first sentence from the first sentence set, obtaining the first character sequence corresponding to the first sentence; step 2, according to the preset window starting point, extracting the second character sequence from the character sequence corresponding to the second text; setting the initial character of the second character sequence as the starting character of the third character sequence; step 3, calculating the editing distance matrix between the first character sequence and the second character sequence; step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 6, setting the next character of the ending character of the second character sequence as the starting point of the preset window; and skipping to execute the step 2." ; Here, the repeated movement of the preset window is seen as second movement processing.); and using, in response to the similarity between the second enhanced text segment and the first text segment being greater than or equal to a second set threshold, the second enhanced text segment as the enhanced second text segment (Chen, Page 12: step 4, judging whether the last row in the editing distance matrix has the matrix element whose matrix element value is less than the preset threshold value; if yes, executing step 5; if not, executing step 6; step 5, setting the character in the second character sequence corresponding to the matrix element with the minimum matrix element value of the last row in the editing distance matrix as the ending character of the third character sequence; establishing the synchronous relationship between the audio segment corresponding to the third character sequence and the first sentence; and setting the next character of the ending character of the third character sequence as the preset window starting point; skipping to execute the step 1 until all the first sentences are matched"; Here, the editing distance being less than a preset threshold is seen as being equivalent to a similarity being greater than a threshold.).
Allowable Subject Matter
Claims 4, 12, and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 4, the claim defines a specific and detailed process of repeatedly moving the endpoint of a text segment to obtain numerous enhanced text segments that is not disclosed by any references found during the prior art search, and would not have been obvious to one of ordinary art before the effective filing date of the claimed invention based on a combination of references found during the prior art search.
Regarding claims 12 and 20, the claims recite comparable limitations to those of claim 4, and thus contain allowable subject matter for the same reasons as discussed above with regards to claim 4.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYLER J BECKER whose telephone number is (703)756-1271. The examiner can normally be reached M-Th, 7:15am-5:45pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TYLER BECKER/ Examiner, Art Unit 2657
/DANIEL C WASHBURN/ Supervisory Patent Examiner, Art Unit 2657
Prosecution Insights