DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 07/28/2026 have been fully considered but they are not persuasive. Regarding arguments on pages 7-8 of the Remarks, Examiner notes that the encoding step is not characterized as a mental process, but would instead be a mathematical calculation. An encoder is generally understood to compress data to a lower dimension signal. The step of encoding the signal to form an output including the detected prosody and phonetic components is a mathematical calculation, while the actual encoder itself would generally be performed by a processor. Therefore, the encoder is considered a generic computing component, as no specialized hardware is described in the claims. Therefore, since the steps are considered abstract, they cannot also be considered an improvement to or significantly more than the abstract ideas.
Regarding arguments on pages 9-10 of the Remarks, Examiner notes that the published Skarnitzl paper cites the dates of 30 Sept – 2 Oct 2024 as the dates of the Symposium. Going to the website for the Symposium (https://isaph2024.ut.ee/program/) reveals a program and a book of abstracts available for download, which were presented on the dates listed above. The abstract provided by Skarnitzl et al. corresponding to the paper is included in this Office Action. The citations to Skarnitzl in the Office Action are updated to include the corresponding citation to the abstract in brackets []. The updated citations show that the teachings of Skarnitzl used in the Office Action were publicly available before the effective filing date of the invention.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Using the subject matter eligibility test from page 74621 of the Federal Register Notice titled “2014 Interim Guidance on Patent Subject Matter Eligibility,” a two-step process is performed. Under step 1, the claims are analyzed to determine if the claim is directed to a process, machine, article of manufacture, or composition of matter. In this case, claims 1-18 are directed to a device, which is a machine or an article of manufacture; claim 19 is directed to a method, which is a process; and claim 20 is directed to a computer-readable medium, which is a machine or an article of manufacture. Step 2A (part 1 of the Mayo test), using the guidance from pages 50-57 of the Federal Register Vol. 84 No. 4 from Monday, January 7, 2019, requires applying a two-prong inquiry. In Prong One, examiners evaluate whether the claim recites a judicial exception, determining if the claim is directed to a law of nature, a natural phenomenon, or an abstract idea. In this case, claim 1 recites detecting components, performing comparisons, and generating outputs, which are mental processes. In Prong Two, examiners evaluate whether the judicial exception is integrated into a practical application that imposes a meaningful limit on the judicial exception. In this case, additional limitations of storing or obtaining audio are mere extrasolution activity, while structural elements of processor, memory, and storage are generic computing components, and do not integrate the abstract idea into a practical application.
Step 2B (part 2 of the Mayo test) requires analyzing the claims to determine if they recite additional elements that amount to significantly more than the judicial exception. In this case, the claims do not include additional elements that are sufficient to amount to significantly more than the abstract idea itself.
Regarding claims 1 and 19-20, detecting components, performing comparisons, and generating outputs are mental processes, while encoding audio is a mathematical process, which are abstract ideas. For example, a human could detect prosodic and phonetic components, in speech, compare them to a reference, and generate an output. Additional limitations of storing or obtaining audio are mere extrasolution activity, while structural elements of processor, memory, and storage are generic computing components, and do not integrate the abstract idea into a practical application or constitute significantly more.
Regarding claims 2-5, 9-11, the limitations are further clarifications of the above abstract ideas.
Regarding claims 6-8, generating audio is mere extrasolution activity, and does not integrate the abstract ideas into a practical application or constitute significantly more.
Regarding claims 12-13, the use of the factorized speech encoder to perform the abstract idea would be considered using generic computing components to apply the abstract idea, which would not integrate the abstract idea into a practical application or constitute significantly more.
Regarding claim 14, generating scores is either a mental process or a mathematical calculation, both of which are abstract ideas without integration into a practical application and without significantly more.
Regarding claim 15, generating a GUI is mere extrasolution activity, and does not integrate the abstract ideas into a practical application or constitute significantly more.
Regarding claims 16-17, the use of the LLM to perform the abstract idea would be considered using generic computing components to apply the abstract idea, which would not integrate the abstract idea into a practical application or constitute significantly more.
Regarding claim 18, a microphone is a generic computing component, and does not integrate the abstract idea into a practical application or constitute significantly more.
The limitations of the claims, taken alone, do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements individually. Applicable case law cited in the Federal Register includes, but is not limited to: Alice Corp., 134 S. Ct. at 2355-56, Digitech Image Tech., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344 (Fed. Cir. 2014), Benson, 409 U.S. at 63.
See "Preliminary Examination Instructions in view of the Supreme Court Decision in Alice Corporation Pty. Ltd. v. CLS Bank International, et al.," dated June 25, 2014, and the Federal Register notice titled "2014 Interim Guidance on Patent Subject Matter Eligibility" (79 FR 74618).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 9-11, 14, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lurie et al. (US 2022/0180762 A1), hereinafter referred to as Lurie, in view of Skarnitzl et al. (Skarnitzl, R. (2024, January). Training English prosody with manipulated voices: Can both intermediate and advanced learners benefit. In Proceedings of the 5th International Symposium on Applied Phonetics (ISAPh 2024).), hereinafter referred to as Skarnitzl, and further in view of Gupta et al. (US 2004/0006461 A1), hereinafter referred to as Gupta.
Regarding claim 1, Lurie teaches:
A device comprising:
a memory configured to store input audio that corresponds to speech representing a target sentence spoken by a user (Fig. 9 element 1030, para [0042], where memory is used); and
one or more processors coupled to the memory (Fig. 9 element 1020, para [0042], where a processor is used) and configured to:
perform a prosody comparison of a reference prosody component and the detected prosody component (para [0025], where overall phrase similarity scores are determined),
perform a phonetics comparison of a reference phonetic component and the detected phonetic component (para [0025], where individual phoneme similarity scores are determined),
generate an output based on the prosody comparison and the phonetics comparison (Fig. 2 element 214, para [0033], where teachable aspects are selected, and the revised augmented teaching output is presented to the user).
Lurie does not teach:
process, at a speech encoder, the input audio to generate an encoder output that includes at least a detected prosody component and a detected phonetic component;
the reference prosody component based on the target sentence with speech characteristics of the user and having a target pronunciation;
the reference phonetic component based on the target sentence with the speech characteristics of the user and having the target pronunciation; and
Skarnitzl teaches:
the reference prosody component based on the target sentence with speech characteristics of the user and having a target pronunciation (page 94 section 1 last paragraph, page 95 first paragraph [page 73 section II first paragraph], where portions of the user's own recorded speech are modified and used for the pronunciation training);
the reference phonetic component based on the target sentence with the speech characteristics of the user and having the target pronunciation (page 94 section 1 last paragraph, page 95 first paragraph [page 73 section II first paragraph], where portions of the user's own recorded speech are modified and used for the pronunciation training); and
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie by using the user modified voice of Skarnitzl (Skarnitzl page 95 first paragraph [page 73 section II first paragraph]) as the reference version of the diagnostic phrase of Lurie (Lurie para [0029]), as using learners' own voices strengthens neural response and results in more effective learning (Skarnitzl page 94 second column first paragraph [page 74 last paragraph]).
Gupta teaches:
process, at a speech encoder, the input audio to generate an encoder output that includes at least a detected prosody component and a detected phonetic component (para [0042-43], where the recorded user voice is encoded, and the voice stream or signal is scored for articulation, duration, intonation, and voicing);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl by using the encoding and scoring of Gupta (Gupta para [0042-43]) on the input speech of Lurie in view of Skarnitzl, (Lurie para [0024]), in order to perform pronunciation scoring for learning or eliminating an accent (Gupta para [0029-30], [0041]).
Regarding claim 2, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1, wherein the reference prosody component includes multiple reference sample prosody components, each of the multiple reference sample prosody components based on the target sentence with the speech characteristics of the user, a respective distinct speech manner, and having the target pronunciation (Skarnitzl page 95 first two full paragraphs [page 73 section II first paragraph and bullet points], where the reference samples include a user reading a diagnostic text of 230 words, and is modified to be the user's own voice with different speech manner and pronunciation, as shown in the bullet points).
Regarding claim 3, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 2, wherein the prosody comparison is based on a comparison of the detected prosody component and each of the multiple reference sample prosody components (Lurie para [0025], [0029], where overall phrase similarity scores are determined by comparing the input with the reference).
Regarding claim 4, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1, wherein the reference phonetic component includes multiple reference sample phonetic components, each of the multiple reference sample phonetic components based on the target sentence with the speech characteristics of the user, a respective distinct speech manner, and having the target pronunciation (Skarnitzl page 95 first two full paragraphs [page 73 section II first paragraph and bullet points], where the reference samples include a user reading a diagnostic text of 230 words, and is modified to be the user's own voice with different speech manner and pronunciation, as shown in the bullet points).
Regarding claim 5, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 4, wherein the phonetics comparison is based on a comparison of the detected phonetic component and each of the multiple reference sample phonetic components (Lurie para [0025], [0029], where individual phoneme similarity scores are determined by comparing the input with the reference).
Regarding claim 6, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1, wherein the one or more processors are configured to generate reference audio that corresponds to synthesized speech that represents the target sentence having the speech characteristics of the user and having the target pronunciation, wherein the reference prosody component and the reference phonetic component are based on the reference audio (Skarnitzl page 95 third paragraph [page 73 section II second paragraph], where the participants listen to the modified stimuli, interpreted as generated reference audio corresponding to synthesized speech).
Regarding claim 9, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 6, wherein the reference audio includes multiple reference audio samples, each of the multiple reference audio samples corresponds to synthesized speech that represents the target sentence having the speech characteristics of the user, a respective distinct speech manner, and having the target pronunciation (Skarnitzl page 95 first and third paragraphs [page 73 section II first paragraph], where nine or ten stimuli are used for each participant, each stimulus comprising between one and three prosodic phrases).
Regarding claim 10, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 9, wherein the reference prosody component includes multiple reference sample prosody components, wherein each of the multiple reference sample prosody components is based on a respective one of the multiple reference audio samples, and wherein the prosody comparison is based on a comparison of the detected prosody component and each of the multiple reference sample prosody components (Skarnitzl page 95 first and third paragraphs [page 73 section II first paragraph], where nine or ten stimuli are used for each participant, each stimulus comprising between one and three prosodic phrases, and Lurie para [0025], [0029], where overall phrase similarity scores are determined by comparing the input with the reference).
Regarding claim 11, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 9, wherein the reference phonetic component includes multiple reference sample phonetic components, wherein each of the multiple reference sample phonetic components based on a respective one of the multiple reference audio samples, and wherein the phonetics comparison is based on a comparison of the detected phonetic component and each of the multiple reference sample phonetic components (Skarnitzl page 95 first and third paragraphs [page 73 section II first paragraph], where nine or ten stimuli are used for each participant, each stimulus comprising between one and three prosodic phrases, and Lurie para [0025], [0029], where individual phoneme similarity scores are determined by comparing the input with the reference).
Regarding claim 14, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1, wherein the one or more processors are configured to:
generate a prosody score based on the prosody comparison (Lurie para [0025], where overall phrase similarity scores are determined); and
generate a phonetic score based on the phonetics comparison, wherein the output is based on the prosody score and the phonetic score (Lurie para [0025], where individual phoneme similarity scores are determined, and Fig. 2 element 214, para [0033], where teachable aspects are selected based on the scores).
Regarding claim 18, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1, further comprising a microphone configured to receive the input audio (Lurie Fig. 1 element 110, para [0025], where the audio input recorder is interpreted as a microphone).
Regarding claim 19, Lurie teaches:
A method comprising:
obtaining, at a device, input audio that corresponds to speech representing a target sentence spoken by a user (para [0025], where a user audio input recorder receives speech of a user speaking diagnostic phrases);
performing, at the device, a prosody comparison of a reference prosody component and the detected prosody component (para [0025], where overall phrase similarity scores are determined),
performing, at the device, a phonetics comparison of a reference phonetic component and the detected phonetic component (para [0025], where individual phoneme similarity scores are determined),
generating, at the device, an output based on the prosody comparison and the phonetics comparison (Fig. 2 element 214, para [0033], where teachable aspects are selected, and the revised augmented teaching output is presented to the user).
Lurie does not teach:
processing, at a speech encoder of the device, the input audio to generate an encoder output that includes at least a detected prosody component and a detected phonetic component;
the reference prosody component based on the target sentence with speech characteristics of the user and having a target pronunciation;
the reference phonetic component based on the target sentence with the speech characteristics of the user and having the target pronunciation; and
Skarnitzl teaches:
the reference prosody component based on the target sentence with speech characteristics of the user and having a target pronunciation (page 94 section 1 last paragraph, page 95 first paragraph [page 73 section II first paragraph], where portions of the user's own recorded speech are modified and used for the pronunciation training);
the reference phonetic component based on the target sentence with the speech characteristics of the user and having the target pronunciation (page 94 section 1 last paragraph, page 95 first paragraph [page 73 section II first paragraph], where portions of the user's own recorded speech are modified and used for the pronunciation training); and
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie by using the user modified voice of Skarnitzl (Skarnitzl page 95 first paragraph [page 73 section II first paragraph]) as the reference version of the diagnostic phrase of Lurie (Lurie para [0029]), as using learners' own voices strengthens neural response and results in more effective learning (Skarnitzl page 94 second column first paragraph [page 74 last paragraph]).
Gupta teaches:
processing, at a speech encoder of the device, the input audio to generate an encoder output that includes at least a detected prosody component and a detected phonetic component (para [0042-43], where the recorded user voice is encoded, and the voice stream or signal is scored for articulation, duration, intonation, and voicing);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl by using the encoding and scoring of Gupta (Gupta para [0042-43]) on the input speech of Lurie in view of Skarnitzl, (Lurie para [0024]), in order to perform pronunciation scoring for learning or eliminating an accent (Gupta para [0029-30], [0041]).
Regarding claim 20, Lurie teaches:
A non-transitory computer-readable medium storing instructions (Fig. 9 element 1050, para [0042], where a computer readable storage medium is used) that, when executed by one or more processors, cause the one or more processors to:
obtain input audio that corresponds to speech representing a target sentence spoken by a user (para [0025], where a user audio input recorder receives speech of a user speaking diagnostic phrases);
perform a prosody comparison of a reference prosody component and the detected prosody component (para [0025], where overall phrase similarity scores are determined),
perform a phonetics comparison of a reference phonetic component and the detected phonetic component (para [0025], where individual phoneme similarity scores are determined),
generate an output based on the prosody comparison and the phonetics comparison (Fig. 2 element 214, para [0033], where teachable aspects are selected, and the revised augmented teaching output is presented to the user).
Lurie does not teach:
process, at a speech encoder, the input audio to generate an encoder output that includes at least a detected prosody component and a detected phonetic component;
the reference prosody component based on the target sentence with speech characteristics of the user and having a target pronunciation;
the reference phonetic component based on the target sentence with the speech characteristics of the user and having the target pronunciation; and
Skarnitzl teaches:
the reference prosody component based on the target sentence with speech characteristics of the user and having a target pronunciation (page 94 section 1 last paragraph, page 95 first paragraph [page 73 section II first paragraph], where portions of the user's own recorded speech are modified and used for the pronunciation training);
the reference phonetic component based on the target sentence with the speech characteristics of the user and having the target pronunciation (page 94 section 1 last paragraph, page 95 first paragraph [page 73 section II first paragraph], where portions of the user's own recorded speech are modified and used for the pronunciation training); and
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie by using the user modified voice of Skarnitzl (Skarnitzl page 95 first paragraph [page 73 section II first paragraph]) as the reference version of the diagnostic phrase of Lurie (Lurie para [0029]), as using learners' own voices strengthens neural response and results in more effective learning (Skarnitzl page 94 second column first paragraph [page 74 last paragraph]).
Gupta teaches:
process, at a speech encoder, the input audio to generate an encoder output that includes at least a detected prosody component and a detected phonetic component (para [0042-43], where the recorded user voice is encoded, and the voice stream or signal is scored for articulation, duration, intonation, and voicing);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl by using the encoding and scoring of Gupta (Gupta para [0042-43]) on the input speech of Lurie in view of Skarnitzl, (Lurie para [0024]), in order to perform pronunciation scoring for learning or eliminating an accent (Gupta para [0029-30], [0041]).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lurie, in view of Skarnitzl, and Gupta, and further in view of Ye et al. (US 2021/0304769 A1), hereinafter referred to as Ye.
Regarding claim 7, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 6,
Lurie in view of Skarnitzl and Gupta does not teach:
wherein the one or more processors are configured to generate, at a personalized text-to-speech engine, the reference audio based on the target sentence and a user speech embedding corresponding to the user.
Ye teaches:
wherein the one or more processors are configured to generate, at a personalized text-to-speech engine, the reference audio based on the target sentence and a user speech embedding corresponding to the user (Fig. 11, para [0097], where a personalized TTS uses a speaker embedding).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl and Gupta by using the personalized TTS of Ye (Ye para [0097]) to generate the reference audio of Lurie in view of Skarnitzl and Gupta (Skarnitzl page 95 first paragraph), so that speech data can be produced from random scripts or text (Ye para [0097]).
Claim(s) 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lurie, in view of Skarnitzl, and Gupta, and further in view of Gong et al. (Gong, Y., Chen, Z., Chu, I. H., Chang, P., & Glass, J. (2022, May). Transformer-based multi-aspect multi-granularity non-native english speaker pronunciation assessment. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 7262-7266). IEEE.), hereinafter referred to as Gong.
Regarding claim 12, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1, wherein the encoder output further includes a detected speaker vocal characteristics component of the speech (Gupta para [0072], where the voicing is interpreted as the detected speaker vocal characteristics)
Lurie in view of Skarnitzl and Gupta does not teach:
wherein the speech encoder corresponds to a factorized speech encoder, .
Gong teaches:
wherein the speech encoder corresponds to a factorized speech encoder (page 3 section 3.3 first paragraph, where the model is based on a factorized neural network).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl and Gupta by using the encoder of Gong (Gong page 2 Fig. 1) to generate the prosody and phonetic components of Lurie in view of Skarnitzl and Gupta (Lurie para [0024], Skarnitzl page 95 section 2.2 first paragraph), in order to capture contextual information between phonemes and words of an utterance and improvement assessment tasks (Gong page 1 section 1 last paragraph).
Regarding claim 13, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1, wherein the reference audio corresponds to synthesized speech that represents the target sentence having the speech characteristics of the user and having the target pronunciation (Skarnitzl page 95 first two full paragraphs [page 73 section II first paragraph and bullet points], where the input audio is modified to be the user's own voice with different speech manner and pronunciation, as shown in the bullet points),
Lurie in view of Skarnitzl and Gupta does not teach:
wherein the one or more processors are configured to process, at the speech encoder, reference audio to generate a reference encoder output that includes at least the reference prosody component and the reference phonetic component,
wherein the encoder output includes a detected speaker vocal characteristics component, and wherein the reference encoder output includes a reference speaker vocal characteristics component.
Gong teaches:
wherein the one or more processors are configured to process, at the speech encoder, reference audio to generate a reference encoder output that includes at least the reference prosody component and the reference phonetic component (page 2 Fig. 1, where the encoder has outputs including prosodic scores and phone scores),
wherein the encoder output includes a detected speaker vocal characteristics component, and wherein the reference encoder output includes a reference speaker vocal characteristics component (page 2 Fig. 1, where the outputs include accuracy, fluency, and completeness scores, which correspond to the speaker vocal characteristics component).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl and Gupta by using the encoder of Gong (Gong page 2 Fig. 1) to generate the prosody and phonetic components of Lurie in view of Skarnitzl and Gupta (Lurie para [0024], Skarnitzl page 95 section 2.2 first paragraph), in order to capture contextual information between phonemes and words of an utterance and improvement assessment tasks (Gong page 1 section 1 last paragraph).
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lurie, in view of Skarnitzl, and Gupta, and further in view of Du et al. (US 2007/0048697 A1), hereinafter referred to as Du.
Regarding claim 15, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1,
Lurie in view of Skarnitzl and Gupta does not teach:
wherein the one or more processors are configured to generate the output including a graphical user interface (GUI) that indicates results of at least the prosody comparison or the phonetics comparison aligned with respective phonemes of the target sentence.
Du teaches:
wherein the one or more processors are configured to generate the output including a graphical user interface (GUI) that indicates results of at least the prosody comparison or the phonetics comparison aligned with respective phonemes of the target sentence (Fig. 11, para [0083-84], where a GUI displays a phoneme by phoneme analysis with feedback information).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl and Gupta by using the GUI of Du (Du para [0083-84]) to present the output of Lurie in view of Skarnitzl and Gupta (Lurie para [0033]), in order to provide a visual indication as to how well a given letter or letter combination was pronounced (Du para [0084]).
Claim(s) 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lurie, in view of Skarnitzl, and Gupta, and further in view of Fu et al. (Fu, K., Peng, L., Yang, N., & Zhou, S. (2024). Pronunciation assessment with multi-modal large language models. arXiv preprint arXiv:2407.09209.), hereinafter referred to as Fu.
Regarding claim 16, Lurie in view of Skarnitzl and Gupta teaches:
The device of claim 1,
to generate feedback on a presentation that includes at least the target sentence, wherein the feedback includes at least one of speech speed feedback, pronunciation suggestion, or speech duration feedback (Lurie para [0025], where the machine learning system is used to provide pronunciation feedback based on comparing the user input to the reference).
Lurie in view of Skarnitzl and Gupta does not teach:
wherein the one or more processors are configured to provide the output to a large language model (LLM)
Fu teaches:
wherein the one or more processors are configured to provide the output to a large language model (LLM) (page 2 section 2.3, where pronunciation assessment is performed using an LLM)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lurie in view of Skarnitzl and Gupta by using the LLM of Fu (Fu page 2 section 2.3) as the machine learning to analyze the output of Lurie in view of Skarnitzl and Gupta (Lurie para [0033]), in order to provide a simpler way to evaluate learners' pronunciation (Fu page 4 section 4.2 middle paragraph).
Regarding claim 17, Lurie in view of Skarnitzl, Gupta, and Fu teaches:
The device of claim 16, wherein the one or more processors are configured to provide the input audio and reference audio to the LLM to generate the feedback, wherein the reference audio corresponds to synthesized speech that represents the target sentence having the speech characteristics of the user and having the target pronunciation (Lurie para [0025], where the machine learning system is used to provide pronunciation feedback based on comparing the user input to the reference, and Fu page 2 section 2.3, where pronunciation assessment is performed using an LLM).
Allowable Subject Matter
Claim 8 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the cited prior art does not teach the limitations of the claim. Specifically, none of the cited prior art teaches that the personalized text-to-speech engine includes an end-to-end speech synthesis model that is based on variational interference with adversarial learning for end-to-end speech synthesis (VITS), in combination with the other limitations of the claims. Hence, none of the cited prior art, either alone or in combination thereof, teaches the combination of limitations found in the claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 2015/0039315 A1 para [0091] teaches comparing a computed prosodic score to a reference prosodic score; US 2018/0082607 A1 para [0007] teaches a pronunciation learning system that allows comparison of users’ speech characteristics to a model speaker, such as pitch, volume, and duration.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRYAN S BLANKENAGEL whose telephone number is (571)270-0685. The examiner can normally be reached 8:00am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRYAN S BLANKENAGEL/Primary Examiner, Art Unit 2658
