DETAILED ACTION
Introduction
1. This office action is in response to Applicant's submission of 12/29/2025. In this filing, the Applicant cancelled claims 13-20 and added new claims 21-28, which was in reply to the restriction/election office action of 10/27/2025. Thus, claims 1-12 and 21-28 are currently pending and examined below. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
2. The drawings filed on 10/04/2023 have been accepted and considered by the Examiner.
Information Disclosure Statement
3. The Information Statement (IDS) filed on 07/14/2025 has been accepted/considered and is in compliance with the provisions of 37 CFR 1.97.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
4. Claims 1-12 and 21-28 are rejected under 35 U.S.C. 101 as being nothing more than an abstract idea. As an example, regarding claim 1, the limitations of receiving original audio and original closed captioning text in one or more original languages, generating translation of the original audio into an alternative language that is different from the one or more original languages, determining a visual style of the original closed captioning text and outputting the content and an overlay comprising the alternative closed captioning text in the visual style of the original closed captioning text are all activities that a human being can accomplish using their mind and at most pen/paper. Hence, all these steps fall under the category of mental processes. These steps are drafted at a high level of generality without tying it to a specific technological improvement and the computing device recited herein can be a general-purpose computing device. Accordingly, this claim recites an abstract idea.
This judicial exception is not integrated into a practical application because the
recitation of a computing device, an edge device, a processor, a memory, a computer readable medium and/or a general-purpose machine learning model merely read to generalized computer components, based upon the claim interpretation wherein the structure is interpreted using the specification. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using generalized computer components to generate, extract, determine, and generate, amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is therefore not patent eligible.
Claims 2-12, merely provide certain details of the mental processes outlined above, such as changing the font, color, location, proximity, amount and rate of the output text etc. These are all steps which themselves can also be accomplished by a human being with at most the aid of a pen/paper and hence also do not amount to significantly more than the judicial exception. Claims 25-28 are computer readable medium (CRM) claims corresponding to method claims 1-12 and therefore are also rejected under 35 U.S.C. 101 for at least the reasons outlined above. Similarly, claims 21-24 are system claims corresponding to the method claims 1-12 and therefore are also rejected under 35 U.S.C. 101 for at least the reasons outlined above.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(2) The claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
5. Claims 1-12 and 21-28 are rejected under 35 U.S.C. 102 (a) (1) as being anticipated by Barbello (U.S. Patent Application Publication # 2023/0376699 A1).
With regards to claim 1, Barbello teaches a method comprising receiving, by an application executing on a computing device, content comprising original audio and original closed captioning text in one or more original languages (Paragraphs 19-22 and figure 1, disclose incoming media including but not limited to video content e.g., recorded or live, and audio content e.g., a podcast. The text version of the audio can be shown on a display device along with a translation of said text);
generating, based on recognition of speech of the original audio, alternative closed captioning text comprising a translation of the original audio into an alternative language that is different from the one or more original languages (Paragraphs 19-22 and figure 1, disclose incoming media including but not limited to video content e.g., recorded or live, and audio content e.g., a podcast. The text version of the audio can be shown on a display device along with a translation of said text);
determining a visual style of the original closed captioning text (Para 40, teaches that the rendering models are used to cause the translated text to substantially match one or more visual characteristics e.g., size, font, style, format, color etc. of native content of the application);
and outputting, via a display device, the content and an overlay comprising the alternative closed captioning text in the visual style of the original closed captioning text (Paragraphs 19-22 and figure 1, further teach that the user can resize the displayed text and move around the screen. The user can also apply the system UI overlay to on-screen text to re-render the text as translated text in another language).
With regards to claim 2, Barbello teaches the method of claim 1, wherein the computing device comprises an edge device (See figure 2 for various edge devices).
With regards to claim 3, Barbello teaches the method of claim 1, wherein the generating, based on recognition of speech of the original audio, alternative closed captioning text comprising a translation of the original audio into the alternative language comprises determining, based on recognition of a language of speech detected in proximity to the display device, that the alternative language of the alternative closed captioning text matches the language of the speech detected in proximity to the display device (Paragraphs 46-47, teach that the translated word can then be presented in the overlay or in a separate overlay that may be positioned proximate to the selected word. The translation and the overlay maybe presented at the top of the virtual keyboard. However, the translation and the overlay can be presented at any suitable location on the display).
With regards to claim 4, Barbello teaches the method of claim 1, wherein the determining the visual style of the original closed captioning text comprises determining an onscreen location for the overlay based on an onscreen location of the original closed captioning text (Paragraphs 46-47 and figure 6, teach the translated word can then be presented in the overlay or in a separate overlay that may be positioned proximate to the selected word. An example implementation in figure 6 illustrates a display in different instances. Chat messages are received and displayed in Portuguese. The application includes an input box for a user to enter text, e.g., an outgoing text message. If automatic translation for incoming messages is enabled the automatic translation for outgoing messages can also be enabled. Here, the user has entered a draft message by providing typed input via a keyboard or by providing a voice command via a microphone of the electronic device, where the voice command is transcribed into the draft message. As the draft message is entered, or upon completion of the draft message, the electronic device may translate the draft message and provide a translation on the display in an overlay. In one example, the translation and the overlay are presented at the top of the virtual keyboard. However, the translation and the overlay can be presented at any suitable location on the display).
With regards to claim 5, Barbello teaches the method of claim 1, wherein the determining the visual style of the original closed captioning text comprises determining a color of the alternative closed captioning text based on a color of the original closed captioning text (Para 40, teaches that the rendering models are used to cause the translated text to substantially match one or more visual characteristics e.g., size, font, style, format, color etc. of native content of the application).
With regards to claim 6, Barbello teaches the method of claim 1, wherein the determining the visual style of the original closed captioning text comprises determining a font of the alternative closed captioning text based on a font of the original closed captioning text (Para 40, teaches that the rendering models are used to cause the translated text to substantially match one or more visual characteristics e.g., size, font, style, format, color etc. of native content of the application).
With regards to claim 7, Barbello teaches the method of claim 1, wherein the determining the visual style of the original closed captioning text comprises determining an amount of the alternative closed captioning text to display on the overlay based on an amount of the original closed captioning text that is outputted (Para 46, teaches that based on the user selection, the on-device real-time translation can be applied to a single term, multiple terms, a phrase, multiple phrases, or all text displayed on the display device).
With regards to claim 8, Barbello teaches the method of claim 1, wherein the outputting, via a display device, the content and an overlay comprising the alternative closed captioning text in the visual style of the original closed captioning text comprises determining a rate of outputting the alternative closed captioning text on the overlay based on a rate at which the original audio is outputted (Para 22, teaches that these techniques for real-time translation can be implemented across different applications running on the electronic device, including instant-messaging applications, audio or video players, and live-stream video applications. Para 45, further teaches that the translated text can be included in an overlay over the copied text together with the copied text, such that the overlay includes both the copied text in the original human language and the translated text in the target human language simultaneously. Both these provisions would inherently require determination of audio output rate).
With regards to claim 9, Barbello teaches the method of claim 1, wherein the recognition of the speech in the original audio is based on use of a machine learning model configured to recognize speech (Para 32, teaches an ASR transcription module. Para 34, teaches translation models e.g., cascaded set of models, including machine learning models trained on human languages and translations between the human languages).
With regards to claim 10, Barbello teaches the method of claim 1, wherein the content comprises indications of times at which dialog in the original audio is spoken, and wherein the alternative closed captioning text is outputted at one or more times at which the dialog in the original audio is spoken (Para 22, teaches that these techniques for real-time translation can be implemented across different applications running on the electronic device, including instant-messaging applications, audio or video players, and live-stream video applications. Para 45, further teaches that the translated text can be included in an overlay over the copied text together with the copied text, such that the overlay includes both the copied text in the original human language and the translated text in the target human language simultaneously. Also see the example illustrated in figure 1).
With regards to claim 11, Barbello teaches the method of claim 1, wherein the overlay covers the original closed captioning text (Para 19, teaches that the user can apply the system UI overlay to on-screen text to re-render the text as translated text in another language, with the re-rendering being near-invisible and appearing as native content within an application).
With regards to claim 12, Barbello teaches the method of claim 1, wherein the overlay is outputted next to the original closed captioning text (Paragraphs 46-47, teach that the translated word can then be presented in the overlay or in a separate overlay that may be positioned proximate to the selected word. The translation and the overlay maybe presented at the top of the virtual keyboard. However, the translation and the overlay can be presented at any suitable location on the display).
With regards to claims 21-24, these are system claims for the corresponding method claims 1-12. These two sets of claims are related as method and apparatus of using the same, with each claimed system element's function corresponding to the claimed method step. Accordingly, claims 21-24 are similarly rejected under the same rationale as applied above with respect to method claims 1-12.
With regards to claims 25-28, these are CRM claims for the corresponding method claims 1-12. These two sets of claims are related as method and CRM of using the same, with each claimed CRM element's function corresponding to the claimed method step. Accordingly, claims 25-28 are similarly rejected under the same rationale as applied above with respect to method claims 1-12.
Conclusion
6. The following prior art, made of record but not relied upon, is considered pertinent to applicant's disclosure: Flanagan (U.S. Patent Application Publication # 2008/0052069 A1), Kim (U.S. Patent Application Publication # 2023/0343050 A1). These references are also included in the PTO-892 form attached with this office action.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. If you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). In case you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEERAJ SHARMA whose contact information is given below. The examiner can normally be reached on Monday to Friday 8 am to 5 pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis-Desir can be reached on 571-272-7799 (Direct Phone). The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
/NEERAJ SHARMA/
Primary Examiner, Art Unit 2659
571-270-5487 (Direct Phone)
571-270-6487 (Direct Fax)
neeraj.sharma@uspto.gov (Direct Email)