DETAILED ACTION
Introduction
1. A response was filed in this application on 06/25/2026 after the non-final rejection of 03/25/2026. Claims 1-4, 21-23 and 25-27 are amended while no claims are cancelled or added in this latest submission by the Applicant. Thus, claims 1-12 and 21-28 are currently pending for reconsideration by the Examiner and are examined below. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to arguments
2. The Applicant’s arguments have been fully considered but they are moot in light of new grounds of rejections as necessitated by amendments.
Response to amendments
3. The rejection under 35 U.S.C. 101 is withdrawn in light of the amendments presented by the Applicant further in view of the arguments presented in this latest submission.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1-12 and 21-28 are rejected under 35 U.S.C. 103 as being unpatentable over Barbello (U.S. Patent Application Publication # 2023/0376699 A1) in view of Pornprasitsakul (U.S. Patent Application Publication # 2014/0201631 A1).
With regards to claim 1, Barbello teaches a method comprising receiving, by an application executing on a computing device, content comprising original audio and original closed captioning text in one or more original languages (Paragraphs 19-22 and figure 1, disclose incoming media including but not limited to video content e.g., recorded or live, and audio content e.g., a podcast. The text version of the audio can be shown on a display device along with a translation of said text);
generating, based on recognition of a language of speech detected in proximity to a display device, alternative closed captioning text comprising a translation of the original audio into an alternative language that is different from the one or more original languages (Paragraphs 19-22 and figure 1, disclose incoming media including but not limited to video content e.g., recorded or live, and audio content e.g., a podcast. The text version of the audio can be shown on a display device along with a translation of said text. Paragraphs 46-47, teach that the translated word can then be presented in the overlay or in a separate overlay that may be positioned proximate to the selected word. The translation and the overlay maybe presented at the top of the virtual keyboard. However, the translation and the overlay can be presented at any suitable location on the display);
determining a visual style of the original closed captioning text (Para 40, teaches that the rendering models are used to cause the translated text to substantially match one or more visual characteristics e.g., size, font, style, format, color etc. of native content of the application);
generating an overlay comprising the alternative closed captioning text in the visual style of the original closed captioning text (Paragraphs 19-22 and figure 1, further teach that the user can resize the displayed text and move around the screen. The user can also apply the system UI overlay to on-screen text to re-render the text as translated text in another language);
Barbello may not explicitly detail outputting, via the display device, the content and the overlay wherein the overlay is output in an onscreen location that corresponds to an onscreen location of the original closed captioning text. This is taught by Pornprasitsakul (Para 48, teaches that the caption input field for a media segment is preferably presented in conjunction e.g., beside, surrounded by a common frame, etc. with the current version of the caption for the media segment. The caption input field is preferably a transparent layer superimposed on top of the media player, but can alternatively be semi-opaque, be superimposed over the video portion of the media player, superimposed over a section of the video portion, or have any other suitable property. The position of the caption input field is preferably fixed in relation to the media player, but can alternatively be adjustable, wherein the caption is preferably displayed at the adjusted position of the caption input field. The adjusted positions can be limited to the top, bottom, and sides of the media player or unlimited. When the caption input field is mobile, the user-determined location of the caption input field is preferably also recorded within the caption file. Alternatively, the caption input field can be arranged beside the media player, or positioned in any other suitable relation. A separate caption input field is preferably rendered for each media segment. Caption input fields can be rendered as a part of the caption stream, wherein the plurality of caption input fields scroll within a frame of the caption stream. Progressing caption input field of a caption stream preferably includes scrolling the caption stream such that caption input field that correspond to the current play positions of the media file are shown and optionally highlighted in frame of the caption stream interface. The caption input field preferably displays the received input of a media segment only for the timeframe of the defined media segment when the media is played. The media can play behind the caption input field, enabling a preview of the media segment. The caption input field preferably functions substantially similar to a text input field);
Barbello and Pornprasitsakul can be considered as analogous art as they belong to a similar field of endeavor in captioning systems. It would thus have been obvious to one having ordinary skill in the art to advantageously combine the teachings of Pornprasitsakul with those of Barbello as shown above to create an easy and efficient interface to translate captions for a media file (Pornpasitsakul, para 20).
With regards to claim 2, Barbello teaches the method of claim 1, wherein the computing device comprises an edge device (See figure 2 for various edge devices).
With regards to claim 3, Barbello teaches the method of claim 1, further comprising determining that the alternative language of the alternative closed captioning text matches the language of the speech detected in proximity to the display device (Paragraphs 46-47, teach that the translated word can then be presented in the overlay or in a separate overlay that may be positioned proximate to the selected word. The translation and the overlay maybe presented at the top of the virtual keyboard. However, the translation and the overlay can be presented at any suitable location on the display).
With regards to claim 4, Barbello teaches the method of claim 1, further comprising determining the onscreen location for the overlay based on the onscreen location of the original closed captioning text (Paragraphs 46-47 and figure 6, teach the translated word can then be presented in the overlay or in a separate overlay that may be positioned proximate to the selected word. An example implementation in figure 6 illustrates a display in different instances. Chat messages are received and displayed in Portuguese. The application includes an input box for a user to enter text, e.g., an outgoing text message. If automatic translation for incoming messages is enabled the automatic translation for outgoing messages can also be enabled. Here, the user has entered a draft message by providing typed input via a keyboard or by providing a voice command via a microphone of the electronic device, where the voice command is transcribed into the draft message. As the draft message is entered, or upon completion of the draft message, the electronic device may translate the draft message and provide a translation on the display in an overlay. In one example, the translation and the overlay are presented at the top of the virtual keyboard. However, the translation and the overlay can be presented at any suitable location on the display).
With regards to claim 5, Barbello teaches the method of claim 1, wherein the determining the visual style of the original closed captioning text comprises determining a color of the alternative closed captioning text based on a color of the original closed captioning text (Para 40, teaches that the rendering models are used to cause the translated text to substantially match one or more visual characteristics e.g., size, font, style, format, color etc. of native content of the application).
With regards to claim 6, Barbello teaches the method of claim 1, wherein the determining the visual style of the original closed captioning text comprises determining a font of the alternative closed captioning text based on a font of the original closed captioning text (Para 40, teaches that the rendering models are used to cause the translated text to substantially match one or more visual characteristics e.g., size, font, style, format, color etc. of native content of the application).
With regards to claim 7, Barbello teaches the method of claim 1, wherein the determining the visual style of the original closed captioning text comprises determining an amount of the alternative closed captioning text to display on the overlay based on an amount of the original closed captioning text that is outputted (Para 46, teaches that based on the user selection, the on-device real-time translation can be applied to a single term, multiple terms, a phrase, multiple phrases, or all text displayed on the display device).
With regards to claim 8, Barbello teaches the method of claim 1, wherein the outputting, via a display device, the content and an overlay comprising the alternative closed captioning text in the visual style of the original closed captioning text comprises determining a rate of outputting the alternative closed captioning text on the overlay based on a rate at which the original audio is outputted (Para 22, teaches that these techniques for real-time translation can be implemented across different applications running on the electronic device, including instant-messaging applications, audio or video players, and live-stream video applications. Para 45, further teaches that the translated text can be included in an overlay over the copied text together with the copied text, such that the overlay includes both the copied text in the original human language and the translated text in the target human language simultaneously. Both these provisions would inherently require determination of audio output rate).
With regards to claim 9, Barbello teaches the method of claim 1, wherein the recognition of the speech in the original audio is based on use of a machine learning model configured to recognize speech (Para 32, teaches an ASR transcription module. Para 34, teaches translation models e.g., cascaded set of models, including machine learning models trained on human languages and translations between the human languages).
With regards to claim 10, Barbello teaches the method of claim 1, wherein the content comprises indications of times at which dialog in the original audio is spoken, and wherein the alternative closed captioning text is outputted at one or more times at which the dialog in the original audio is spoken (Para 22, teaches that these techniques for real-time translation can be implemented across different applications running on the electronic device, including instant-messaging applications, audio or video players, and live-stream video applications. Para 45, further teaches that the translated text can be included in an overlay over the copied text together with the copied text, such that the overlay includes both the copied text in the original human language and the translated text in the target human language simultaneously. Also see the example illustrated in figure 1).
With regards to claim 11, Barbello teaches the method of claim 1, wherein the overlay covers the original closed captioning text (Para 19, teaches that the user can apply the system UI overlay to on-screen text to re-render the text as translated text in another language, with the re-rendering being near-invisible and appearing as native content within an application).
With regards to claim 12, Barbello teaches the method of claim 1, wherein the overlay is outputted next to the original closed captioning text (Paragraphs 46-47, teach that the translated word can then be presented in the overlay or in a separate overlay that may be positioned proximate to the selected word. The translation and the overlay maybe presented at the top of the virtual keyboard. However, the translation and the overlay can be presented at any suitable location on the display).
With regards to claims 21-24, these are system claims for the corresponding method claims 1-12. These two sets of claims are related as method and apparatus of using the same, with each claimed system element's function corresponding to the claimed method step. Accordingly, claims 21-24 are similarly rejected under the same rationale as applied above with respect to method claims 1-12.
With regards to claims 25-28, these are CRM claims for the corresponding method claims 1-12. These two sets of claims are related as method and CRM of using the same, with each claimed CRM element's function corresponding to the claimed method step. Accordingly, claims 25-28 are similarly rejected under the same rationale as applied above with respect to method claims 1-12.
Conclusion
5. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). The Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). The following prior art, made of record but not relied upon, is considered pertinent to applicant's disclosure: Effinger (U.S. Patent # 11070891 B1), Yurick (U.S. Patent Application Publication # 2007/0011012 A1). These references are also included in the PTO-892 form attached with this office action.
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. If you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). In case you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEERAJ SHARMA whose contact information is given below. The examiner can normally be reached on Monday to Friday 8 am to 5 pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis-Desir can be reached on 571-272-7799 (Direct Phone). The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
/NEERAJ SHARMA/
Primary Examiner, Art Unit 2659
571-270-5487 (Direct Phone)
571-270-6487 (Direct Fax)
neeraj.sharma@uspto.gov (Direct Email)
Prosecution Insights