Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the Amendment filed on 11/26/2025. Claims 1, 9 and 15 have been amended. Claim 6 has been canceled. Claims 1-5, 7-21 are pending in the case. Claims 1, 9, and 15 are independent claims.
Response to Arguments
Applicant's arguments filed on 11/26/2025 have been fully considered but they are not persuasive.
Applicant argues that the cited arts do not disclose the amendment limitation.
Examiner respectfully disagrees.
Here, output, by an extension of the web browser, the first audio signal to an operating system of the client device for transmission to a driver of a speaker device; and output, by the extension of the web browser, the second audio signal directly to the driver of the speaker device; ([1629] of Lecoeuche, “MIME type of the recording. MIME types can be specified such as "audio/wav" for WAV (RIFF header) 8 kHz 8-bit mono mu-law [PCM] single channel or "audio/basic" for Raw (headerless) 8 kHz 8-bit mono mu-law [PCM] single channel. If unspecified, defaults to G.711 wave file.” It is implied that the audio is from the driver of the speaker when the user is able to hear the audio in the wave format. A person of ordinary skills in the art would have understood that the driver of the speaker is what is broadcast the sound from audio files on the system so the user can hear it in a wave form)
Hence, the cited arts recite the current amendments.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7-21 are rejected under 35 U.S.C. 103 as being unpatentable over Lecoeuche (US 20050091059 A1) in view of Dube (US 10789956 B1) and in further view of Won (US 20170177171 A1).
Referring to claim 1, Lecoeuche discloses a system for navigating and completing a web form using audio, the system comprising: one or more memories; and one or more processors, communicatively coupled to the one or more memories, (Fig. 2 of Lecoeuche) configured to:
receive input to trigger audio navigation of a web form loaded by a web browser, wherein the web form comprises hypertext markup language (HTML) code; ([0066] and [0057] of Lecoeuche, HTML code including different information fields executed by code. And “by use of stylus 33 of device 30, the event "onClick" is initiated which calls or executes function "talk" in script portion 272. This action activates a grammar used for speech recognition that is associated with the type of data generally expected in field 250. This type of interaction, which involves more than one technique of input (e.g. voice and pen-click/roller) is referred as "multimodal")
generate, using a text-to-speech library of the web browser of a client device, a first audio signal based on a first label indicated in the HTML code and associated with a first input element of the web form; ([0062] of Lecoeuche, “Such controls are commonly formed in a library and used by controls 302 to perform a particular visual task. Library 300A includes methods for generating the desired client markup, event handlers, etc. Examples of visual controls 302 include a “Label" control that provides a selected text label on a visual display such as the label" Credit Card Submission" 304 in FIG. 5. Another example of a higher level visual control 302 is a "Textbox", which allows data to be entered in a data field such as is indicated at 250 in FIG. 5. The existing visual controls 302 are also well-known. In the first approach for extending server side plug-in module controls to include recognition and/or audible prompting, each of the visual controls 302 would include further parameters or attributes related to recognition or audible prompting. In the case of the “label" control, which otherwise provides selected text on a visual display, further attributes may include whether an audio data file will be rendered or text-to-speech conversion will be employed as well as the location of this data file. A library 300B, similar to library 300A, includes further markup information for performing recognition and/or audible prompting. Each of the visual controls 302 is coded so as to provide this information to the controls 300B as appropriate to perform the particular task related to recognition or audible prompting.”)
output, by an extension of the web browser, the first audio signal to an operating system of the client device for transmission to a driver of a speaker device; ([1629] of Lecoeuche, “MIME type of the recording. MIME types can be specified such as "audio/wav" for WAV (RIFF header) 8 kHz 8-bit mono mu-law [PCM] single channel or "audio/basic" for Raw (headerless) 8 kHz 8-bit mono mu-law [PCM] single channel. If unspecified, defaults to G.711 wave file.” It is implied that the audio is from the driver of the speaker when the user is able to hear the audio in the wave format. A person of ordinary skills in the art would have understood that the driver of the speaker is what is broadcast the sound from audio files on the system so the user can hear it in a wave form)
generate, by the extension of the web browser, of the client device, a trigger that causes monitoring detection of speech associated with first audio; ([0611]-[0620] of Lecoeuche, triggering of the monitoring detection of speech associated with the first audio, such as recording of user’s telephone call, is done by receiving the call with set parameters, such as max timeout, initial timeout, etc…)
record, based on the trigger transmitted by the extension of the web browser, the first audio after generating the first audio signal; ([0061] of Lecoeuche, recorded audio file can be translated to text)
generate, using a speech-to-text library of the web browser, a first transcription of the first audio; (Lecoeuche does not specifically disclose “speech-to-text library” based on recording of audio data, but col. 9, line 9-17 of Dube discloses “After training the text-to-speech model, the text-to-speech platform may provide the text-to-speech model to another device. For example, the text-to-speech platform may provide the text-to-speech model to a device of an entity of the particular industry. In this way, computing resources may be conserved, which may otherwise be used to record, process, store, and/or transfer audio data obtained via a dedicated recording process and then generate a speech-to-text model based on the audio data obtained via a dedicated recording process.” Lecoeuche and Dube are analogous art because both references concern using speech recognition to process data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lecoeuche’s text-to-speech technique with speech-to-text method taught by Dube. The motivation for doing so would have been effectively process both audio and text as input and output to assist user with data process)
modify the first input element of the web form based on the first transcription; ([0062] of Lecoeuche, further modification/prompting for additional task associated with the audio data)
generate, using the text-to-speech library of the web browser, a second audio signal based on a second label indicated in the HTML code and associated with a second input element of the web form; (see citation above regarding the first audio signal)
output, by the extension of the web browser, the second audio signal directly to the driver of the speaker device; ([1629] of Lecoeuche, “MIME type of the recording. MIME types can be specified such as "audio/wav" for WAV (RIFF header) 8 kHz 8-bit mono mu-law [PCM] single channel or "audio/basic" for Raw (headerless) 8 kHz 8-bit mono mu-law [PCM] single channel. If unspecified, defaults to G.711 wave file.” It is implied that the audio is from the driver of the speaker when the user is able to hear the audio in the wave format. A person of ordinary skills in the art would have understood that the driver of the speaker is what is broadcast the sound from audio files on the system so the user can hear it in a wave form)
record second audio after generating the second audio signal; generate, using the speech-to-text library of the web browser, a second transcription of the second audio; (see citation above regarding the first audio signal)
modify the second input element of the web form based on the second transcription; ([0235] of Lecoeuche, “new improvement for the multi-modal scenario allows the user to change the value for the textbox either through voice recognition or through standard graphical user interface methods such as with a keyboard, a handwriting recognition, etc.”)
generate, using the text-to-speech library of the web browser, a third audio signal based on a submission button indicated in the HTML code; ([0234] of Lecoeuche, “ it is common to include a "submit” button 532 when the user believes that he/she has entered all required information. As indicated above, the user may not have chosen to confirm that a particular value was properly recognized for a given textbox; however, upon activation of the submit button 532 the author of the application may at that time force confirmation of the previously ignored confirmation prompt, or choose to accept the recognized input as correct and return the form to the server for processing.”)
record third audio after generating the third audio signal; generate, using the speech-to-text library of the web browser, a third transcription of the third audio; (see citation above regarding the first audio signal) and
activate the submission button of the web form based on the third transcription. ([0234] of Lecoeuche, “ it is common to include a "submit” button 532 when the user believes that he/she has entered all required information. As indicated above, the user may not have chosen to confirm that a particular value was properly recognized for a given textbox; however, upon activation of the submit button 532 the author of the application may at that time force confirmation of the previously ignored confirmation prompt, or choose to accept the recognized input as correct and return the form to the server for processing.”)
Lecoeuche in view of Dube do not specifically disclose triggers detection of speech/voice “wherein the trigger is transmitted by the extension of the web browser”.
However, Won discloses triggers detection of speech/voice wherein the trigger is transmitted by the extension of the web browser ([0060] of Won, where the user input, such as user’s voice/speech, that triggers certain detection of the voice/speech is done through integration of the web browser extension with the web browser application.)
Lecoeuche and Dube and Won are analogous art because both references concern using speech recognition to process data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lecoeuche’s text-to-speech technique with speech-to-text method taught by Dube and adding browser extension to the web application as taught by Won. The motivation for doing so would have been effectively process both audio and text as input and output to assist user with data process.
Referring to claim 2, Lecoeuche in view of Dube and Won disclose the system of claim 1, wherein the one or more processors are further configured to:
record fourth audio after generating the second audio signal; generate, using the speech-to-text library of the web browser, a fourth transcription of the fourth audio; ([0061] of Lecoeuche, recorded audio file can be translated to text and Lecoeuche does not specifically disclose “speech-to-text library” based on recording of audio data, but col. 9, line 9-17 of Dube discloses “After training the text-to-speech model, the text-to-speech platform may provide the text-to-speech model to another device. For example, the text-to-speech platform may provide the text-to-speech model to a device of an entity of the particular industry. In this way, computing resources may be conserved, which may otherwise be used to record, process, store, and/or transfer audio data obtained via a dedicated recording process and then generate a speech-to-text model based on the audio data obtained via a dedicated recording process.”. Lecoeuche and Dube are analogous art because both references concern using speech recognition to process data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lecoeuche’s text-to-speech technique with speech-to-text method taught by Dube. The motivation for doing so would have been effectively process both audio and text as input and output to assist user with data process) and
repeat the second audio signal based on the fourth transcription being associated with a repeat command, wherein the second audio is recorded after the second audio signal is repeated. ([0078] of Lecoeuche, command controls such as hep, repeat, cancel, etc..)
Referring to claim 3, Lecoeuche in view of Dube and Won disclose the system of claim 1, wherein the one or more processors are further configured to:
record fourth audio after generating the second audio signal;
generate, using the speech-to-text library of the web browser, a fourth transcription of the fourth audio; repeat the first audio signal based on the fourth transcription being associated with a backward command; record fifth audio after repeating the first audio signal;
generate, using the speech-to-text library of the web browser, a fifth transcription of the fifth audio; and re-modify the first input element of the web form based on the fifth transcription. (see above citations for geniting audio and modify input based on speech-to-text transcription. ([0235] of Lecoeuche, “new improvement for the multi-modal scenario allows the user to change the value for the textbox either through voice recognition or through standard graphical user interface methods such as with a keyboard, a handwriting recognition, etc.”))
Referring to claim 4, Lecoeuche in view of Dube and Won disclose the system of claim 1, wherein the one or more processors are further configured to:
generate, using the text-to-speech library of the web browser, a fourth audio signal based on a third label indicated in the HTML code and associated with a third input element of the web form; record fourth audio after generating the fourth audio signal; generate, using the speech-to-text library of the web browser, a fourth transcription of the fourth audio; (see above citations for geniting audio and modify input based on speech-to-text transcription) and
skip the third input element of the web form based on the fourth transcription being associated with a skip command. (Even though Lecoeuche does not specifically disclose “skip” commend, but [0078] of Lecoeuche, command controls such as hep, repeat, cancel, etc.. a personal of ordinary skills in the art would have modified Lecoeuche to also try “skip” commend)
Referring to claim 5, Lecoeuche in view of Dube and Won disclose the system of claim 1, wherein the one or more processors are further configured to:
identify the first label indicated in the HTML code based at least in part on a tag associated with the first input element. ([0053] of Lecoeuche, “the extensions (also commonly known as "tags") are a small set of XML elements, with associated attributes and DOM object properties, events and methods, which may be used in conjunction with a source markup document to apply a recognition and/or audible prompting interface, DTMF or call control to a source page.”)
Referring to claim 7, Lecoeuche in view of Dube and Won disclose the system of claim 1, wherein the one or more processors are further configured to:
receive an indication of the web form; and transmit a request for the HTML code using the web browser in response to the indication of the web form. ([0052] of Lecoeuche, “a HTML markup language code is illustrated. The HTML code includes a body portion 270 and a script portion 272. Entry of information in each of the fields 250, 252 and 254 is controlled or executed by code portions 280, 282 and 284, respectively. Referring first to code portion 280, on selection of field 250, for example, by use of stylus 33 of device 30, the event "onClick" is initiated which calls or executes function "talk" in script portion 272. This action activates a grammar used for speech recognition that is associated with the type of data generally expected in field 250. This type of interaction, which involves more than one technique of input (e.g. voice and pen-click/roller) is referred as "multimodal".”)
Referring to claim 8, Lecoeuche in view of Dube and Won disclose the system of claim 1, wherein the input to trigger audio navigation of the web form is based on a mouse click, a keyboard entry, a touchscreen interaction, or an audio command. ([0066] and [0057] of Lecoeuche, HTML code including different information fields executed by code. And “by use of stylus 33 of device 30, the event "onClick" is initiated which calls or executes function "talk" in script portion 272. This action activates a grammar used for speech recognition that is associated with the type of data generally expected in field 250. This type of interaction, which involves more than one technique of input (e.g. voice and pen-click/roller) is referred as "multimodal")
Referring to claim 9, Lecoeuche discloses a method for navigating and completing a web form using audio (Fig. 2 of Lecoeuche) comprising:
generating, by a user device and using a text-to-speech library of a web browser of the client device, a first audio signal based on a first label associated with a first input element of a web form; ([0062] of Lecoeuche, “Such controls are commonly formed in a library and used by controls 302 to perform a particular visual task. Library 300A includes methods for generating the desired client markup, event handlers, etc. Examples of visual controls 302 include a “Label" control that provides a selected text label on a visual display such as the label" Credit Card Submission" 304 in FIG. 5. Another example of a higher level visual control 302 is a "Textbox", which allows data to be entered in a data field such as is indicated at 250 in FIG. 5. The existing visual controls 302 are also well-known. In the first approach for extending server side plug-in module controls to include recognition and/or audible prompting, each of the visual controls 302 would include further parameters or attributes related to recognition or audible prompting. In the case of the “label" control, which otherwise provides selected text on a visual display, further attributes may include whether an audio data file will be rendered or text-to-speech conversion will be employed as well as the location of this data file. A library 300B, similar to library 300A, includes further markup information for performing recognition and/or audible prompting. Each of the visual controls 302 is coded so as to provide this information to the controls 300B as appropriate to perform the particular task related to recognition or audible prompting.” Lecoeuche does not specifically disclose “speech-to-text library” based on recording of audio data, but col. 9, line 9-17 of Dube discloses “After training the text-to-speech model, the text-to-speech platform may provide the text-to-speech model to another device. For example, the text-to-speech platform may provide the text-to-speech model to a device of an entity of the particular industry. In this way, computing resources may be conserved, which may otherwise be used to record, process, store, and/or transfer audio data obtained via a dedicated recording process and then generate a speech-to-text model based on the audio data obtained via a dedicated recording process.” Lecoeuche and Dube are analogous art because both references concern using speech recognition to process data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lecoeuche’s text-to-speech technique with speech-to-text method taught by Dube. The motivation for doing so would have been effectively process both audio and text as input and output to assist user with data process)
generating, by the user device and using a speech-to-text library of the web browser, a first transcription of first audio recorded after the first audio signal is played, wherein the first audio is recorded based on a trigger, that is generated by an extension of the web browser of the client device, and that causes monitoring detection of speech associated with the first audio; ([0611]-[0620] of Lecoeuche, triggering of the monitoring detection of speech associated with the first audio, such as recording of user’s telephone call, is done by receiving the call with set parameters, such as max timeout, initial timeout, etc…)
modifying the first input element of the web form based on the first transcription; ([0062] of Lecoeuche, further modification/prompting for additional task associated with the audio data)
generating, by the user device and using the text-to-speech library of the web browser, a second audio signal based on a second label associated with a second input element of the web form; generating, by the user device and using the speech-to-text library of the web browser, a second transcription of second audio recorded after the second audio signal is played; (col. 9, line 9-17 of Dube discloses “After training the text-to-speech model, the text-to-speech platform may provide the text-to-speech model to another device. For example, the text-to-speech platform may provide the text-to-speech model to a device of an entity of the particular industry. In this way, computing resources may be conserved, which may otherwise be used to record, process, store, and/or transfer audio data obtained via a dedicated recording process and then generate a speech-to-text model based on the audio data obtained via a dedicated recording process.”)
modifying the second input element of the web form based on the second transcription; ([0235] of Lecoeuche, “new improvement for the multi-modal scenario allows the user to change the value for the textbox either through voice recognition or through standard graphical user interface methods such as with a keyboard, a handwriting recognition, etc.”)
receiving, at the user device, input associated with submitting the web form; (see citation above regarding the first audio signal) and
activating a submission clement of the web form based on the input. ([0234] of Lecoeuche, “ it is common to include a "submit” button 532 when the user believes that he/she has entered all required information. As indicated above, the user may not have chosen to confirm that a particular value was properly recognized for a given textbox; however, upon activation of the submit button 532 the author of the application may at that time force confirmation of the previously ignored confirmation prompt, or choose to accept the recognized input as correct and return the form to the server for processing.”)
Lecoeuche in view of Dube do not specifically disclose triggers detection of speech/voice “wherein the trigger is transmitted by the extension of the web browser”.
However, Won discloses triggers detection of speech/voice wherein the trigger is transmitted by the extension of the web browser ([0060] of Won, where the user input, such as user’s voice/speech, that triggers certain detection of the voice/speech is done through integration of the web browser extension with the web browser application.)
Lecoeuche and Dube and Won are analogous art because both references concern using speech recognition to process data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lecoeuche’s text-to-speech technique with speech-to-text method taught by Dube and adding browser extension to the web application as taught by Won. The motivation for doing so would have been effectively process both audio and text as input and output to assist user with data process.
Referring to claim 10, Lecoeuche in view of Dube and Won disclose the method of claim 9, further comprising:
receiving feedback associated with the first audio signal or the second audio signal; and
updating the text-to-speech library based on the feedback. ([0240] of Lecoeuche, the system can update the highlighted portion from the user as the new recognized portion/update the highlight portion)
Referring to claim 11, Lecoeuche in view of Dube and Won disclose the method of claim 9, wherein the first input element comprises a text box. ([0236] of Lecoeuche, “If the user, activates any of the buttons 550, 552, 554 or 556, the control algorithm can ascertain which collection has been identified for resetting using a simple function (in a manner similar to that discussed above with respect to identifying textboxes) and then remove the corresponding values in the associated SemanticItems for that collection and/or change the status information to signify that new data is required.”)
Referring to claim 12, Lecoeuche in view of Dube and Won disclose the method of claim 9, wherein the second input element comprises a drop-down menu or a list of radio buttons, and the second audio signal is further based on a plurality of options associated with the second input element. ([0095] of Lecoeuche, drop down list associated with selections of inputs)
Referring to claim 13, Lecoeuche in view of Dube and Won disclose the method of claim 12, wherein modifying the second input element comprises:
selecting an option, from the plurality of options, based on the second transcription. ([0095] of Lecoeuche, drop down list associated with selections of inputs and commend selected=”true”)
Referring to claim 14, Lecoeuche in view of Dube and Won disclose the method of claim 9, wherein the web form comprises hypertext markup language (HTML) code or cascading style sheets (CSS) code. ([0066] and [0057] of Lecoeuche, HTML code including different information fields executed by code. And “by use of stylus 33 of device 30, the event "onClick" is initiated which calls or executes function "talk" in script portion 272. This action activates a grammar used for speech recognition that is associated with the type of data generally expected in field 250. This type of interaction, which involves more than one technique of input (e.g. voice and pen-click/roller) is referred as "multimodal")
Referring to claim 15, Lecoeuche discloses the non-transitory computer-readable medium storing a set of instructions for navigating and completing a web form using audio, the set of instructions comprising: one or more instructions that, when executed by one or more processors of a device, cause the device to:
generate, using a text-to-speech library of a web browser of the client device, a first audio signal based on a label associated with an input element of a web form; ([0062] of Lecoeuche, “Such controls are commonly formed in a library and used by controls 302 to perform a particular visual task. Library 300A includes methods for generating the desired client markup, event handlers, etc. Examples of visual controls 302 include a “Label" control that provides a selected text label on a visual display such as the label" Credit Card Submission" 304 in FIG. 5. Another example of a higher level visual control 302 is a "Textbox", which allows data to be entered in a data field such as is indicated at 250 in FIG. 5. The existing visual controls 302 are also well-known. In the first approach for extending server side plug-in module controls to include recognition and/or audible prompting, each of the visual controls 302 would include further parameters or attributes related to recognition or audible prompting. In the case of the “label" control, which otherwise provides selected text on a visual display, further attributes may include whether an audio data file will be rendered or text-to-speech conversion will be employed as well as the location of this data file. A library 300B, similar to library 300A, includes further markup information for performing recognition and/or audible prompting. Each of the visual controls 302 is coded so as to provide this information to the controls 300B as appropriate to perform the particular task related to recognition or audible prompting.” Lecoeuche does not specifically disclose “speech-to-text library” based on recording of audio data, but col. 9, line 9-17 of Dube discloses “After training the text-to-speech model, the text-to-speech platform may provide the text-to-speech model to another device. For example, the text-to-speech platform may provide the text-to-speech model to a device of an entity of the particular industry. In this way, computing resources may be conserved, which may otherwise be used to record, process, store, and/or transfer audio data obtained via a dedicated recording process and then generate a speech-to-text model based on the audio data obtained via a dedicated recording process.”. Lecoeuche and Dube are analogous art because both references concern using speech recognition to process data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lecoeuche’s text-to-speech technique with speech-to-text method taught by Dube. The motivation for doing so would have been effectively process both audio and text as input and output to assist user with data process)
generate, using an extension of the web browser of the client device, a trigger that causes monitoring detection of speech associated with first audio; ([0611]-[0620] of Lecoeuche, triggering of the monitoring detection of speech associated with the first audio, such as recording of user’s telephone call, is done by receiving the call with set parameters, such as max timeout, initial timeout, etc…)
generate, using a speech-to-text library of the web browser, a first transcription of the first audio recorded after the first audio signal is played, (col. 9, line 9-17 of Dube discloses “After training the text-to-speech model, the text-to-speech platform may provide the text-to-speech model to another device. For example, the text-to-speech platform may provide the text-to-speech model to a device of an entity of the particular industry. In this way, computing resources may be conserved, which may otherwise be used to record, process, store, and/or transfer audio data obtained via a dedicated recording process and then generate a speech-to-text model based on the audio data obtained via a dedicated recording process.”.) wherein the first audio is recorded based on the trigger; ([0611]-[0620] of Lecoeuche, triggering of the monitoring detection of speech associated with the first audio, such as recording of user’s telephone call, is done by receiving the call with set parameters, such as max timeout, initial timeout, etc…)
modify the input element of the web form based on the first transcription; ([0062] of Lecoeuche, further modification/prompting for additional task associated with the audio data)
generate, using a speech-to-text library of the web browser, a second transcription of second audio recorded after modifying the input element; ([0235] of Lecoeuche, “new improvement for the multi-modal scenario allows the user to change the value for the textbox either through voice recognition or through standard graphical user interface methods such as with a keyboard, a handwriting recognition, etc.”)
repeat the first audio signal based on the second transcription being associated with a backward command; ([0098] of Lecoeuche, “Command controls 310B are user utterances common in voice-only dialogs which typically have little semantic import in terms of the question asked, but rather seek assistance or effect navigation, e.g. help, cancel, repeat, etc.” here repeat the first audio is based on the commend to go back to the audio again)
generate, using the speech-to-text library of the web browser, a third transcription of third audio recorded after the first audio signal is repeated; (col. 9, line 9-17 of Dube discloses “After training the text-to-speech model, the text-to-speech platform may provide the text-to-speech model to another device. For example, the text-to-speech platform may provide the text-to-speech model to a device of an entity of the particular industry. In this way, computing resources may be conserved, which may otherwise be used to record, process, store, and/or transfer audio data obtained via a dedicated recording process and then generate a speech-to-text model based on the audio data obtained via a dedicated recording process.") and
re-modify the input element of the web form based on the third transcription. ([0235] of Lecoeuche, “new improvement for the multi-modal scenario allows the user to change the value for the textbox either through voice recognition or through standard graphical user interface methods such as with a keyboard, a handwriting recognition, etc.”)
Lecoeuche in view of Dube do not specifically disclose triggers detection of speech/voice “wherein the trigger is transmitted by the extension of the web browser”.
However, Won discloses triggers detection of speech/voice wherein the trigger is transmitted by the extension of the web browser ([0060] of Won, where the user input, such as user’s voice/speech, that triggers certain detection of the voice/speech is done through integration of the web browser extension with the web browser application.)
Lecoeuche and Dube and Won are analogous art because both references concern using speech recognition to process data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lecoeuche’s text-to-speech technique with speech-to-text method taught by Dube and adding browser extension to the web application as taught by Won. The motivation for doing so would have been effectively process both audio and text as input and output to assist user with data process.
Referring to claim 16, Lecoeuche in view of Dube and Won disclose the non-transitory computer-readable medium of claim 15, wherein the first audio comprises speech with letters. (1501 of Lecoeuche, “the case for event values specified in the StartEvent and StopEvent attributes must be exactly as those events are defined. For example, the onmouseup and onmousedown events are specified in all lower case letters.”)
Referring to claim 17, Lecoeuche in view of Dube and Won disclose the non-transitory computer-readable medium of claim 15, wherein the first audio comprises speech with words. ([0005] of Lecoeuche, spoken words is part of the speech)
Referring to claim 18, Lecoeuche in view of Dube and Won disclose the non-transitory computer-readable medium of claim 15, wherein the one or more instructions, that cause the device to modify the input element based on the first transcription, cause the device to: insert the first transcription into the input element. ([0386] of Lecoeuche)
Referring to claim 19, Lecoeuche in view of Dube and Won disclose the non-transitory computer-readable medium of claim 15, wherein the one or more instructions, that cause the device to modify the input element based on the first transcription, cause the device to: determine that the first transcription matches an option associated with the input element; and select the option using the input element. ([0095] of Lecoeuche, drop down list associated with selections of inputs and commend selected=”true”)
Referring to claim 20, Lecoeuche in view of Dube and Won disclose the non-transitory computer-readable medium of claim 19, wherein the one or more instructions, that cause the device to determine that the first transcription matches the option, cause the device to:
determine that a similarity score based on the first transcription and the option satisfies a similarity threshold. ([0061] and [0105] of Lecoeuche, confidence score threshold)
Referring to claim 21, Lecoeuche in view of Dube and Won disclose the system of claim 1, wherein the trigger is transmitted based on at least one of:
An amount of time that has passed after outputting the first audio signal, or receiving a signal indicating that the first audio signal ended. ([0602] of Lecoeuche, “initialTimeout: Optional. Maximum time (in milliseconds) of silence allowed at the beginning of a recording.”)
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure:
Larsen et al (US 20120236201 A1)
Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
In the interests of compact prosecution, Applicant is invited to contact the examiner via electronic media pursuant to USPTO policy outlined MPEP § 502.03. All electronic communication must be authorized in writing. Applicant may wish to file an Internet Communications Authorization Form PTO/SB/439. Applicant may wish to request an interview using the Interview Practice website: http://;www.uspto.gov/patent/laws-and-regulations/interview-practice.
Applicant is reminded Internet e-mail may not be used for communication for matters under 35 U.S.C. § 132 or which otherwise require a signature. A reply to an Office action may NOT be communicated by Applicant to the USPTO via Internet e- mail. If such a reply is submitted by Applicant via Internet e-mail, a paper copy will be placed in the appropriate patent application file with an indication that the reply is NOT ENTERED. See MPEP § 502.03(II).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAIMEI JIANG whose telephone number is (571)270-1590. The examiner can normally be reached M-F 9-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached on 571-270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAIMEI JIANG/Primary Examiner, Art Unit 2142
Prosecution Insights