DETAILED ACTION
This office action is in response to Applicant’s amendment filed on 6/9/2026. Claims 1, 14, 17, and 20 were amended. Claims 1-20 are pending in the application of which Claims 1, 14, and 20 are independent and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments filed in the Amendment filed 6/9/2026 (herein “Amendment”) with respect to the 35 USC §103 rejection raised in the previous office action have been fully considered but are moot in view of the new grounds of rejection which was necessitated by applicant’s amendment. Therefore, the previous rejection has been withdrawn. However, upon further consideration, a new ground of rejection is introduced for independent claims further adding Fu et al. (US20260181244A1) to the combination of Li and Merg.
Please see prior art section below for more detail including updated citations and obviousness rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 6, 8, 10, 14, and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (Toward Interactive Dictation”, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics), and in further view of Merg, and Fu et al. (US 20260181244 A1)(herein "Fu").
Li, and Merg were applied in the previous Office Action.
Regarding claims 1, and 14 Li teaches [A computer-implemented method for providing generative content for one or more voice samples using one or more generative artificial intelligence (Al) models, comprising: -claim 1], and [A system comprising: a processor; and a non-transitory computer memory comprising instructions that, when executed by the processor, cause the system to perform operations of: - claim 14] (Li, Introduction, page 15320:” we experiment with two strategies for implementing the proposed system: one that uses a pre-trained language model to directly predict the edited text given unedited text and a command, and another that interprets the command as a program specifying how to edit. … we also experimented with two choices of pre-trained language model: a small finetuned T5 model and a large prompted GPT3 model.”, and section 6 where experimentation and results are discussed, which inherently allude to having computer program, a non-transitory computer readable storage medium having program instructions embodied therewith, a system comprising a memory device for storing program code; and a processor device operatively coupled to the memory device for running the program code.)
receiving a first voice sample and a second voice sample in a voice capture session; (Li, Section 3, part b, page 15321:” Segmentation When the current transcript changes, the system can update its segmentation. It does so by partitioning the current transcript U into a sequence of segments ui, labeling each as being either a dictation or a command.”, and Figure 1:” A user writes an email using speech input, interleaving dictation (a,c) and commanding (b,d). Top shows the continuous user utterance, while bottom shows the document state at each point of the utterance. Dictations are transcribed verbatim, while commands are interpreted and executed. “)
based on determining that the first voice sample is a speech-to-text dictation, providing a first text string for display of the first voice sample; (Li, Section 3 Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and Section 5, part 2, Page 15325: Each dictation segment is directly spliced into the document at the current cursor position.”, and Figure 3, page 15324 depicts all the relevant texts including the output of speech-to-text dictation, command, etc.”, and see D2 in Figure 2.)
based on determining that the second voice sample is a voice command, (Li, Section 3, Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and in Figure 2 “u2” represents command, which in this case the command is classified as capitalization command “Capitalize the S&E speak”).
providing the text modification prompt to the generative AI model; and (Li, Section 5.2, Page 15326:” For each Ui that is predicted as a command segment, we first predict the normalized utterance u’I, we then interpret u’I in context to predict either the document state di or an update program Pi. We then update the document state accordingly. We experiment with two ways of implementing the two steps: we either fine-tune two separate T5-base models that run in a pipeline for each command, or we prompt GPT3 to generate both the normalized utterance and the interpretation output in a single inference step.”)
Li does not teach, however, Fu teaches determining a command classification type of the second voice sample by: providing data corresponding to the second voice sample to a generative Al model within a command type classification prompt that instructs the generative Al model to determine a command classification type for the second voice sample; and receiving, from the generative Al model, the command classification type for the second voice sample; generating a text modification prompt based on the command classification type by: selecting a prompt type corresponding to the command classification type; and including the first text string and a second text string based on the second voice sample in the text modification prompt; (Fu, Par. 0068:” The AI model 128 may be trained and retrained to interpret various types of voice commands and user speech. For example, if the voice command 132 indicates, “Hey Camera, brighten the face by 10%,” then the textual prompt 218 input to the AI image generator 220 includes the user instructions 134 to “lighten a group of pixels in a region that corresponds to a face.” The AI model 128 outputs the modified image 136 to replace the image preview 120 within the viewfinder 118. As another example, the voice command 132 indicates, “Hey Camera, remove the traffic lights.” The modified image 136 output from the AI model 128 omits image features of the image preview 120 that are classified by the AI model 128 to be traffic lights. As other examples, the voice command 132 indicates, “Hey Camera, enable low-light pro mode settings,” “Hey Camera, enable low-light pro mode settings,” “Hey Camera, make sky bluer and increase contrast,” “Hey Camera, increase texture of background without changing the brightness,” “Hey Camera, blur the background and focus on foreground of the image,” “Hey Camera, remove the people in the background,” “Hey Camera, brighten the stars in the sky,” and “Hey Camera, remove the shadows on my face.” The modified image 136 output from the AI model 128 alters image features of the image preview 120 to generate the modified image 136 that satisfies the user instructions 134.”)
fu is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li further in view of Fu to provide data corresponding to the second voice sample to a generative Al model within a command type classification prompt that instructs the generative Al model to determine a command classification type for the second voice sample; and receiving, from the generative Al model, the command classification type for the second voice sample; generating a text modification prompt based on the command classification type by: selecting a prompt type corresponding to the command classification type; and including the first text string and a second text string based on the second voice sample in the text modification prompt. Motivation to do so would improve performance and configure the AI model to respond to user requests quickly (Fu, Par. 0042).
Li, as modified above, does not teach, however Merg teaches providing a modified first text string for display based on receiving the modified first text string from the generative AI model in response to the text modification prompt. (Merg, Par. 0112:” … After the text has been modified, the modified text can then be displayed using user interface 410; e.g., the modified text can replace the entered text in text entry region 420.”)
Merg is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Merg to provide a modified first text string for display based on receiving the modified first text string from the generative AI model in response to the text modification prompt. Motivation to do so would increase efficiency, enhanced quality and relevance, and the ability to offer personalized and dynamic user experiences.
Regarding claim 2, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, further teaches wherein the voice capture session is a single session that captures multiple dictation sentences from a user associated with a client device. (Li, Page 15319, Figure 1 depicts multiple dictation sentences:” just wanted to ask about the event on the 23rd , on Friday the 23rd. Is the event still on? Change “the event” to “it” in the last sentence.)
Regarding claim 6, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, further teaches determining that the first voice sample is a speech-to-text dictation based on not identifying a predetermined action word in the first text string. (Li, Section 3 Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and Section 5, part 2, Page 15325: Each dictation segment is directly spliced into the document at the current cursor position.”, and Figure 3, page 15324 depicts all the relevant texts including the output of speech-to-text dictation, command, etc.”) Note: Figure 1 already determine that the first voice sample is a speech-to-text dictation and does not contain the command.
Regarding claim 8, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, further teaches providing the first text string of the first voice sample for display within a message composition field of a messaging thread user interface associated with a messaging thread between multiple users. (Li, Section 3 Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and Section 5, part 2, Page 15325: Each dictation segment is directly spliced into the document at the current cursor position.”, and Figure 3, page 15324 depicts all the relevant texts including the output of speech-to-text dictation, command, etc.”, and see D2 in Figure 2.)
Regarding claim 10, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, further teaches based on determining that the first voice sample is a speech-to-text dictation, causing display of the first text string in a first user interface field; and (Li, Section 3 Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and Section 5, part 2, Page 15325: Each dictation segment is directly spliced into the document at the current cursor position.”, and Figure 3, page 15324 depicts all the relevant texts including the output of speech-to-text dictation, command, etc.”, and see D2 in Figure 2.)
based on determining that the second voice sample is a voice command, causing display of at least a portion of the second text string in a second user interface field concurrent with displaying the first text string. (Li, Section 3 Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and Section 5, part 2, Page 15325: Each dictation segment is directly spliced into the document at the current cursor position.”, and Figure 3, page 15324 depicts all the relevant texts including the output of speech-to-text dictation, command, etc.”, and see U2 in Figure 2.)
Regarding claim 16, Li, as modified above, teaches the system of claim 14.
Li, as modified above, further teaches generating the second text string using a speech-to-text conversion model; and (Li, Page 15319, Figure 1: “A user writes an email using speech input, interleaving dictation (a, c) and commanding (b, d). Top shows the continuous user utterance, while bottom shows the document state at each point of the utterance. Dictations are transcribed verbatim, while commands are interpreted and executed. Our system supports openended commanding (i.e., b, d both invoke a replace operation but use vastly different phrasing).”) Note: Figure 1 shows both first and second text strings transcribed.
identifying a predetermined action word in the second text string indicating the second voice sample as a voice command. (Li, Section 3 Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and Section 5, part 2, Page 15325: Each dictation segment is directly spliced into the document at the current cursor position.”, and Figure 3, page 15324 depicts all the relevant texts including the output of speech-to-text dictation, command, etc.”) Note: Figure 1 already determine that the second voice sample is a voice command since it contains a predetermined action word (i.e. in b is “asking”, while in d is “checking”).
Regarding claim 17, Li, as modified above, teaches the system of claim 14.
Li, as modified above, further teaches converting the second voice sample to the second text string; (Li, Page 15319, Figure 1: “A user writes an email using speech input, interleaving dictation (a, c) and commanding (b, d). Top shows the continuous user utterance, while bottom shows the document state at each point of the utterance. Dictations are transcribed verbatim, while commands are interpreted and executed. Our system supports openended commanding (i.e., b, d both invoke a replace operation but use vastly different phrasing).”) Note: Figure 1 shows both first and second text string transcribed.
providing the second text string to the generative AI model within a classification generative AI model prompt; and (Li, Section 5.2, Page 15326:” For each Ui that is predicted as a command segment, we first predict the normalized utterance u’I, we then interpret u’I in context to predict either the document state di or an update program Pi. We then update the document state accordingly. We experiment with two ways of implementing the two steps: we either fine-tune two separate T5-base models that run in a pipeline for each command, or we prompt GPT3 to generate both the normalized utterance and the interpretation output in a single inference step.”)
receiving the command classification type from the generative AI model. (Li, Section 3, Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and in Figure 2 “u2” represents command, which in this case the command is classified as capitalization command “Capitalize the S&E speak”).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu and Merg , and in further view of Shang (US 20250384892A1).
Shang was applied in the previous Office Action.
Regarding claim 3, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, does not teach, however, Shang teaches wherein the voice capture session includes multiple continuous microphone activation sessions corresponding to the first text string. (Shang, Par. 0035:” … the first audio is an audio captured by the audio capture device. For example, in a conference scenario, a plurality of microphones may be disposed in the conference room, and the first audios may be audios captured by the plurality of microphones.”)
Shang is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Shang to wherein the voice capture session includes multiple continuous microphone activation sessions corresponding to the first text string. Motivation to do so would improve a playback effect of the audio (Shang, Par. 0058).
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu and Merg, and in further view of DeLuca (US20120198005A1).
DeLuca was applied in the previous Office Action.
Regarding claim 4, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, further teaches wherein: a first user provides the first voice sample and the second voice sample; and(Li, Section 4.1, Page 15323:” We asked the demonstrator to play both the role of the user (issuing the speech stream), and also the roles of the segmentation, normalization, and interpretation parts of the system (Figures 2b-d). Thus, we collect actual ASR results, while asking the demonstrator to demonstrate gold predictions for segmentation, normalization, and interpretation.”, and Section 3, part b, page 15321:” Segmentation When the current transcript changes, the system can update its segmentation. It does so by partitioning the current transcript U into a sequence of segments ui, labeling each as being either a dictation or a command.”, and Figure 1:” A user writes an email using speech input, interleaving dictation (a, c) and commanding (b, d). Top shows the continuous user utterance, while bottom shows the document state at each point of the utterance. Dictations are transcribed verbatim, while commands are interpreted and executed. “)
Li, as modified above, does not teach, however, Deluca teaches the voice capture session is associated with a messaging thread on a mobile device between the first user and at least one additional user. (DeLuca, Par. 0039:” … a portable electronic device 50 which can be used for communicating messages with similar devices, as discussed in greater detail below. … portable electronic device 50 can include, without limitation, a cellular telephone, a portable email paging device, …”, and Par. 0066:” A number of messages in a thread algorithm measure an average number of messages, including the originating message and replies, that are associated with the contact for a given thread. Such an algorithm thus parses the message into a plurality of sub-messages, and correlates with other messages previously received from the same contact.”)
DeLuca is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of DeLuca to have the voice capture session is associated with a messaging thread on a mobile device between the first user and at least one additional user. Motivation to do so would provide for performing message analytics upon at least a portion of a first message (DeLuca, Par. 0003).
Regarding claim 13, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, further teaches generating, based on identifying the voice command of the second voice sample [[as a text tone change classification]], (Li, Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and see D2 in Figure 2.)
the text modification prompt that includes instructing the generative AI model to change [[a tone]] of the first text string based on a context included in the second text string.(Li, Section 5.2, Page 15326:” For each Ui that is predicted as a command segment, we first predict the normalized utterance u’I, we then interpret u’I in context to predict either the document state di or an update program Pi. We then update the document state accordingly. We experiment with two ways of implementing the two steps: we either fine-tune two separate T5-base models that run in a pipeline for each command, or we prompt GPT3 to generate both the normalized utterance and the interpretation output in a single inference step.”)
Li, as modified above, does not teach, however, Deluca teaches text tone change classification. (DeLuca, Par. 0077:” … The fourth indicator provides a general suggestion for modification to the message, which is identified as "Increase Formality"; generally suggesting that the tone of the message 808a should be increased in formality.”)
DeLuca is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of DeLuca to perform text tone change classification. Motivation to do so would provide for performing message analytics upon at least a portion of a first message (DeLuca, Par. 0003).
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu and Merg, and in further view of Mertens (US2023125194A1).
Mertens was applied in the previous Office Action.
Regarding claims 5, and 15 Li, as modified above, teaches the computer-implemented method and system of claims 1 and 14 respectively.
Li, as modified above, does not teach, however Mertens teaches wherein the first voice sample and the second voice sample are associated with adding text and content to a [digital document – claim 5], [word-processing document – claim 15]. (Mertens, Par. 0052:” … Generating improved sentence structures may comprise adding tokens, words, or other structures, deleting them, or substituting different ones.”, and Par. 0060:” … words from the sentence structures, or adding one or more additional words to the sentence structures when generating the one or more sentences.”)
Mertens is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Mertens to wherein the first voice sample and the second voice sample are associated with adding text and content to a document. Motivation to do so would improve or enhance the text derived from the original speech (Mertens, Par. 0096).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu and Merg, and in further view of Rajan (US 10971151 B1).
Rajan was applied in the previous Office Action.
Regarding claim 7, Li, as modified above, teaches the computer-implemented method of claim 6.
Li, as modified above, does not teach, however, Rajan teaches determining that the first voice sample is a speech-to-text dictation based on: analyzing the first text string with a classification model to identify an intent of the first text string; and (Rajan, Col. 6, line 59 – 67:” Event 624 may represent performing speech recognition on the audio information while the user is uttering a spoken command. State 606 may represent comprising the text form of the spoken command based on event 624. Upon detection that the user's utterance of the spoken command has terminated (i.e., event 626), system 100 may begin intent processing. Intent processing may be via various intent extractors and/or other methods of identifying the spoken command.”)
determining that the intent of the first text string is a speech-to-text dictation. (Rajan, Col. 6, ll. 59-61:” Event 624 may represent performing speech recognition on the audio information while the user is uttering a spoken command.”, and Col. 7, ll. 26-28:” Event 630 may occur in response to determining the intent of the spoken command (i.e., state 608) includes transcription dictation.”)
Rajan is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Rajan to analyze the first text string with a classification model to identify an intent of the first text string; and determining that the intent of the first text string is a speech-to-text dictation. Motivation to do so would allow a spoken instance of a predetermined keyword to be present before, in the middle, or after a spoken command from the user (Rajan, Col. 1, ll. 33-35).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu and Merg, and in further view of Kim (US 20230410788 A1).
Kim was applied in the previous Office Action.
Regarding claim 9, Li, as modified above, teaches the computer-implemented method of claim 8.
Li, as modified above, does not teach, however, Kim teaches not receiving user input to send the first text string to a recipient user associated with the messaging thread before receiving the second voice sample within the voice capture session. (Kim, Par. 0074:” … the first overlapping utterance 312 obtained from the first uttered voice 310 with the second overlapping utterance 322 obtained from the second uttered voice 320. In this regard, the processor 2420 may optionally or additionally add a silence period (e.g., a short pause period or a silence period) 332 of a specified length between the first overlapping utterance 312 and the second overlapping utterance 322.”)
Kim is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Kim to not receiving user input to send the first text string to a recipient user associated with the messaging thread before receiving the second voice sample within the voice capture session. Motivation to do so would allow at least two people to talk over the phone simultaneously (Kim, Par. 0004).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu and Merg, and in further view of Seo (US 20230410788 A1).
Seo was applied in the previous Office Action.
Regarding claim 11, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, does not teach, however, Seo teaches determining that the second voice sample is a voice command based on analyzing the second voice sample to identify voice characteristics indicating the second voice sample as a voice command. (Seo, Par. 0009:” According to an aspect of the disclosure, a speech recognition device includes a microphone, and a processor configured to receive a voice signal through the microphone, generate voice characteristic data by analyzing the voice signal by using a data recognition model based on a neural network, determine whether the voice signal is voice uttered from a user or voice output from an external device based on the voice characteristic data, and when the voice signal is determined as the voice uttered from the user, determine the voice signal as a voice command of the user and perform an operation corresponding to the voice command.”)
Seo is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Seo to determine that the second voice sample is a voice command based on analyzing the second voice sample to identify voice characteristics indicating the second voice sample as a voice command. Motivation to do so would provide a recognition result according to the purpose of recognizing the data (Seo, Par. 0246).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu, and Merg, and in further view of Rajan, and Seo.
Regarding claim 12, Li, as modified above, teaches the computer-implemented method of claim 1.
Li, as modified above, further teaches generating the second text string using a speech-to-text conversion model; (Li, Page 15319, Figure 1: “A user writes an email using speech input, interleaving dictation (a, c) and commanding (b, d). Top shows the continuous user utterance, while bottom shows the document state at each point of the utterance. Dictations are transcribed verbatim, while commands are interpreted and executed. Our system supports openended commanding (i.e., b, d both invoke a replace operation but use vastly different phrasing).”) Note: Figure 1 shows both first and second text string transcribed.
Li, as modified above, does not teach, however, Rajan teaches analyzing the second text string with a classification model to identify an intent of the second text string; and (Rajan, Col. 6, line 59 – 67:” Event 624 may represent performing speech recognition on the audio information while the user is uttering a spoken command. State 606 may represent comprising the text form of the spoken command based on event 624. Upon detection that the user's utterance of the spoken command has terminated (i.e., event 626), system 100 may begin intent processing. Intent processing may be via various intent extractors and/or other methods of identifying the spoken command.”)
Rajan is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Rajan to analyze the second text string with a classification model to identify an intent of the second text string. Motivation to do so would allow a spoken instance of a predetermined keyword to be present before, in the middle, or after a spoken command from the user (Rajan, Col. 1, ll. 33-35).
Li, as modified above, does not teach, however, Seo teaches determining that the intent of the second text string is a voice command. (Seo, Par. 0009:” According to an aspect of the disclosure, a speech recognition device includes a microphone, and a processor configured to receive a voice signal through the microphone, generate voice characteristic data by analyzing the voice signal by using a data recognition model based on a neural network, determine whether the voice signal is voice uttered from a user or voice output from an external device based on the voice characteristic data, and when the voice signal is determined as the voice uttered from the user, determine the voice signal as a voice command of the user and perform an operation corresponding to the voice command.”)
Seo is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Seo to determine that the intent of the second text string is a voice command. Motivation to do so would provide a recognition result according to the purpose of recognizing the data (Seo, Par. 0246).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Li, Fu and Merg, and in further view of Maschmeyer (US 20240320432 A1).
Maschmeyer was applied in the previous Office Action.
Regarding claim 19, Li, as modified above, teaches the system of claim 14.
Li, as modified above, does not teach, however, Maschmeyer teaches wherein providing the modified first text string for display includes providing the modified first text string in a separate user interface text field that is apart from a user interface text field displaying the first text string. (Maschmeyer, Par. 0008:” FIGS. 3B-3D illustrate example user interfaces for displaying suggested modifications to a textual description.”, and Par. 0031:” … In this example, the rating 312 is generated for a textual description input by a user at a text input box 314.”, and Par. 0036:” … the suggested modification generator 220 uses the rating model 205 to rewrite at least a portion of the received textual description 105 to improve its score. FIG. 3D illustrates an example user interface 340 displaying a rewritten textual description 342.”)
Maschmeyer is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Maschmeyer to wherein providing the modified first text string for display includes providing the modified first text string in a separate user interface text field that is apart from a user interface text field displaying the first text string. Motivation to do so would provide an automated mechanism to improve textual descriptions (Maschmeyer, Par. 0041).
PNG
media_image1.png
390
630
media_image1.png
Greyscale
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Li (“ Toward Interactive Dictation”), and in further view of Fu (US20260181244A1), Deluca (US20120198005A1) and Merg (US20170364870A1).
Regarding claim 20, Li, teaches A computer-implemented method for providing generative content for one or more voice samples using one or more generative artificial intelligence (AI) models, comprising: (Li, Introduction, page 15320:” we experiment with two strategies for implementing the proposed system: one that uses a pre-trained language model to directly predict the edited text given unedited text and a command, and another that interprets the command as a program specifying how to edit. … we also experimented with two choices of pre-trained language model: a small finetuned T5 model and a large prompted GPT3 model.”)
receiving, from a first user, a first voice sample and a second voice sample in a voice capture session [[associated with a messaging thread on a device between the first user and at least one additional user]]; (Li, Section 3, part b, page 15321:” Segmentation When the current transcript changes, the system can update its segmentation. It does so by partitioning the current transcript U into a sequence of segments ui, labeling each as being either a dictation or a command.”, and Figure 1:” A user writes an email using speech input, interleaving dictation (a, c) and commanding (b, d). Top shows the continuous user utterance, while bottom shows the document state at each point of the utterance. Dictations are transcribed verbatim, while commands are interpreted and executed. “)
based on determining that the first voice sample is a speech-to-text dictation, (Li, Section 3 Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and Section 5, part 2, Page 15325: Each dictation segment is directly spliced into the document at the current cursor position.”, and Figure 3, page 15324 depicts all the relevant texts including the output of speech-to-text dictation, command, etc.”, and see D2 in Figure 2.)
providing a first text string of the first voice sample for display within a message composition field [[of the messaging thread]]; (Li, Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and see D2 in Figure 2.)
based on determining that the second voice sample is a voice command, (Li, Section 3, Page 15322:” When Ui is a dictation segment, no prediction is needed: the state update simply inserts the current segment at the cursor. However, when Ui is a command segment, predicting the state update that the user wanted requires a text understanding model. Note that commands can come in many forms. Commonly they are imperative commands, as in Figure 1d. But one can even treat speech repairs such as Figure 1b as commands, in a system that does not handle repairs at stage (a) or (c).”, and in Figure 2 “u2” represents command, which in this case the command is classified as capitalization command “Capitalize the S&E speak”).
providing the text modification prompt to the generative AI model (Li, Section 5.2, Page 15326:” For each Ui that is predicted as a command segment, we first predict the normalized utterance u’I, we then interpret u’I in context to predict either the document state di or an update program Pi. We then update the document state accordingly. We experiment with two ways of implementing the two steps: we either fine-tune two separate T5-base models that run in a pipeline for each command, or we prompt GPT3 to generate both the normalized utterance and the interpretation output in a single inference step.”)
based on a context included in the second text string; (Li, Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and see D2 in Figure 2.)
based on receiving a modified first text string from the generative AI model in response to the text modification prompt, providing the modified first text string with the tone of the first text string changed for display; and (Li, Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and see D2 in Figure 2.).
wherein the text modification prompt instructs the generative AI model to change a tone of the first text string based on a context included in the second text string; ( Li, Section 4.1, step 2, Page 15323:” All labeled segments are displayed in the right column of the UI. After the demonstrator has finished speaking, they fill in the normalized text for each command segment. (The segment shows original and normalized text in the ASR and Gold ASR fields.)”, and see D2 in Figure 2.).
Li, does not teach, however, Fu teaches determining a command classification type for the second voice sample by: providing data corresponding to the second voice sample to a generative AI model within a command type classification prompt that instructs the generative AI model to determine a command classification type for the second voice sample; and receiving, from the generative AI model, a [[text tone change classification]] as the command classification type for the second voice sample; generating a text modification prompt based on the [[text tone change classification]] by: selecting a prompt type corresponding to the [[text tone change classification]]; and including the first text string and a second text string based on the second voice sample in the text modification prompt, (Fu, Par. 0068:” The AI model 128 may be trained and retrained to interpret various types of voice commands and user speech. For example, if the voice command 132 indicates, “Hey Camera, brighten the face by 10%,” then the textual prompt 218 input to the AI image generator 220 includes the user instructions 134 to “lighten a group of pixels in a region that corresponds to a face.” The AI model 128 outputs the modified image 136 to replace the image preview 120 within the viewfinder 118. As another example, the voice command 132 indicates, “Hey Camera, remove the traffic lights.” The modified image 136 output from the AI model 128 omits image features of the image preview 120 that are classified by the AI model 128 to be traffic lights. As other examples, the voice command 132 indicates, “Hey Camera, enable low-light pro mode settings,” “Hey Camera, enable low-light pro mode settings,” “Hey Camera, make sky bluer and increase contrast,” “Hey Camera, increase texture of background without changing the brightness,” “Hey Camera, blur the background and focus on foreground of the image,” “Hey Camera, remove the people in the background,” “Hey Camera, brighten the stars in the sky,” and “Hey Camera, remove the shadows on my face.” The modified image 136 output from the AI model 128 alters image features of the image preview 120 to generate the modified image 136 that satisfies the user instructions 134.”)
fu is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li further in view of Fu to determine a command classification type for the second voice sample by: providing data corresponding to the second voice sample to a generative AI model within a command type classification prompt that instructs the generative AI model to determine a command classification type for the second voice sample; and receiving, from the generative AI model, a [[text tone change classification]] as the command classification type for the second voice sample; generating a text modification prompt based on the [[text tone change classification]] by: selecting a prompt type corresponding to the [[text tone change classification]]; and including the first text string and a second text string based on the second voice sample in the text modification prompt. Motivation to do so would improve performance and configure the AI model to respond to user requests quickly (Fu, Par. 0042).
Li, as modified above, does not teach, however DeLuca teaches voice captured associated with a messaging thread on a device between the first user and at least one additional user (DeLuca, Par. 0066:” A number of messages in a thread algorithm measure an average number of messages, including the originating message and replies, that are associated with the contact for a given thread. Such an algorithm thus parses the message into a plurality of sub-messages, and correlates with other messages previously received from the same contact.”)
text tone change classification, (DeLuca, Par. 0077:” … The fourth indicator provides a general suggestion for modification to the message, which is identified as "Increase Formality"; generally suggesting that the tone of the message 808a should be increased in formality.”)
DeLuca is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, further in view of DeLuca to voice captured associated with a messaging thread on a device between the first user and at least one additional user, and text tone change classification. Motivation to do so would provide for performing message analytics upon at least a portion of a first message (DeLuca, Par. 0003).
Li, as modified above, does not teach, however, Merg teaches replacing the first text string with the modified first text string within the message composition field of the messaging thread. (Merg, Par. 0112:” … After the text has been modified, the modified text can then be displayed using user interface 410; e.g., the modified text can replace the entered text in text entry region 420.”).
Merg is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Li, as modified above, further in view of Merg to replace the first text string with the modified first text string within the message composition field of the messaging thread. Motivation to do so would increase efficiency, enhanced quality and relevance, and the ability to offer personalized and dynamic user experiences.
Allowable Subject Matter
Claim 18 is objected to as being dependent upon a rejected base claim, but would be allowable if written in independent form including all of the limitations of the base claim and any intervening claims.
Claim 18, recites:” wherein command classification types included in the classification generative AI model prompt include a text tone change classification, an auto-compose text classification, a user query classification, an image creation classification, and a meme generation classification.”, which is allowable over the prior art. The closest teachings to the indicated allowable subject matter are the references that are cited in the current office action. However, none of the prior art of record teach the limitation as stated above specifically the underlined as shown including all supporting limitations thereof. Therefore, claim 18 is allowable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Narayanan (US 20250181849 A1) teaches in Par. 0009:” … method including: (i) receiving, by a processor, text content from an entity that stores the text content as a data object associated with the entity, (ii) generating, by the processor, a prompt for a large language model that comprises the text content and directions for modifying the text content, (iii) providing, by the processor, the prompt to the large language model, (iv) executing, by the processor, the large language model, the execution causing creation of modified text content in accordance with the directions for modifying the text content from the prompt; (v) receiving, by the processor from the large language model, the modified text content, and (vi) creating, by the processor, a new data object that stores the modified text content in association with the entity.”
Examiner's Note: Examiner has cited particular columns and line numbers and/or paragraph numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DARIOUSH AGAHI, P.E. whose telephone number is (408)918-7689. The examiner can normally be reached Monday - Thursday and alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DARIOUSH AGAHI, P.E.
Primary Examiner
/DARIOUSH AGAHI/Primary Examiner, Art Unit 2656
Prosecution Insights