DETAILED ACTION
Introduction
Applicant's submission filed on 05/14/2026 has been entered. Claims 1, 3-26 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The response filed on 05/14/2026 has been correspondingly accepted and considered in this Office Action. Claims 1, 3-26 have been examined. Claim 2 has been cancelled. Applicant’s amendments to claim 1, 25 and 26, indicating a series of steps are being used to create this second request and then based on this second request determining if a certain criteria is satisfied and later executed by the digital assistant and that assistant acts on that by providing an output overcome the 35 U.S.C 101 rejections previously set forth in the Non-Final Office Action mailed 02/17/2026. The dependent claims 3-24 overcome the 35 U.S.C 101 rejections previously set forth in the Non-Final Office Action mailed 02/17/2026 based on their dependency to the amended claim 1 respectively. Therefore, the above referenced rejections under 35 U.S.C. 101 are withdrawn.
Response to Arguments
Applicant's arguments filed 05/14/2026 have been fully considered as follows:
Applicant’s arguments with respect to claim 1 (also representative of claims 25 and 26) state that
“However, Grupen fails to disclose determining a second request is intended for a digital assistant, "wherein the second request is intended for the digital assistant when a ranking of the second request [based on a second portion of the audio stream] is higher than a ranking of the first request [based on a first portion of the audio stream]" as required by amended representative claim 1. (Emphasis added). Accordingly, Grupen does not cure the deficiencies of Fabrizio”
The examiner respectfully disagrees, Grupen teaches “If the electronic device determines that the second natural-language input satisfies the dialog criteria, at block 835, the electronic device outputs a second natural-language output associated with the first dialog flow, and optionally, performs one or more tasks specified by the dialog flow. Consider for instance, the following dialog exchange: User: “I want to repair my phone” Electronic device: “Okay. What would you like to repair?” User: “My display is cracked.”” in Grupen, col 43 line 63 – col 44 line 4, the ranking of the second request is based on the first request and is more specific than the first request and hence higher ranked. Therefore, Grupen teaches when the second request is intended for the digital assistant, wherein the second request is intended for the digital assistant when a ranking of the second request is higher than a ranking of the first request and therefore, the rejections of Claims 1, 25 and 26 are rejected under 35 U.S.C. 103 are sustained and further updated accordingly.
In response to the art rejection(s) of the remainder of dependent claims are rejected under 35 U.S.C 103, in case said claims are correspondingly discussed and/or argued for at least the same rationale presented in Remarks filed 05/14/2026, Examiner respectfully notes as follows. For completeness, should the mentioned claims be likewise traversed for similar reasons to independent claims 1, 25 and 26 correspondingly, Examiner respectfully directs Applicant to the same previous supra reasons provided in the response directed towards claims 1, 25 and 26 correspondingly discussed above. For at least the same supra provided reasons, Examiner likewise respectfully disagrees, and Applicant's arguments have been fully considered but they are not persuasive.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 3, 5-7 and 17-26 are rejected under 35 U.S.C. 103 as being unpatentable over Fabbrizio et. al. US PgPub. 2008/0184164 in view of Grupen et. al. US Patent 10,733,982.
Regarding claim 1, Fabbrizio teaches a non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device (see Fabbrizio, [0011]), cause the electronic device to: receive an audio stream; (see Fabbrizio, [0102] teaches S0 to state S1 which receives the user input); generate, based on a first portion of the audio stream, a first request, wherein the first request has a first type of predefined state (see Fabbrizio, [0102] If user input is received at state S.sub.1, that requires confirmation, the flow transitions to state S.sub.2 that performs a confirmation interaction with the user, the spoken dialog system can confirm user input ( first type of predefined state)); in accordance with a determination that the first request is intended for a digital assistant operating on the electronic device: change the first type of predefined state of the first request to a second type of predefined state different from the first type of predefined state (see Fabbrizzio, Fig. 5B, [0102] S2 transitions to S3 /S4 or Done.. or S1 transitions to S6 S7 Fail (second type of predefined state))); generate, based on a second portion of the audio stream that is received after the first portion of the audio stream, a second request (see Fabbrizio, [0098] when receives input for context switch (second portion of audio stream)); and in accordance with a determination, based on the second request, that a first set of criteria is satisfied: change the second type of predefined state of the first request to a third type of predefined state different from the first type of predefined state and the second type of predefined state (see Fabbrizio, [0098] comparing the input with context shift ( first set of criteria), set the state that the shift describes(third type of predefined state)); provide an output, generated by the digital assistant, based on the second request state that the shift describes (see Fabbrizio, [0084-0086] discusses further processing with context switches and outputs by spoken systems by the flow controller ). However Fabbrizio fails to teach wherein the first set of criteria includes a first criterion that is satisfied when the second request is intended for the digital assistant, wherein the second request is intended for the digital assistant when a ranking of the second request is higher than a ranking of the first request.
However, Grupen teaches wherein the first set of criteria includes a first criterion that is satisfied when the second request is intended for the digital assistant, wherein the second request is intended for the digital assistant when a ranking of the second request is higher than a ranking of the first request (see Grupen, Fig. 8 steps 825, 830, col 43 lines 48- col 44 line 37 discusses request for electronic device ( intended for digital assistant) and determines whether the second natural-language input satisfies dialog criteria( both correspond to a same task as indicated in specifications) associated with the selected dialog flow ( ranking, if criteria met: higher ranking)).
Fabbrizio and Grupen are considered to be analogous to the claimed invention because they relate to spoken dialog processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Fabbrizio on the routines of dialog manager with the processing of audio for digital assistant teachings of Grupen to improve user to interaction with a digital assistant using natural-language dialog ( see Grupen, col 2 lines 29-35).
Regarding claim 3, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1. Fabbrizio teaches change the candidate state of the second request to an active state of the second request (see Fabbrizio, [0098] when receives input for context switch (second request), comparing the input with context shift, set the state that the shift describes(active state of second request)). Grupen also teaches in accordance with a determination that the second request is intended for the digital assistant, change the candidate state of the second request to an active state of the second request (see Grupen, Fig. 8, steps 825-835 discusses the processing of the second requests based on the dialog flow parameters(states) ). The same motivation to combine as claim 1 applies.
Regarding claim 5, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1, further teaches wherein: the first type of predefined state of the first request is a candidate state of the first request (see Fabbrizio, [0102] teaches S0 to state S1 to state S2 (candidate state)); and the second type of predefined state of the first request is an active state of the first request (see Fabbrizio, [0102] S2 transitions to S3 /S4 (Active state)or S1 transitions to S6 S7 (Active state)).
Regarding claim 6, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1, further teaches wherein the third type of predefined state of the first request is a canceled state of the first request (see Fabbrizio, [0102] teaches invalid prompt(S4) (canceled state)).
Regarding claim 7, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 6, further teaches wherein the first set of criteria includes a second criterion that is satisfied when the second request includes a continuation of the first request (see Fabbrizio, [0101-102] If input is received appropriately, then the flow may transition to the done state from S1 (continuation of first request)).
Regarding claim 17, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1, further teaches in accordance with a determination that a second set of criteria is satisfied, generating the second request based on speech input in the first portion of the audio stream and speech input in the second portion of the audio stream (see Fabbrizio, [0084-0086] discusses further processing with context switches and outputs by spoken systems by the flow controller).
Regarding claim 18, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 17, further teaches wherein the second set of criteria includes a fourth criterion that is satisfied when the first request has an active state or when the first request has a candidate state (see Fabbrizio, [0101-102] If input is received appropriately, then the flow may transition to the done state from S1(active) or S2 (candidate) or S5).
Regarding claim 19, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1, further teaches in accordance with a determination that a third set of criteria is satisfied, generate a third request based on speech input in the second portion of the audio stream, wherein the generated third request is not based on speech input in the first portion of the audio stream (see Fabbrizio, [0101-102] If user input received needs confirmation, the flow transitions to S2 to perform the task based confirmation based on S3 or S4; third request based on second portion ).
Regarding claim 20, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 19, further teaches wherein the third set of criteria includes a fifth criterion that is satisfied when at least one of: the speech input in the first portion of the audio stream is followed by a pause in speech of at least a predetermined duration (see Fabbrizio, [0101] the state transition to silent( pause) while the input is received); and the speech input in the first portion of the audio stream is determined to include a complete utterance (see Fabbrizio, [0101-102] In the input prompt state S1 handles silence, interaction continues correctly(progress to done state), rejection, a wrong category and a confirmation interaction with the user).
Regarding claim 21, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1, further teaches the second request, when generated, has a first type of predefined phase (see Fabrizio, Fig. 7, step 702); when the second request has the first type of predefined phase, the electronic device is prohibited from performing a first set of actions corresponding to the second request (see Fabrizio, Fig. 7, 704 and [0098] the table can be programmed to shift to another FC or not change the context(prohibit)); and the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to: in accordance with a determination that a phase transition criterion is satisfied: change the first type of predefined phase of the second request to a second type of predefined phase different from the first type of predefined phase, wherein when the second request has the second type of predefined phase, the electronic device is permitted to perform the first set of actions.(see Fabbrizio, Fig. 7, 708 and it processed per the FC ).
Regarding claim 22, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 21, further teaches wherein the phase transition criterion is satisfied when the second request is determined to be complete (see Fabbrizio, [0102] done state).
Regarding claim 23, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 21, further teaches wherein the phase transition criterion is satisfied when the second request currently has an active state or currently has a background state (see Fabbrizio, [0101-102] If input is received appropriately, then the flow may transition to the done state from S1(active) or S2 (candidate) or S5(background state)).
Regarding claim 24, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1, further teaches wherein only a single request can have the second type of predefined state at a time (see Fabbrizio, [0101-102] the RTN can only be in a particular predefined state at a time as indicated).
Regarding claim 25, is directed to a device claim corresponding to the non-transitory computer-readable storage medium claim presented in claim 1 and is rejected under the same grounds stated above regarding claim 1.
Regarding claim 26, is directed to a method claim corresponding to the non-transitory computer-readable storage medium claim presented in claim 1 and is rejected under the same grounds stated above regarding claim 1.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Fabbrizio et. al. US PgPub. 2008/0184164 in view of Grupen et. al. US Patent 10,733,982 further in view of Rastrow et. al. US Patent 12,236,950.
Regarding claim 4, Fabbrizio in view of Grupen a non-transitory computer-readable storage medium of claim 1, Fabbrizio further teaches change the first type of predefined state of the first request to a canceled state of the first request, wherein the canceled state of the first request is different from the first type of predefined state of the first request and the second type of predefined state of the first request (see Fabbrizio, [0102] discusses processing of wrong category(S7), rejection action(S6), invalid prompt(S4) to fail the processing different from (S2), Fig. 5B). However Fabrizio in view of Grupen fails to teach in accordance with a determination that the first request is not intended for the digital assistant .
However, Rastrow teaches in accordance with a determination that the first request is not intended for the digital assistant(see Rastrow, col 30 lines 14-18, If the speech is not determined to be system-directed (564: No) (e.g. SDD result 551 indicates no system-directed speech), the system may cancel (566) further processing for the audio data.).
Fabbrizio, Grupen and Rastrow are considered to be analogous to the claimed invention because they relate to spoken dialog processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Fabbrizio in view of Grupen on the routines of dialog manager with the detection of device directed speech for digital assistant teachings of Rastrow to improve speech processing and other audio-based user interface systems ( see Rastrow, col 4 lines 10-25).
Claims 8-11, 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Fabbrizio et. al. US PgPub. 2008/0184164 in view of Grupen et. al. US Patent 10,733,982 further in view of Kennewick, US PgPub. 2021/0082397.
Regarding claim 8, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1. However, Fabbrizio in view of Grupen fail to teach receive a second audio stream; generate, based on a first portion of the second audio stream, a third request, wherein the third request has a fourth type of predefined state; in accordance with a determination that the third request is intended for the digital assistant operating on the electronic device: change the fourth type of predefined state of the third request to a fifth type of predefined state different from the fourth type of predefined state; generate, based on a second portion of the second audio stream that is received after the first portion of the second audio stream, a fourth request; and in accordance with a determination, based on the fourth request, that a second set of criteria is satisfied: change the fifth type of predefined state of the third request to a sixth type of predefined state different from the fourth type of predefined state and the fifth type of predefined state, wherein the sixth type of predefined state of the third request is a completed state of the third request; and provide an output, generated by the digital assistant, based on the fourth request.
However, Kennewick teaches receive a second audio stream (see Kennewick, Fig. 502); generate, based on a first portion of the second audio stream, a third request, wherein the third request has a fourth type of predefined state (see Kennewick, [0087, 0095,0096, 0101 ] additional input is processed in 320. After determining an intent for additional speech input(input speech processed as in step 306) and processing the dialog session based on state analysis, (steps, 402->404->408 (forth type)) in accordance with a determination that the third request is intended for the digital assistant operating on the electronic device: change the fourth type of predefined state of the third request to a fifth type of predefined state different from the fourth type of predefined state (see Kennewick,[0102] request clarification ( fifth state)); generate, based on a second portion of the second audio stream that is received after the first portion of the second audio stream, a fourth request (see Kennewick, [0103] further processes additional input ); and in accordance with a determination, based on the fourth request, that a second set of criteria is satisfied: change the fifth type of predefined state of the third request to a sixth type of predefined state different from the fourth type of predefined state and the fifth type of predefined state, wherein the third sixth type of predefined state of the first third request is a completed state of the first third request (see Kennewick, [0103] goes back to 402 to 404 to 406 as indicated in [0097-0100] At 404, the dialog system determines whether the dialog session is in a first state. In the first state, the user may be unlikely to provide additional feedback); and provide an output, generated by the digital assistant, based on the fourth request (see Kennewick [0054] responses that the dialog system may output include “Anything else?”, “That will be $39.78, “The capitol of California is Sacramento,” and so forth).
Fabbrizio, Grupen and Kennewick are considered to be analogous to the claimed invention because they relate to spoken dialog processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Fabbrizio in view of Grupen on the routines of dialog manager with the multiple response teachings of Kennewick to improve natural flow to the dialog with user using natural-language dialog system( see Kennewick, [0004]).
Regarding claim 9, Fabbrizio in view of Grupen further in view of Kennewick teach a non-transitory computer-readable storage medium of claim 8. Kennewick further teaches wherein the third request corresponds to a first task(see Kennewick, [0109] At 506, the dialog system determines whether the subsequent speech input(third request) is relevant to previous speech input (first task) ), and wherein the second set of criteria include a second criterion that is satisfied when the forth request corresponds to a second task different from the first task and when the forth request is received after the first task completes(see Kennewick [0110] At 508, if the dialog system determines that the subsequent speech input is relevant to the previous speech input, then the dialog system processes the subsequent speech input with the previous speech input).
Regarding claim 10, Fabbrizio in view of Grupen further in view of Kennewick teach a non-transitory computer-readable storage medium of claim 8. Kennewick further teaches wherein the second set of criteria includes a third criterion that is satisfied when the second audio stream includes a cessation of speech for a predetermined duration that follows the third request (see Kennewick, [0097- 0099] length of pause to determine cessation of speech).
Regarding claim 11, is directed to a device claim corresponding to the non-transitory computer-readable storage medium claim presented in claim 8 and is rejected under the same grounds stated above regarding claim 8. Kennewick further teaches wherein the third sixth type of predefined state of the third request is a background state of the third request(see Kennewick, [0104] The third state(background) is a state in which additional speech input is likely (e.g., it is appropriate to continue the dialog session and wait for additional speech input) ).
Regarding claim 13, Fabbrizio in view of Grupen further in view of Kennewick teach a non-transitory computer-readable storage medium of claim 11, Fabbrizio further teaches wherein the second set of criteria includes a third criterion that is satisfied when the forth request specifies to pause a task corresponding to the third request (see Fabbrizio, [0034] teaches implementation of pause for the RTN state to pause before exiting to get new user input ( second request specifies a pause to the task)).
Regarding claim 14, Fabbrizio in view of Grupen further in view of Kennewick a non-transitory computer-readable storage medium of claim 11, Fabbrizio further teaches after changing the fifth type of predefined state of the first request to the background state of the third request: receive a user input requesting to continue the first request (see Fabbrizio, [0101], when the active state S1 if silence in input goes to S2 (background) and then input receives further input before timer expires); and in response to receiving the user input requesting to continue the third request: change the background state of the third request to a second active state of the third request (see Fabbrizio, [0102] changes from S5 to S1 to S2/S6/S7 (second active state)); and provide a second output, generated by the digital assistant, based on the third request (see Fabbrizio, [0102] transition to done ( provide the second output)).
Regarding claim 15, Fabbrizio in view of Grupen further in view of Kennewick a non-transitory computer-readable storage medium of claim 14, Kennewick further teaches before receiving the user input requesting to continue the third request, provide a third output including a prompt to continue the third request (see Kennewick, [0076-0077] At 312, if a response is to be generated at 310, then the dialog system may generate and output a response. The dialog system may execute a task and/or provide feedback to the user. The dialog system prepares and provides the response, “OK, where to?” while continuing to listen to additional speech input.).
Claims 12 are rejected under 35 U.S.C. 103 as being unpatentable over Fabbrizio et. al. US PgPub. 2008/0184164 in view of Grupen et. al. US Patent 10,733,982 further in view of Kennewick, US PgPub. 2021/0082397 further in view of Selfridge et al, Continuously Predicting and Processing Barge-in During a Live Spoken Dialogue Task " Proceedings of the SIGDIAL 2013 Conference. 2013.
Regarding claim 12, Fabbrizio in view of Grupen further in view of Kennewick teach a non-transitory computer-readable storage medium of claim 11, however, fails to teach wherein the first request corresponds to a third task, and wherein the second set of criteria includes a second criterion that is satisfied when the fourth request corresponds to a second task different from the first task and when the fourth request is received before the first task completes.
However Selfridge teaches wherein the first request corresponds to a third task, and wherein the second set of criteria includes a second criterion that is satisfied when the forth request corresponds to a second task different from the first task and when the forth request is received before the first task completes (see Selfridge, sect 3.3 when the speech ( second request) is a barge in(before first task completes) during the prompt, will classify as Understood Barge-In (UBI) and change to new speech act or if not understood then it is discarded and the recognition is NUBI as shown in Fig. 1, 2).
Fabbrizio, Grupen, Kennewick and Selfridge are considered to be analogous to the claimed invention because they relate to spoken dialog processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Fabbrizio in view of Grupen in view of Kennewick on the routines of dialog manager with the barge-in processing of input during system speech teachings of Selfridge to improve the spoken dialog systems to demonstrate more human-like turn-taking behavior ( see Selfridge, sect 1).
Claims 16 are rejected under 35 U.S.C. 103 as being unpatentable over Fabbrizio et. al. US PgPub. 2008/0184164 in view of Grupen et. al. US Patent 10,733,982 further in view of Gruber et. al. US Patent 9,318,108 .
Regarding claim 16, Fabbrizio in view of Grupen teach a non-transitory computer-readable storage medium of claim 1, Grupen discusses affordance(see Grupen, col 27 lines 32-37). However, Fabbrizio in view of Grupen fail to teach while receiving the audio stream: in accordance with a determination that the audio stream includes speech input and that the first request has the first type of predefined state: display an affordance corresponding to the digital assistant in a first state; and in accordance with a determination that the audio stream includes speech input and that the first request has the second type of predefined state display the affordance corresponding to the digital assistant in the first state.
However, Gruber teaches while receiving the audio stream: in accordance with a determination that the audio stream includes speech input and that the first request has the first type of predefined state: display an affordance corresponding to the digital assistant in a first state (see Gruber, Fig. 14); and in accordance with a determination that the audio stream includes speech input and that the first request has the second type of predefined state display the affordance corresponding to the digital assistant in the first state (see Gruber, Fig. 15-17).
Fabbrizio, Grupen and Gruber are considered to be analogous to the claimed invention because they relate to spoken dialog processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Fabbrizio in view of Grupen on the routines of dialog manager with the interfaces for digital assistant teachings of Gruber to present users with consistent interfaces of different devices and improve ease of usage ( see Gruber, col 1, lines 28-42).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Konzelmann, US PgPub. 2023/0053341 teaches examples of determining whether to cause natural conversation output to be provided for presentation to a user in response to determining that the user paused in providing of a spoken utterance and/or determining when to fulfill the spoken utterance (see Konzelmann, Fig. 5A-D).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NANDINI SUBRAMANI whose telephone number is (571)272-3916. The examiner can normally be reached Monday - Friday 12:00pm - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh M Mehta can be reached at (571)272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NANDINI SUBRAMANI/Examiner, Art Unit 2656
/BHAVESH M MEHTA/Supervisory Patent Examiner, Art Unit 2656
Prosecution Insights