Detailed Action
Notice of Pre-AIA or AIA status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. § 119(a)–(d), which papers have been placed of record in the file.
Information Disclosure Statement
The information disclosure statements filed on September 19, 2024 and March 14, 2025 comply with the provisions of 37 C.F.R. § 1.97, 1.98, and MPEP § 609, and therefore have been placed in the application file. The information referred to therein has been considered as to the merits.
Specification
The disclosure is objected to because of the following informalities:
(1) In paragraph [0059] “may be not able to” is a misplaced negation. Proper English syntax requires the negation (“not”) to attach to the first finite verb of the clause. Thus, “not” must immediately follow “may.”
(2) In paragraph [0066] “meeting time has -determined” is missing a word (likely “meeting time has been determined”).
(3) The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. The following titles are suggested:
Task Processing Based on Target Application Availability
Determining Feedback Information for a Target Task Based on Application Availability
Digital Assistant Interaction and Feedback Generation Based on Target Application Availability
Appropriate correction is required.
Claim Objections
The present continuous tense of “causing” in the fourth line of claim 10 disagrees with the tense of “when executed.” The word “causing” should be replaced with “cause.”
Claim Interpretation
The broadest reasonable interpretation of the “computer readable storage medium” of claim 20 is that it is limited to non-transitory embodiments only, because claim 20 uses the reference-back word “The,” and points back to claim 19, thereby incorporating the “non-transitory” limitation from its parent claim by reference.
Claim Rejections – 35 U.S.C. § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4–10, and 13–19 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by U.S. Patent Application Publication No. 2019/0371329 A1 (“D’Souza”).
Claim 1
D’Souza discloses:
A method for task processing, comprising:
Reference is made to FIG. 10, which illustrates an anticipatory process 1050. See D’Souza ¶¶ 166–167.
determining a target task to be executed at a target device based on an interaction between a user and a digital assistant,
In response individual 2 speaking an utterance 1004 containing a task (“call me a car using Taxi”), “backend system 100 may generate text data representing the audio data, determine an intent of utterance 1004 from the text data, and may attempt to determine an application that can service utterance 1004.” D’Souza ¶ 166.
wherein the digital assistant runs on the target device;
As required by claim 1, the applications 262 that perform the task and the program modules 258, 260, 264 responsible for processing and responding to the utterance 1004 (i.e., the claimed digital assistant) are all executed by and run on the same device (backend system 100). See D’Souza ¶ 73 and FIG. 2A.
in response to determining that the target task is associated with a target application on the target device, determining availability of the target application, the availability of the target application indicating whether the digital assistant is permitted to execute the target task using the target application; and
The NLU module 260 of backend system 100 performs steps 1052–1054 of determining which applications are enabled for a particular user account, and which applications are not enabled for the account. D’Souza ¶ 167. For instance, continuing with the Taxi example, “[a]t step 1054, NLU module 260 may determine that the requested application, ‘Taxi,’ is not currently enabled for the user account.” D’Souza ¶ 167.
determining feedback information for the target task based on the determined availability of the target application.
“Due to the requested application not being currently enabled, at step 1056, text data of a response may be received by TTS module 264 from NLU module 260, where the response indicates that the requested application is not current enabled.” D’Souza ¶ 168.
Claim 4
D’Souza discloses the method of claim 1, wherein determining the availability of the target application comprises:
determining whether the target device has registered the target application with the digital assistant; and
“At step 1052, NLU module 260 may access user accounts module 268 to determine which applications are currently enabled for a particular user account. For example, user accounts module 268 may include an applications module 282, which stores a listing of all the applications that are enabled for each user profile stored within user accounts module 268.” D’Souza ¶ 167.
in response to determining that the target device has registered the target application with the digital assistant, determining the availability based on registration information of the target application.
For applications that are enabled, those applications register “various keywords and sentence frameworks” with NLU module 260 that NLU module 260 uses to route the user’s utterance to the registered application. D’Souza ¶¶ 151–153.
Claim 5
D’Souza discloses the method of claim 4,
wherein the target application comprises a plurality of functions, the registration information indicates one or more functions of the plurality of functions that are registered to the digital assistant,
Each of the applications may register multiple different “intents” with the system, each “intent” corresponding to a different function performed by the respective application. D’Souza ¶¶ 90, 99, 105, and 109.
and determining the availability based on the registration information for the target application comprises: determining, from the plurality of functions, a target function for executing the target task; and
“An intent classification (‘IC’) module 274 may parse the query to determine an intent or intents for each identified domain, where the intent corresponds to the action to be performed that is responsive to the query.” D’Souza ¶ 88.
in response to the registration information indicating that the target function is registered, determining that the digital assistant is permitted to execute the target task using the target application, or in response to the registration information indicating that the target function is not registered, determining that the digital assistant is not permitted to execute the target task using the target application.
“For instance, the user account associated with electronic device 10 may have three applications currently enabled, Skill 1 application 1022, Movie Trivia application 1024, and Calendar application 1026. At step 1054, NLU module 260 may determine that the requested application, ‘Taxi,’ is not currently enabled for the user account.” D’Souza ¶ 167. This causes process 1050 to engage in a process of helping the user enable the skill rather than executing the non-enabled skill. D’Souza ¶ 168.
Claim 6
D’Souza discloses the method of claim 4, further comprising:
triggering registration of the target application with the digital assistant in response to a predetermined event at the target device.
“[I]ndividual 2 may further say utterance 1008, ‘Alexa—Enable Taxi,’ which may cause the application, “Taxi,” to be enabled for the individual's user account on backend system 100.” D’Souza ¶ 168.
Claim 7
D’Souza discloses the method of claim 1, wherein determining the feedback information for the target task comprises:
in response to determining that the digital assistant is not permitted to execute the target task using the target application, generating prompt information that instructs the user to enable the target application for the digital assistant.
“Due to the requested application not being currently enabled, at step 1056, text data of a response may be received by TTS module 264 from NLU module 260, where the response indicates that the requested application is not current enabled. Furthermore, the response may also include an option for the individual operating the requesting device (e.g., individual 2 operating electronic device 10) to have the requested application enabled.” D’Souza ¶ 168.
Claim 8
D’Souza discloses the method of claim 1, wherein determining the feedback information for the target task comprises:
in response to determining that the digital assistant is permitted to execute the target task using the target application, determining guidance information for executing the target task based on the interaction.
In addition to determining the intent of or action corresponding to an utterance, NLU module 260 may further determine, e.g., from the same utterance, “the pertinent pieces of information in the text that allow an action to be completed.” D’Souza ¶ 82.
Claim 9
D’Souza discloses the method of claim 1,
wherein the target application is selected from a set of candidate applications.
“Each domain specific NLU pipeline will create its own domain specific NLU results, for example Results A for domain A 222, Results B for domain B 224, Results C for domain C 226, and so on. The different results may then be input into a domain ranking component 240, which may ranks the different results for the different domains, and selects what the system believes to be the most applicable results given the input text and other factors. Those highest ranking results may then be used to execute a command, perform one or more actions, or obtain information responsive to a user query, or otherwise respond to the input text.” D’Souza ¶ 108; see also D’Souza ¶ 128 (more explicitly describing ranking the applications responsive to an utterance).
Claims 10 and 13–18
Claims 10 and 13–18 are directed to an electronic device comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit that cause the electronic device to perform exactly the same method as recited in corresponding claims 1 and 4–9. D’Souza discloses the method of claims 1 and 4–9, and further discloses the electronic device comprising the processing unit, memory, and instructions for performing the same. See D’Souza ¶¶ 56–60 and 176.
Therefore, D’Souza anticipates claims 10 and 13–18 based on those findings.
Claim 19
Claim 19 recites a broader but fully encompassing version of the memory portion of the electronic device of claim 10. Therefore, claim 19 is rejected over all of the findings set forth in the rejection of claim 10 with respect to the memory and its program instructions.
Claim Rejections – 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were effectively filed absent any evidence to the contrary. Applicant is advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned at the time a later invention was effectively filed in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 2, 3, 11, 12, and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over D’Souza as applied to claims 1, 10, and 19 above, and further in view of U.S. Patent Application Publication No. 2016/0154792 A1 (“Sarikaya”).
Claim 2
D’Souza teaches the method of claim 1, and while process 300 of D’Souza’s disclosure is nearly identical to that of claim 2—i.e., they both involve a multi-turn dialog with the virtual assistant to determine the user’s ultimate intent—the end result of process 300 is to determine which application the user wishes to enable for the virtual assistant based on the multi turn dialog, rather than determining which application corresponds to the user’s command based on the multi turn dialog.
Sarikaya, however, teaches a method 400 for determining the target task to be executed at the target device, which comprises:
determining a type of task to be executed based on a first user input from the user;
“In one example, the first turn of a session may include the natural language expression ‘send a text message,’” and statistical system 100 may ascertain information from this first turn in the session, such as “predicting the slot type to be ‘text_message’ from the ‘send a text message’.” Sarikaya ¶ 50.
generating, according to the type, a response of the digital assistant to the first user input, the response instructing the user to provide additional information about the task to be executed;
“The response/action by the statistical system 100 after processing the natural language expression (e.g., as described above) may be, “and say what.’” Sarikaya ¶ 50.
receiving a second user input for the response; and
“In this regard, the second turn of the session may include a natural language expression such as, ‘what will the weather be like tomorrow,’” i.e., the user’s response to “and say what.” Sarikaya ¶ 50.
determining the target task based on the first user input and the second user input.
“The statistical system 100 may use the system response ‘and say what’ in addition to information from the first turn in the session (e.g., predicting the slot type to be ‘text_message’ from the ‘send a text message’ expression) to determine that the ultimate goal and/or intent of the user 102 is to send a text message asking what the weather will be like rather than incorrectly predicting the natural language expression ‘what will the weather be like tomorrow,’ to be a weather request for the statistical system 100.” Sarikaya ¶ 50.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to improve or supplement D’Souza’s NLP module(s) with Sarikaya’s statistical system 100 (or even simply the multi-turn model), thereby providing a way for the multi-turn dialog in D’Souza’s system to be used for determining actual commands to perform (rather than merely enabling the skills). One would have been motivated to combine Sarikaya with D’Souza because Sarikaya directly recognizes and solves a problem that arises in D’Souza’s disclosure of adding new skills to a virtual assistant: “as new spoken language inputs (e.g., queries) are received and new rules are written to handle new queries, the system becomes more complex, and in some cases, may cause already existing rules to be broken. Accordingly, embodiments described herein include contextual models for supporting and/or handling complicated multi-turn scenarios in contextual language understanding.” Sarikaya ¶ 17.
Claim 3
D’Souza and Sarikaya teach the method of claim 2, wherein determining the target task based on the first user input and the second user input comprises:
generating a first prompt and a second prompt based on the first user input and the second user input, respectively;
Statistical system 100 extracts each of the user’s responses as “contextual information” that is to be input into a multi-turn model. Sarikaya ¶¶ 50–51; see also Sarikaya ¶ 20 (“contextual information may include information extracted from each turn in a session. For example, the information extracted may include a domain prediction, an intent prediction, and slot types predicted (e.g., the results) from a previous turn (e.g., a previous natural language expression/request from the current session).”).
providing a combination of the first prompt and the second prompt to a machine learning model; and
“As discussed above, the components of the multi-turn model 140 may process the natural language expression using contextual information.” Sarikaya ¶ 51. Multi-turn model is a machine learning model because it is “traied over time,” e.g., from the various multi-turn sessions. Sarikaya ¶ 51; see also Sarikaya ¶ 17 (more explicitly defining the models as using “machine learning based techniques,” such as “artificial neural networks, Bayesian classifiers, and/or genetically derived algorithms”).
determining the target task based on a model output of the machine learning model for the first prompt and the second prompt.
As mentioned above, “[t]he statistical system 100 may use the system response ‘and say what’ in addition to information from the first turn in the session (e.g., predicting the slot type to be ‘text_message’ from the ‘send a text message’ expression) to determine that the ultimate goal and/or intent of the user 102 is to send a text message asking what the weather will be like rather than incorrectly predicting the natural language expression ‘what will the weather be like tomorrow,’ to be a weather request for the statistical system 100.” Sarikaya ¶ 50.
More concretely, the multi-turn model (and/or a combination of models including the multi-turn model) make a prediction about the ultimate goal of the user by calculating scores for potential matches. See Sarikaya ¶¶ 25–26.
Claims 11 and 12
Claims 11 and 12 are directed to an electronic device comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit that cause the electronic device to perform exactly the same method as recited in corresponding claims 2 and 3. D’Souza and Sarikaya teach the method of claims 2 and 3, and further teach the electronic device comprising the processing unit, memory, and instructions for performing the same. See D’Souza ¶¶ 56–60 and 176.
Therefore, claims 11 and 12 are obvious over the combination of D’Souza with Sarikaya for the reasons set forth above (taken in conjunction with the additional findings about the hardware).
Claim 20
Claim 20 recites a broader but fully encompassing version of the memory portion of the electronic device of claim 11. Therefore, claim 20 is rejected over all of the findings set forth in the rejection of claim 11 with respect to the memory and its program instructions, and the reasons to combine the references.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Justin R. Blaufeld whose telephone number is (571)272-4372. The examiner can normally be reached M-F 9:00am - 4:00pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James K Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Justin R. Blaufeld
Primary Examiner
Art Unit 2151
/Justin R. Blaufeld/Primary Examiner, Art Unit 2151
Prosecution Insights