DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the amendment filed on 11/24/2025. Claims 1, 8, 9, 16, 17, and 21-29 are pending in the case.
Applicant Response
In Applicant’s response dated 04/07/2025 Applicant amended Claims 1, 9 and 17 argued against all objections and rejections previously set forth in the Office Action dated 06/25/2025.
Continued Examination under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/24/2025 has been entered.
Examiner Comments
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8-9, 16-17, and 21-29 are rejected under 35 U.S.C. 103 as being unpatentable over Gruber (Pub. No.: US 20180350353 A1, Pub. Date: Dec. 6, 2018) in view of Khan (Pub. No.: US 20170228240 A1, Pub. Date: 2017-08-10) in further view of Agrawal (Pub. No.: US 20160225371 A1, Pub. Date: 2016-08-04) in further view of Malone (Pub. No.: US 20130329086 A1 Date Pub: 2013-12-12)
Regarding independent Claim 1,
Gruber teaches a computer-implemented method for performing digital assistant operations (see Abstract, illustrating multi-part voice command for a virtual assistant device), comprising:
receiving, by a digital assistant device, a command that includes a plurality of keywords (see Gruber: Fig.8, [0243], “speech input can be received from a user (e.g., from microphone 213 of FIG. 2). In some examples, the speech input can be directed to a virtual assistant, and can include one actionable command (e.g., “What's the weather going to be today?”) or multiple actionable commands (e.g., “Navigate to Jessica's house and send her a message that I'm on my way.”).” See also Fig.9, [0248], describing an example of a command that consist a plurality of keywords, “main keywords can be identified in the text string, and the string can be parsed based on their location.”)
receiving, by the digital assistant device, a first action dataset and a second action dataset from a server based on the received command (see Gruber: Fig.8 [0252], “first probability that the first candidate substring corresponds to a first actionable command (a first action dataset) and a second probability that the second candidate substring corresponds to a second actionable command (a second action dataset) can be determined.”, for example, as [0243] states - the digital assistant receives multiple actionable commands (e.g., “Navigate to Jessica's house and send her a message that I'm on my way, the first actionable command (a first action dataset) is the navigate to Jessica house and the second actionable command(second action dataset) is sending a message), wherein
the first action dataset is associated with a first mobile application installed on the digital assistant device (see Gruber: Fig.9, [0248], In the example of FIG. 9, the word “the word “navigate” can correspond to a virtual assistant domain for providing maps, turn-by-turn navigation instructions, or other navigation assistance to a user.”, i.e., navigation app is the first mobile application installed on the digital assistant, for example navigating to Jessica house and sending a message are the action datasets that are executed by the messaging app and the navigation app), and the second action dataset is associated with a second mobile application installed on the digital assistant device (see Gruber: Fig.9, [0248], “the word “send” can correspond to a messaging domain or multiple virtual assistant domains (e.g., email, instant messages, text messages, messaging applications, or the like)” i.e., email app is the second mobile application installed on the digital assistant device),
wherein each of the first action dataset and the second action dataset is generated by a command template that includes the plurality of keywords in the received command as parameters that correspond to the first mobile application or the second mobile application (see Gruber: Fig.9, [0248], “each candidate substring can be analyzed in view of user request templates of a virtual assistant. For example, a virtual assistant can process user requests in part by matching a spoken request to a template. The template can have associated processes, tasks, algorithms, or the like that a virtual assistant can use to handle a user's request. In addition, a template can have designated variables were entities, arguments, or the like can be expected (e.g., a contact name in the template “Call [contact]. Each candidate substring can thus be compared to some or all of the various templates of a virtual assistant.”)
wherein an action dataset is a data structure that includes a command template (see Gruber: Fig.9, [0248], a template can have designated variables were entities, arguments, or the like can be expected (e.g., a contact name in the template “Call [contact]. Each candidate substring can thus be compared to some or all of the various templates of a virtual assistant.”)
determining, by the digital assistant device, that the first action dataset is most [probable] command based on the first mobile application associated with the first action dataset, contextual information associated with a user of digital assistant device and a recently-opened application history of the digital assistant device (see Gruber: Fig.9, [0252], “at block 808, a first probability that the first candidate substring corresponds to a first actionable command and a second probability that the second candidate substring corresponds to a second actionable command can be determined. In some examples, each candidate substring can be analyzed to determine a probability that it corresponds to a valid, actionable command.” … [0297], “user data including contacts, preferences, location, and the like can be used to interpret voice commands. The various processes discussed herein can also be modified in various other ways according to user preferences, contacts, text, usage history, profile data, demographics, or the like ( contextual information) . In addition, such preferences and settings can be updated over time based on user interactions (e.g., frequently uttered commands, frequently selected applications, etc.). Gathering and use of user data that is available from various sources can be used to improve the delivery to users of invitational content or any other content that may be of interest to them.”)
in response to determining that the first action dataset is most [probable] to the received command (see Gruber: Fig.8, [0252], “a first probability that the first candidate substring corresponds to a first actionable command and a second probability that the second candidate substring corresponds to a second actionable command can be determined. In some examples, each candidate substring can be analyzed to determine a probability that it corresponds to a valid, actionable command.”), reproducing, by the digital assistant device, a set of touch operations utilizing the first mobile application based on instructions included in the first action dataset (see Gruber: Fig.8, [0281], “at block 814, a first process associated with the first intent and a second process associated with the second intent can be executed. With user intents determined for each candidate substring (or some candidate substrings), the processes associated with the user intents can be executed. For example, messages can be composed and sent, emails can be deleted, notifications can be dismissed, or the like (i.e., set of touch operations performed the first mobile applications). In some examples, multiple tasks or processes can be associated with individual user intents, and the various tasks or processes can be executed at block 814. In other examples, the virtual assistant can engage the user in a dialogue to acquire additional information as necessary for completing task flows.”).
wherein the instructions included in the first action dataset contain the recorded event sequence data that corresponds to the set of touch operations reproduced by the digital assistant device utilizing the first mobile application (see Gruber: Fig.7A, [0236], describing “task flow processing module 736 can proceed to perform the ultimate task associated with the actionable intent. Accordingly, task flow processing module 736 can execute the steps and instructions in the task flow model according to the specific parameters contained in the structured query. For example, the task flow model for the actionable intent of "restaurant reservation" can include steps and instructions for contacting a restaurant.”, i.e. recorded series of events associated with the task/action of reserving a restaurant are reproduced by the digital assistant device utilizing the mobile app.). see also [0281], stating “For example, messages can be composed and sent, emails can be deleted, notifications can be dismissed, or the like.”, i.e. sending email, reserving restaurant and/or invoking navigation task are set of touch operations that are reproduced by the digital assistance device.)
As shown above, Gruber teaches that combination of semantic coherence, template matching, and/or service testing to determine a probability that a candidate substring corresponds to an actionable command. In other words, Gruber teaches selecting the action dataset based on determined probability of the action data set to the received command. Examiner notes that Gruber teaches determining the most probable action command (action dataset) based on contextual information and historical interaction information.
Gruber does not teach the system wherein:
determining, by the digital assistant device, that the first action dataset is most relevant command based on the first mobile application associated with the first action dataset, contextual information associated with the digital assistant device and a recently-opened application history of the digital assistant device ; and
determining that the first action dataset is most relevant to the received command includes parsing the contextual information to identify keywords relevant to the plurality of keywords in the received command.
However, Khan teaches the digital assistant operations method comprising:
determining, by the digital assistant device, that the first action dataset is most relevant command (see Khan: Fig.2, [0048], “Analyzing the data store (operation 240) may include identifying one or more possible responses to the user's input, such as fetching the answer to the user's query. Analyzing the data store (operation 240) can also include identifying the most relevant response(s) for the particular user. User-specific information in the data store, such as search history, calendar, location, social media data, can be used to identify or rank the most relevant responses.”), based on the first mobile application associated with the first action dataset, contextual information associated with a user of digital assistant device and a recently-opened application history of the digital assistant device (see Khan: Fig.2, [0028], Services data 190 may also include the various applications (“apps”) on the device 104, Contextual data 184 include data related or specific to the received data input, Search data 186 may include search history associated with a user, an account, a profile, a device, etc.), wherein determining that the first action dataset is most relevant to the received command includes parsing the contextual information to identify keywords relevant to the plurality of keywords in the received command (see Khan: Fig.2, [0043], “operation 232 includes identifying what the input is requesting as well as identifying the context of the request. For example, the context is determined by identifying key words within the input. Then those key words are matched with words pre-associated with one or more contexts. Additionally, context may also include other aspects surrounding the input, such as the time of day, whether the user is travelling, the user's location, etc. Analyzing the text may also include semantically analyzing the text to determine the user's intent and/or the context of the input.” … “Analyzing the input may include using speech recognition to convert the speech to text. Then the text may be analyzed to determine the context of the input and/or semantically analyzed to determine an intent.”, i.e. semantically analyzing elements involves parsing/identifying features relevant to the received command ). Examiner notes that Khan teaches the system that receives user input and performs semantic analysis to determine context and intent, which involve extracting keywords or semantic tokens form the input and contextual information as shown in Figure 4, [0043].
Because Gruber and Khan are in the same/similar field of endeavor of digital assistant supported on a computing device and providing users with information, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Gruber to include the method that determine the most relevant action dataset or operation of digital assistant device based on contextual information that include mobile application associated with the action dataset and recently-opened application history of the digital assistant device and parsing the contextual information to identify keywords relevant to the plurality of keywords in the received command as taught by Khan. After modification of Gruber, the digital assistant device that selects the actionable command or the operation of the digital assistant based on probability of valid that the actionable command or operation of the digital assistant operation based on probably of valid actionable command can also determine most relevant operation or actionable command based on contextual information that comprise digital assistant device installed application associated with the action dataset and a recently-opened application and parsing the contextual information to identify keywords relevant to the plurality of keywords in the received command as taught by Khan. One would have been motivated to make such a combination in order to provide users with resource efficient user experience and contextually-relevant action and useful user interface applications.
Gruber and Khan does not teach or suggest the digital assistant operations method comprising an action dataset is a data structure that includes:
application identifying information,
recorded event sequence data that includes inputs and attributes for the inputs,
a hash value for the action dataset that is based on the action dataset
However, Agrawal teaches the method comprising an action dataset is a data structure that includes:
application identifying information (see Agrawal: Fig.5, [0064], discussing operation system information to location field has identifying information associated with the application, e.g., either OS or text messaging). See also Agrawal, Fig. 13, (location identifies Outlook), Agrawal does disclose that needing the correct version of the software is important and being able to determine whether the software is the current version and most updated. See e.g., [0123], discussing that when the user updates software, it will determine current version and bundle any updated voice commands with it).
recorded event sequence data that includes inputs and attributes for the inputs (see Agrawal: Fig.12, [0110], “the voice command generator module 344 determines one or more command terms and alternative terms for some or all of the viewable element manipulations represented in the viewable element manipulation sequence 352 in a manner similar to that described above with reference to block 414 of method 400 of FIG. 4. Further, at block 1126 the voice command generator module 344 determines a base command syntax in a manner similar to that described above with reference to block 416 of method 400.”
command templates (see Agrawal: Fig.9, [0088], “At block 906, the command mapper module 350 accesses a stored voice command set (template) for an operation.” … [0092], “The voice command will include variables (parameter) associated with command actions, and thus part of the emulation process includes injecting the variables at the appropriate point in the emulation process. To illustrate, a voice command of “send email to Sam Jones” would be recognized by the command mapper module 350 as having the recipient variable “Sam Jones”, and thus the manipulation emulator module 348 would emulate a user's manipulation of the to: field of the email message being composed by emulating keyboard input so as to insert the character string “Sam Jones” into the to: field and
Because Gruber, Khan and Agrawal are in the same/similar field of endeavor of digital assistants’ device and virtual assistants that provide an intuitive interface between users and electronic devices, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Gruber to include an action dataset is a data structure that includes application identifying information, and recorded event sequence data that includes inputs and attributes for the inputs,
as taught by Agrawal. One would be motivated to make such a combination to provide the advantage of adding additional translation information to provide for a flexible translation of commands.
Gruber, Khan and Agrawal does not teach the method comprising an action dataset is a data structure that includes a hash value for the action dataset that is based on the action dataset.
However, Malone teaches teach the method comprising an action dataset is a data
structure that includes a hash value for the action dataset that is based on the action dataset (see Malone: Fig.1, [0025], “Hashing 122 is a technique that generates a digital "signature" for a data set, such that any party wishing to verify the correctness of the data set can easily recalculate the hash and compares it with the previously calculated hash. Hash algorithms have the further property that it is computationally difficult to find multiple data sets that have the same hash value.”)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having Agrawal-Anupam-Malone before them to modify the datasets of Agrawal-Anupam to generate hashes as discussed by Malone. One would be motivated to do so for higher security and to be able to verify the correctness of the dataset. (see Malone: [0025].)
Regarding Claim 8,
Gruber, Gandhi, Agrawal, and Malone teaches all the limitations of Claim 1. Gandhi further teaches the method wherein the first action dataset is determined most relevant to the received command based further on an application usage history of the digital assistant device (see Khan: Fig.2, [0048], “Analyzing the data store (operation 240) may include identifying one or more possible responses to the user's input, such as fetching the answer to the user's query. Analyzing the data store (operation 240) can also include identifying the most relevant response(s) for the particular user. User-specific information in the data store, such as search history, calendar, location, social media data, can be used to identify or rank the most relevant responses.”),
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Gruber to include the method wherein the action dataset is determined most relevant to the received command based further on an application usage history of the digital assistant device as taught by Khan. After modification of Gruber, the digital assistant device that selects the actionable command or the operation of the digital assistant based on probability of valid that the actionable command or operation of the digital assistant operation based on probably of valid actionable command can also determine most relevant operation or actionable command based on contextual information that comprise digital assistant device installed application associated with the action dataset and a recently-opened application taught by Khan. One would have been motivated to make such a combination in order to provide users with resource efficient user experience and contextually-relevant action and useful user interface applications.
Regarding independent Claims 9 and 17,
Claims 9 is directed to a non-transitory computer storage medium and Claim 17 is directed to a system claim and the claim limitations have similar/same technical features as Claim 1 and are rejected under the same rationale.
Regarding Claim 16,
Claim 16 is directed to a non-transitory computer storage medium claim and the claim limitations has similar/same technical features as Claim 8 and is rejected under the same rationale.
Regarding Claim 21,
Gruber, Khan, Agrawal, and Malone teaches all the limitations of Claim 1. Gruber further teaches the method wherein the server is configured to select each of the first action dataset and the second action dataset from a plurality of action datasets based on a comparison of the command and a set of command templates in each action dataset of the plurality of action datasets (see Gruber: Fig.9, [0254] and [0259], “Instead of or in combination with semantic coherence, each candidate substring can be analyzed in view of user request templates of a virtual assistant. For example, a virtual assistant can process user requests in part by matching a spoken request to a template. The template can have associated processes, tasks, algorithms, or the like that a virtual assistant can use to handle a user's request. In addition, a template can have designated variables where entities, arguments, or the like can be expected (e.g., a contact name in the template “Call [contact].”). Each candidate substring can thus be compared to some or all of the various templates of a virtual assistant.”), … see Fig.012, [0279], “a user may be likely to issue a certain set of commands related to email management when viewing emails 1142 in email application 1250. For example, potential user requests related to email management can include replying, deleting, moving, filing, forwarding, marking as read, marking as unread, marking as spam, archiving, blocking a sender, and the like. When determining user intent for substrings, these potential user requests can be given additional weight or priority, or can be used as a comparison (e.g., comparing templates) to accurately interpret user commands. In particular, based on emails 1142 appearing on display 1030, potential user requests associated with email management can be used in interpreting user commands based on at least some likelihood that a user will issue commands associated with what is displayed.”)
Regarding Claim 22,
Gruber, Khan, Agrawal, and Malone teaches all the limitations of Claim 1. Gruber further teaches the method wherein the first action dataset is determined most relevant to the received command based further on a determination that at least a portion of location data associated with the digital assistant device corresponds to the command template of the first action dataset (see Gruber: Fig.9, [0256], “all templates can be used. In a virtual assistant, templates associated with providing maps, guidance, or navigation can include “Navigate to [location],” “Get me directions to [location],” “Show me a map of [location],” or the like. In comparing candidate substring 922 to such navigation-related templates, at least one matching template can be identified: “Navigate to [location],” where the location variable can allow the words “Jessica's house” to be ignored for purposes of comparison. In this example, at least one matching template can be identified, so a yes result or a high similarity score or probability can be produced.”)
Regarding Claim 23,
Gruber, Khan, Agrawal, and Malone teaches all the limitations of Claim 1. Gruber further teaches the method wherein the first action dataset is determined most relevant to the received command based further on a determination that at least a portion of message data associated with the digital assistant corresponds to the command template of the first action dataset (see Gruber: Fig.9, [0256], “The template can have associated processes, tasks, algorithms, or the like that a virtual assistant can use to handle a user's request. In addition, a template can have designated variables where entities, arguments, or the like can be expected (e.g., a contact name in the template “Call [contact].”). Each candidate substring can thus be compared to some or all of the various templates of a virtual assistant.”)
Regarding Claims 24, 25 and 26
Claims 24, 25, and 26 are directed to a non-transitory computer storage medium claim and the claim limitations have similar/same technical features as Claim 21-23 and are rejected under the same rationale.
Regarding Claims 27, 28 and 29
Claims 27, 28, and 29 are directed to a system claim and the claim limitations have similar/same technical features as Claim 21-23 and are rejected under the same rationale.
Response to Arguments
Claim Rejections - 35 U.S.C. § 103,
Applicant's prior art arguments with respect to the currently amended independent claims and the dependent claims have been fully considered but they are moot in view of the new grounds of rejection presented above. Applicant is respectfully referred to the complete rejections presented above and the newly cited portions of the references previously relied upon. Examiner further notes that Applicant’s arguments are mere allegations that the cited art does not teach the limitations of the independent claim as amended and do not explicitly show any deficiencies with the previously cited art of the record in relationship with the newly recited limitations.
Thus, Examiner respectfully reasserts that the combination of Gruber, Khan, Agrawal, and Malone sufficiently teaches all the limitations recited in the independent claims, as amended, and therefore the claims are still rejected under 35 U.S.C. 103 as being unpatentable over Gruber, Khan, Agrawal, and Malone.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PGPUB
NUMBER:
INVENTOR-INFORMATION:
TITLE / DESCRIPTION
US 9449105 B1
Taylor; David C.
Title: User-context-based Search Engine
Description: This invention relates to a data extraction tool and, more particularly, to novel systems and methods for searching, organizing, and presenting information stored in electronic format.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZELALEM W SHALU whose telephone number is (571)272-3003. The examiner can normally be reached M- F 0800am- 0500pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached on (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Zelalem Shalu/Examiner, Art Unit 2145
/CHAU T NGUYEN/Primary Examiner, Art Unit 2145