DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5 and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As per Claim 5 (and similarly claim 13):
It is not clear if “identified in the graph definition of the function” in lines 2-3 of claim 5 is supposed to refer to:
1. “the requested inputs” (in which case “the requested inputs identified in the graph definition of the function” lacks antecedent basis because, in claim 1, it is the parameters that are “identified in the graph definition” and “the requested inputs” correspond to the parameters, but are not, themselves, identified in the graph definition)
Or
2. “each respective input” (in which case there is no antecedent basis issue for “the requested inputs identified in the graph definition of the function”, but this is not necessarily the most intuitive interpretation of the claim language because “identified in the graph definition of the function” appears immediately after “the requested inputs”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 3, 4, 7, 9, 10, 11, 12, 15, 17, and 18, is/are rejected under 35 U.S.C. 103 as being unpatentable over Bharadwaj (US 2020/0258509).
As per Claim 1 (and similarly claim 9):
Bharadwaj suggests A method, comprising: receiving a request to execute a function in a software application through a conversational user interface; retrieving, from a knowledge engine, a graph definition of the function, the graph definition of the function identifying parameters used in executing the function; iteratively requesting input through the conversational user interface for each parameter of the parameters identified in the graph definition of the function based on a traversal of the graph definition of the function; determining that the requested inputs corresponding to the parameters identified in the graph definition of the function have been provided through the conversational user interface; and executing the function using the requested inputs as parameters for executing the function (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9; [all paragraphs and Figures are cited for each limitation with “key” paragraphs and Figures pertaining to each limitation identified below, i.e. all other paragraphs and Figures not specifically referenced for any particular limitation are eligible to provide context and additional support]
“A method, comprising: receiving a request to execute a function in a software application through a conversational user interface;”: paragraphs 1, 3, 36, 56, 112-114, 129, 131-133, and 135; Figures 6-9; Figure 6 depicts a process/”method” that includes, among other things, “recei[ving] user input” [element 302], where Figures 7-9 and paragraph 113 [particularly the “top” user input in each of Figures 7-9] describe examples of user input which are each “a request to” perform/”execute” an action/”function”, where the action/”function” can be interpreted as “a function in a software application” in the sense that it is an action/”function” that an automated assistant “software application” [see paragraph 1] is capable of performing [such that the “software application” logically contains the programming/software that enables the automated assistant “software application” to perform/”execute” the action/”function”], and where the user’s request/input can be interpreted as being “receiv[ed]… through a conversational user interface” [i.e. received through “user interface” which allows the automated assistant to receive natural language inputs from the user and to communicate natural language outputs/prompts to the user, thereby conducting a “conversation” between the user and the automated assistant]. For claim 9, paragraphs 129, 131-133, 135 [which describe generic computer elements and functions including a processor, a memory, and software modules executed by a processor, and different types of generic computer devices] and Figures 7-9 [which appear to depict a user interacting with a user device that is performing the dialog functions of Figure 6] suggests a corresponding generic computer system embodiment that reads on claim 9. Additionally/alternatively [relevant for claims 7 and 15], a text equivalent of a user’s spoken request received as a result of converting the user’s spoken request into text via speech/voice recognition [see paragraphs 1, 3, 36, and 112] can be interpreted as “a request to execute a function in a software application” which is “receiv[ed]… through a conversational user interface” which includes speech/voice recognition.
“retrieving, from a knowledge engine, a graph definition of the function, the graph definition of the function identifying parameters used in executing the function;”: paragraphs 56, 61, 71-75, 79-82, 95-101, 107-110, 113-114, and 117; Figures 2 and 6-9; Paragraph 61 describes accessing dependency graph data structures which are associated with different actions/intents, and which are stored in a database by a dialog engine, paragraph 113 describes where a user’s input is determined to be associated with an action that has an associated dependency graph data structure, paragraphs 71-73, 79-82, and 95-101 describe where a dependency graph data structure comprises nodes, including nodes that identify “an assistant method that collects one or more parameters” by prompting a user to select/specify information for node-specific parameters, and paragraphs 107-110 describe where determining whether any parameters needed to perform the action are unset/remain-to-be-determined. These portions suggest where “a graph definition of the function” [i.e. a dependency graph data structure corresponding to the action/”function” requested by a user’s input, where the data that forms the dependency graph data structure’s data can be interpreted as a “definition” of what forms the dependency “graph” data structure] is accessed [suggested to be “retrieved” from the database in which the dependency graph data structure is stored so that the dependency graph data structure can be used] from the collective set of the dialog engine and the database that stores the dependency graph data structures [where the collective set of the dialog engine and the database can be interpreted as a “knowledge engine” that contains dialog and dependency graph data structure “knowledge”], and where the accessed/”retrieved” dependency graph data structure includes assistant method nodes “identifying parameters used in executing the function” [e.g. date, time, party size parameters needed to “book a table at O’Briens”/make-a-reservation in the example of paragraphs 71-74 and 79-82, where a reservation booking action/”function” logically cannot be properly “executed” without at least specified date and time parameters, and see also paragraphs 75, 107, and 117 which describe where parameters are “necessary for”/”needed to”, and are thus at least suggested to be used for a Reserve action method that makes a final reservation], where, for the purposes of this prior art rejection, “parameters”/”the parameters identified in the graph definition” are mapped to a plurality of unset parameters that are identified in a dependency graph data structure and that are needed to perform/”execute” the action/”function”.
“iteratively requesting input through the conversational user interface for each parameter of the parameters identified in the graph definition of the function based on a traversal of the graph definition of the function;”: Figures 2 and 6-9; paragraphs 71-74, 79-82, 95-101, 105, and 107-110; As just discussed, for the purposes of this prior art rejection, “parameters”/”the parameters identified in the graph definition” are mapped to a plurality of unset parameters that are identified in a dependency graph data structure and that are needed to perform/”execute” the action/”function”. Paragraphs 107-110 and Figures 6-9 suggest where a user is “iteratively” asked/”requested” for inputs that set/specify values/information for unset parameters corresponding to assistant method nodes in the dependency graph data structure [i.e. where each “iteration” asks the user for an input corresponding to an unset one or more parameters, where the “one” embodiment of “one or more parameters” suggests an embodiment where every individual parameter has its own “iteration”] until there are no remaining unset parameters. Figures 7-9 depict where the user inputs are requested via a dialog/”conversation” with the user that starts with a user requesting an action/”function” [suggesting that the requests for user input are made through “the conversational user interface” through which the “request to execute a function” is “received”]. Paragraphs 105 and 113-114 describe where a dependency graph data structure is “traversed”, where determining that no input data relating to a parameter has been provided is done “by accessing and traversing the dependency graph data structure”, and where a user is prompted for a parameter for which no input data has been provided, which suggests where the iterative requesting of inputs from the user for the unset parameters is “based on a traversal of the graph definition of the function”.
“determining that the requested inputs corresponding to the parameters identified in the graph definition of the function have been provided through the conversational user interface; and executing the function using the requested inputs as parameters for executing the function”: Figures 2 and 6-9; paragraphs 71-75, 79-82, 95-101, 107-110, 117, and 120-121; Paragraph 110 particularly describes where a final action method is executed “once all parameters have been set” and where one example of executing the final action method “us[es] the parameters determined during the dialog session as inputs” and Figure 6 depicts where a final action method is executed in response to a “no” decision made during “Any unset parameters?” in block 312. Figures 7-9 and paragraph 110 also describe where a response which reports the outcome of an action is provided to the user, and paragraphs 120-121 describe where a BuyTicket action is executed based on a number of tickets specified by a user’s input, where it is at least suggested that the previously-unset parameter values are used to perform the action because the values identify critical information needed to properly fulfill the user’s request [see also paragraphs 75, 107, and 117 which describe where parameters are “necessary for”/”needed to”, and are thus at least suggested to be used for a Reserve action method that makes a final reservation]. These portions suggest where the automated assistant “determin[es] that the requested inputs corresponding to the parameters identified in the graph definition of the function have been provided through the conversational user interface” [i.e. determines that the user, through the conversational user interface that allows the user and the automated assistant to conduct a dialog/”conversation”, has provided all of the input values/information that the automated assistant requested, where the input values/information correspond to the unset parameters that correspond to assistant method nodes in the dependency graph data structure corresponding to the action requested by the user] and where “the function” is “execut[ed]…using the requested inputs as parameters for executing the function” [e.g. where the purchasing of tickets in Figure 9 uses the “three” quantity to purchase 3 tickets, and where the reservation in Figure 7 is made using the “four” value for the “how many people?” parameter and using the “Saturday” and “8:00pm” values for the date and time parameters, such that the system performs the action in accordance with the criteria specified by the user])
As per Claim 17:
Bharadwaj suggests A method, comprising: receiving a request to execute a function in a software application through a graphical conversational user interface; retrieving, from a knowledge engine, a graph definition of the function, the graph definition of the function identifying parameters used in executing the function; retrieving, from the knowledge engine, a user interface definition for the function; iteratively requesting input through the graphical conversational user interface for each parameter of the parameters identified in the graph definition of the function based on (1) a traversal of the graph definition of the function and (2) the user interface definition for the function; determining that the requested inputs corresponding to the parameters identified in the graph definition of the function have been provided through the graphical conversational user interface; executing the function using the requested inputs as parameters for executing the function; and transmitting, to the graphical conversational user interface, a user interface definition including a result of executing the function (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9; [all paragraphs and Figures are cited for each limitation with “key” paragraphs and Figures pertaining to each limitation identified below, i.e. all other paragraphs and Figures not specifically referenced for any particular limitation are eligible to provide context and additional support]
“A method, comprising: receiving a request to execute a function in a software application through a graphical conversational user interface;”: paragraphs 1, 56, 109, 113-114, 129, 131-133, and 135; Figures 6-9; Figure 6 depicts a process/”method” that includes, among other things, “recei[ving] user input” [element 302], where Figures 7-9 and paragraph 113 [particularly the “top” user input in each of Figures 7-9] describe examples of user input which are each “a request to” perform/”execute” an action/”function”, where the action/”function” can be interpreted as “a function in a software application” in the sense that it is an action/”function” that an automated assistant “software application” [see paragraph 1] is capable of performing [such that the “software application” logically contains the programming/software that enables the automated assistant “software application” to perform/”execute” the action/”function”], and where the user’s request/input can be interpreted as being “receiv[ed]… through a conversational user interface” [i.e. received through “user interface” which allows the automated assistant to receive natural language inputs from the user and to communicate natural language outputs/prompts to the user, thereby conducting a “conversation” between the user and the automated assistant]. For claim 13, paragraph 109 and Figures 7-8 [which appear to depict a user device’s display of a conversation] describe where prompts/responses/natural-language-outputs are presented in visual form [suggesting an embodiment where the “conversational user interface” is “graphical”].
“retrieving, from a knowledge engine, a graph definition of the function, the graph definition of the function identifying parameters used in executing the function;”: paragraphs 10, 56, 61, 71-75, 79-82, 88-89, 95-101, 107-110, 113-114, and 117; Figures 2, 4, 3E-3G, 5E-5I, and 6-9; Paragraph 61 describes accessing dependency graph data structures which are associated with different actions/intents, and which are stored in a database by a dialog engine, paragraph 113 describes where a user’s input is determined to be associated with an action that has an associated dependency graph data structure, paragraphs 71-73, 79-82, and 95-101 describe where a dependency graph data structure comprises nodes, including nodes that identify “an assistant method that collects one or more parameters” by prompting a user to select/specify information for node-specific parameters, and paragraphs 107-110 describe where determining whether any parameters needed to perform the action are unset/remain-to-be-determined. These portions suggest where “a graph definition of the function” [i.e. a dependency graph data structure corresponding to the action/”function” requested by a user’s input, where the data that forms the dependency graph data structure’s data can be interpreted as a “definition” of what forms the dependency “graph” data structure] is accessed [suggested to be “retrieved” from the database in which the dependency graph data structure is stored so that the dependency graph data structure can be used] from the collective set of the dialog engine and the database that stores the dependency graph data structures [where the collective set of the dialog engine and the database can be interpreted as a “knowledge engine” that contains dialog and dependency graph data structure “knowledge”], and where the accessed/”retrieved” dependency graph data structure includes assistant method nodes “identifying parameters used in executing the function” [e.g. date, time, party size parameters needed to “book a table at O’Briens”/make-a-reservation in the example of paragraphs 71-74 and 79-82, where a reservation booking action/”function” logically cannot be properly “executed” without at least specified date and time parameters, and see also paragraphs 75, 107, and 117 which describe where parameters are “necessary for”/”needed to”, and are thus at least suggested to be used for a Reserve action method that makes a final reservation], where, for the purposes of this prior art rejection, “parameters”/”the parameters identified in the graph definition” are mapped to a plurality of unset parameters that are identified in a dependency graph data structure and that are needed to perform/”execute” the action/”function”.
“retrieving, from the knowledge engine, a user interface definition for the function;”: paragraphs 10, 56, 61, 71-75, 79-82, 88-89, 95-101, 107-110, 113-114, and 117; Figures 2, 4, 3E-3G, 5E-5I, and 6-9; Figures 2 and 4 are described [by paragraphs 71 and 89] as depicting dependency graph data structures, paragraph 73 describes where “Each node… identifies an action method… or an assistant method”, Figures 3E-3G and 5E-5I are described [by paragraphs 88 and 97-101] as specifications for assistant method nodes of the dependency graph data structures in Figures 2 and 4, Figures 3E-3G and 5E-5I also depict respective prompts to ask the user for respective parameters, and paragraph 75, 107, and 117 describes where parameters can be needed for performing/executing an action. These portions also suggest an embodiment where the dependency graph data structures include assistant method nodes which identify [paragraph 73, i.e. identify but do not include] assistant method specifications [since Figures 3E-3G and 5E-5I which depict assistant method specifications in separate Figures relative to Figures 2 and 4, and whose “names” are referenced in nodes of Figures 2 and 4], and where the assistant method specifications are also stored by the “knowledge engine” [the collective-set-of-the-dialog-engine-and-the-database/“knowledge engine”, as discussed above, since the assistant method specifications contain prompt information “used to manage the conduct of a conversation” and used “to control the interaction between a user and an automated assistant” which, as per paragraph 61, are functions that the dialog engine is used to perform] and are “retrieved” from where they are stored when they need to be used. These portions thus suggest “retrieving, from the knowledge engine” [retrieving from the assistant method specifications’ storage location[s] in the collective set of the dialog engine and the database, where the dialog engine is suggested to use the assistant method specifications to conduct a conversation with the user] “a user interface definition for the function” [the collective set of assistant method specifications identified by the dependency graph data structure corresponding to the action/”function” requested by the user input, which “define” how the system is to “interface” with the “user” in order to obtain parameter[s] and which are necessary “for” performing/executing “the function”]
“iteratively requesting input through the graphical conversational user interface for each parameter of the parameters identified in the graph definition of the function based on (1) a traversal of the graph definition of the function and (2) the user interface definition for the function;”: paragraphs 10, 56, 61, 71-75, 79-82, 88-89, 95-101, 105, 107-110, 113-114, and 117; Figures 2, 4, 3E-3G, 5E-5I, and 6-9; As discussed above, for the purposes of this prior art rejection, “parameters”/”the parameters identified in the graph definition” are mapped to a plurality of unset parameters that are identified in a dependency graph data structure and that are needed to perform/”execute” the action/”function”. Paragraphs 107-110 and Figures 6-9 suggest where a user is “iteratively” asked/”requested” for inputs that set/specify values/information for unset parameters corresponding to assistant method nodes in the dependency graph data structure [i.e. where each “iteration” asks the user for an input corresponding to an unset one or more parameters, where the “one” embodiment of “one or more parameters” suggests an embodiment where every individual parameter has its own “iteration”] until there are no remaining unset parameters. Figures 7-9 depict where the user inputs are requested via a dialog/”conversation” with the user that starts with a user requesting an action/”function” [suggesting that the requests for user input are made through “the conversational user interface” through which the “request to execute a function” is “received”, where paragraph 109 and Figures 7-8 suggest an embodiment where the “conversational user interface” is “graphical”]. Paragraphs 105 and 113-114 describe where a dependency graph data structure is “traversed”, where determining that no input data relating to a parameter has been provided is done “by accessing and traversing the dependency graph data structure”, and where a user is prompted for a parameter for which no input data has been provided, which suggests where the iterative requesting of inputs from the user for the unset parameters is “based on a traversal of the graph definition of the function”. Also, as discussed above, the collective set of assistant method specifications identified by the dependency graph data structure corresponding to the action/”function” requested by the user input, which “define” how the system is to “interface” with the “user” in order to obtain parameter[s] and which are necessary “for” performing/executing “the function”, can be interpreted as “the user interface definition for the function” and since the assistant method specifications in Figures 3E-3G, 5E-5I include a prompt which is suggested to define how the automated assistant asks the user for a parameter [see e.g. Figure 3F which includes a “how many people” prompt and the “Sure, for how many people?” communication by the automated assistant in Figure 7], the “iterative requesting” is also suggested to be “based on… the user interface definition for the function” [i.e. the user is asked for parameters according to the prompt information in the collective set of assistant method specifications identified by the dependency graph data structure corresponding to the action/”function” requested by the user input]
“determining that the requested inputs corresponding to the parameters identified in the graph definition of the function have been provided through the graphical conversational user interface; executing the function using the requested inputs as parameters for executing the function; and transmitting, to the graphical conversational user interface, a user interface definition including a result of executing the function”: Figures 2, and 6-9; paragraphs 71-75, 79-82, 95-101, 107-110, 117, and 120-121; Paragraph 110 particularly describes where a final action method is executed “once all parameters have been set” and where one example of executing the final action method “us[es] the parameters determined during the dialog session as inputs” and Figure 6 depicts where a final action method is executed in response to a “no” decision made during “Any unset parameters?” in block 312. Figures 7-9 and paragraph 110 also describe where a response which reports the outcome of an action is provided to the user, and paragraphs 120-121 describe where a BuyTicket action is executed based on a number of tickets specified by a user’s input, where it is at least suggested that the previously-unset parameter values are used to perform the action because the values identify critical information needed to properly fulfill the user’s request [see also paragraphs 75, 107, and 117 which describe where parameters are “necessary for”/”needed to”, and are thus at least suggested to be used for a Reserve action method that makes a final reservation]. These portions suggest where the automated assistant “determin[es] that the requested inputs corresponding to the parameters identified in the graph definition of the function have been provided through the conversational user interface” [i.e. determines that the user, through the conversational user interface that allows the user and the automated assistant to conduct a dialog/”conversation” and that is suggested to be “graphical” by paragraph 109 and Figures 7-8, has provided all of the input values/information that the automated assistant requested, where the input values/information correspond to the unset parameters that correspond to assistant method nodes in the dependency graph data structure corresponding to the action requested by the user], where “the function” is “execut[ed]…using the requested inputs as parameters for executing the function” [e.g. where the purchasing of tickets in Figure 9 uses the “three” quantity to purchase 3 tickets, and where the reservation in Figure 7 is made using the “four” value for the “how many people?” parameter and using the “Saturday” and “8:00pm” values for the date and time parameters, such that the system performs the action in accordance with the criteria specified by the user], and where “a user interface definition including a result of executing the function” is “transmit[ed], to the graphical conversational user interface” [i.e. where the words of a response which reports the outcome of an action is transmitted to and displayed through the graphical conversational user interface to the user, where the words of the response which reports the outcome are “define” how to “interface” with the “user” to report the outcome and “includ[e] a result of executing the function”, e.g., “Your reservation is confirmed” in Figure 7 is a result of executing the reservation action/”function”])
As per Claim 2 (and similarly claims 10 and 18):
Bharadwaj suggests wherein the graph definition of the function comprises a plurality of nodes, each respective node being associated with a respective input parameter for the function and a respective user interface control to display in the [graphical] conversational user interface to obtain a value for the respective input parameter (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9;
Figures 2 and 4 and paragraphs 71, 73, 79-82, 89, and 95-101 describe where a dependency graph data structure [i.e. “the graph definition of the function”] includes “a plurality of nodes” that each correspond to “a respective input parameter” and that each identify a corresponding assistant method. Paragraphs 75, 107, 110, and 117 and Figures 6-9 describe where parameters are unset and are used/needed to perform an action [such that the parameters are “for the function” requested by the user]. Figures 2, 4, 6-9, 3E-3G and 5E-5I, and paragraphs 79-82 and 95-101 describe where the assistant method specifications for the assistant methods identified by the assistant method nodes each include prompt information that “controls” how the automated assistant “interfaces” with the “user” to ask for, and thereby “to obtain a value for the respective input parameter” [i.e. each assistant method specification corresponds to an assistant method identified by a respective node and is used to prompt the user for a value for the parameter corresponding to the respective node]. Paragraph 109 and Figures 7-8 describe an embodiment where the prompts for parameters can be in visual [i.e. “displayed”] form [and thereby suggest an embodiment where the prompt/”user interface control” is “to be displayed”, and for claim 13, where the “conversational user interface” is “graphical”].)
As per Claim 3 (and similarly claim 11):
Bharadwaj suggests wherein: each respective node of the plurality of nodes further indicates whether the respective input parameter is a mandatory input for the function, and iteratively requesting input through the conversational user interface for parameters identified in the graph definition of the function comprises repeating requests for mandatory inputs for the function until the mandatory inputs are provided through the conversational user interface (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9;
Paragraph 102 describes where parameters may be mandatory or may be optional, paragraph 73, 79-82, and 95-101, and Figures 2 and 4 describe where nodes of a dependency graph data structure contain parameter-specific information [including what parameter to obtain and an assistant method identification that identifies an assistant method used to prompt the user for the parameter], paragraphs 107 and 117 describe where parameters may be “needed to perform the action”/”necessary to execute”-an-action, and paragraphs 107-110 and Figure 6 describes prompting a user for parameters until there are no more unset parameters. These portions suggest an embodiment where the nodes which each contain parameter-specific information also each contain a designation of whether the node’s corresponding parameter is mandatory or optional, and also suggest where, if the user does not provide a mandatory parameter that he/she is prompted/”requested” to provide, a prompt/”request” for the value is “repeated” [because if the user does not provide the mandatory parameter, then the parameter remains “unset” which would lead block 312 of Figure 6 to prompt the user again until the “Any unset parameters?” decision in block 312 is “no”, and because an action that needs the mandatory parameter logically cannot be performed/executed without the user providing mandatory parameter through “the conversational user interface”]).
As per Claim 4 (and similarly claim 12):
Bharadwaj suggests retrieving, from the knowledge engine, a user interface definition for the function (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9;
Figures 2 and 4 are described [by paragraphs 71 and 89] as depicting dependency graph data structures, paragraph 73 describes where “Each node… identifies an action method… or an assistant method”, Figures 3E-3G and 5E-5I are described [by paragraphs 88 and 97-101] as specifications for assistant method nodes of the dependency graph data structures in Figures 2 and 4, Figures 3E-3G and 5E-5I also depict respective prompts to ask the user for respective parameters, and paragraph 75, 107, and 117 describes where parameters can be needed for performing/executing an action. These portions also suggest an embodiment where the dependency graph data structures include assistant method nodes which identify [paragraph 73, i.e. identify but do not include] assistant method specifications [since Figures 3E-3G and 5E-5I which depict assistant method specifications in separate Figures relative to Figures 2 and 4, and whose “names” are referenced in nodes of Figures 2 and 4], and where the assistant method specifications are also stored by the “knowledge engine” [the collective-set-of-the-dialog-engine-and-the-database/“knowledge engine”, as discussed above, since the assistant method specifications contain prompt information “used to manage the conduct of a conversation” and used “to control the interaction between a user and an automated assistant” which, as per paragraph 61, are functions that the dialog engine is used to perform] and are “retrieved” from where they are stored when they need to be used. These portions thus suggest “retrieving, from the knowledge engine” [retrieving from the assistant method specifications’ storage location[s] in the collective set of the dialog engine and the database, where the dialog engine is suggested to use the assistant method specifications to conduct a conversation with the user] “a user interface definition for the function” [the collective set of assistant method specifications identified by the dependency graph data structure corresponding to the action/”function” requested by the user input, which “define” how the system is to “interface” with the “user” in order to obtain parameter[s] and which are necessary “for” performing/executing “the function”])
As per Claim 7 (and similarly claim 15):
Bharadwaj suggests wherein the request to execute the function comprises a request received through a voice recognition engine (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9;
As discussed in the rejection of claims 1 and 9, above:
Additionally/alternatively [relevant for claims 7 and 15], a text equivalent of a user’s spoken request received as a result of converting the user’s spoken request into text via speech/voice recognition [see paragraphs 1, 3, 36, and 112] can be interpreted as “a request to execute a function in a software application” which is “receiv[ed]… through a conversational user interface” which includes speech/voice recognition.
Paragraphs 1, 3, 36, and 112, and Figure 9 thus suggest an embodiment where “the request to execute the function” is a text equivalent of a spoken request [also “a request”] which is “received through a voice recognition engine” [i.e. received as a result of converting the spoken request into the text equivalent of the spoken request, where, given the context of paragraph 3 and the lack of description of identifying who is speaking rather than identifying what is being spoken, voice recognition appears to be used, in this reference, as a synonym for “speech recognition”])
Claim(s) 6, 14, and 19, is/are rejected under 35 U.S.C. 103 as being unpatentable over Bharadwaj, as applied to Claims 1, 9, and 17, above, and further in view of Binder et al. (US 2014/0222436), hereafter Binder, and Daly et al. (US 2019/0228098), hereafter Daly
As per Claim 6 (and similarly claims 14 and 19):
Bharadwaj suggests wherein executing the function using the requested inputs comprises executing the function based on the requested inputs… (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9;
Paragraph 110 particularly describes where a final action method is executed “once all parameters have been set” and where one example of executing the final action method “us[es] the parameters determined during the dialog session as inputs” and Figure 6 depicts where a final action method is executed in response to a “no” decision made during “Any unset parameters?” in block 312. Figures 7-9 and paragraph 110 also describe where a response which reports the outcome of an action is provided to the user, and paragraphs 120-121 describe where a BuyTicket action is executed based on a number of tickets specified by a user’s input, where it is at least suggested that the previously-unset parameter values are used to perform the action because the values identify critical information needed to properly fulfill the user’s request [see also paragraphs 75, 107, and 117 which describe where parameters are “necessary for”/”needed to”, and are thus at least suggested to be used for a Reserve action method that makes a final reservation]. These portions suggest where “the function” is “execut[ed]…using the requested inputs as parameters for executing the function”/”execut[ed]… based on the requested inputs” [e.g. where the purchasing of tickets in Figure 9 uses the “three” quantity to purchase 3 tickets, and where the reservation in Figure 7 is made using the “four” value for the “how many people?” parameter and using the “Saturday” and “8:00pm” values for the date and time parameters, such that the system performs the action in accordance with the criteria specified by the user])
Bharadwaj does not, but Binder suggests wherein executing the function using the requested inputs comprises executing the function based on the requested inputs and a…graph defining an order of operations for the function (paragraphs 42-43, 78, 92, 94-95, 99;
Binder, like Bharadwaj, describes an assistant system which determines a user’s intent from spoken/textual natural language input and performs a corresponding action/function to satisfy the user’s request [paragraphs 42-43] and describes asking a user for additional information needed to perform an action like a restaurant reservation [paragraphs 94-95, 99]. Binder further at least suggests where speech-recognized words are associated with an “actionable intent” which “represents a task that can be performed by the digital assistant… and has an associated task flow”, where the “associated task flow is a series of programmed actions and steps that the digital assistant… takes in order to perform the task” [paragraph 78], where the actionable intent is represented by a structured query which includes parameters for one or more nodes and where necessary parameters are not initially specified and are requested from the user [paragraphs 92 and 94-95] and where performing a task associated with an actionable intent includes performing a plurality of steps based on parameters in the structured query [paragraph 99], where the steps of the task flow model are suggested to be steps of a task flow [since the steps in paragraph 99 are “take[n] in order to perform the” restaurant reservation “task”, see paragraph 78], and where the nature of the steps suggests that the steps are performed in the order that they are numbered in paragraph 99 [at a minimum, a form cannot or should not be submitted in step 3 until after the parameter information is entered into the form in step 2, and entering information into a form on a website in step 2 logically cannot be performed prior to logging onto a server hosting the website in step 1]. Binder thus suggests where executing a task/”function” based on requested inputs is based on a task flow which defines a sequence/”order” of operations used to perform the task/”function”, where the task flow can be interpreted as a task’s “graph” since it includes an ordered sequence of operations [similar to the dependency graphs in Bharadwaj].
Binder thus suggests “wherein executing the function using the requested inputs comprises executing the function based on the requested inputs and a…graph defining an order of operations for the function”: where Bharadwaj’s system performs/executes an action/”function” by using the assistant method parameters obtained by prompting the user, and by performing a sequence of operations whose order is defined in a task flow/”graph” that specifies operations used to perform the action/”function” and an order of the operations).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of effective filing to perform a simple substitution of one type of action performance with another because the prior art teaches the claimed invention except for the substitution of action performance which does not necessarily perform an action based on a graph defining an order of operations for the action with action performance which does. Binder suggests that action performance which does performs an action based on a graph defining an order of operations for the action was known in the art. One of ordinary skill in the art could have substituted one type of action performance with another to obtain the predictable results of an automated assistant which receives a user input requesting an action to be performed, determines the action to be performed from the user input, uses a dependency graph data structure associated with the determined action to request parameters needed to perform the action from the user, and which performs the determined action using the requested parameters (as per Bharadwaj) where the action is performed based on a task flow which specifies a sequence of operations for performing the action (as suggested by Binder).
Bharadwaj, in view of Binder, do not, but Daly suggests wherein executing the function using the requested inputs comprises executing the function based on the requested inputs and a calculation graph defining an order of operations for the function (paragraph 17;
Daly teaches where a user’s question includes parameters [similar to requests in Bharadwaj and Binder] and more specifically describes where the user’s request asks for a mathematical calculation which appears to be sufficiently complex as to require multiple steps/calculations [paragraph 17, e.g., subtracting the down payment from the item price, performing the APR calculations, etc.].
Daly thus suggests “wherein executing the function using the requested inputs comprises executing the function based on the requested inputs and a calculation graph defining an order of operations for the function”: where the user’s request in the Bharadwaj/Binder combination is for a mathematical calculation action/task and where the task flow [which can be interpreted as a “graph” which contains a sequence/”order” of operations, as discussed above in the portion of this rejection of claims 6, 14, and 19 based on Binder] used to perform the mathematical calculation action/task is a mathematical “calculation” task-flow/”graph”).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of effective filing to perform a simple substitution of one type of user-requested action with another because the prior art teaches the claimed invention except for the substitution of a user-requested action which is not necessarily a mathematical calculation with a user-requested action which is. Daly teaches that a user-requested action which is a mathematical calculation was known in the art. One of ordinary skill in the art could have substituted one type of user-requested action with another to obtain the predictable results of an automated assistant which receives a user input requesting an action to be performed, determines the action to be performed from the user input, uses a dependency graph data structure associated with the determined action to request parameters needed to perform the action from the user, and which performs the determined action using the requested parameters (as per Bharadwaj) where the action is performed based on a task flow which specifies a sequence of operations for performing the action (as suggested by Binder) where the action asks for a mathematical calculation (as per Daly).
Claim(s) 8, 16, and 20, is/are rejected under 35 U.S.C. 103 as being unpatentable over Bharadwaj, as applied to Claims 1, 9, and 17, above, and further in view of Kumar et al. (US 2018/0137401), hereafter Kumar.
As per Claim 8 (and similarly claims 16 and 20):
Bharadwaj suggests wherein receiving the request to execute the function through the [graphical] conversational user interface comprises: receiving an utterance through the [graphical] conversational user interface;…and identifying the function based on… in the [graphical] conversational user interface (paragraphs 1, 3, 7, 10-12, 36, 56, 61, 71-75, 79-82, 88-89, 95-103, 105, 107-110, 112-117, 120-121, 129, 131-133, 135; Figures 2, 4, 3E-3G, 5E-5I, and 6-9;
Figure 6 [element 302] and Figure 9, and paragraphs 1 and 36 describe an embodiment of receiving a user input that requests an action which “receiv[es] an utterance” [i.e. receives a spoken/voice input] “through the [graphical] conversational user interface” [i.e. received through “user interface” which allows the automated assistant to receive natural language inputs from the user and to communicate natural language outputs/prompts to the user, thereby conducting a “conversation” between the user and the automated assistant, where, for claim 20, paragraph 109 and Figures 7-8 describe an embodiment where the prompts for parameters can be in visual [i.e. “displayed”] form, thereby suggesting an embodiment where the “conversational user interface” is “graphical”].
Paragraphs 103, 105, and 107-110 and Figure 6-9 at least suggests where an intent of a user’s input is determined and where “the function” [which is requested by the user and which is associated with the dependency graph data structure used to request parameters from the user] is “identified” by processing/analyzing the user input using natural language understanding information which can be interpreted as information “in the [graphical] conversational user interface” [since the automated assistant logically cannot interpret/understand what a user requests without knowledge/information that allows it to understand the words input by the user, and the suggested natural language understanding information is information used to conduct a conversation/dialog with a user such that it can be interpreted as being part of the “conversational user interface” which is used to conduct a conversation/dialog with the user])
Bharadwaj does not, but Kumar suggests wherein receiving the request to execute the function through the [graphical] conversational user interface comprises: receiving an utterance through the [graphical] conversational user interface; identifying an intent from the received utterance; and identifying the function based on a mapping of functions and intents in the [graphical] conversational user interface (Figures 5, 7-8 and 11, paragraphs 40, 59-60;
Bharadwaj describes determining an intent from a user input and determining/identifying an action to be performed from a user input, but does not specifically describe where the action/”function” is identified based on a mapping between the action/”function” and the intent of the user input [in Bharadwaj, the intent seems to be the action, not something that is mapped to the action].
Kumar similarly describes a system which maps text phrases [at least suggested to be natural language text phrases] to intents and which performs an action and generates a response [paragraphs 38 and 40, and Figure 8 and 11, where paragraph 38 describes “text phrases submitted by the security analyst” and Figure 11 depicts examples of phrases associated with an “Analyst” which include natural language phrases]. Kumar further suggests where determining an action corresponding to a user input is performed more particularly by determining an intent associated with the user input, and then mapping the determined intent to a corresponding action [Figures 5 and 8; paragraphs 59-60]
Kumar thus suggests “wherein receiving the request to execute the function through the [graphical] conversational user interface comprises: receiving an utterance through the [graphical] conversational user interface; identifying an intent from the received utterance; and identifying the function based on a mapping of functions and intents in the [graphical] conversational user interface”: where Bharadwaj’s system, instead of identifying an action/”function” directly by determining the intent of a spoken user input [i.e. “utterance”], determines/”identifies” “an intent from” the spoken-user-input/”utterance”, and then “identif[ies] the function” by mapping the determined intent to the action/”function” that the user asked the automated system to perform/execute, where the mapping of the determined intent to the action/”function” is “based on a mapping of functions and intents” [e.g. based on a table mapping intents to actions like the one in Kumar, Figure 8] “in the [graphical] conversational user interface” [since the automated assistant logically cannot interpret/understand what a user requests without knowledge/information that allows it to understand the words input by the user, and the suggested natural language understanding information is information used to conduct a conversation/dialog with a user such that it can be interpreted as being part of the “conversational user interface” which is used to conduct a conversation/dialog with the user, and the intent determination and intent-to-action mapping information is used to understand what the user is requesting the automated assistant to do])
Therefore, it would have been obvious to one of ordinary skill in the art at the time of effective filing to perform a simple substitution of one type of action determination with another because the prior art teaches the claimed invention except for the substitution of action determination which does not necessarily determine an intent from a natural language user input and then map the determined intent to corresponding action with action determination which does. Kumar suggests that action determination which determines an intent from a natural language user input and then map the determined intent to corresponding action was known in the art. One of ordinary skill in the art could have substituted one type of action determination with another to obtain the predictable results of an automated assistant which receives a user input requesting an action to be performed, determines the action to be performed from the user input, uses a dependency graph data structure associated with the determined action to request parameters needed to perform the action from the user, and which performs the determined action using the requested parameters (as per Bharadwaj) where the action is determined by determining an intent of the user input and mapping the determined intent to the determined action (as suggested by Kumar).
Allowable Subject Matter
The following is a statement of reasons for the indication of allowable subject matter:
As per Claim(s) 5 (and similarly claim[s] 13), the prior art of record does not teach or suggest the combination of all limitations in claim(s) 1, 4, and 5 together, including (i.e. in combination with the remaining limitations in claim[s] 1, 4, and 5) wherein the user interface definition for the function specifies, for each respective input of the requested inputs identified in the graph definition of the function, a prompt to display in the conversational user interface and a type of user interface control through which the respective input is to be entered.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12,106,013, hereafter Parent Patent 1. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of this application are rendered obvious by the claims of Parent Patent 1.
Claim 1 is broader than Claim 1 of Parent Patent 1 (the 2nd to last limitation of Claim 1 does not recite where the “determining” is “based on a completeness graph associated with the function” but is otherwise identical to Claim 1 of Parent Patent 1).
Claim 9 is broader than Claim 9 of Parent Patent 1 (the 2nd to last limitation of Claim 9 does not recite where the “determining” is “based on a completeness graph associated with the function” but is otherwise identical to Claim 9 of Parent Patent 1).
Claim 17 is broader than Claim 1 of Parent Patent 1 (the 3rd to last limitation of Claim 17 does not recite where the “determining” is “based on a completeness graph associated with the function” but is otherwise identical to Claim 17 of Parent Patent 1).
Claims 2-8, 10-16, and 18-20 are identical to Claims 2-8, 10-16, and 18-20, of Parent Patent 1 (and are therefore rejected under non-statutory double patenting, but are not rejected under statutory double patenting because they depend on independent claims 1, 9, and 17 which differ from Claims 1, 9, and 17 of Parent Patent 1).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC YEN whose telephone number is (571)272-4249. The examiner can normally be reached M-F 12:00PM -8:30PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, RICHEMOND DORVIL can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
EY 6/6/2026
/ERIC YEN/ Primary Examiner, Art Unit 2658
Prosecution Insights