Detailed Action
Notice of Pre-AIA or AIA status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The reply filed on April 24, 2026 (“Response”) is not fully responsive to the prior Office action because it is missing, for every rejection, remarks “specifically pointing out how the language of the claims patentably distinguishes them from the references,” remarks “clearly point[ing] out the patentable novelty which [the Applicant] thinks the claims present in view of the state of the art,” and remarks that “also show how the amendments avoid [the] references” cited in the Office Action. 37 C.F.R § 1.111.
The Response merely opines that the Examiner was “optimistic” about the present amendment when they were discussed during an interview. However, “[a]n interview does not remove the necessity for reply to Office actions as specified in § 1.111.” 37 C.F.R § 1.133(b). Furthermore, while the Examiner does not necessarily disagree with the Applicant’s characterization of the Examiner’s high-level understanding of the prior art’s applicability to the orally-proposed amendment, the Examiner respectfully notes that the April 24 amendments were not presented in writing to the Examiner prior to that interview, and that the Examiner cautioned the Applicant’s representative that he could not agree to anything that isn’t presented in writing ahead of time. Moreover, given the length of the prior art cited in this application, and the divergent nature of the three independent claims, it is not possible to give adequate and complete consideration of each combination of references with each divergent set of claims during the span of an interview. Nevertheless, the Applicant will observe that several grounds of rejection in this Office Action are withdrawn in favor of others, and that, while the remaining rejections continue to rely upon some of the same teachings from several of the references, other portions are newly cited herein, responsive to the newly amended claim limitations.
Since the omission of arguments appears to be bona fide on the current record, the Examiner will waive the requirement for this amendment, and simply update all of the rejections to properly respond to the claims as amended.
Accordingly, with all claims rejected, the Applicant’s request for a notice of allowance is respectfully denied at this time.
Claim Objections
The Office objects to claims 11, 12, and 17 for containing the following informalities. Appropriate correction is required.
Claim 11
PNG
media_image1.png
158
992
media_image1.png
Greyscale
The amendment to claim 11 appears to inadvertently add a comma just prior to a semicolon, which is grammatically incorrect. See screenshot below:
For the sake of legibility, this Office Action will not reproduce the comma when mentioning this part of the claim.
Claim 12
Claim 12 contains an informality that was present both before and after the amendment (the Examiner did not notice it until now): the word “content” is missing on the third line (“the second assistant content characterizes a different suggestion”).
Claim 17
PNG
media_image2.png
78
747
media_image2.png
Greyscale
The amendment to claim 17 inserts a comma but simultaneously deletes the necessary space following that comma, creating a typographical error.
Claim Rejections – 35 U.S.C. § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1–5, 11, and 12 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by U.S. Patent Application Publication No. 2018/0365025 A1 (“Almecija”).
Claim 1
Almecija discloses:
A method implemented by one or more processors, the method comprising:
“FIG. 6 shows a process 600 for determining a user experience level and adapting a user interface, according to an embodiment. Steps within process 600 may be executed by CPU/GPU 154 or processing abilities of various components within the components within the user experience system 104.” Almecija ¶ 38.
determining, based on client interaction data, a prior interaction a user has had with an automated assistant that is accessible via a computing device, wherein the client interaction data is generated based on the prior interaction between the user and the automated assistant;
“At step 606, user experience system 104 retrieves user UI interaction history and profile. Each user has a profile that is dynamically created. This profile includes their user interface actions, history, and preferences, as well as other information about them that may affect how a UI is adapted.” Almecija ¶ 42.
selecting, based on the prior interaction the user had with the automated assistant, a particular interaction cohort for classifying the prior interaction,
“At step 608, user experience system 104, through user experience learning component 142 in an embodiment, applies a learning component to assign and/or update one or more user groupings.” Almecija ¶ 43. Keep in mind, while these are “user groupings,” the data being grouped is prior interaction data, because the profiles to which the user groupings are assigned or updated store data about the user’s “user interface actions.” Almecija ¶ 42. “For example, a user may have a similar way of working most sessions, but one session clicking buttons fast and furiously (i.e. shorter intervals between button clicks and faster mouse travel). Thus, the system can learn to group the user into a ‘speed needed’ or ‘urgent’ grouping that may dynamically update the user interface to show only the one anticipated next action or even automate some actions that would normally be clicks by the user in order to save time.” Almecija ¶ 43.
wherein the particular interaction cohort is selected from a plurality of interaction cohorts that vary according to a determined frequency of usage of a particular feature of the automated assistant;
“User experience learning component 142 learns and groups individuals across all of the various software applications and over time. Thus, it learns and develops an understanding of overall usage of the UI to group certain patterns and usages with similar patterns and usages,” Almecija ¶ 43, such as “buttons clicked, screens interacted (mouse or touch interactions with the screens in an embodiment), and number of monitors of interaction for the user.” Almecija ¶ 86.
These groupings include groupings that concern the frequency with which a segment of users utilize one of the particular features mentioned earlier. For example, as shown in FIG. 7, one grouping may apply “if the user has used the software application less than five times and appears focused on only performing one type of task based on the buttons and menus they are accessing,” another may apply “if a user has had a specific type of user interface output when using the software application many times in the past because they only perform one type of task,” and yet another may apply “if there is an emergency situation and a new user is trying to complete a task they have not done before and are not likely to do again.” Almecija ¶ 90.
subsequent to selecting the particular interaction cohort for classifying the prior interaction:
As shown in FIG. 6, steps 612 and 614 (discussed next) are performed subsequent to steps 606 and 608 (discussed earlier). See Almecija FIG. 6.
generating, based on the particular interaction cohort selected for the prior interaction, assistant content for rendering at an interface of the computing device or a separate computing device,
“At step 612, user experience system 104, through UI adaptive component 148 in an embodiment, adapts a user interface per user experience level and/or assigned grouping.” Almecija ¶ 46.
wherein the assistant content includes natural language content instructing the user how to utilize the particular feature of the automated assistant,
Almecija discloses many types of adaptations for step 612, but one such adaptation includes “the dynamic providing of hints to help the user navigate or otherwise use the UI.” Almecija ¶ 64. In particular, “FIG. 15 shows UI hinting in the form of a box providing a textual hint to the user, providing helpful information on how the user may choose to use the software. User interface 1500 has been adapted to show hint box 1510 in this example. Hint box 1510 informs the user that ‘Based on other users in your field, you may consider adjusting contrast next to improve the medical understanding of the image.’” Almecija ¶ 66.
wherein the assistant content is to be audibly rendered via one or more speakers of the computing device or the separate computing device,
“And the hint box may include a video or audio of the technique being taught or explained.” Almecija ¶ 66; see also Almecija ¶ 46 (disclosing that step 612—the step mapped to the claimed “generating” step that corresponds to this wherein clause—may also involve “changing paradigms (e.g., visual to audible).”).
and wherein different assistant content is generated for other users associated with other interaction cohorts of the plurality of interaction cohorts;
“At step 612, user experience system 104, through UI adaptive component 148 in an embodiment, adapts a user interface per user experience level and/or assigned grouping. Adapting the user interface can mean re-sizing the screen, changing the layout, reducing or adding buttons, changing menus, altering what content is shown, changing fonts, changing paradigms (e.g. visual to audible), changing icons, re-arranging UI assets, and more.” Almecija ¶ 46. An additional example of adapting the user interface includes “the dynamic providing of hints to help the user navigate or otherwise use the UI.” Almecija ¶¶ 64–66.
and causing the computing device or the separate computing device to audibly render the assistant content at the interface,
“At step 614, user experience system 104, through UI output component 150 in an embodiment, outputs the adapted UI to user IO 102.” Almecija ¶ 47. As mentioned earlier, this “may include a video or audio of the technique being taught or explained.” Almecija ¶ 66.
in furtherance of informing the user about how to utilize the particular feature of the automated assistant.
“Such hint boxes are ways that the system can help educate beginner users and allow them to understand the software better.” Almecija ¶ 66.
Claim 2
Almecija discloses the method of claim 1, further comprising:
subsequent to causing the computing device or the separate computing device to audibly render the assistant content: determining, based on subsequent client interaction data, a subsequent interaction, or lack of interaction, between the user with the automated assistant, wherein the subsequent client interaction data is generated based on the subsequent interaction, or lack of interaction, between the user and the automated assistant;
After adapting the UI in accordance with the method described in the rejection of claim 1, “[t]he system can ask the user what their UI preferences are and if certain adapted UIs have been helpful,” Almecija ¶ 87, and therefore receive a response to those questions.
and causing one or more trained machine learning models to be further trained based on the subsequent interaction, or lack of interaction, between the user and the assistant content,
“This feedback can be put under historical usage factors when trying to understand how to best learn what the best adapted UI is to output in the current session.” Almecija ¶ 87. Note that these “historical usage factors” refer to nodes in a “first node layer in [a] neural network 700” shown in FIG. 7. Almecija ¶ 85.
wherein generating the assistant content for rendering interface involves utilizing the one or more trained machine learning models.
Step 608 of method 600 is performed using the neural network 700, including the first node layer 702. See Almecija ¶¶ 43 and 85.
Claim 3
Almecija discloses the method of claim 2, further comprising:
subsequent to causing the computing device or the separate computing device to audibly render the assistant content at the interface: selecting, based on the subsequent interaction of the user, a separate interaction cohort from the plurality of interaction cohorts, wherein the separate interaction cohort corresponds to a more experienced user cohort relative to the particular interaction cohort.
“In an embodiment, the user experience system can provide an adapted user interface for beginner users such as FIG. 4 and then adapt for experienced users such as in FIG. 3.” Almecija ¶ 75. Specifically, with respect to the hint box example given in the rejection of claim 1, the beginner hint boxes “might not be shown to advanced users or the text in the hint box would be to a more advanced technique of using the software.” Almecija ¶ 66.
Claim 4
Almecija discloses the method of claim 2, further comprising:
subsequent to causing the computing device or the separate computing device to render the assistant content: selecting, based on the subsequent interaction of the user, a separate interaction cohort from the plurality of interaction cohorts, wherein the separate interaction cohort corresponds to a less experienced user cohort relative to the particular interaction cohort.
Almecija’s group assignments are task/context specific, so, a user who is initially assigned to an experienced grouping may be moved to a more novice grouping in response to the system 104 detecting that the user’s interactions correspond to a task with which the user is inexperienced. See Almecija ¶ 90. For example, “[w]hen a user is considered advanced because they have used a software application hundreds of times but only use it for completing one function, or one task comprised of multiple functions that have been automated as discussed below, the user experience system may present a very simple user interface of only one button and one imaging window.” Almecija ¶ 77. This applies to the type of hint box help offered as well—the software will offer beginner help or advanced help depending on the classification. See Almecija ¶ 66.
Claim 5
Almecija discloses the method of claim 1,
wherein the client interaction data indicates multiple different features of the automated assistant that the user has utilized via the computing device or the separate computing device.
“Each user has a profile that is dynamically created. This profile includes their user interface actions, history, and preferences, as well as other information about them that may affect how a UI is adapted.” Almecija ¶ 42.
Claim 11
Almecija discloses:
A method implemented by one or more processors, the method comprising:
“FIG. 6 shows a process 600 for determining a user experience level and adapting a user interface, according to an embodiment. Steps within process 600 may be executed by CPU/GPU 154 or processing abilities of various components within the components within the user experience system 104.” Almecija ¶ 38.
determining, based on client interaction data, a first interaction between a user and a first feature of an automated assistant and a second interaction between the user and a second feature of the automated assistant;
“At step 606, user experience system 104 retrieves user UI interaction history and profile. Each user has a profile that is dynamically created. This profile includes their user interface actions, history, and preferences, as well as other information about them that may affect how a UI is adapted.” Almecija ¶ 42.
It should be understood that the words “actions, history, and preferences” are all plural, thus, Almecija does indeed disclose both first and second interactions among the data. It should also be understood that the claimed “automated assistant” corresponds to Almecija’s user interface in general.
selecting, based on the first interaction and the second interaction, distinct interaction cohorts for classifying the first interaction and the second interaction,
“At step 608, user experience system 104, through user experience learning component 142 in an embodiment, applies a learning component to assign and/or update one or more user groupings.” Almecija ¶ 43. Keep in mind, while these are “user groupings,” the data being grouped is prior interaction data, because the profiles to which the user groupings are assigned or updated store data about the user’s “user interface actions.” Almecija ¶ 42.
To be clear, Almecija discloses selecting first and second cohorts for first and second interactions in two different ways, and either one is sufficient to disclose the claim language.
Option 1: the first and second interactions and their respective cohorts are distinct as to time that they are performed, irrespective of which feature receives the interaction. “For example, a user may have a similar way of working most sessions, but one session clicking buttons fast and furiously (i.e. shorter intervals between button clicks and faster mouse travel). Thus, the system can learn to group the user into a ‘speed needed’ or ‘urgent’ grouping that may dynamically update the user interface to show only the one anticipated next action or even automate some actions that would normally be clicks by the user in order to save time.” Almecija ¶ 43.
Option 2: the first and second interactions and their respective cohorts are distinct as to the actual feature receiving the interaction, irrespective of their timing. See Almecija ¶ 44 (“The user experience determination is not always a single determination. User experience determination component may determine the user’s experience level with the particular UI screen/workflow”) and Almecija ¶ 91.
wherein each interaction cohort of the distinct interaction cohorts is selected from a plurality of interaction cohorts that vary according to respective frequency of usage of the first feature of the automated assistant and the second feature of the automated assistant;
“User experience learning component 142 learns and groups individuals across all of the various software applications and over time. Thus, it learns and develops an understanding of overall usage of the UI to group certain patterns and usages with similar patterns and usages,” Almecija ¶ 43, such as “buttons clicked, screens interacted (mouse or touch interactions with the screens in an embodiment), and number of monitors of interaction for the user.” Almecija ¶ 86.
subsequent to selecting the distinct interaction cohorts for classifying the first interaction and the second interaction:
As shown in FIG. 6, steps 612 and 614 (discussed next) are performed subsequent to steps 606–610 (discussed earlier). See Almecija FIG. 6.
generating first assistant content based on a first interaction cohort of the interaction cohorts, and second assistant content based on a second interaction cohort of the interaction cohorts, wherein the first assistant content and the second assistant content are generated for rendering at an interface of the computing device or a separate computing device,
“At step 612, user experience system 104, through UI adaptive component 148 in an embodiment, adapts a user interface per user experience level and/or assigned grouping.” Almecija ¶ 46.
wherein the first assistant content includes first natural language content instructing the user how to utilize the first feature of the automated assistant, wherein the second assistant content includes second natural language content instructing the user how to utilize the second feature of the automated assistant,
Almecija discloses many types of adaptations for step 612, but one such adaptation includes “the dynamic providing of hints to help the user navigate or otherwise use the UI.” Almecija ¶ 64. In particular, “FIG. 15 shows UI hinting in the form of a box providing a textual hint to the user, providing helpful information on how the user may choose to use the software.” Almecija ¶ 66. Thus, for features where the user’s interactions classify him into the beginner group, the hint box text “can help educate beginner users and allow them to understand the software better,” while, for features where the user’s interactions classify him as an expert, “the text in the hint box would be to a more advanced technique of using the software.” Almecija ¶ 66.
and wherein the first assistant content and the second assistant content are to be audibly rendered via one or more speakers of the computing device or the separate computing device;
“And the hint box may include a video or audio of the technique being taught or explained.” Almecija ¶ 66; see also Almecija ¶ 46 (disclosing that step 612—the step mapped to the claimed “generating” step that corresponds to this wherein clause—may also involve “changing paradigms (e.g., visual to audible).”).
causing the computing device or the separate computing device to audibly render the first assistant content, in furtherance of informing the user about how to utilize the first feature of the automated assistant; and causing the computing device or the separate computing device to audibly render the second assistant content, in furtherance of informing the user about how to utilize the second feature of the automated assistant.
“At step 614, user experience system 104, through UI output component 150 in an embodiment, outputs the adapted UI to user IO 102.” Almecija ¶ 47. As mentioned earlier, this “may include a video or audio of the technique being taught or explained.” Almecija ¶ 66. “Such hint boxes are ways that the system can help educate beginner users and allow them to understand the software better,” or help advanced users learn about additional features. Almecija ¶ 66.
Claim 12
Almecija discloses the method of claim 11,
wherein the first assistant content characterizes a suggestion regarding the first feature of the automated assistant and the second assistant characterizes a different suggestion regarding a second feature of the automated assistant.
“[T]he text in the hint box would be to a more advanced technique of using the software” for an advanced user, as compared to the text for a beginner user, where the text simply “help[s] educate” them and “allow[s] them to understand the software better.” Almecija ¶ 66.
Claim Rejections – 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were effectively filed absent any evidence to the contrary. Applicant is advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned at the time a later invention was effectively filed in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
I. Almecija and Paulus teach claims 6 and 14–16.
Claims 6 and 14–16 are rejected under 35 U.S.C. § 103 as being unpatentable over Almecija as applied to claims 1 and 11 above, and further in view of U.S. Patent No. 9,785,534 B1 (“Paulus”).
For the sake of clarity and legibility, all quotes from the Paulus reference in this rejection have been modified to reduce all UPPERCASE PART NAMES to lowercase.
Claim 6
Almecija teaches the method of claim 1, but does not explicitly disclose “determining that the user is estimated to reduce engagement with the automated assistant at a particular time, or within a threshold duration of the particular time, wherein causing the computing device or the separate computing device to render the assistant content at the interface is performed at the particular time.”
Paulus, however, teaches a method for providing assistance to a user of a software application that involves selecting a particular interaction cohort to provide more tailored assistance to the user, and further teaches:
determining that the user is estimated to reduce engagement with the automated assistant at a particular time, or within a threshold duration of the particular time, wherein causing the computing device or the separate computing device to audibly render the assistant content at the interface is performed at the particular time.
If “a ‘yes’ determination is made at user at risk of abandoning the interactive software system? operation 323,” then “process flow proceeds through to select one or more appropriate user experience components to transform the user experience provided through the interactive software system to a user experience customized to facilitate progress and prevent abandonment of the interactive software system operation 325.” Paulus col. 36 ll. 41–54.
The user experience components include “content delivery messages,” Paulus col. 30 ll. 4–9, and those content delivery messages include a “specific wording of text [or] audio” to communicate with the user. Paulus col. 14 ll. 26–46.
Claim 14
Almecija and Paulus teach the method of claim 11, further comprising:
subsequent to selecting the distinct interaction cohorts for classifying the first interaction and the second interaction: determining that the user is estimated to reduce engagement with the first feature or the second feature at a particular time, or within a threshold duration of the particular time, wherein causing the computing device or the separate computing device to audibly render the first assistant content and/or the second assistant at the interface is performed in response to determining that the user is estimated to reduce engagement with the first feature or the second feature.
The interactive software system groups questions that it asks of the user into different respective topics. Paulus col. 20 ll. 25–47. (In this rejection, each group is mapped to a respective one of the first and second features). Then, the system can evaluate each group’s different likelihood to cause the user to abandon the interactive software system, and provide a user experience that avoids “any difficult/unpleasant questions and/or suggestions” related to a specific topic. Paulus col. 38 ll. 12–47.
Much like both Almecija and the claimed invention, part of providing this user experience includes providing “content delivery messages,” Paulus col. 30 ll. 4–9, and those content delivery messages may include a “specific wording of text [or] audio” to communicate with the user. Paulus col. 14 ll. 26–46.
Claim 15
Almecija and Paulus teach the method of claim 11, further comprising:
subsequent to selecting the distinct interaction cohorts for classifying the first interaction and the second interaction: determining that the user is estimated to reduce engagement with the computing device or the separate computing device at a particular time, or within a threshold duration of the particular time, wherein causing the computing device or the separate computing device to audibly render the first assistant content and/or the second assistant content is performed in response to determining that the user is estimated to reduce engagement with the computing device or the separate computing device.
If “a ‘yes’ determination is made at user at risk of abandoning the interactive software system? operation 323,” then “process flow proceeds through to select one or more appropriate user experience components to transform the user experience provided through the interactive software system to a user experience customized to facilitate progress and prevent abandonment of the interactive software system operation 325.” Paulus col. 36 ll. 41–54.
Much like both Almecija and the claimed invention, part of providing this user experience includes providing “content delivery messages,” Paulus col. 30 ll. 4–9, and those content delivery messages may include a “specific wording of text [or] audio” to communicate with the user. Paulus col. 14 ll. 26–46.
Claim 16
Claim 16 requires (1) all of the elements of claim 15, and (2) that the one or more speakers of the audible interface to be “integral to a vehicle.”
Regarding (1), Almecija and Paulus teach the method of claim 15 for the reasons given in the previous rejections.
Regarding (2), both Almecija and Paulus teach playing audible instructions for a user, see Almecija ¶¶ 46 and 66, and Paulus col. 14 ll. 26–46, and Almecija further teaches that its computer system (environment 100) includes a “speaker 116,” Almecija ¶ 36, and that environment 100 may be in a vehicle Almecija ¶106. Thus, the only difference between the prior art and the invention of claim 16 is the further instruction for the known speaker 116 to be integral with the known vehicle arrangement.
Such a difference would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention because making components integral with one another is a per se rationale for modifying a device under 35 U.S.C. § 103. MPEP § 2144.04 (subsection (V.)(B.)) (citing In re Larson, 340 F.2d 965, 968 (CCPA 1965)).
II. Almecija, Paulus, and Wolverton teach claims 7 and 8.
Claims 7 and 8 are rejected under 35 U.S.C. § 103 as being unpatentable over Almecija and Paulus as applied to claim 6 above, and further in view of U.S. Patent Application Publication No. 2014/0136187 A1 (“Wolverton”).
Claim 7
Almecija and Paulus teach the method of claim 6. Almecija further teaches that the computing device may be intended for a vehicle, and recommends an arrangement that considers whether/when a user is switching from a vehicle context to a non-vehicle context, or vice versa, in order to determine what kind of assistance to provide. See, e.g., Almecija ¶106 (explaining the system will change user interface paradigms “when a user is going from their home into their car”); ¶ 86 (changing the type of help provided depending on whether the user is using a smartphone or a desktop computer); ¶ 88 (“the system may know that the user wants more advanced features in their office user IO and would like less advanced features when moving in a mobile context”); and ¶ 107.
Paulus additionally teaches the engagement reduction limitation of claim 7, i.e.,
wherein determining that the user is estimated to reduce engagement with the automated assistant is based on available interaction data indicating current or past engagement of the user with the vehicle computing device and/or the automated assistant.
Paulus uses an “abandonment indicator,” which helps determine if the user is “at risk of abandoning the interactive software system,” Paulus col. 36 ll. 41–54, and teaches that the abandonment indicator may be based on data describing “the force with which the user touches hardware associated with the interactive software system” and “the speed with which the user touches hardware associated with the interactive software system.” Paulus col. 17 ll. 57–60 and col. 18 ll. 5–23.
Thus, the difference between claim 7 on one hand, and the Almecija-Paulus combination on the other, is a vehicle computing device that controls a vehicle.
Wolverton, like Almecija and Paulus, teaches a method for assisting users with software on a computer, but Wolverton further teaches:
the computing device is a vehicle computing device that is directly attached to, and controls, a vehicle
“Referring to FIG. 1, a vehicle personal assistant 112 is embodied in a computing system 100 as computer software, hardware, firmware, or a combination thereof,” and more particularly, “may be embodied as an in-vehicle computing system (e.g., an ‘in-dash’ system).” Wolverton ¶ 36; see also ¶¶ 125 and 135. “The vehicle personal assistant 112 may also be configured to disable or otherwise limit entertainment options within the vehicle 104 while the vehicle 104 is in motion.” Wolverton ¶ 141.
wherein determining that the user is estimated to reduce engagement with the automated assistant is based on available interaction data indicating current or past engagement of the user with the vehicle computing device and/or the automated assistant.
The personal assistant 112 “may analyze the duration of the pause between inputs 102 and determine an appropriate response thereto. For pauses of longer duration, the method 500 may simply abandon the dialog and wait for the user to provide new input 102 at a later time. For pauses of shorter duration, the method 500 may issue a reminder to the user or prompt the user to continue the thought.” Wolverton ¶ 118.
Additionally, the personal assistant 12 may have an input classifier 134, which “analyzes the computer-readable representations of the inputs 102 as prepared by the input recognizer/interpreter 130, and classifies the inputs 102 according to rules or templates that may be stored in the vehicle-specific conversation model 132 or the vehicle context model 116,” to anticipate a situation where “a vehicle driver may simply give up on an inquiry if an answer is not received within a reasonable amount of time.” Wolverton ¶ 52.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to improve Almecija and/or Paulus’s respective user assistance methods by extending them to the context of vehicle computers, as taught by Wolverton. One would have been motivated to make such a modification because “vehicle features that may be very helpful to a vehicle user may remain underused or not used at all” without providing such assistance. Wolverton ¶ 1.
Claim 8
Paulus and Wolverton teach the method of claim 7,
wherein the available interaction data indicates that, at the particular time, the user has ceased controlling the vehicle within a threshold duration of time, and/or the vehicle is parked or stopped.
“In some embodiments, the method 200 determines how to prompt the user for clarification based on the current vehicle context. For example, if the vehicle 104 is parked, the method 200 may prompt the user via spoken natural language, text, graphic, and/or video.” Wolverton ¶ 97.
III. Almecija, Paulus, and Biswas teach claims 9 and 10.
Claims 9 and 10 are rejected under 35 U.S.C. § 103 as being unpatentable over Almecija and Paulus as applied to claim 6 above, and further in view of U.S. Patent Application Publication No. 2022/0300392 A1 (“Biswas”).
Claim 9
Almecija and Paulus teach the method of claim 6, and while neither reference explicitly discloses that their respective underlying applications “facilitate internet searching,” Paulus further teaches that:
the automated assistant is an application
The abandonment indicator, which helps determine if the user is “at risk of abandoning the interactive software system,” Paulus col. 36 ll. 41–54, may include data describing “the force with which the user touches hardware associated with the interactive software system” and “the speed with which the user touches hardware associated with the interactive software system.” Paulus col. 17 ll. 57–60 and col. 18 ll. 5–23.
Although Almecija and Paulus do not explicitly say that their underlying software systems “facilitate[] internet searching,” Biswas, teaches a similar method involving predicting how a user will react to an event in a software application, and mitigating the reaction with remedial action, see Biswas Abstract, and further teaches:
the automated assistant is an application that facilitates internet searching,
As shown in FIG. 1, Biswas’s method collects user interaction data 102 to make predictions about a user’s interactions with a software product. User interaction data 102 includes “link selection activity,” from links that are “included in a set of search results displayed by the web browser.” Biswas ¶ 29.
and wherein determining that the user is estimated to reduce engagement with the automated assistant is based on available interaction data indicating current or past engagement of the user with the application.
“In accordance with embodiments of the present disclosure, the user interaction data 102 can be current data (e.g., activity data associated with a current activity, a number of activities in a current session, etc.), which can be obtained and used with the user reaction prediction models 112 to detect a user reaction, or reactions, 114 (e.g., a current user reaction, or reactions).” Biswas ¶ 23.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to apply Almecija and/or Paulus’s user assistance facilities to the field of applications that facilitate internet searching, as taught by Biswas (which has the same goal of predicting and mitigating poor user reactions to the software). One would have been motivated to follow Biswas’s lead of applying this type of help system to internet searching software because “[a] negative user experience with a website can result in the user limiting a current visit and any future visits to the website.” Biswas ¶ 1.
Claim 10
Almecija, Paulus and Biswas teach the method of claim 9, and Paulus further teaches:
the available interaction data indicates that, at the particular time, the application has, or has not, received an input from the user within a threshold duration of time.
The abandonment indicator includes data describing “the speed with which the user touches hardware associated with the interactive software system.” Paulus col. 17 ll. 57–60 and col. 18 ll. 5–23.
IV. Almecija and Aggarwal teach claim 13.
Claim 13 is rejected under 35 U.S.C. § 103 as being unpatentable over Almecija as applied to claim 12 above, and further in view of U.S. Patent Application Publication No. 2022/0100465 A1 (“Aggarwal”).
Claim 13
Almecija teaches the method of claim 12,
wherein the first feature corresponds to
Almecija’s system considers whether/when a user is switching from a vehicle context to a non-vehicle context, or vice versa, in order to determine what kind of assistance to provide for the different features. See, e.g., Almecija ¶106 (explaining the system will change user interface paradigms “when a user is going from their home into their car”); ¶ 86 (changing the type of help provided depending on whether the user is using a smartphone or a desktop computer); ¶ 88 (“the system may know that the user wants more advanced features in their office user IO and would like less advanced features when moving in a mobile context”); and ¶ 107.
Almecija does not appear to explicitly disclose that the first feature actually corresponds to controlling such a vehicle.
Aggarwal, however, teaches a method that is similar to that of the parent claims (e.g., providing help for a virtual assistant with respect to multiple features),
wherein the first feature corresponds to controlling, via the automated assistant, a vehicle that the computing device is attached,
“In some examples, when a user interacts with a particular type of computing device 210 (e.g., a mobile phone, vehicle head unit, etc.), assistant module 222 [or 122] may assign a higher relevancy score to actions which the particular type of computing device is configured to perform.” Aggarwal ¶ 84. Actions that assistant module 222 is capable of performing include “start[ing] the user’s vehicle.” Aggarwal ¶¶ 41 and 114.
and the second feature corresponds to controlling, via the automated assistant, a separate application from the automated assistant.
“Assistant modules 122 may rely on other applications (e.g., third-party applications), services, or other devices (e.g., televisions, automobiles, watches, home automation systems, entertainment systems, etc.) to perform actions or services for an individual.” Aggarwal ¶ 27; see also ¶ 59. The performance of these commands using other, third-party applications are among the actions that the assistant modules track in order to provide suggestions for executing those commands, when they are relevant. See, e.g., Aggarwal ¶¶ 80, 89, and 92.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to apply Paulus’s overall user help facility to an assistant that performs vehicle commands and commands from separate applications, as taught by Aggarwal.
Such a combination would have been obvious because it involves nothing more than the use of a known technique to improve similar devices, methods, or products in the same way. See Intel Corp. v. PACT XPP Schweiz AG, 61 F.4th 1373, 1380-81, 2023 USPQ2d 297 (Fed. Cir. 2023) citing KSR Int’l Co. v. Teleflex, Inc. 550 U.S. 398, 417 (2007).
Consistent with the guidance for this rationale in MPEP § 2143 (subsection (I.)(C.)), the relevant findings of fact for this conclusion are supported by a preponderance of the evidence, as follows:
(1) The prior art contained a “base” device, method, and product upon which the claimed invention can be seen as an “improvement.” The evidence for this finding includes all of the findings from the rejections of claims 11 and 12, which provide a correspondence between the elements that claim 13 incorporates from its parent claims by reference to the Almecija prior art reference.
(2) The prior art contained a “comparable” device, method, and product that is not the same as the base device, but that has been improved in the same way as the claimed invention. The evidence for this finding is provided above, via the citations to Aggarwal’s disclosure.
(3) One of ordinary skill in the art could have applied the known “improvement” technique in the same way to the “base” device, method, or product, and the results would have been predictable to one of ordinary skill in the art. The evidence for this finding is that both prior art references already disclose each of the elements of the claimed invention, with Aggarwal directly instructing the skilled artisan to employ the same in a vehicle, and with respect to third party applications.
Therefore, based on the above findings, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to apply Almecija’s overall user help facility to an assistant that performs vehicle commands and commands from separate applications, as taught by Aggarwal.
V. Almecija and Srenger teach claims 17 and 20.
Claim(s) 17 and 20 are rejected under 35 U.S.C. § 103 as being unpatentable over Almecija in view of U.S. Patent Application Publication No. 2006/0218506 A1 (“Srenger”).
Claim 17
Almecija discloses:
A method implemented by one or more processors, the method comprising:
“FIG. 6 shows a process 600 for determining a user experience level and adapting a user interface, according to an embodiment. Steps within process 600 may be executed by CPU/GPU 154 or processing abilities of various components within the components within the user experience system 104.” Almecija ¶ 38.
determining, based on client interaction data, a prior interaction a user has had with a particular feature of an automated assistant that is accessible via a computing device, wherein the client interaction data is generated based on the prior interaction between the user and the automated assistant;
“At step 606, user experience system 104 retrieves user UI interaction history and profile. Each user has a profile that is dynamically created. This profile includes their user interface actions, history, and preferences, as well as other information about them that may affect how a UI is adapted.” Almecija ¶ 42.
For example, the aforementioned data, via at least the user interface actions, may include user interface actions that the user performs to complete a specific task by clicking on various icons or buttons. Almecija ¶¶ 56 and 86. Any given one of these falls within the scope of the claimed “particular feature.”1
Another example of a particular feature tracked by the user experience system 104 is whether the user uses a smartphone version of the user interface, and whether the user uses “the option to use two side-by-side screens in a desktop environment.” Almecija ¶ 86.
User experience system 104 also registers “how many times the user has used certain help menus and for what types of issues, what tasks (series of actions) that user has performed, and past user outputted user interfaces have been presented to that user.” Almecija ¶ 87.
selecting, based on the prior interaction the user had with the particular feature of the automated assistant, a first interaction cohort for classifying the user,
“At step 608, user experience system 104, through user experience learning component 142 in an embodiment, applies a learning component to assign and/or update one or more user groupings.” Almecija ¶ 43.
wherein the first interaction cohort is selected from a plurality of interaction cohorts that vary according to a determined frequency of usage of the particular feature of the automated assistant;
“User experience learning component 142 learns and groups individuals across all of the various software applications and over time. Thus, it learns and develops an understanding of overall usage of the UI to group certain patterns and usages with similar patterns and usages,” Almecija ¶ 43, such as “buttons clicked, screens interacted (mouse or touch interactions with the screens in an embodiment), and number of monitors of interaction for the user.” Almecija ¶ 86.
These groupings include groupings that concern the frequency with which a segment of users utilize one of the particular features mentioned earlier. For example, as shown in FIG. 7, one grouping may apply “if the user has used the software application less than five times and appears focused on only performing one type of task based on the buttons and menus they are accessing,” another may apply “if a user has had a specific type of user interface output when using the software application many times in the past because they only perform one type of task,” and yet another may apply “if there is an emergency situation and a new user is trying to complete a task they have not done before and are not likely to do again.” Almecija ¶ 90.
determining, based on additional client interaction data, a stoppage in usage of the particular feature of the automated assistant for a threshold duration of time; selecting, based on the stoppage in usage of the particular feature of the automated assistant for the threshold duration of time, a second interaction cohort, that differs from the first interaction cohort, from the plurality of interaction cohorts for classifying the user;
The words “stoppage” and “duration” are italicized above to highlight a difference between the claimed invention and the prior art: user experience system 104 reassigns users to new and different user groups upon determining that they refrain from using certain features, and also reassigns users to the different groups when their usage of a feature over a period of time passes a threshold, but Almecija does not explicitly disclose the combination of these two concepts together, i.e., measuring the duration of time for a stoppage of use, and comparing that duration to a threshold.
Specifically, as shown in FIG. 6, user experience system 104 repeatedly loops between step 606 of retrieving more interaction history from the user and step 608 of updating the user’s groups based on the new information, so that it “learns and groups individuals across all of the various software applications and over time.” Almecija ¶ 43. “The user experience learning component 142 can automatically determine changes and patterns in the user's UI behavior and needs.” Lamecija ¶ 43.
One example of refraining from use over a period of time is the side-by-side monitor example mentioned earlier: “if the user has the option to use two side-by-side screens in a desktop environment but uses the right monitor 90% of the time in the current session, the user may prefer future adapted user interface outputs to focus more of the interaction on the right monitor.” Almecija ¶ 86.
subsequent to selecting the the second interaction cohort:
As shown in FIG. 6, steps 612 and 614 (discussed next) are performed subsequent to steps 606 and 608 (discussed earlier). See Almecija FIG. 6.
generating, based on the second interaction cohort, assistant content for rendering at an interface of the computing device or a separate computing device,
“At step 612, user experience system 104, through UI adaptive component 148 in an embodiment, adapts a user interface per user experience level and/or assigned grouping.” Almecija ¶ 46.
wherein the assistant content includes natural language content instructing the user how to utilize the particular feature of the automated assistant,
Almecija discloses many types of adaptations for step 612, but one such adaptation includes “the dynamic providing of hints to help the user navigate or otherwise use the UI.” Almecija ¶ 64. In particular, “FIG. 15 shows UI hinting in the form of a box providing a textual hint to the user, providing helpful information on how the user may choose to use the software. User interface 1500 has been adapted to show hint box 1510 in this example. Hint box 1510 informs the user that ‘Based on other users in your field, you may consider adjusting contrast next to improve the medical understanding of the image.’” Almecija ¶ 66.
wherein the assistant content is to be audibly rendered via one or more speakers of the computing device or the separate computing device,
“And the hint box may include a video or audio of the technique being taught or explained.” Almecija ¶ 66; see also Almecija ¶ 46 (disclosing that step 612—the step mapped to the claimed “generating” step that corresponds to this wherein clause—may also involve “changing paradigms (e.g., visual to audible).”).
and causing the computing device or the separate computing device to audibly render the assistant content,
“At step 614, user experience system 104, through UI output component 150 in an embodiment, outputs the adapted UI to user IO 102.” Almecija ¶ 47. As mentioned earlier, this “may include a video or audio of the technique being taught or explained.” Almecija ¶ 66.
in furtherance of reminding the user how to utilize the particular feature of the automated assistant.
“Such hint boxes are ways that the system can help educate beginner users and allow them to understand the software better.” Almecija ¶ 66.
As mentioned above, Almecija differs from the claimed invention in that Almecija does not explicitly check whether the duration of a stoppage of usage of a feature meets a threshold duration of time.
However, reminding a user about a feature that has not been used for at least a threshold amount of time in furtherance of reminding the user how to utilize the particular feature, was a known technique prior to the effective filing date of the claimed invention. In particular, Srenger teaches that, after a user has successfully used a command a number of times, “if a user has not used a command within a predetermined time period, such as one week,” then “the command is reinstated to the list of menu items” of a help menu, in furtherance of resolving a problem where “the user may have forgotten how to use the command.” Srenger ¶ 23.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to improve Almecija’s user experience system 104 with Srenger’s technique of reminding a user how to use a feature in response to the user stopping usage of that feature for a threshold period of time. One would have been motivated to improve user experience system 104 with Srenger’s technique based on an explicitly recognized need in the art for “a user interface with a menu system that can be automatically adapted, based on usage pattern, to provide efficient assistance and an enhanced user experience.” Srenger ¶ 6.
Claim 20
Almecija and Srenger teach the method of claim 17,
wherein the first interaction cohort corresponds to users who have had more interactions with the particular feature of the automated assistant than another user who is assigned to the second interaction cohort.
As shown in FIG. 7, based on the UI interaction data, user experience system 104 will determine the probability that a user should be assigned to the “long time user typical task” grouping 704, versus the probability that the user should be assigned to the grouping 704 of “non-typical task/situation” for “a task they have not done before and are not likely to do again.” Almecija ¶ 90.
VI. Almecija, Srenger, and Matsubara teach claim 18.
Claim 18 is rejected under 35 U.S.C. § 103 as being unpatentable over Almecija and Srenger as applied to claim 17 above, and further in view of U.S. Patent Application Publication No. 2005/0125233 A1 (“Matsubara”).
Claim 18
Almecija and Srenger teach the method of claim 17,
wherein the computing device is integral to a vehicle
“The mobile telephone, includes conventional cellular phone hardware (also not represented for simplicity) such as processors and user interfaces that are integrated into the vehicle.” Srenger ¶ 15.
Neither Almecija nor Srenger explicitly disclose a feature of the computing device that involves controlling the vehicle using the automated assistant.
Matsubara, however, teaches a computing device (“main device 1” in FIG. 1),
wherein the computing device is integral to a vehicle
Main device 1 is “a vehicle mounted control apparatus.” Matsubara ¶ 20.
and the particular feature involves controlling the vehicle using the automated assistant.
Main device 1 has an “interface 10” to perform a function of “relaying the operational status signals of electronic devices of the car and control signals to these electronic devices, for example, a control device of an air conditioner, head lights, and sensors for detecting the on-off states of a wiper and the head lights (all of which are not shown), between the control section 2 of the apparatus and these electronic devices.” Matsubara ¶ 22. As shown in FIG. 6B, when the user needs help understanding how to use main device 1 to issue commands, “the control section 2 reads from the memory 3 a method of uttering a voice command relating to the selected command and displays it on the display device 6,” e.g., a prompt that instructs the user to operate a steering wheel button and to “[u]tter the names of the prefecture and the name of facility consecutively.” Matsubara ¶ 41.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to extend the principles of Almecija’s user experience system 104 to the field of vehicles and vehicle controls, as directly suggested by Matsubara. One would have been motivated to extend these principles to vehicles because, “when a command by a voice (hereinafter referred to as ‘voice command’) is entered to a car navigation apparatus, depending on user’s vocal conditions (for example, the level of voice uttered by a user), the car navigation apparatus cannot recognize the voice in some cases,” and users have difficulty understanding why the voice recognition failed, Matsubara ¶ 7, necessitating a solution “that can inform a user about the state of recognition of a voice command uttered by the user in such a way that the user can easily understand the state of recognition.” Matsubara ¶ 8.
VII. Almecija, Srenger, and Smith teach claim 19.
Claim 19 is rejected under 35 U.S.C. § 103 as being unpatentable over Almecija and Srenger as applied to claim 17 above, and further in view of U.S. Patent Application Publication No. 2008/0300884 A1 (“Smith”).
Claim 19
Almecija and Srenger teach the method of claim 17, but since the rejection of claim 17 maps the claimed “automated assistant” to the same software that is also providing the assistance for using the software, Almecija necessarily does not anticipate an arrangement where “one or more features involve controlling a separate application via the automated assistant.”
Additionally, while Almecija does disclose a mechanism for switching its user interface paradigm from touch (or mouse) to voice, see Almecija ¶¶ 101, 102, and 106, and even acknowledges that “[s]ome users prefer or only can use . . . voice input and shun[] keyboard input,” Almecija ¶ 3, Almecija’s hinting feature does not appear to include “natural language content specifying a spoken utterance to provide to the automated assistant for controlling the separate application.”
Smith, however, teaches a method that renders assistant content at the interface, in furtherance of informing the user about one or more features employed by, or not employed by, the user during the prior interaction and/or the separate prior interaction,
wherein the one or more features involve controlling a separate application via the automated assistant,
As shown in FIG. 1, Smith teaches a general purpose computer 104 with an “[a]udio command interface 108 [that] determines whether voice data corresponds to an audio command, including “to access and control a native application 112.” Smith ¶ 19. Native application 112 is one of a plurality of applications on the general purpose computer 104 that are distinct from the audio command interface 108; audio command interface 108 must access their respective APIs to convert the voice commands into machine-usable commands for whichever of the native applications 112 are to be controlled. See Smith ¶ 21.
and the assistant content includes natural language content specifying a spoken utterance to provide to the automated assistant for controlling the separate application.
“Audio command interface 108 can also provide a list of available commands to the person using mobile device 102, such as by presenting prompts to the person, by allowing the person to request a list of available audio commands, or in other suitable manners.” Smith ¶ 19.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to extend Almecija’s user experience system 104 to work with additional applications, rather than embedding the feature directly within a single software application, considering such a suggestion was already known, via Smith’s disclosure. One would have been motivated to follow Smith’s suggestion to extend Almecija’s user experience system to additional applications because systems for using voice commands that “are application-specific” (as is the case with Almecija’s system) are inconvenient, because they “require the person to have multiple mobile devices and/or systems to remotely access and control the different applications at a computer.” Smith ¶ 2.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 C.F.R. § 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 C.F.R. § 1.17(a)) pursuant to 37 C.F.R. § 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Justin R. Blaufeld whose telephone number is (571)272-4372. The examiner can normally be reached M-F 9:00am - 4:00pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James K Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Justin R. Blaufeld
Primary Examiner
Art Unit 2151
/Justin R. Blaufeld/Primary Examiner, Art Unit 2151
1 It should be understood that by tracking many different interactions with several different particular features of the user interface, user experience system 104 effectively determines a prior interaction with at least one particular feature as required by the claim, because the “comprising” transitional phrase allows for the prior art to disclose additional, non-recited elements, MPEP § 2111.03, and also because the plain meaning of “a particular feature” includes one or more particular features. Baldwin Graphic Systems, Inc. v. Siebert, 512 F.3d 1338, 1342 (Fed. Cir. 2008).
Prosecution Insights