DETAILED ACTION
This action is in response to the application filed 9/26/2024 and subsequent amendment filed 10/23/2025.
Claims 1-27 have been submitted for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 2, 11-17, 19-21, and 24-27 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Bentell et al. (US 2021/0407165), hereinafter Bentell.
As per claim 1, Bentell teaches the following:
an electronic device, comprising:
one or more processors, (see Fig. 4, 402);
a memory, (see Fig. 4, 412);
an input device, (see Fig. 4, 408); and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, (see Fig. 4, 416), the one or more programs including instructions for:
while receiving a natural-language input from a user:
providing a visual output at a first location at an edge of the display, wherein a visual characteristic of the visual output is based on an environment of the electronic device. As Bentell shows in Fig. 2C, a visual indication is provided at a bottom edge of the display. Bentall teaches in paragraph [0028], and corresponding Fig. 2A, that one or more portions of the radius change in accordance with properties of a voice command (natural-language input), such as a direction of the command, volume of the command, etc. (environment of device);
after ceasing to receive the natural-language input:
ceasing to provide the visual output. As Bentell teaches in paragraph [0054], and corresponding Fig. 2B, in response to detecting an end of the voice command, the icon is animated in a second fashion, which is distinct from a first fashion of when the command is being received, i.e., the first visual output is the first animation and ceases to be provided; and
providing an output responsive to the natural-language input. As Bentell teaches in paragraph [0057], and corresponding Fig. 2Bc, after processing the voice command, a confirmation state is entered where a result of the processing is provided (“Play my favorites” in the example given)
Regarding claim 2, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the visual output includes an animation responsive to the natural-language input. As Bentell teaches in paragraph [0028], and corresponding Fig. 2A, portions of the given icon are warped in accordance with properties of the voice command.
Regarding claim 11, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for:
while receiving the natural-language input from the user, detecting a volume of the natural-language input; and
in response to detecting the volume of the natural-language input:
in accordance with a determination that the volume of the natural-language input is a first volume, providing the visual output with a third intensity; and in accordance with a determination that the volume of the natural-language input is a second volume, different from the first volume, providing the visual output with a fourth intensity. As Bentell teaches in paragraph [0005], the visual feedback is updated according to the user’s voice via a sine wave, where the sine wave may changes based on a volume of the user’s voice command.
Regarding claim 12, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for:
while providing the visual output at the first location of the edge of the display, detecting movement of the user from a first location relative to the electronic device to a second location relative to the electronic device. As Bentell teaches in paragraph [0026], the dynamic animation changes according to direction of the voice command; and
in response to detecting movement of the user:
moving the visual output from the first location of the edge of the display to a second location of the edge of the display, wherein the second location is different from the first location. As Bentell teaches in paragraph [0005], an animation may change based on a direction, e.g., where the user is located relative to the electronic device. Therefore, using Fig. 2C as an example, the animation may move along an edge of the display as follows:
PNG
media_image1.png
364
376
media_image1.png
Greyscale
Regarding claim 13, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for:
while providing the visual output at the first location of the edge of the display, detecting movement of another user; and in response to detecting movement of another user, forgo moving the visual output. Bentell teaches of a variety of sensors in paragraph [0088], including “sensors/devces for sensing and measuring various environmental conditions” which would be capable of detecting movement of one or more user. As Bentell is directed to utilizing voice input for determining animation, and does not explicitly teach of utilizing “movement of a user” in altering the animation, the animation would not move regarding movement of a second user.
The examiner would like to note that the limitation of “forgo moving the visual output” seems unclear within claim 13, as neither claim 1 nor 13 ever mentions “moving the visual output”.
Regarding claim 14, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for:
while providing the visual output at the first location of the edge of the display, detecting a change in a gaze of the user; and in response to detecting the change in the gaze of the user, continuing to provide the visual output at the first location. Bentell teaches of a variety of sensors in paragraph [0088], including “sensors/devces for sensing and measuring various environmental conditions” which would be capable of detecting a user’s gaze. As Bentell is directed to utilizing voice input for determining animation, and makes no explicit teaching of utilizing a user’s gaze in display, the system would therefore continue to output the visual animations regardless of where a user’s gaze may be.
The examiner would like to again note that, similar to claim 13, the limitations of claim 14 appear to simply state the device has the capability to detect a user’s gaze, but does not utilize the information in any claimed fashion.
Regarding claim 15, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for: while receiving a second natural-language input from the user: providing a second visual output at a third location of the edge of the display, wherein a second visual characteristic of the second visual output is based on the environment of the electronic device. As Bentell shows in Fig. 2C, a visual indication is provided at a bottom edge of the display. Bentall teaches in paragraph [0028], and corresponding Fig. 2A, that one or more portions of the radius change in accordance with properties of a voice command (natural-language input), such as a direction of the command, volume of the command, etc. (environment of device). Therefore, the animations of Bentell may occur differently among different inputs by the user given the environment changes among the inputs.
Regarding claim 16, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the display has at least two or more edges; and the first location of the edge of the display is at two adjacent edges of the at least two or more edges. As Bentell shows in Fig. 2, the icon is presented along a bottom edge, as well as approaching the left and right edges.
Regarding claim 17, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for:
while providing the visual output at the first location of the edge of the display, detecting a first movement of the user from the first location relative to the electronic device to a third location relative to the electronic device;
in response to detecting the first movement of the user, displaying an animation of the visual output moving from the first location of the edge of the display to a fourth location of the edge of the display, wherein the first location and the fourth location are on a first edge of the display. As Bentell teaches in paragraph [0026], the dynamic animation changes according to a direction of a voice command. Therefore, using Fig. 2C as an example (reproduced below), an animation may change from a first location to a forth location along a first edge (left) of the display:
PNG
media_image2.png
350
415
media_image2.png
Greyscale
detecting a second movement of the user from the third location relative to the electronic device to a fourth location relative to the electronic device; and
in response to detecting the second movement of the user, displaying an animation of the visual output moving from the fourth location of the edge of the display to a fifth location of the edge of the display , wherein the fifth location is on a second edge of the display different from the first edge of the display. As Bentell teaches in paragraph [0026], the dynamic animation changes according to a direction of a voice command. Therefore, using Fig. 2C as an example (reproduced below), an animation may change from the fourth location along the first edge (left) to a fifth location along a second edge (right):
PNG
media_image3.png
350
494
media_image3.png
Greyscale
Regarding claim 19, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for: while receiving the natural-language input from the user, detecting one or more properties of the natural-language input;
in response to detecting one or more properties of the natural-language input:
in accordance with a determination that the one or more properties of the natural- language input satisfies a first set of criteria, animating the visual output at a first speed. As Bentell teaches in paragraph [0074], a sin wave may be damped over time. Therefore, upon the natural-language input occurring over a first period of time (first set of criteria), the wave travels at a first speed; and
in accordance with a determination that the one or more properties of the natural- language input satisfies a second set of criteria, different from the first set of criteria, animating the visual output at a second speed, different from the first speed. As Bentell teaches in paragraph [0074], a sin wave may be damped over time. Therefore, upon the natural-language input occurring longer than a first period of time and extending into a second period of time (second set of criteria), the wave travels at a second speed, different than the first.
Regarding claim 20, Bentell teaches the device of claim 19 as described above. Bentell further teaches the following:
the speed of the animation is based on properties of the environment of the electronic device. As Bentell teaches in paragraph [0074], a sin wave may be damped over time, wherein a time of voice input is interpreted as occurring within the environment of the device.
Regarding claim 21, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the environment of the electronic device is detected by one or more sensors in communication with the electronic device. See paragraph [0088].
Regarding claim 24, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for:
while continuing to provide the visual output, processing the natural-language input. As Bentell shows in Fig. 2Bb, and corresponding paragraph [0055], while the device is in a “processing state”, where the icon maintains display, i.e., is continued to be provided.
Regarding claim 25, Bentell teaches the device of claim 1 as described above. Bentell further teaches the following:
the one or more programs further including instructions for:
processing the natural-language input while receiving the natural-language input; and in accordance with a determination that processing the natural-language input finishes prior to providing the visual output, forgoing providing the visual output. As Bentell teaches in paragraphs [0056] and [0057], and corresponding Fig. 2B, an animation is presented throughout receiving and processing input, where animation is ceased during a confirmation state. Therefore, the animation is interpreted to be foregone upon processing completion.
As per claim 26, Bentell teaches the following:
a non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device that includes a display and one or more input devices, (see Fig. 4).
The remaining limitations of claim 26 are substantially similar to those of claim 1 and are rejected using the same reasoning.
As per claim 27, the limitations of claim 27 are substantially similar to those of claim 1 and are rejected using the same reaonsing.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 18, 22, and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bentell as applied to claim 1, in view of Klein et al. (US 2014/0207452)**.
Regarding claim 3, Bentell teaches the device of claim 1 as described above. While Bentell teaches of a voice input indication icon, Bentell does not explicitly discuss underlying content or independence of settings. In a similar field of endeavor, Klein teaches of a method of modifying appearance of an animation based upon direction of a speaking user (see abstract). Klein further teaches the following:
wherein properties of the visual output are independent of the settings of the display of the electronic device. As Klein shows in Fig. 1, the speech indicator 114 is presented as an overlay and therefor independent of settings of main display area 110. This is further exemplified in Fig. 4D, where a main display area is dimmed while the speech indicator is not.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the overlaying method of Klein. One of ordinary skill would have been motivated to have made such modification because presenting the indicator as an overly benefits users in providing a persistent view of speech recognition, regardless of primary content being viewed.
Regarding claim 18, Bentell teaches the device of claim 1 as described above. However, Bentell does not explicitly teach of “moving lights”. In a similar field of endeavor, Klein teaches of a method of modifying appearance of an animation based upon direction of a speaking user (see abstract). Klein further teaches the following:
wherein the visual output at the first location of the edge of the display includes an animation of moving lights. As Klein teaches in paragraphs [0016] and [0017], graphical feedback indicators for speech recognition may comprise any suitable number and arrangement of visual feedback elements, including iconography, images, etc., where the examiner interprets this teaching to encompass “moving lights”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the iconography of Klein. One of ordinary skill would have been motivated to have made such modification because the varied iconography of Klein would benefit a user in a more custom display and not being locked as a single possible display style of Bentell.
Regarding claim 22, Bentell teaches the device of claim 1 as described above. However, Bentell does not explicitly teach of a gradient of light. In a similar field of endeavor, Klein teaches of a method of modifying appearance of an animation based upon direction of a speaking user (see abstract). Klein further teaches the following:
the visual output includes a gradient of light. As Klein teaches in paragraph [0041], an outer ring may provide feedback to the user with an appearance such as difference color, texture, etc., which the examiner interprets as encompassing a “gradient”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the appearance variance of Klein. One of ordinary skill would have been motivated to have made such modification because as Klein teaches in paragraph [0041], varying different aspects of the display benefits a user in relaying a variety of information to the user.
Regarding claim 23, modified Bentell teaches the device of claim 22 as described above. However, as described above, Bentell does not explicitly teach of a gradient of light. Klein further teaches the following:
wherein the gradient of light comprises at least a first portion of bright light proximate to the user and a second portion of dim light further from the user. As Klein shows in Fig. 1, speech indicator 114, a white portion (bright light) is used to indicate a direction of a speaking user, while a darker portion (dim light) is displayed on a remaining area. The examiner would like to note that a color fading from bright to dim is, by definition, a gradient.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the appearance variance of Klein. One of ordinary skill would have been motivated to have made such modification because as Klein teaches in paragraph [0041], varying different aspects of the display benefits a user in relaying a variety of information to the user.
Claim(s) 4-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bentell as applied to claim 1, in view of Van Os et al. (US 2015/0382047), hereinafter Van.
Regarding claim 4, Bentell teaches the device of claim 1 as described above. However, Bentell does not explicitly tach of not modifying a second portion of the display. In a similar field of endeavor, Van teaches of displaying a virtual assistant (see abstract). Van teaches the following:
wherein the edge of the display is a first portion of the display, the one or more programs further including instructions for:
in response to detecting the natural-language input from the user, providing the visual output on the first portion of the display without modifying a second portion of the display, different from the first portion of the display. As Van shows in the transition from Fig. 4A to Fig. 4B, the speech interface 484 (first portion) does not modify a video 480 (second portion) being displayed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the non-modifying overlapped display of Van. One of ordinary skill would have been motivated to have made such modification because such overlap would benefit users in preserving a view of a current activity while voice input is being provided.
Regarding claim 5, Bentell teaches the device of claim 1 as described above. However, Bentell does not explicitly tach of an interaction between the visual output and interface objects. In a similar field of endeavor, Van teaches of displaying a virtual assistant (see abstract). Van teaches the following:
wherein providing the visual output includes displaying an interaction between the visual output and one or more user interface objects on the display. As Van teaches in paragraph [0112], the currently viewed content may be contracted while an overlapping interface is displayed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the non-modifying overlapped display of Van. One of ordinary skill would have been motivated to have made such modification because such overlap would benefit users in preserving and overall view of a current activity while voice input is being provided.
Regarding claim 6, Bentell teaches the device of claim 1 as described above. While Bentell teaches in paragraph [0088] of the use of light sensors, Bentell does not explicitly tach of generating output with factors depending on a light threshold. In a similar field of endeavor, Van teaches of displaying a virtual assistant (see abstract). Van teaches the following:
the one or more programs further including instructions for:
while receiving the natural-language input from the user:
detecting a lighting level of the environment of the electronic device; and
in response to detecting the lighting level of the environment:
in accordance with a determination that the lighting level of the environment is above a predetermined threshold, generating the visual output with a first set of factors; and in accordance with a determination that the lighting level of the environment is below the predetermined threshold, generating the visual output with a second set of factors. As Van teaches in paragraph [0153], ambient lighting around a device may be utilized in determining which device to output content too. This teaching at least suggests that the same light detection method may be utilized in determining which device to output the voice input interface to.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the output determination based on ambient light of Van. One of ordinary skill would have been motivated to have made such modification because the output determination of Van benefits users in providing visual output at an easier to view/optimal fashion.
Regarding claim 7, Bentell teaches the device of claim 6 as described above. However, as discussed above, Bentell does not explicitly tach of generating output with factors depending on a light threshold. Van teaches the following:
the first set of factors includes a first opacity, a first intensity, a first brightness, and/or a first scale, and the second set of factors includes a second opacity, a second intensity, a second brightness, and/or a second scale. As Van teaches in paragraph [0153], ambient lighting around a device may be utilized in determining which device to output content too. As applicant’s limitation is directed to different measurements of light, this is interpreted to be at least suggested by Van’s “ambient lighting” detection.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the output determination based on ambient light of Van. One of ordinary skill would have been motivated to have made such modification because the output determination of Van benefits users in providing visual output at an easier to view/optimal fashion.
Regarding claim 8, Bentell teaches the device of claim 7 as described above. Bentell in view of Van teaches the following:
the one or more programs further including instructions for:
after generating the visual output with the first set of factors:
detecting a change in the lighting level of the environment of the electronic device; and in response to detecting the change in the lighting level of the environment:
in accordance with a determination that the lighting level of the environment changed from above the predetermined threshold to below the predetermined threshold, modifying the visual output by changing the first brightness to the second brightness, wherein the first brightness is greater than the second brightness; and in accordance with a determination that the lighting level of the environment remained above the predetermined threshold, forgo modifying the visual output with the first set of factors. As Bentell teaches in paragraph [0026], animation is dynamically altered based upon aspects detected by the device. Van teaches in paragraph [0153], that output may be determined based upon ambient lighting of the device. Therefore, upon the modification, the output may be dynamically altered based upon changes in the ambient lighting.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the output determination based on ambient light of Van. One of ordinary skill would have been motivated to have made such modification because the output determination of Van benefits users in providing visual output at an easier to view/optimal fashion.
Claim(s) 9 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bentell as applied to claim 1, in view of Kakapuri (US 2017/0045936).
Regarding claim 9, Bentell teaches the method of claim 1 as describe above. While Bentell teaches of the use of proximity sensors in paragraph [0088], Bentell does not explicitly teach of altering an intensity of visual output based upon proximity of the user. Kakapuri teaches the following:
the one or more programs further including instructions for:
while receiving the natural-language input from the user, detecting a proximity of the user to the electronic device. As Kakapuri teaches in the abstract, a first sensor determines a distance of a user from the device; and
in response to detecting the proximity of the user to the electronic device:
in accordance with a determination that the proximity of the user to the electronic device is a first distance, providing the visual output with a first intensity; and in accordance with a determination that the proximity of the user to the electronic device is a second distance, different from the first distance, providing the visual output with a second intensity, different from the first intensity. As Kakapuri shows in Fig. 3, 330, and corresponding paragraph [0030], the brightness of a display (intensity) is automatically adjusted based upon proximity of a user.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the brightness based on proximity of Kakapuri. One of ordinary skill would have been motivated to have made such modification because as Kakapuri teaches in paragraph [0008], such automatic adjustments benefit users in reducing eye strain.
Regarding claim 10, modified Bentell teaches the method of claim 9 as describe above. However, as described above, Bentell does not explicitly teach of altering an intensity of visual output based upon proximity of the user. Kakapuri further teaches the following:
the one or more programs further including instructions for:
detecting a change in the proximity of the user to the electronic device from the first distance to the second distance; and in response to detecting the change in the proximity of the user to the electronic device, modifying the visual output from the first intensity to the second intensity. As Kakapuri shows in the flowchart of Fig. 4, the brightness setting is changed based upon a proximity change.
It would have been obvious to one of ordinary skill in the art before the effective filing date of applicant’s claimed invention to have modified the speech icon of Bentell with the brightness based on proximity of Kakapuri. One of ordinary skill would have been motivated to have made such modification because as Kakapuri teaches in paragraph [0008], such automatic adjustments benefit users in reducing eye strain.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Aigner (US 2015/0029225), teaches of animating edges of display based upon orientation of device.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GREGORY A DISTEFANO whose telephone number is (571)270-1644. The examiner can normally be reached Monday - Friday: 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Bashore can be reached at 5712424088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GREGORY A. DISTEFANO/
Examiner
Art Unit 2174
/WILLIAM L BASHORE/ Supervisory Patent Examiner, Art Unit 2174
Prosecution Insights