DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Action is responsive to the Application filed on 5/29/2024. Claims 1-20 are pending in the case.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-11 are rejected under 35 U.S.C. 103 as being unpatentable over Prokofieva et al. (US 20160091967 A1, hereinafter Prokofieva) in view of Vennström et al. (US 20180157323 A1, hereinafter Vennström), and further in view of Zhuang et al. (US 20250054466 A1, hereinafter Zhuang).
As to independent claim 1, Prokofieva teaches an electronic device (“The one or more user device(s) 104 can represent a diverse variety of device types…Example mobile computers can include laptop computers, tablet computers, wearable computers,” paragraph 0029) comprising:
a display (Display 108 in Fig. 1);
at least one processor (Processing unit(s) 208, paragraph 0030); and
memory for storing instructions that, when executed by the at least one processor individually or collectively (“computer-readable media 210 such as memory” paragraph 0030), cause the electronic device to:
receive an utterance of a user (“Block 504 illustrates receiving a user utterance referring to a first visual element of the one or more visual elements in the visual context.” paragraph 0075);
obtain information on user interface (UI) components displayed on the display, based on context information of the electronic device (“Block 402 illustrates identifying visual elements that are available for receiving user interaction in a visual context. As described above, the visual context can include a web browser, conversational interaction system, or some other visual context for displaying visual elements.” Paragraph 0069, 0075,0042);
determine a UI component for which a state is to be updated by the utterance among the UI components, based on agaze feature and the utterance (“Block 512 illustrates determining a particular visual element of the visual elements that is referred to in the user utterance. As described above, the determining can be based at least in part on the lexical features and gaze features.” Paragraph 0079); and
update the state of the determined UI component based on the utterance (“based at least in part on identifying the intended visual element, the analysis module 218 can cause the computerized conversational system to perform the action and/or task associated with the intended visual element.” Paragraph 0079).
Prokofieva does not appear to expressly teach map weights to the UI components based on the information on the UI components and gaze information of the user; and
determine a UI component for which a state is to be updated by the utterance among the UI components, based on a weight mapping result and the utterance.
Vennström teaches based on the information on the UI components and gaze information of the user (“Distance information as described in connection with FIG. 5C may be a part of the information on which the weighting of interaction elements/parts of the UI is assigned. In embodiments, the identification rule may comprise distance information in combination with any of the other weighting, selection or identification information presented in method embodiments herein […] For each interaction element: In step S321: retrieve or calculate weight information for the interaction element, based on the properties of the interaction element. Examples are presented in connection with FIG. 5A. The position and size of the interaction elements may be retrieved from a memory 115, provided via an API connected with the GUI, or be determined using image processing methods known in the art. In step S322: retrieve or calculate weights for a selected area comprising one or more measured, calculated or predicted gaze points 410, based on a mathematical function.” Paragraph 0096-0101,0095); and
determine a UI component for which a state is to be updated based on a weight (“In step S325: determine a resulting weighting value from the generated set of weighted value [....] In an embodiment, the highest of the resulting weighting values is selected. Thereby, the interaction element that has the highest resulting weighting value from all the interaction elements is identified or selected [….] the interaction element may in some embodiments after selection or identification be highlighted in the GUI, or activated, or simply identified by the system.” paragraph 0108-0115).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
Vennstrom teaches retrieve or calculate weight value/information for the interaction element. However, Prokofieva and Vennström do not appear to expressly teach map weights to the UI component.
Zhuang teaches map weights to the regions in an image (“the gaze area data indicates coordinate positions of a gaze point of the user and/or one or more surrounding pixels thereof in the image. The first grid matrix weight map data indicates an attention level of the user to regions in the image. FIG. 3 shows a schematic diagram of the grid matrix weight map provided according to some embodiments of the disclosure. As shown in FIG. 3, in the grid matrix weight map, the gaze point of the user can have the highest weight (e.g. 3), adjacent pixels near it can have higher weights (e.g. 2), and edge pixels away from the gaze point of the user can have lower weights (e.g. 1). The weights of each pixel can decrease uniformly or unevenly from the gaze point, with pixels closer to the gaze point having higher weights and relatively dense grids.” paragraph 0041-0043).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Zhuang. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 2, Prokofieva teaches the electronic device of claim 1, Prokofieva further teaches wherein the context information of the electronic device comprises information on an application driven in the electronic device at a point in time at which the utterance is received (“the techniques described herein include identifying visual elements that are available for user interaction in a visual context, such as a web browser, application interface, or some other conversational system.” Paragraph 0003,0017).
As to dependent claim 3. Prokofieva teaches the electronic device of claim 1, Prokofieva does not appear to expressly teach wherein the information on the UI components comprises position information of UI components displayed on the display.
Vennström teaches wherein the information on the UI components comprises position information of UI components displayed on the display (“Examples are presented in connection with FIG. 5A. The position and size of the interaction elements may be retrieved from a memory 115, provided via an API connected with the GUI, or be determined using image processing methods known in the art.” Paragraph 0100).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 4. Prokofieva teaches the electronic device of claim 1, Prokofieva further teaches wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
obtain information associated with a trajectory of a gaze or eye of the user from a wearable electronic device connected to the electronic device (trajectory of gaze input 112 obtained in Fig. 1, “Example mobile computers can include, wearable computers,” paragraph 0029); and
obtain the gaze information based on the information associated with the trajectory of the gaze or eye of the user (“Gaze can represent a direction in which a user's eyes are facing during a speech input 110. The tracking component 106 can track user gaze to generate gaze input 112. Gaze input 112 can include eye gaze input, head pose input, and/or nose pointing input.” Paragraph 0023).
As to dependent claim 5, Prokofieva does not appear to expressly teach the electronic device of claim 1, wherein
the electronic device is a head mounted display (HMD) comprising an augmented reality (AR) device, a virtual reality (VR) device, and a mixed reality (MR) device, and
the gaze information corresponds to a result obtained by tracking eyes of the user by the electronic device.
Vennström teaches the electronic device is a head mounted display (HMD) comprising an augmented reality (AR) device, a virtual reality (VR) device, and a mixed reality (MR) device (“The term display device as used herein may refer to a screen, a head-mounted display, a head-up display, eye glasses or other type of display configured to display a projection onto a suitable surface.” Paragraph 0018,0050), and
the gaze information corresponds to a result obtained by tracking eyes of the user by the electronic device (“If the user is using or wearing any kind of display providing an augmented reality representation of the real-world scene, an interaction element that the user is able to interact with, or an element that the user is looking at according to the improved measured gaze point calculated using any of the embodiments presented herein, may be highlighted or marked.” Paragraph 0050)
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 6. Prokofieva teaches the electronic device of claim 1, Prokofieva further teaches wherein the gaze information corresponds to a focus of a user who gazes at an arbitrary point on the display (“The extraction module 216 can identify fixation points representing where a user's 102 gaze lands in a visual context.” paragraph 0049). Vennström further teaches gaze position 410 on the display as shown in Fig. 5C.
As to dependent claim 7. Prokofieva teaches the electronic device of claim 6, Prokofieva does not appear to expressly teach wherein the weights are set based on a distance between the focus and a UI component.
Vennström teaches wherein the weights are set based on a distance between the focus and a UI component (“Distance information as described in connection with FIG. 5C may be a part of the information on which the weighting of interaction elements/parts of the UI is assigned.” Paragraph 0096, Fig. 5c, 0095).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 8. Prokofieva teaches the electronic device of claim 6, Prokofieva does not appear to expressly teach wherein the weights are set to be inversely proportional to a distance between the focus and a UI component.
Vennström teaches wherein the weights are set to be inversely proportional to a distance between the focus and a UI component (“FIG. 5C shows two interaction elements 500, 540 having a respective selected point 520, 530. The measured gaze point 410 is at a certain time instance located between said interaction elements 500, 540. In embodiments illustrated in FIG. 5c, there is provided a method for determining which interaction element 500, 540 the user wants to select, the method comprising calculating the distance d1, d2 from the respective selected points 520, 530 to the measured gaze point 410,” paragraph 0095).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 9. Prokofieva teaches the electronic device of claim 6, Prokofieva does not appear to expressly teach wherein the instructions,
when executed by the processor individually or collectively, cause the electronic device to: when the focus corresponds to an arbitrary UI component, map a weight of “0” to a UI component other than the arbitrary UI component.
Vennström teaches when executed by the processor individually or collectively, cause the electronic device to: when the focus corresponds to an arbitrary UI component, area defined for a specific interaction element, wherein the selection area at least partly overlaps the area of the interaction element. Typically, the selection area coincides with or comprises the area of the interaction element. The size of the selection area depends on the weighting of the interaction element, based on the properties of the interaction element, the interaction context,” Paragraph 0069, this implies that a UI element other the arbitrary UI element is mapped to lowest weight value).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
Vennstrom teaches retrieve or calculate weight value/information for the interaction element. However, Prokofieva and Vennström do not appear to expressly teach map a weight of “0” to a UI component other than the arbitrary UI component.
Zhuang teaches map a weight of “0” to a region of an image other than the arbitrary region (“The weights of each pixel can decrease uniformly or unevenly from the gaze point, with pixels closer to the gaze point having higher weights and relatively dense grids.” paragraph 0041-0043).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Zhuang. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 10. Prokofieva teaches the electronic device of claim 1, Prokofieva does not appear to expressly teach wherein the instructions, when executed by the processor individually or collectively, cause the electronic device to:
determine whether to update a state by the utterance, sequentially starting from a UI component to which a highest weight value
Vennström teaches determine whether to update a state, sequentially starting from a UI component to which a highest weight is mapped (“In step S325: determine a resulting weighting value from the generated set of weighted value [....] In an embodiment, the highest of the resulting weighting values is selected. Thereby, the interaction element that has the highest resulting weighting value from all the interaction elements is identified or selected [….] the interaction element may in some embodiments after selection or identification be highlighted in the GUI, or activated, or simply identified by the system.” paragraph 0108-0115).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
Vennstrom teaches retrieve or calculate weight value/information for the interaction element. However, Prokofieva and Vennström do not appear to expressly teach map highest weights to the UI component.
Zhuang teaches map highest weights to a region of an image (“the gaze area data indicates coordinate positions of a gaze point of the user and/or one or more surrounding pixels thereof in the image. The first grid matrix weight map data indicates an attention level of the user to regions in the image. FIG. 3 shows a schematic diagram of the grid matrix weight map provided according to some embodiments of the disclosure. As shown in FIG. 3, in the grid matrix weight map, the gaze point of the user can have the highest weight (e.g. 3), adjacent pixels near it can have higher weights (e.g. 2), and edge pixels away from the gaze point of the user can have lower weights (e.g. 1). The weights of each pixel can decrease uniformly or unevenly from the gaze point, with pixels closer to the gaze point having higher weights and relatively dense grids.” paragraph 0041-0043).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Zhuang. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 11. Prokofieva teaches the electronic device of claim 1, Prokofieva further teaches wherein the instructions are triggered when a sustained gaze of the user at one of the UI components is detected (“Extracting gaze features can include fixation points (e.g., centroid fixation point and/or any fixation point) associated with the gaze input 304 at predetermined times.” Paragraph 0071).
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Prokofieval et al. in view of Vennström et al., Zhuang et al., and further in view of Kim (US 20210193142 A1).
As to dependent claim 12. Prokofieva teaches the electronic device of claim 1, Prokofieva does not appear to expressly teach wherein the instructions are triggered when an utterance comprising a wake-up word of a voice assistant is received prior to the utterance.
Kim teaches wherein the instructions are triggered when an utterance comprising a wake-up word of a voice assistant is received prior to the utterance (“In general, when a unique trigger or wake-up word is received, each voice assistant recognizes the same and enters a mode for listening a subsequent speech from a user 90 who requests a service” paragraph 0050).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Kim. One would have been motivated to make such a combination to improve user experience.
As to dependent claim 13. Prokofieva teaches the electronic device of claim 1, Prokofieva does not appear to expressly teaches wherein the instructions are triggered when the utterance comprises a wake-up word of a voice assistant.
Kim teaches wherein the instructions are triggered when the utterance comprises a wake-up word of a voice assistant (“In general, when a unique trigger or wake-up word is received, each voice assistant recognizes the same and enters a mode for listening a subsequent speech from a user 90 who requests a service” paragraph 0050).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Kim. One would have been motivated to make such a combination to improve user experience.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Prokofieval et al. in view of Vennström et al., Zhuang et al., and further in view of Baron (US 20130257780).
As to dependent claim 14. Prokofieva teaches the electronic device of claim 1, Prokofieval does not appear to expressly teach wherein the instructions are triggered when a touch input to the display is received.
Baron teaches wherein the instructions are triggered when a touch input to the display is received (“An electronic device may receive a touch selection of an element on a touch screen. In response, the electronic device may enter a listening mode for a voice command spoken by a user of the device.” Abstract).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Baron. One would have been motivated to make such a combination to improve user experience.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 15 and 17-18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shapiro et al. (US 9377852, hereinafter Shapiro).
As to independent claim 15, Shapiro discloses an electronic device comprising:
a display (display 112);
at least one processor (processor 104); and
memory for storing instructions that, when executed by the at least one processor individually or collectively (“In the present disclosure, the methods disclosed may be implemented as sets of instructions or software readable by a device.” Col. 7, lines 57-58), cause the electronic device to:
track a gaze of a user at the display (Eye tracking sensor 102 detecting a gaze of a user within a display at step 502 in Fig. 5, Col. 5 lines 64-67);
receive an utterance of the user (“The method 500 may also support the use of voice recognition by providing context to a voice command. In this embodiment, the step of receiving an input from the user related to the item 506 includes receiving a voice command from the user. A further step of the method 500 may include determining a meaning of the voice command based on the voice command and the item.” Col. 6 lines 35-41. “After looking at the item he wishes to change, the user may issue a voice command such as “accelerate to 190.” Col. 6 lines 47-49,); and
update a state of a user interface (UI) component displayed on the display, based on the utterance (“The additional step of the method 500 of determining a meaning of the voice command based on at least the voice command and the item may be completed by a voice recognition system.” Col. 6 lines 57-60),
wherein the UI component is a UI component corresponding to the user's gaze before a point in time at which the utterance is received (“Using the method 500, his gaze will be detected and correlated to an airspeed control item within the display, as shown in FIG. 4 where the location 114 of the user's gaze is within the airspeed control display (item 108) and more particularly in the area of a speed of 190. After looking at the item he wishes to change, the user may issue a voice command such as “accelerate to 190.” The voice command may be issued either simultaneously upon gazing/looking at the item, or may be issued after.” Col. 6 lines 43-51).
As to dependent claim 17, Shapiro further discloses wherein the instructions, when executed by the processor individually or collectively, cause the electronic device to:
obtain information associated with a trajectory of a gaze or eye of the user from a wearable electronic device connected to the electronic device (“In another embodiment, the eye tracking sensor 102 is incorporated within an existing helmet or mask system used by the user, or within a head up display. In another embodiment, the eye tracking sensor 102 includes a sensor placed around the eyes to measure movement.” Col. 3 lines 34-38); and
obtain gaze information of the user in real time based on the information associated with the trajectory of the gaze or eye of the user (detecting a gaze of user within a display at 502 in Fig. 5).
As to dependent claim 18, Shapiro further discloses, wherein
the electronic device is a head mounted display (HMD) comprising an augmented reality (AR) device, a virtual reality (VR) device, and a mixed reality (MR) device (“In another embodiment, the eye tracking sensor 102 is incorporated within an existing helmet or mask system used by the user, or within a head up display. In another embodiment, the eye tracking sensor 102 includes a sensor placed around the eyes to measure movement.” Col. 3 lines 34-38, “the display includes at least one of a Head Up display or a head mounted display.” Claim 3), and
the instructions, when executed by the processor, individually or collectively, cause the electronic device to obtain gaze information of the user in real time (detecting a gaze of user within a display at 502 in Fig. 5).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shapiro et al. in view of Vennström et al., and further in view of Zhuang et al.
As to dependent claim 16, Shapiro teaches the electronic device of claim 15, Shapiro further teaches wherein the instructions, when executed by the processor individually or collectively, cause the electronic device to:
obtain information on UI components displayed on the display, based on context information of the electronic device (Fig. 4 displaying an airspeed control item within a display in the user interface system of a flight display shown in Fig. 2);
update the state of the determined UI component based on the utterance (“The additional step of the method 500 of determining a meaning of the voice command based on at least the voice command and the item may be completed by a voice recognition system.” Col. 6 lines 57-60).
Shapiro does not appear to expressly teach wherein the instructions, when executed by the processor individually or collectively, cause the electronic device to:
map weights to the UI components based on the information on the UI components and gaze information of the user;
determine a UI component for which a state is to be updated by the utterance among the UI components, based on a weight mapping result and the utterance; and
Vennström teaches based on the information on the UI components and gaze information of the user (“Distance information as described in connection with FIG. 5C may be a part of the information on which the weighting of interaction elements/parts of the UI is assigned. In embodiments, the identification rule may comprise distance information in combination with any of the other weighting, selection or identification information presented in method embodiments herein […] For each interaction element: In step S321: retrieve or calculate weight information for the interaction element, based on the properties of the interaction element. Examples are presented in connection with FIG. 5A. The position and size of the interaction elements may be retrieved from a memory 115, provided via an API connected with the GUI, or be determined using image processing methods known in the art. In step S322: retrieve or calculate weights for a selected area comprising one or more measured, calculated or predicted gaze points 410, based on a mathematical function.” Paragraph 0096-0101,0095, Fig. 5C); and
determine a UI component for which a state is to be updated based on a weight (“In step S325: determine a resulting weighting value from the generated set of weighted value [....] In an embodiment, the highest of the resulting weighting values is selected. Thereby, the interaction element that has the highest resulting weighting value from all the interaction elements is identified or selected [….] the interaction element may in some embodiments after selection or identification be highlighted in the GUI, or activated, or simply identified by the system.” paragraph 0108-0115).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Prokofieva to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
Vennström teaches retrieve or calculate weight value/information for the interaction element. However, Prokofieva and Vennström do not appear to expressly teach map weights to the UI component.
Zhuang teaches map weights to the regions in an image (“the gaze area data indicates coordinate positions of a gaze point of the user and/or one or more surrounding pixels thereof in the image. The first grid matrix weight map data indicates an attention level of the user to regions in the image. FIG. 3 shows a schematic diagram of the grid matrix weight map provided according to some embodiments of the disclosure. As shown in FIG. 3, in the grid matrix weight map, the gaze point of the user can have the highest weight (e.g. 3), adjacent pixels near it can have higher weights (e.g. 2), and edge pixels away from the gaze point of the user can have lower weights (e.g. 1). The weights of each pixel can decrease uniformly or unevenly from the gaze point, with pixels closer to the gaze point having higher weights and relatively dense grids.” paragraph 0041-0043).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Shapiro to comprise the teachings of Zhuang. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
As to dependent claim 20, Shapiro teaches the electronic device of claim 16, Shapiro does not appear to expressly teach wherein the gaze information corresponds to a focus of a user who gazes at an arbitrary point on the display.
Vennström teaches wherein the gaze information corresponds to a focus of a user who gazes at an arbitrary point on the display (gaze position 410 in Fig. 5C).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Shapiro to comprise the teachings of Vennström. One would have been motivated to make such a combination in order to track the eye gaze efficiently and accurately.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Shapiro et al. in view of Kim.
As to dependent claim 19, Shapiro teaches the electronic device of claim 15, Shapiro does not appear to expressly teach wherein updating of the state of the UI component is triggered when the utterance comprises a wake-up word of a voice assistant. Shapiro does not appear to expressly teaches wherein updating of the state of the UI component is triggered when the utterance comprises a wake-up word of a voice assistant.
Kim teaches wherein updating of the state of the UI component is triggered when the utterance comprises a wake-up word of a voice assistant (“In general, when a unique trigger or wake-up word is received, each voice assistant recognizes the same and enters a mode for listening a subsequent speech from a user 90 who requests a service” paragraph 0050).
Accordingly, it would have been obvious to a person ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Shapiro to comprise the teachings of Kim. One would have been motivated to make such a combination in order to improve user experience.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Gatzke et al. US 12353796 B2 Controlling Audibility of Voice Commands Based on Eye Gaze Tracking.
Prasad et al. US 20250004544 A1 Gaze initiated actions.
Pu et al. US 20220284904 A1 Text Editing using voice and gesture inputs for assistant systems.
Giraldi et al. US 10186086 B2 A user may virtually interact with the user interface via natural user input in the form of a combination of eye gaze and wearer input.
Vertegaal et al. Gaze assisted input for an electronic device.
Powderly et al. US 20200363865 A1 Multimodal task execution and text editing for a wearable system.
Park et al. US 9547371 B2 An eye gaze sensing unit to sense eye positions and eye gaze directions of a user, and a hand shape and determine object as target selected by user.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHELET SHIBEROU whose telephone number is (571)270-7493. The examiner can normally be reached Monday-Friday 9:00 AM-5:00 PM Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHELET SHIBEROU/Primary Examiner, Art Unit 2171