DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1 and 5 have been amended as per the amendment filed on 3/23/2026.
Claims 1, 3, 5-9, 13, 16, 17, 19, and 22-23 are pending and prosecuted.
Specification
The amendment to paragraphs [0007] and [0042] of the specification, filed on 3/23/2026, has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al., US Patent Publication 2024/0310902, in further view of McKenzie et al., US Patent Publication 2017/0322623, and in further view of Rockel et al., US Patent Publication 2022/0101593, henceforth known as Rockel, and in further view of Moll et al., US Patent Publication 2023/0377223, henceforth known as Moll.
Regarding Claim 1, Wang discloses a head-mounted device (Abstract; Figure 2; [0025]; a computing device), comprising:
a head-mounted support structure (Figure 2a-e; [0025]; a wearable device 110);
a camera in the head-mounted support structure, wherein the camera is configured to determine first and second hand movements of a first hand, wherein the first hand movement is at a first time, the second hand movement is at a second time (Figures 1 and 2a-e; [0025]; [0030-0031]; [0040]; [0074]; one or more sensors 111 that include cameras for detecting hand gestures. The examiner considers different selection input of letters as different hand movements);
a gaze tracker in the head-mounted support structure, wherein the gaze tracker is configured to determine a gaze location (Figures 1 and 2a-e; [0025-0026]; [0030-0031]; [0040]; [0074]; eye tracking sensors that can output gaze tracking signals that are used with a target predictor 116 which can predict which virtual or real-world object that the user intends to target); and
a display in the head-mounted support structure, wherein the display is configured to display a virtual keyboard and to display a text input (Figures 1 and 2a-e; [0026]; [0029-0032]; [0040]; [0074]; a display 113 that displays a virtual keyboard 204 and a text entry box 206 (display text input) where the user uses a combination of gaze and hand gesture to perform a selection input of the letter they wish to target).
However, Wang doesn’t explicitly disclose wherein the camera is configured to detect a secondary input of a second hand,
wherein the first hand movement is a finger pinch at a first time
the gaze tracker is further configured to determine a gaze swipe input based on gaze locations between the first and second times beginning at the finger pinch, and
wherein the display is configured to display a text input in response to the gaze swipe input curve and to the secondary input.
McKenzie et al., US Patent Publication 2017/0322623, teaches a hybrid text input system, including a user eye swipe input to a virtual keyboard in a virtual reality environment, together with a controller input. The user’s gaze on a letter of a keyboard is detected by the eye graze tracking device of the HMD, and the user may apply a touch and hold input to a touch surface. The initial gaze is directed at a letter, such as H on the virtual keyboard, and a swiping action such as by a eye swiping action (gaze tracker is further configured to determine a gaze swipe input based on gaze locations) along the virtual keyboard, causes the corresponding characters and/or text to be entered in the text entry box 480 (wherein the display is configured to display a text input in response to the gaze swipe input curve) (Figure 7A-7E: [0044-0050]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the disclosure of Wang to further include the teachings of McKenzie in order to provide the gaze tracker is further configured to determine a gaze swipe input based on gaze locations between the first and second times, and wherein the display is configured to display a text input in response to the gaze swipe input curve. The motivation to combine these analogous arts is because McKenzie teaches a hybrid text input system that utilizes user eye swipe input for characters/text entry (McKenzie: [0044];).
However, the combination of Wang and McKenzie doesn’t explicitly teach
wherein the camera is configured to detect a secondary input of a second hand,
wherein the first hand movement is a finger pinch at a first time
the gaze tracker is further configured to determine a gaze swipe input based on gaze locations between the first and second times beginning at the finger pinch,
wherein the display is configured to display a text input in response to the gaze swipe input curve and to the secondary input.
However, Rockel et al., US Patent Publication 2022/0101593, teaches performing a first operation with respect to a first user interface object in accordance with the first user input includes: in accordance with a determination that the first user input includes (e.g., is, includes, starts with, ends with, etc.) a predefined selection gesture, selecting the first user interface object as a target for a subsequent input (e.g., a drag gesture while the pinch gesture is maintained, a flick gesture while the pinch gesture is maintained, a drag gesture after the predefined selection gesture is terminated, etc.) received from the first user ([0135]; [0269];).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Wang and McKenzie to further include the teachings Rockel such a pinch gesture on a letter is used to start the process for causing the corresponding characters and/or text to be entered in the text entry box 480 through the swiping action such as by the eye swiping action in order to provide wherein the first hand movement is a finger pinch at a first time, and the gaze tracker is further configured to determine a gaze swipe input based on gaze locations between the first and second times beginning at the finger pinch. The motivation to combine these arts is because Rockel describes providing an improved method and interface for interacting with a three-dimensional environment and facilitating the user’s use of the computer systems when interacting with the three-dimensional environment, thereby increasing the effectiveness, efficiency, and user safety and satisfaction with such computer systems (Rockel: [0012];).
However, the combination of Wang, McKenzie, and Rockel doesn’t explicitly teach wherein the camera is configured to detect a secondary input of a second hand,
wherein the display is configured to display a text input in response to the gaze swipe input curve and to the secondary input.
Moll et al., US Patent Publication 2023/0377223, teaches an AR system that detects a defined text selection gesture 448 being made by the user using one of their hands as a selection hand 464 ([0055];).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Wang, McKenzie, and Rockel to further include the teachings of Moll such that one or more sensors 111 that include cameras for detecting hand gestures is capable of detecting either hand of the user for different selection input of letters in order to provide wherein the camera is configured to detect a secondary input of a second hand, wherein the display is configured to display a text input in response to the gaze swipe input curve and to the secondary input. The motivation to combine these analogous arts is because Moll teaches the use of a text selection gesture can be made by the user using one of the their hands as a selection hand (Moll: [0055];).
Regarding Claim 3, The combination of Wang, McKenzie, Rockel, and Moll teaches the head-mounted device further comprising: control circuitry configured to determine the text input based on the gaze swipe input curve (Wang: [0069-0071]; processors capable of providing the features described, McKenzie: Figure 3, 7A-7E: [0019-0024]; [0044-0050]; the eye swipe’s taken by the eye gaze tracking device 165 of HMD are processed and translated into corresponding text input via the components (control circuitry) as seen in Figure 3 ).
Regarding Claim 9, The combination of Wang, McKenzie, Rockel, and Moll teaches wherein the virtual keyboard has keys arranged in columns, and positions of the keys in each column are configured to be arranged as the gaze swipe input curve is generated (Wang: Figures 1 and 2a-e; [0026]; [0029-0032]; [0040]; [0074]; a display 113 that displays a virtual keyboard 204 and a text entry box 206 (display text input). The virtual keyboard 204 is arranged in columns, and the positions of the keys are arranged as the swipe of the gaze occurs).
Claim(s) 6, 7 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al., US Patent Publication 2024/0310902, in further view of McKenzie et al., US Patent Publication 2017/0322623, henceforth known as McKenzie, in further view of Rockel et al., US Patent Publication 2022/0101593, henceforth known as Rockel, in further view of Moll et al., US Patent Publication 2023/0377223, and in further view of Dash, US Patent Publication 2017/0090747
Regarding Claim 6, The combination of Wang, McKenzie, Rockel, and Moll doesn’t explicitly teach wherein the camera is configured to image a user’s hands, and the display is configured to display the virtual keyboard overlaid on the user’s hands.
However, Dash, US Patent Publication 2017/0090747, discloses the use of a superimposing a virtual keyboard displayed onto a user’s hands with use of sensors 212 for enabling hand/finger detection (Figure 1; [0021-0022]; [0025-0030];).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify, the combinational disclosure of Wang, McKenzie, Rockel, and Moll to further include the teachings of Dash in order to provide wherein the camera is configured to image a user’s hands, and the display is configured to display the virtual keyboard overlaid on the user’s hands. The motivation to combine these analogous art is to provide a simple method and associated system capable of associating data input with a system (Dash: [0007];)
Regarding Claim 7, The combination of Wang, McKenzie, Rockel, Moll, and Dash teaches wherein the camera is configured to detect movement of fingers on the user’s hands (Wang: Figures 1 and 2a-e; [0025]; [0030-0031]; [0040]; [0074]; one or more sensors 111 that include cameras for detecting hand gestures), and the control circuitry is configured to determine the text input based on the gaze swipe input curve and the movement of the fingers (Wang: Figures 1 and 2a-e; [0025]; [0030-0031]; [0040]; [0069-0071]; [0074]; McKenzie: Figure 3, 7A-7E: [0019-0024]; [0044-0050]; the eye swipe’s taken by the eye gaze tracking device 165 of HMD are processed and translated into corresponding text input via the components (control circuitry) as seen in Figure 3 of McKenzie, along with the detected hand gesture for the selection input of letters).
Allowable Subject Matter
Claims 5 and 8 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 13, 16, 17, 19, 22, and 23 are allowed.
The following claims are drafted by the examiner and considered to distinguish patentably over the art of record in this application, Claims 1-23 (and are based on the original filed claims) are presented to applicant for consideration:
1. A head-mounted device, comprising:
a head-mounted support structure;
a camera in the head-mounted support structure, wherein the camera is configured to determine first and second hand movements;
a gaze tracker in the head-mounted support structure, wherein the gaze tracker is configured to determine a gaze location; and
a display in the head-mounted support structure, wherein the display is configured to display a virtual keyboard and to display a text input in response to the gaze location and the first and second hand movements,
wherein the first hand movement is at a first time, the second hand movement is at a second time, the gaze tracker is further configured to determine a gaze swipe input based on gaze locations between the first and second times, and the display is further configured to display the text input in response to the gaze swipe input and the first and second hand movements,
wherein the gaze swipe input comprises a gaze swipe input curve, the head-mounted device further comprising: control circuitry configured to determine the text input based on the gaze swipe input curve,
wherein the first and second hand movements comprise movements with a first hand, the camera is further configured to detect a secondary input with a second hand, and the secondary input is configured to delete at least a portion of the text input, to select an autofill recommendation pause the gaze tracker determination of the gaze swipe input curve,
wherein determining the text input comprises determining the text input based on the secondary input.
2. (Cancelled)
3. (Cancelled)
4. The head-mounted device of claim [[3]] 1, wherein the camera is configured to detect a finger pinch, and the gaze tracker is configured to determine the gaze swipe input curve beginning at the finger pinch.
5. (Cancelled)
6. The head-mounted device of claim [[3]] 1, wherein the camera is configured to image a user’s hands, and the display is configured to display the virtual keyboard overlaid on the user’s hands.
7. The head-mounted device of claim 6, wherein the camera is configured to detect movement of fingers on the user’s hands, and the control circuitry is configured to determine the text input based on the gaze swipe input curve and the movement of the fingers.
8. The head-mounted device of claim 7, wherein each of the fingers corresponds to a respective row of the virtual keyboard, and the control circuitry is configured to determine that the text input is from the respective row based on the movement of the respective finger.
9. The head-mounted device of claim 3, wherein the virtual keyboard has keys arranged in columns, and positions of the keys in each column are configured to be arranged as the gaze swipe input curve is generated.
10. The head-mounted device of claim 1, wherein the first hand movement is at a first time, the second hand movement is at a second time, and the camera is further configured to detect a hand swipe input based on additional hand movements between the first time and the second time.
11. The head-mounted device of claim 10, wherein the hand swipe input comprises a hand swipe input curve between the first and second times, the head-mounted device further comprising: control circuitry configured to determine the text input based on the hand swipe input curve.
12. The head-mounted device of claim 11, wherein the virtual keyboard has keys arranged in columns, and positions of the keys in each column are configured to be arranged as the hand swipe input curve is generated.
13. A method of determining a text input using a head-mounted device having a display, a gaze tracker, and a camera, the method comprising: displaying a virtual keyboard with the display;
determining a gaze location on the virtual keyboard with the gaze tracker;
determining first and second hand movements with the camera; and determining the text input based on the gaze location and the first and second hand movements;
wherein determining the first and second hand movements comprises determining the first and second hand movements at respective first and second times, the method further comprising:
determining a gaze swipe input with the gaze tracker based on gaze locations between the first time and the second time, wherein determining the text input comprises determining the text input based on the gaze swipe input;
determining the gaze swipe input comprises determining a gaze swipe input curve between the first and second times, and wherein determining the text input comprises determining the text input based on the gaze swipe input curve
wherein the first and second hand movements comprise movements with a first hand, the method further comprising:
detecting a secondary input from a second hand with the camera, and the secondary input is configured to delete at least a portion of the text input, to select an autofill recommendation, or to pause the gaze tracker determination of the gaze swipe input curve,
wherein determining the text input comprises determining the text input based on the secondary input.
14.(Cancelled)
15. (Cancelled)
16. (Cancelled).
17. The method of claim [[16]] 13, wherein detecting the secondary input comprises detecting an additional hand movement.
18. The method of claim 13, wherein determining the first and second hand movements comprises determining the first and second hand movements at respective first and second times, further comprising: determining a hand swipe input with the camera based on hand locations between the first time and the second time, wherein determining the text input comprises determining the text input based on the hand swipe input.
19. A head-mounted device having a front and a rear, comprising:
a support structure;
a first display in the support structure configured to display an image to a first eye box at the rear;
a second display in the support structure configured to display an image to a second eye box at the rear;
a camera at the front, wherein the camera is configured to detect first and second hand movements at respective first and second times;
a gaze tracker in the support structure, wherein the gaze tracker comprises a light emitter and a light detector configured to determine gaze locations; and
control circuitry configured to display a virtual keyboard using the first and second displays, to determine a gaze swipe input based on the gaze locations between the first time and the second time, and to determine a text input in response to the gaze swipe input;
wherein the first and second hand movements comprise movements of a first hand;
wherein the camera is further configured to detect a movement of a second hand, and wherein the control circuitry is configured to delete at least a portion of the text input, to select an autofill recommendation, or to pause the gaze tracker determination of the gaze swipe input curve,
wherein determining the text input comprises determining the text input based on the secondary input.
20. (Cancelled)
21. (Cancelled)
22. The head-mounted device of claim 19, wherein the camera is further configured to take an image of a user’s hands, and the first and second displays are configured to display the virtual keyboard overlaid on the user’s hands.
23. The head-mounted device of claim 22, wherein the camera is further configured to detect movement of fingers on the user’s hands, and the control circuitry is configured to determine the text input based on the gaze swipe input and the movement of the fingers.
Response to Arguments
Applicant’s arguments with respect to claims 1, 3, 5-9, 13, 16, 17, 19, and 22-23 have been fully considered, but they are directed to claims as amended, and therefore are moot in view of the new grounds of rejection presented above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK F MARINELLI whose telephone number is (571)270-3383. The examiner can normally be reached Monday - Friday: 8:00AM - 5:00PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, George Eng can be reached at (571)-272-7495. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK F MARINELLI/Primary Examiner, Art Unit 2699
Prosecution Insights