DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) were filed on 4/8/2025, 8/12/2025, and 10/21/2025. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Drawings
The drawings filed 9/21/2023 were accepted.
Claim Objections
Claim 14 is objected to because of the following informalities: “detecting that a fifth gaze input directed to the first object” should be replaced with “detecting that a fifth gaze input is directed to the first object”. Appropriate correction is required.
Allowable Subject Matter
Claims 14-21, 23-33, and 35 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 36 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
The claim does not fall within at least one of the four categories of patent eligible subject matter because the claim falls outside the scope of patent-eligible subject matter at least because the claimed "computer-readable storage medium" in light of the supporting disclosure is broad enough to encompass transitory embodiments. The specification does not define the “computer-readable storage medium.” A reasonable interpretation of the "computer-readable storage medium" includes a signal. See MPEP 2106(I). Examiner suggests using "non-transitory computer-readable storage medium" to overcome the rejection.
Non-limiting examples of claims that are not directed to one of the statutory categories: transitory forms of signal transmission (for example, a propagating electrical or electromagnetic signal per se), In re Nuijten, 500 F.3d 1346, 1357, 84 USPQ2d 1495, (Fed.Cir. 2007).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 9, 12, 13, 22, 34, 36, and 37 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Haine et al (US11592899B1; filed 10/28/2021).
With regards to claim 1, Haine et al discloses A method, comprising: at a computer system that is in communication with a first display generation component and one or more input devices (Haine et al, col 2, line 67-col 3, line 1: “Eye-controlled user interfaces track a user's eye movement;” abstract: “a button within a display”): displaying, via the first display generation component, a first object in a first view of a three-dimensional environment (Haine et al, Col 1, lines 9-10: “user gaze fixation within a display such as an augmented reality (“AR”) or virtual reality (“VR”) display”), wherein the first object includes at least a first portion of the first object and a second portion of the first object (Haine et al, Fig. 10-12: Each selectable element can be interpreted as a portion of a larger object, such as object 1190 in Fig. 11B, which includes multiple selectable portions (begin/cancel) and the rest of the object outside of those portions);
while displaying the first object, detecting, via the one or more input devices, a first gaze input that meets first criteria, wherein the first criteria require that the first gaze input is directed to the first portion of the first object in order for the first criteria to be met (Haine et al, Col 4, lines 65-68: “A user gaze occurs when a user fixates at a location within a display for a time equal to or exceeding a threshold.” The criteria can be interpreted as either the gaze being detected within the boundary of the interactable portion, or that the gaze fixation exceeded a threshold period. Fig. 4: Shows the steps of exceeding a threshold time period to cause display of additional element);
in response to detecting that the first gaze input meets the first criteria, displaying a first control element that corresponds to a first operation associated with the first object, wherein the first control element was not displayed prior to detecting that the first gaze input met the first criteria (Haine et al, Col 5, lines 1-4: “For purposes of triggering the display of a confirmation element or activating a button, the user gaze may appear within a button/confirmation element 410, at an edge or line of a button/confirmation element 420 or outside of a button/confirmation element at a set distance;” Fig. 2: The control element (Confirmation Element) is only displayed after step 1);
while displaying the first control element, detecting, via the one or more input devices, a first user input directed to the first control element (Haine et al, Fig. 2: “User performs action with gaze at confirmation element”); and
in response to detecting the first user input directed to the first control element, performing the first operation with respect to the first object (Haine et al, abstract: “The multi-step activation process comprises displaying an confirmation element in response to a first user gaze at a button. The button is subsequently activated in response to a second user gaze at the confirmation element.”).
With regards to claim 3, which depends on claim 1, Haine et al discloses while displaying the first object, detecting, via the one or more input devices, a third gaze input, directed to the second portion of the first object that is distinct from the first portion of the first object, that does not meet the first criteria; and in response to detecting that the third gaze input does not meet the first criteria, forgoing display of the first control element that corresponds to the first operation associated with the first object (Haine et al, Fig. 14: Step 1420, which displays the confirmation element, only occurs if the user “fixates on button for greater than interval T1;” This could be a result of the user not gazing at the button (first portion), and instead gazing at another portion as shown in Fig. 11A, which shows a user being able to select either being or cancel with their gaze).
With regards to claim 9, which depends on claim 1, Haine et al discloses wherein the first portion of the first object includes a first sub-portion of the first object and a second sub-portion of the first object, wherein the first sub-portion of the first object and the second sub-portion of the first object are separated by a third sub-portion of the first object that is not included in the first portion of the first object, and the first criteria require that the first gaze input is directed to at least one of the first sub-portion and the second sub-portion of the first object in order for the first criteria to be met (Haine et al, Figs. 10, 11A, 11B, 12, 13: All figures show the sub-portion that the user must gaze upon to cause the confirmation element to appear; Under this interpretation, the first sub-portion can be interpreted as Begin button 1120 (in fig. 11A), the third sub-portion can be interpreted as the part of the object 1110 directly above it, and the second sub-portion can be interpreted as the part of the object directly above the third sub-portion).
With regards to claim 12, which depends on claim 1, Haine et al discloses while displaying the first object, detecting, via the one or more input devices, a fourth gaze input, directed to a respective portion of the first object (Haine et al, Fig. 11A: Shows a user interface object with multiple buttons (portions) that the user is gazing at);
in response to detecting the fourth gaze input: in accordance with a determination that the fourth gaze input is directed to the first portion of the first object and meets the first criteria with respect to the first portion of the first object, displaying the first control element (Haine et al, Fig. 11A: gaze is directed at Cancel, which is interpreted as the first portion, and a boundary and confirmation element are displayed);
in accordance with a determination that the fourth gaze input is directed to a third portion of the first object, different from the first portion of the first object, displaying a second control element that corresponds to a second operation associated with the first object, wherein the second control element was not displayed prior to detecting that the fourth gaze input directed to the third portion of the first object (Haine et al, Fig. 11A: User’s gaze moves from Cancel to Begin, and the control element (the border with the confirmation element 1136) is displayed after the user’s gaze is held over each portion);
while displaying the second control element, detecting a third user input directed to the second control element; and in response to detecting the third user input directed to the second control element, performing the second operation with respect to the first object (Haine et al, Col 7, lines 11-15: “A multi-step activation of the CANCEL button 1130 occurs when the user fixates a subsequent gaze at the confirmation element 1136. A similar multi-step activation process occurs with the BEGIN button 1120.” The selection of the confirmation element, which activates the selected button, is interpreted as the selection which results in the performing of the second operation).
With regards to claim 13, which depends on claim 1, Haine et al discloses wherein the first control element is a close affordance (Haine et al, Fig. 13: control element is interpreted as the confirmation element 1315, which is associated with the “dismissal glyph”);
and the method includes: while displaying the close affordance, detecting, via the one or more input devices, a fourth user input directed to the close affordance; and in response to detecting the fourth user input directed to the close affordance, closing the first object, including ceasing display of the first object in the first view of the three- dimensional environment (Haine et al, Col 7, lines 59-62: “A user subsequently gazes at the confirmation element 1315 to complete the multi-step dismissal resulting the in the block 1310 being removed from the display”).
With regards to claim 22, which depends on claim 1, Haine et al discloses wherein displaying the first control element that corresponds to the first operation associated with the first object in response to detecting that the first gaze input meets the first criteria with respect to the first portion of the first object includes: in accordance with a determination that the first object is displayed as a two-dimensional object in the three-dimensional environment (Haine et al, col 1, lines 9-10: “user gaze fixation within a display such as an augmented reality (“AR”) or virtual reality (“VR”) display;” Fig. 14: The invention determines the fixation time and compares it to T1, which necessarily means there is a comparison between gaze position and position of the portion being displayed), displaying the first control element at a first position with a first spatial relationship to the first object (Haine et al, col 4, lines 3-5: “The confirmation element 220 is generated within the display in a manner to visually show an association with the button 215 to the user”).
With regards to claim 34, which depends on claim 1, Haine et al discloses the first criteria require that the first gaze input is maintained on the first portion of the first object for at least a first threshold amount of time in order for the first criteria to be met; and, displaying the first control element includes displaying the first control element after the first gaze input is directed to the first portion of the first object for at least the first threshold amount of time (Haine et al, Fig. 14: “Fixate on button for greater than interval T1” causes the confirmation element to appear in step 1420).
Claim 36 recites substantially similar limitations to claim 1 and is thus rejected along the same rationale.
Claim 37 recites substantially similar limitations to claim 1 and is thus rejected along the same rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 4-8, and 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Haine et al in view of Weinberger et al (US20140049462A1; filed 8/20/2012).
With regards to claim 2, which depends on claim 1, Haine et al discloses wherein displaying the first object in the first view of the three-dimensional environment includes displaying an… in the first view of the three-dimensional environment (Haine et al, Fig. 1: Multiple selectable elements which appear to represent applications (115-121) are shown in the user’s field of view; Col 1, lines 9-10: “user gaze fixation within a display such as an augmented reality (“AR”) or virtual reality (“VR”) display”).
However, Haine et al does not disclose displaying an application window of a first application.
Weinberger et al teaches displaying an application window of a first application (Weinberger et al, paragraph 32: “FIG. 3 is an illustration of possible placement of windows in a display;” paragraph 15: “application window”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Haine et al and Weinberger et al such that the UI objects include application windows. This would have enabled the invention to direct user input to UI elements including windows with their gaze (Weinberger et al, paragraph 2: “It may require cumbersome actions such as moving a mouse, clicking or performing keyboard shortcuts to switch active windows. However, these approaches are inefficient and also are approximations or proxies for determining where the user's attention is, or which window the user wants to interact with”).
With regards to claim 4, which depends on claim 1, Haine et al discloses wherein the first control element is a first… affordance; and the method includes: while displaying the first… affordance, detecting, via the one or more input devices, a second user input directed to the first… affordance; and in response to detecting the second user input directed to the first… affordance, resizing the first object.
Haine et al does not disclose resize affordance… resize affordance… resize affordance… resizing the first object.
However, Weinberger et al teaches resize affordance… resize affordance… resize affordance… resizing the first object (Weinberger et al, paragraph 15: “once selected, the UI element also becomes available for input, such as, movement, typing into, resizing, minimizing, closing, and so on”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Haine et al and Weinberger et al such that the user can resize the object using the gaze input. This would have enabled the invention to direct user input to UI elements including windows with their gaze (Weinberger et al, paragraph 2: “It may require cumbersome actions such as moving a mouse, clicking or performing keyboard shortcuts to switch active windows. However, these approaches are inefficient and also are approximations or proxies for determining where the user's attention is, or which window the user wants to interact with”).
With regards to claim 5, which depends on claim 4, Haine et al does not disclose yet Weinberger et al teaches detecting the second user input directed to the first resize affordance includes detecting a direction of movement of the second user input directed to the first resize affordance; and in response to detecting the second user input directed to the first resize affordance, resizing the first object, includes: in accordance with a determination that the direction of movement of the second user input is a first direction, increasing a size of the first object; and in accordance with a determination that the direction of movement of the second user input is a second direction different from the first direction, decreasing the size of the first object. (Weinberger et al, paragraph 35: “Therefore, when the user clicks or drags these elements, it is the window manager that takes the appropriate actions, such as moving or resizing the windows;” The dragging to resize a window resizes it based on the movement and direction of the dragging).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Haine et al and Weinberger et al such that the user can resize the object. This would have enabled the invention to direct user input to UI elements including windows with their gaze (Weinberger et al, paragraph 2: “It may require cumbersome actions such as moving a mouse, clicking or performing keyboard shortcuts to switch active windows. However, these approaches are inefficient and also are approximations or proxies for determining where the user's attention is, or which window the user wants to interact with”).
With regards to claim 6, which depends on claim 4, Haine et al does not disclose yet Weinberger et al teaches detecting the second user input directed to the first resize affordance includes detecting an amount of movement of the second user input directed to the first resize affordance; and in response to detecting the second user input directed to the first resize affordance, resizing the first object, includes: in accordance with a determination that the amount of movement of the second user input is a first amount of movement, changing a size of the first object to a first size that is selected based on the first amount of movement of the second user input; and in accordance with a determination that the amount of movement of the second user input is a second amount of movement different from the first amount of movement, changing the size of the first object to a second size different from the first size, the second size selected based on the second amount of movement of the second user input (Weinberger et al, paragraph 35: “Therefore, when the user clicks or drags these elements, it is the window manager that takes the appropriate actions, such as moving or resizing the windows;” The dragging to resize a window resizes it based on the movement and direction of the dragging).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Haine et al and Weinberger et al such that the user can resize the object using the gaze input. This would have enabled the invention to direct user input to UI elements including windows with their gaze (Weinberger et al, paragraph 2: “It may require cumbersome actions such as moving a mouse, clicking or performing keyboard shortcuts to switch active windows. However, these approaches are inefficient and also are approximations or proxies for determining where the user's attention is, or which window the user wants to interact with”).
With regards to claim 7, which depends on claim 4, Haine et al discloses detecting the first user input directed to the first control element includes detecting a first air gesture directed to the first control element; and the first object is resized in response to detecting the first air gesture directed to the first control element (Haine et al, col 1, lines 32-37: “This interface may rely exclusively on eye-controlled interactions or may use a combination of eye-controlled interactions with other types of user control such as hand-gestures, hand controllers, head movement or other types of movement that is translated into the virtual environment.”).
With regards to claim 8, which depends on claim 4, Haine et al discloses wherein the first portion of the first object includes a first corner of the first object, and the first criteria require that the first gaze input is directed to the first corner of the first object in order for the first criteria to be met (Haine et al, col 7, lines 55-56: “In a first example, a block 1310 is provided with a glyph 1311 at the upper corner”).
With regards to claim 10, which depends on claim 9, Haine et al discloses detecting the first user input while displaying the first control element includes detecting the first user input directed to the first sub-portion of the first object or the second sub-portion of the first object (Haine et al, Figs. 10, 11A, 11B, 12, 13: All figures show the sub-portion that the user must gaze upon to cause the confirmation element to appear; Under this interpretation, the first sub-portion can be interpreted as Begin button 1120 (in fig. 11A), the third sub-portion can be interpreted as the part of the object 1110 directly above it, and the second sub-portion can be interpreted as the part of the object directly above the third sub-portion).
Haine et al does not disclose yet Weinberger et al teaches performing the first operation with respect to the first object in response to detecting the first user input directed to the first control element includes: in accordance with a determination that the first user input is directed to the first sub-portion of the first object, changing a size of the first object to a first size while maintaining a position of a center of the first object; and in accordance with a determination that the first user input is directed to the second sub-portion of the first object, changing the size of the first object to a second size while maintaining the position of the center of the first object in the three-dimensional environment (Weinberger et al, paragraph 35: “Therefore, when the user clicks or drags these elements, it is the window manager that takes the appropriate actions, such as moving or resizing the windows;” The dragging to resize a window resizes it along the edge without moving the center).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Haine et al and Weinberger et al such that the user can resize the object. This would have enabled the invention to direct user input to UI elements including windows with their gaze (Weinberger et al, paragraph 2: “It may require cumbersome actions such as moving a mouse, clicking or performing keyboard shortcuts to switch active windows. However, these approaches are inefficient and also are approximations or proxies for determining where the user's attention is, or which window the user wants to interact with”).
With regards to claim 11, which depends on claim 9, Haine et al does not disclose yet Weinberger et al teaches the first portion of the first object corresponds to a first edge of the first object and does not correspond to a second edge of the first object; and performing the first operation with respect to the first object in response to detecting the first user input directed to the first control element includes: changing a size of the first object by moving the first edge of the first object while maintaining a position of the second edge of the first object in the three-dimensional environment. (Weinberger et al, paragraph 35: “Therefore, when the user clicks or drags these elements, it is the window manager that takes the appropriate actions, such as moving or resizing the windows;” The dragging to resize a window resizes it based on the dragging of the edge or corner of the window).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Haine et al and Weinberger et al such that the user can resize the object. This would have enabled the invention to direct user input to UI elements including windows with their gaze (Weinberger et al, paragraph 2: “It may require cumbersome actions such as moving a mouse, clicking or performing keyboard shortcuts to switch active windows. However, these approaches are inefficient and also are approximations or proxies for determining where the user's attention is, or which window the user wants to interact with”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Weber et al (US20160195924A1): Teaches activating affordances based on gaze detection on links/buttons.
Cederlund et al (US20150130740A1): Teaches gaze tracking for user input.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRODERICK C ANDERSON whose telephone number is (313)446-6566. The examiner can normally be reached Monday-Tuesday, Thursday-Saturday 9-5 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at 5712724124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.C.A/Examiner, Art Unit 2178
/STEPHEN S HONG/Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights