DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) were filed on 10/17/2024, 12/18/2024, 1/2/2025, 4/2/2025, 4/2/2025, 5/1/2025, 6/4/2025, 6/26/2025, 7/14/2025, 8/5/2025, 8/28/2025, 9/18/2025, 10/7/2025, 10/22/2025, 11/26/2025, 12/23/2025, 1/7/2026, and 5/5/2026. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Drawings
The drawings filed 9/21/2023 were accepted.
Allowable Subject Matter
Claims 2-3, 5-6, and 10-21 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The use of specific musical progressions (octaves, perfect fifths, major keys) to indicate sequential gaze inputs, as well as distinct audio feedback that are differentiated based on whether the input was merely a gaze or a selection were not taught by the prior art found by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 22, 24, and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lopez et al (US20140226131A1; filed 2/14/2014) in view of Lumsden et al (US20110021318A1; filed 7/19/2010).
With regards to claim 1, Lopez et al teaches a computer system configured to communicate with one or more display generation components and one or more input devices (Lopez et al, abstract: “eye tracking control… objects are displayed on a display of a device”), comprising:
one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for (Lopez et al, paragraph 86: “In example embodiments, one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein”):
displaying, via the one or more display generation components, a first gaze enrollment user interface, wherein the first gaze enrollment user interface includes a first plurality of gaze target elements, including a first gaze target element and a second gaze target element (Lopez et al, abstract: “One or more objects are displayed on a display of a device, where the one or more objects are associated with a function unrelated to a calculation of one or more calibration parameters;” Fig. 6A-6C: shows objects used for gaze detection/calibration);
while displaying the first gaze enrollment user interface, detecting, via the one or more input devices, a selection input; and in response to detecting the selection input (Lopez et al, abstract: “While the one or more objects are displayed, eye movement information associated with the user is determined, which indicates eye movement of one or more eye features associated with at least one eye of the user. The eye movement information is associated with a first object location of the one or more objects.”):
in accordance with a determination that the selection input is directed toward the first gaze target element, outputting first audio feedback corresponding to the first gaze target element; and
in accordance with a determination that the selection input is directed toward the second gaze target element… feedback corresponding to the second gaze target element and different from the first audio feedback (Lopez et al, paragraph 62: “In operation 718, the eye tracking layer may return feedback to the user depending on whether the correct number in the PIN was gazed upon. The feedback may be any feedback that notifies the user that the digit gazed upon has been processed (e.g., vibration of the computing device, visual notification on the computing device, a sound, etc.).” The disclosed audio feedback indicates that the correct target element was gazed upon (first gaze target element), but Lopez does not disclose what additional feedback is or is not outputted when other elements are gazed upon).
However, Lopez et al does not disclose outputting second audio feedback corresponding to the second… target element and different from the first audio feedback.
Lumsden et al teaches outputting second audio feedback corresponding to the second… target element and different from the first audio feedback (Kelly et al, abstract: “a reference note (of a musical scale) that provides a marker of the goal or target position of the exercise for the subject, and a sequence of real-time feedback notes that define progress towards the goal;” paragraph 12: “providing audio feedback that permits intuitive distinction between non-goal and goal points within the exercise”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Lopez et al and Lumsden et al such that the feedback for different inputs maps to differing audio feedback. This would have enabled the feedback to guide the user through an exercise (Lumsden et al, paragraph 11: “There remains a need for a simple and efficient audio feedback for training motor skills to a wide audience, especially one that is useful for guiding a subject through a cognitive exercise in which the feedback is interpreted and acted on in real time.”).
With regards to claim 4, which depends on claim 1, Lopez et al discloses the first gaze target element is displayed at a first location in the first gaze enrollment user interface; the second gaze target element is displayed at a second location in the first gaze enrollment user interface different from the first location; the first gaze target element… the second gaze target element (Lopez et al, Fig. 6A-C: multiple gaze target elements (e.g. 602, 612) are displayed at different locations).
However, Lopez et al does not disclose the first… target element corresponds to a first musical note; and the second… target element corresponds to a second musical note different from the first musical note.
Lumsden et al teaches the first… target element corresponds to a first musical note; and the second… target element corresponds to a second musical note different from the first musical note (Kelly et al, abstract: “a reference note (of a musical scale) that provides a marker of the goal or target position of the exercise for the subject, and a sequence of real-time feedback notes that define progress towards the goal;” paragraph 12: “providing audio feedback that permits intuitive distinction between non-goal and goal points within the exercise”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Lopez et al and Lumsden et al such that the feedback for different inputs maps to differing audio feedback. This would have enabled the feedback to guide the user through an exercise (Lumsden et al, paragraph 11: “There remains a need for a simple and efficient audio feedback for training motor skills to a wide audience, especially one that is useful for guiding a subject through a cognitive exercise in which the feedback is interpreted and acted on in real time.”).
With regards to claim 22, which depends on claim 1, Lopez et al discloses in response to detecting the selection input: in accordance with a determination that the selection input is directed toward the first gaze target element, displaying, via the one or more display generation components, a first visual indication indicative of the first gaze target element being selected; and in accordance with a determination that the selection input is directed toward the second gaze target element, displaying, via the one or more display generation components, a second visual indication different from the first visual indication and that is indicative of the second gaze target element being selected (Lopez et al, paragraph 62: “In operation 718, the eye tracking layer may return feedback to the user depending on whether the correct number in the PIN was gazed upon. The feedback may be any feedback that notifies the user that the digit gazed upon has been processed (e.g…. visual notification on the computing device…).” The visual notifications have to be different between the selection of the first and second elements in order for the notification to notify the user that “the digit gazed upon has been processed”).
Claim 24 recites substantially similar limitations to claim 1 and is thus rejected along the same rationale.
Claim 25 recites substantially similar limitations to claim 1 and is thus rejected along the same rationale.
Claim(s) 7-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lopez et al in view of Lumsden et al, and further in view of Kienzle et al (US20180046851A1; filed 8/14/2017).
With regards to claim 7, which depends on claim 1, Lopez et al discloses wherein the selection input includes a gaze input… (Lopez et al, abstract: “gaze information of a user of the device, where the gaze information indicates where the user is looking”).
However, the combination of Lopez et al and Lumsden et al does not teach a gaze input and a gesture input.
Kienzle et al teaches a gaze input and a gesture input (Kienzle et al, abstract: “A first set of signals corresponding to a first signal modality (such as the direction of a gaze) during a time interval is collected from an individual. A second set of signals corresponding to a different signal modality (such as hand-pointing gestures made by the individual) is also collected. In response to a command, where the command does not identify a particular object to which the command is directed, the first and second set of signals is used to identify candidate objects of interest”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Lopez et al, Lumsden et al, and Kienzle et al such that the input includes both gaze and gesture inputs. This would have enabled the invention to identify more naturalistic human actions as inputs (Kienzle et al, paragraph 4: “Although some virtual reality systems may enable users to immerse themselves in selected environments, the extent to which naturalistic human behaviors can be used within such systems for general purposes is limited at best.”).
With regards to claim 8, which depends on claim 7, Lopez et al discloses outputting the first audio feedback comprises: outputting a first note in response to detecting the first portion… (Lopez et al, paragraph 62: “In operation 718, the eye tracking layer may return feedback to the user depending on whether the correct number in the PIN was gazed upon. The feedback may be any feedback that notifies the user that the digit gazed upon has been processed (e.g., vibration of the computing device, visual notification on the computing device, a sound, etc.).”).
However, Lopez et al does not disclose the gesture input includes a first portion of the gesture input and a second portion of the gesture input subsequent to the first portion… detecting the first portion of the gesture input… and outputting a second note different from the first note in response to detecting the second portion of the gesture input
Lumsden et al teaches outputting a second note different from the first note in response to detecting the second portion (Kelly et al, abstract: “a reference note (of a musical scale) that provides a marker of the goal or target position of the exercise for the subject, and a sequence of real-time feedback notes that define progress towards the goal;” paragraph 12: “providing audio feedback that permits intuitive distinction between non-goal and goal points within the exercise”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Lopez et al, Lumsden et al, and Kienzle et al such that the feedback for different inputs maps to differing audio feedback. This would have enabled the feedback to guide the user through an exercise (Lumsden et al, paragraph 11: “There remains a need for a simple and efficient audio feedback for training motor skills to a wide audience, especially one that is useful for guiding a subject through a cognitive exercise in which the feedback is interpreted and acted on in real time.”).
Kienzle et al teaches the gesture input includes a first portion of the gesture input and a second portion of the gesture input subsequent to the first portion… detecting the first portion of the gesture input… detecting the second portion of the gesture input (Kienzle et al, paragraph 5: “One or both of the data sets obtained may include timing information in some embodiments, e.g., the particular times at which various gestures were made or when the gaze direction remained steady for a certain amount of time may be indicated in the data sets.”)
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Lopez et al, Lumsden et al, and Kienzle et al such that the input includes both gaze and gesture inputs. This would have enabled the invention to identify more naturalistic human actions as inputs (Kienzle et al, paragraph 4: “Although some virtual reality systems may enable users to immerse themselves in selected environments, the extent to which naturalistic human behaviors can be used within such systems for general purposes is limited at best.”).
With regards to claim 9, which depends on claim 1, the combination of Lopez et al and Lumsden et al does not discloses wherein the selection includes a gaze input that is maintained on a respective gaze target element for a threshold duration of time.
However, Kienzle et al teaches wherein the selection includes a gaze input that is maintained on a respective gaze target element for a threshold duration of time (Kienzle et al, paragraph 5: “One or both of the data sets obtained may include timing information in some embodiments, e.g., the particular times at which various gestures were made or when the gaze direction remained steady for a certain amount of time may be indicated in the data sets”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Lopez et al, Lumsden et al, and Kienzle et al such that the inputs are identified based on a threshold amount of time they are held. This would have enabled the invention to identify steady gaze directions and fixation positions (Kienzle et al, paragraph 78: “an attempt to identify “stable directions” with respect to each of the modalities in the depicted embodiment… eye fixations (representing a steady gaze direction) and/or smooth pursuit eye movements (indicating slight variations in gaze directions) may be detected to identify stable directions”).
Claim(s) 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lopez et al in view of Lumsden et al, and further in view of Beymer et al (US20050243054A1; filed 4/29/2004).
With regards to claim 23, which depends on claim 22, Lopez et al discloses while displaying the first gaze enrollment user interface and prior to detecting the selection input, detecting, via the one or more input devices, a first gaze input (Lopez et al, abstract: “facilitate eye tracking control calibration are provided;” Fig. 6A-C: show the enrollment interface).
However, the combination of Lopez et al and Lumsden et al does not disclose in response to detecting the first gaze input: in accordance with a determination that the gaze input is directed toward the first gaze target element, displaying, via the one or more display generation components, a third visual indication indicative of the gaze of the user being directed toward the first gaze target element, wherein the third visual indication is different from the first visual indication; and in accordance with a determination that the gaze input is directed toward the second gaze target element, displaying, via the one or more display generation components, a fourth visual indication indicative of the gaze of the user being directed toward the second gaze target element, wherein the fourth visual indication is different from the second visual indication.
Beymer et al teaches in response to detecting the first gaze input: in accordance with a determination that the gaze input is directed toward the first gaze target element, displaying, via the one or more display generation components, a third visual indication indicative of the gaze of the user being directed toward the first gaze target element, wherein the third visual indication is different from the first visual indication; and in accordance with a determination that the gaze input is directed toward the second gaze target element, displaying, via the one or more display generation components, a fourth visual indication indicative of the gaze of the user being directed toward the second gaze target element, wherein the fourth visual indication is different from the second visual indication (Beymer et al, Fig. 8A: User gaze and keypress indicates a GUI object (steps 805, 810, 820), then the GUI object is highlighted (step 825). The highlighting of the object is interpreted as the third or fourth visual indications which are indicative of the gazes, depending on which object was gazed upon. The selection (step 845 and 850) of the object activates the object and removes the highlight, which is interpreted as either the first or second visual indication, depending on which object was selected).
It would have been obvious to a person of ordinary skill in the art before the effective filing date to have combined Lopez et al, Lumsden et al, and Beymer et al such that the object the user is gazing at is highlighted prior to selection. This would have enabled the invention to more precisely select the gazed-upon targets (Beymer et al, paragraph 2: “Imprecision can also be reduced by using a probabilistic model possibly comprising the geometry, application state, eye movements subsequent to object highlighting, or object history.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Stellmach et al (US20190324529A1): Teaches a gaze and gesture input to move objects in a GUI.
Lu et al (US20170364149A1): Teaches active calibration for gaze inputs which includes displaying sets of dots simultaneously.
Lee (US20080129520A1): Teaches providing musical notes as feedback to input such as scrolling through a list.
Darrah et al (US10402159B1): Teaches providing music notes as audio feedback for visually impaired users.
Aurongzeb et al (US20240028110A1): Teaches audio confirmation feedback to indicate the activation of gaze tracking.
Coffman et al (US20200201495A1): Teaches using harmonic tones to indicate progress being made in a setup interface.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRODERICK C ANDERSON whose telephone number is (313)446-6566. The examiner can normally be reached Monday-Tuesday, Thursday-Saturday 9-5 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at 5712724124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.C.A/Examiner, Art Unit 2178
/STEPHEN S HONG/Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights