DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-26 are pending.
Claim Rejections - 35 USC § 102
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office Action.
Claims 1, 17-18 and 22-26 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Rohrbacher (US 2019/0294312 A1, IDS).
As to claim 1, Rohrbacher discloses a method (Rohrbacher, FIG. 10, [0057], a “method for controlling a combined eye and gesture tracking system 100”) comprising:
at an electronic device (Rohrbacher, FIG. 1, [0039], “gesture tracking system 100”) in communication with a display generation component (Rohrbacher, FIG. 3, [0039], “graphical user interface 102”; [0045], “the graphical user interface may be depicted on a cluster instrument display of a vehicle, a center console display, an infotainment display, and/or a head-up display (HUD)”) and one or more input devices (Rohrbacher, FIG. 1, [0039], e.g., “camera 101”), including a gaze tracking device (Rohrbacher, e.g., FIG. 9, [0056], a device performing “detecting a user eye gaze 320”):
while displaying, via the display generation component, a user interface (Rohrbacher, FIG. 3, [0039], “graphical user interface 102”; [0045], “the graphical user interface may be depicted on a cluster instrument display of a vehicle, a center console display, an infotainment display, and/or a head-up display (HUD)”) that includes a cursor (Rohrbacher, FIGS. 2-4, [0048], “graphical pointer 120”), detecting, via the one or more input devices (Rohrbacher, FIG. 1, [0039], e.g., “camera 101”), a cursor movement input corresponding to a first movement value (Rohrbacher, FIG. 9, [0056], “detecting a displacement user gesture 330”; FIGS. 2-4, [0051], e.g., “displacement direction 231”); and
in response to detecting the cursor movement input:
in accordance with a determination that the cursor movement input includes input corresponding to movement of the cursor towards a gaze location, in the user interface, of a gaze of a user of the electronic device, wherein the gaze location is detected via the gaze tracking device, moving the cursor in the user interface (Rohrbacher, FIG. 9, [0056], “controlling a rate of movement of the graphical pointer 340”) by a first amount in accordance with the cursor movement input (Rohrbacher, FIG. 9, [0056], “the rate of movement can increase or decrease during the movement, or can be slower respectively faster than the rate of movement of the displacement user gesture 230 at the beginning or end of the movement of the graphical pointer 120, based on the user eye gaze 210”; [0058], “The gravity effect can accelerate when the graphical pointer 120 gets closer towards the eye focus position 130”); and
in accordance with a determination that the cursor movement input includes input corresponding to movement of the cursor away from the gaze location in the user interface, moving the cursor in the user interface by a second amount in accordance with the cursor movement input (Rohrbacher, FIG. 9, [0056], “the rate of movement can increase or decrease during the movement, or can be slower respectively faster than the rate of movement of the displacement user gesture 230 at the beginning or end of the movement of the graphical pointer 120, based on the user eye gaze 210”), wherein the second amount is less than the first amount (Rohrbacher, FIGS. 4-9, [0058], “The gravity effect can accelerate when the graphical pointer 120 gets closer towards the eye focus position 130”; it is reasonably inferred that the “gravity effect slows down when the graphical pointer 120 gets away from the eye focus position 130”).
As to claim 17, Rohrbacher the method of claim 1, wherein the cursor (Rohrbacher, FIGS. 2-4, [0048], “graphical pointer 120”) includes a visual indication of a location of an input focus (Rohrbacher, FIG. 4, [0051], the “eye focus position with an active gravity effect 131”) in the user interface (Rohrbacher, FIG. 4, [0050], “graphical user interface 102”).
As to claim 18, Rohrbacher discloses the method of claim 1, wherein the cursor (Rohrbacher, FIGS. 2-4, [0048], “graphical pointer 120”) includes a visual indication of a location of a selection input focus (Rohrbacher, FIG. 4, [0051], the “eye focus position with an active gravity effect 131” on “graphical items 110” to be selected) in the user interface (Rohrbacher, FIG. 4, [0050], “graphical user interface 102”).
As to claim 22, Rohrbacher discloses the method of claim 1, wherein the movement towards the gaze location in the user interface is towards the gaze location in two dimensions in the user interface (Rohrbacher, see FIG. 4).
As to claim 23, Rohrbacher discloses the method of claim 1, wherein the movement away from the gaze location in the user interface is away from the gaze location in two dimensions in the user interface (Rohrbacher, see FIG. 4).
As to claim 24, Rohrbacher discloses the method of claim 1, further comprising: in response to detecting the cursor movement input:
in accordance with a determination that the cursor movement input includes input corresponding to movement of the cursor towards the gaze location in a first dimension and away from the gaze location in a second dimension, moving the cursor in the user interface towards the gaze location in the first dimension by a third amount, and away from the gaze location in the second dimension by a fourth amount (Rohrbacher, FIG. 9, [0056], “the rate of movement can increase or decrease during the movement, or can be slower respectively faster than the rate of movement of the displacement user gesture 230 at the beginning or end of the movement of the graphical pointer 120, based on the user eye gaze 210”; [0058], “The gravity effect can accelerate when the graphical pointer 120 gets closer towards the eye focus position 130”; the 3rd amount (toward) > the 4th amount (away)); and
in accordance with a determination that the cursor movement input includes input corresponding to movement of the cursor away from the gaze location in the first dimension and towards the gaze location in the second dimension, moving the cursor in the user interface away from the gaze location in the first dimension by a fifth amount that is less than the third amount, and towards the gaze location in the second dimension by a sixth amount that is greater than the fourth amount (Rohrbacher, FIG. 9, [0056], “the rate of movement can increase or decrease during the movement, or can be slower respectively faster than the rate of movement of the displacement user gesture 230 at the beginning or end of the movement of the graphical pointer 120, based on the user eye gaze 210”; [0058], “The gravity effect can accelerate when the graphical pointer 120 gets closer towards the eye focus position 130”; the 5th amount (away) < the 6th amount (toward) & the 4th amount (away) < the 6th amount (toward)).
As to claim 25, it differs from claim 1 only in that it is the electronic device performing the method of claim 1. It recites substantially the same limitations as in claim 1, and Rohrbacher discloses them. Please see claim 1 for detailed analysis.
As to claim 26, it differs from claim 1 only in that it is the non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform the method of claim 1. It recites substantially the same limitations as in claim 1, and Rohrbacher discloses them. Please see claim 1 for detailed analysis.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office Action.
Claims 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Rohrbacher (US 2019/0294312 A1, IDS) in view of Park (US 2019/0354259 A1).
As to claim 3, Rohrbacher does not teach the method of claim 1, wherein moving the cursor in the user interface by the first amount in accordance with the determination that the cursor movement input includes input corresponding to movement of the cursor towards the gaze location is further in accordance with a determination that the cursor movement input is not part of a content selection input in the user interface, the method further comprising: in response to detecting the cursor movement input: in accordance with a determination that the cursor movement input is part of the content selection input in the user interface, moving the cursor in the user interface by a third amount in accordance with the cursor movement input irrespective of whether the cursor movement input includes input corresponding to movement of the cursor towards or away from the gaze location in the user interface.
However, Park teaches the concept that, in response to detecting the cursor movement input: in accordance with a determination that the cursor movement input is part of the content selection input in the user interface (Park, FIG. 1, [0040], “when the user desires to select a point on the virtual object 305, the user's gaze may be positioned on the virtual object 305”), moving the cursor in the user interface by a third amount in accordance with the cursor movement input irrespective of whether the cursor movement input includes input corresponding to movement of the cursor towards or away from the gaze location in the user interface (Park, FIG. 1, [0040], “As the pointer of the control device 200 is closer to the user's gaze, more detailed manipulation may be required. At this time, the electronic device 100 may reduce the sensitivity of the motion of the control device 200, by lowering the movement speed of the pointer of the control device 200. On the other hand, when the pointer of the control device 200 is farther away from the user's gaze, the electronic device 100 may increase the sensitivity of the motion of the control device 200, by increasing the movement speed of the pointer of the control device 200”).
At the time of effective filing date, it would have been obvious to one of ordinary skill in the art to modify the step of moving the cursor in the user interface by the first amount in accordance with the determination that the cursor movement input includes input corresponding to movement of the cursor towards the gaze location (Rohrbacher, FIG. 9, [0056], “the rate of movement can increase or decrease during the movement, or can be slower respectively faster than the rate of movement of the displacement user gesture 230 at the beginning or end of the movement of the graphical pointer 120, based on the user eye gaze 210”; [0058], “The gravity effect can accelerate when the graphical pointer 120 gets closer towards the eye focus position 130”) taught by Rohrbacher to further comprise varying the speed and sensitivity of the motion of the cursor, as taught by Park, in order to provide “an electronic device capable of improving the accuracy and convenience of manipulation by a control device in a virtual space” (Park, [0005]).
As to claim 4, Park teaches the method of claim 3, wherein the user interface includes text content (Park, e.g., see FIG. 1, “object 305” includes texts), and the content selection input in the user interface is a text selection input in the user interface (Park, see FIG. 1). Examiner renders the same motivation as in claim 3.
As to claim 5, Park teaches the method of claim 1, wherein moving the cursor in the user interface by the first amount in accordance with the determination that the cursor movement input includes input corresponding to movement of the cursor towards the gaze location is further in accordance with a determination that the cursor is a first distance from the gaze location in the user interface when the cursor movement input is detected, the method further comprising:
in response to detecting the cursor movement input: in accordance with the determination that the cursor movement input includes input corresponding to movement of the cursor towards the gaze location in the user interface, and in accordance with a determination that the cursor is a second distance, less than the first distance, from the gaze location in the user interface when the cursor movement input is detected, moving the cursor in the user interface by a third amount, less than the first amount, in accordance with the cursor movement input (Park, FIG. 3, [0043], “move first point within virtual space at speed determined based on distance between first point and second point 350”; FIG. 4, [0049], “electronic device 100 may move the first point 301 within the virtual space 300 at a speed determined based on a distance 303 between the first point 301 and the second point 302, in response to the obtaining of motion information”; FIG. 5, [0058], “electronic device 100 may determine the speed based on the pre-stored data and the distance between first point 301 and the second point 302”). Examiner renders the same motivation as in claim 3.
Claims 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Rohrbacher (US 2019/0294312 A1, IDS) in view of Mojaver et al. (US 2018/0220080 A1).
As to claim 14, Rohrbacher does not teach the method of claim 1, wherein detecting the cursor movement input includes detecting, via the one or more input devices, movement of at least a portion of a hand of the user.
However, Mojaver teaches the concept of detecting, via the one or more input devices, movement of at least a portion of a hand of the user (Mojaver, FIGS. 3-4, [0042], e.g., “the processor can utilize appropriate image recognition software to detect the position and posture of the individual fingers on each hand”).
At the time of effective filing date, it would have been obvious to one of ordinary skill in the art to modify the movement of “graphical pointer 120” taught by Rohrbacher to be further associated with the movement of, e.g., “fingers” detected by the camera, as taught by Mojaver, in order to “take advantage of the presence of the operator's hands and fingers within the field of view of the camera” (Mojaver, [0007]) for easier and more intuitive cursor control.
As to claim 15, Mojaver teaches the method of claim 14, wherein detecting the cursor movement input includes detecting, via the one or more input devices, movement of a finger of the hand of the user (Mojaver, FIGS. 3-4, [0042], e.g., “the processor can utilize appropriate image recognition software to detect the position and posture of the individual fingers on each hand”). Examiner renders the same motivation as in claim 14.
As to claim 16, Mojaver teaches the method of claim 14, wherein detecting the cursor movement input includes detecting, via the one or more input devices, movement of at least the portion of the hand of the user while the hand of the user is in a predefined hand shape (Mojaver, FIGS. 3-4, [0042], e.g., “index-point”, “index-middle-point”, “index-trigger”, etc.). Examiner renders the same motivation as in claim 14.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Rohrbacher (US 2019/0294312 A1, IDS) in view of Andersen et al. (US 10,402,081 B1).
As to claim 19, Rohrbacher does not teach the method of claim 1, further comprising: while displaying, via the display generation component, the user interface that includes the cursor, detecting, via the one or more input devices, a scrolling input that includes detecting movement of a predefined portion of the user while the predefined portion of the user is in a respective shape; and in response to detecting the scrolling input, scrolling the user interface in accordance with the movement of the predefined portion of the user while the predefined portion of the user makes the respective shape.
However, Andersen teaches the concept of, while displaying, via the display generation component, the user interface that includes the cursor, detecting, via the one or more input devices, a scrolling input that includes detecting movement of a predefined portion of the user while the predefined portion of the user is in a respective shape (Andersen, FIG. 4, “ FIG. 4 is a diagram of exemplary hand gestures captured by the sensor device 103 and analyzed by the thumb scroll control module 112 to determine scroll speed and scroll direction”); and
in response to detecting the scrolling input, scrolling the user interface in accordance with the movement of the predefined portion of the user while the predefined portion of the user makes the respective shape (Andersen, FIG. 4, “As shown in FIG. 4, a user can place his or her thumb and forefinger within the defined zone of a graphical thumb scroll element 304. To scroll up the graphical content in the associated surface plane, the user can rotate his thumb and forefinger in a forward motion around the graphical thumb scroll element 304—so that the thumb starts below (or in front of) the graphical thumb scroll element 304 and moves upward around the front side of the element, while at the same time the forefinger starts above (or behind) the graphical thumb scroll element 304 and moves downward around the back side of the element”).
At the time of effective filing date, it would have been obvious to one of ordinary skill in the art to modify the “camera 101” taught by Rohrbacher to further recognize the user’s hand gesture to implement the “graphical thumb scroll element 304”, as taught by Andersen, in order to provide “an advantageous virtual thumb scroll user interface element that is easy to operate and enables a user to accurately and precisely control UI elements such as windows, menus, and other controls in VR, MR, and AR environments” (Andersen, col. 1, ll. 40-44).
Claims 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Rohrbacher (US 2019/0294312 A1, IDS) in view of Lin et al. (US 2006/0190832 A1).
As to claim 20, Rohrbacher does not teach the method of claim 1, further comprising: while displaying, via the display generation component, the user interface that includes the cursor, detecting, via the one or more input devices, a scrolling input; and in response to detecting the scrolling input: displaying an animated transition of the cursor fading; and scrolling the user interface in accordance with the scrolling input.
However, Lin teaches the concept of while displaying, via the display generation component, the user interface that includes the cursor, detecting, via the one or more input devices, a scrolling input; and in response to detecting the scrolling input: cursor disappearing; and scrolling the user interface in accordance with the scrolling input (Lin, FIG. 22, [0165], “Referring to (22-5) of FIG. 22, the cursor disappears. The cursor may disappear when the scroll bar 415 is tapped or double-clicked. After the cursor has disappeared, if the scroll bar 415 is touched and dragged in a vertical direction, the webpage is scrolled in the vertical direction as well”).
At the time of effective filing date, it would have been obvious to one of ordinary skill in the art to modify the “graphical pointer 120” taught by Rohrbacher to disappear when scrolling gesture is detected, using, e.g., animated fading effect as an engineering choice, in order to provide “various functions and menus to facilitate the user manipulating the terminal” (Lin, [0006]).
As to claim 21, Lin teaches the method of claim 20, further comprising: while scrolling the user interface in accordance with the scrolling input and while the cursor is not displayed in the user interface, detecting, via the one or more input devices, an end of the scrolling input; and in response to detecting the end of the scrolling input: initiating a process to cease the scrolling of the user interface; and initiating display of the cursor in the user interface via the display generation component (Lin, FIG. 23, [0167], “Referring to (23-2) of FIG. 23, a cursor and a touch input device, for example, a virtual trackball, for shifting the cursor are generated on the webpage. The cursor and the touch input device may be generated in response to a predefined manipulation of the user input unit 140”, e.g., detecting the end of the scrolling input). Examiner renders the same motivation as in claim 20.
Allowable Subject Matter
Claims 2 and 6-13 would be allowable if rewritten to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
As to claim 2, the closest known prior art, i.e., Rohrbacher (US 2019/0294312 A1, IDS), Park (US 2019/0354259 A1), Andersen et al. (US 10,402,081 B1), Lin et al. (US 2006/0190832 A1), West et al. (US 2019/0018498 A1), Publicover et al. (US 2018/0275753 A1, IDS) and Peterson et al. (US 2023/0185368 A1), alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “in accordance with a determination that the cursor movement input is not part of a content creation input in the user interface, the method further comprising: in response to detecting the cursor movement input: in accordance with a determination that the cursor movement input is part of the content creation input in the user interface, moving the cursor in the user interface by a third amount in accordance with the cursor movement input irrespective of whether the cursor movement input includes input corresponding to movement of the cursor towards or away from the gaze location in the user interface”.
As to claim 6, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “detecting, via the one or more input devices, additional cursor movement input in the first direction; and in response to detecting the additional cursor movement input in the first direction: in accordance with a determination that the additional cursor movement input in the first direction is towards the first location, moving the cursor in the user interface by a third amount in accordance with the additional cursor movement input; and in accordance with a determination that the additional cursor movement input in the first direction is away from the first location, moving the cursor in the user interface by a fourth amount, less than the third amount, in accordance with the additional cursor movement input”.
As to claim 7, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “detecting, via the one or more input devices, additional cursor movement input in the first direction; and in response to detecting the additional cursor movement input in the first direction: in accordance with a determination that the additional cursor movement input in the first direction is towards the second location, moving the cursor in the user interface by a third amount in accordance with the additional cursor movement input; and in accordance with a determination that the additional cursor movement input in the first direction is away from the second location, moving the cursor in the user interface by a fourth amount, less than the third amount, in accordance with the additional cursor movement input”.
As to claim 8, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “while detecting a first portion of the second cursor movement input that corresponds to a second movement value, wherein the gaze location is a first location in the user interface and the first portion of the second cursor movement input is towards the first location, moving the cursor in the user interface by a third amount in accordance with the first portion of the second cursor movement input; after detecting the first portion of the second cursor movement input, detecting that the gaze location has changed to a second location in the user interface; and in response to detecting that the gaze location has changed to the second location in the user interface and while detecting a second portion of the second cursor movement input after the first portion of the second cursor movement input, wherein the second portion of the second cursor movement input corresponds to the second movement value and the second portion of the second cursor movement input is away from the second location: during a first time after detecting that the gaze location has changed, moving the cursor in the user interface by a fourth amount, less than the third amount, in accordance with the second portion of the second cursor movement input; and during a second time, after the first time, after detecting that the gaze location has changed, moving the cursor in the user interface by a fifth amount, less than the fourth amount, in accordance with the second portion of the second cursor movement input”.
As to claim 9, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “while displaying, via the display generation component, the user interface that includes the cursor, detecting, via the one or more input devices, a second cursor movement input; and while detecting the second cursor movement input: while detecting a first portion of the second cursor movement input that corresponds to a second movement value, wherein the cursor is moving towards the gaze location in the user interface during the first portion of the second cursor movement input, moving the cursor in the user interface by a third amount in accordance with the first portion of the second cursor movement input; after detecting the first portion of the second cursor movement input and while detecting a second portion of the second cursor movement input after the first portion of the second cursor movement input, wherein the second portion of the second cursor movement input corresponds to the second movement value, and wherein the cursor is moving away from the gaze location in the user interface during the second portion of the second cursor movement input: during a first time after detecting a start of the second portion of the second cursor movement input, moving the cursor in the user interface by a fourth amount, less than the third amount, in accordance with the second portion of the second cursor movement input; and during a second time, after the first time, after detecting the start of the second portion of the second cursor movement input, moving the cursor in the user interface by a fifth amount, less than the fourth amount, in accordance with the second portion of the second cursor movement input”.
As to claim 10, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “detecting, via the one or more input devices, a second cursor movement input corresponding to a second movement value; and in response to detecting the second cursor movement input: in accordance with a determination that the second cursor movement input includes input corresponding to movement of the cursor towards a respective location in the user interface, associated with a location of a predefined portion of the user of the electronic device, wherein the respective location is detected via the one or more input devices, moving the cursor in the user interface by a third amount in accordance with the cursor movement input”.
As to claim 11, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “moving the cursor relative to the gaze location with a simulated physical property, and moving the cursor in the user interface by the second amount in response to detecting the cursor movement input in accordance with the determination that the cursor movement input includes input corresponding to movement of the cursor away from the gaze location includes moving the cursor relative to the gaze location with the simulated physical property”.
As to claim 12, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitation “the simulated physical property is simulated magnetic attraction”.
As to claim 13, the closest known prior art indicated above, alone or in reasonable combination, fails to teach limitations in consideration of the claims as a whole, specifically with respect to the limitations “detecting, via the one or more input devices, a second cursor movement input corresponding to a second movement value greater than the first movement value; and in response to detecting the second cursor movement input: in accordance with a determination that the second cursor movement input includes input corresponding to movement of the cursor towards the gaze location in the user interface, moving the cursor in the user interface by a third amount in accordance with the cursor movement input, wherein the third amount is greater than the first amount; and in accordance with a determination that the second cursor movement input includes input corresponding to movement of the cursor away from the gaze location in the user interface, moving the cursor in the user interface by a fourth amount in accordance with the cursor movement input, wherein the fourth amount is greater than the second amount and less than the third amount”.
Conclusion
The prior arts made of record and not relied upon are considered pertinent to applicant’s disclosure: Peterson et al. (US 2023/0185368 A1) teaches the concept of “detecting, using a camera sensor, a user gaze position … identifying, using a processor, a first location of a cursor on the display screen” (Abs.).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RICHARD J HONG whose telephone number is (571) 270-7765. The examiner can normally be reached on 9:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chanh Nguyen can be reached on (571) 272-7772. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
May 22, 2026
/RICHARD J HONG/Primary Examiner, Art Unit 2623
***
Prosecution Insights