DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Examiner acknowledges Request for Continued Examination and receipt of amendments/arguments filed 1/7/26. The arguments are addressed herein below. Claims 1, 4, 6-8, 12-15, and 18-28 are pending, Claims 2-3, 5, 9-11, and 16-17 are canceled, Claims 1, 6, 8, 13, 15, and 20 are currently amended, and Claims 21-28 are newly added.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4, 6-8, 12-13, 15, 18, 20, 22, 24-25, and 27-28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Davies (US 2019/0354172) in view of Bonnier (US 10,379,612 B1), and in further view of Nishizawa (US 2015/0169096 A1).
Claims 1, 8, 15, and 22: Davies discloses a system comprising: a head-mounted display (HMD) (Fig. 4, ¶ 50) comprising: a display panel (Figs. 4-5a, ¶ 50); and an eye tracking sensor (Fig. 1, ¶ 42 – “FIG. 1 shows an eye tracking system 100 (which may also be referred to as a gaze tracking system), according to an embodiment. The system 100 comprises illuminators 111 and 112 for illuminating the eyes of a user, and an image sensor 113 for capturing images of the eyes of the user”); an eye tracking system sensor (Fig. 1, ¶ 42 – “FIG. 1 shows an eye tracking system 100 (which may also be referred to as a gaze tracking system), according to an embodiment. The system 100 comprises illuminators 111 and 112 for illuminating the eyes of a user, and an image sensor 113 for capturing images of the eyes of the user”); a handheld controller (¶ 59 – “In some embodiments, the system uses a stimulus activation, or trigger signal, to determine that the user is likely interacting with the stimulus. For example, a stimulus activation can include: a mouse click; keyboard input; joystick movement; game pad or hand controller (VR) input; or a timer (i.e., the timing for which an element has been displayed on the screen)”); a processor (¶ 75); and memory (one or more non-transitory computer-readable media - ¶ 75, 96) storing computer-executable instructions that, when executed by the processor (¶ 75), cause the processor to (perform the method): present a user interface element on the display panel (¶ Figs. 6-7c, ¶ 59 – “For example, a stimulus activation can include: a mouse click; keyboard input; joystick movement; game pad or hand controller (VR) input; or a timer (i.e., the timing for which an element has been displayed on the screen). When a stimulus activation such as mouse reference 610 occurs, the system can copy the buffer, e.g. circular buffer 600, for further analysis. In various embodiments, depending on the type of stimulus, the system can: copy the record immediately; copy the record after either a predetermined or variable time period; or suspend changes to the record, i.e. add no new data to the record until it is analyzed.”); receive, from the handheld controller, user input data indicating that a user wearing the HMD has provided user input associated with the user interface element via the handheld controller; receive, from the eye tracking sensor, and in response to receiving the user input data from the handheld controller, eye data associated with one or more eyes of the user (Figs. 7a-c, ¶ 62); and calibrate the eye tracking system based at least in part on the eye data and location data indicating a location on the display panel where the user interface element is presented (Figs. 7a-c, ¶ 62-67).
Davies teaches the above, receive, from the handheld controller, first user input data indicating that a user wearing the HMD has provided first user input via the handheld controller to move a pointer over the user interface element; highlight, in response to receiving the first user input data from the handheld controller, the user interface element on the display panel, receive, from the handheld controller, after highlighting the user interface element, second user input data indicating that the user has provided a second user input via the handheld controller to select the user element; receiving, from the eye tracking sensor, in response to receiving the first user input data from the handheld controller, in response to highlighting the user interface element and in response to receiving the second user input data from the handheld controller, eye data associated with one or more eyes of the user. Davies at least teaches that various modifications can be applied without departing from the overall scope of the invention (¶ 91), the use of a cursor/pointer to select/highlight a user interface element (Figs. 7a-b, ¶ 60-62); and that the invention is applied to aspects pertaining to video games (¶ 61, 78). Furthermore, an analogous art of Bonnier teaches receive, from the handheld controller (Col. 3:51-Col. 4:7), first user input data indicating that a user wearing the HMD (Figs. 1-2, Col. 2:36-60, Col. 4:38-50, Col. 7:62-Col. 8:17) has provided first user input via the handheld controller to move a pointer (46) over the user interface element (44); highlight, in response to receiving the first user input data from the handheld controller, the user interface element on the display panel (Figs. 3-4, Col. 4:65-Col. 5:63, Col. 8:18-40 - here the first user input that corresponds to movement of the pointer over the user interface element, wherein the pointing being placed over the user interface element highlights the user interface element), receive, from the handheld controller, after highlighting the user interface element, second user input data indicating that the user has provided a second user input via the handheld controller to select the user element (Figs. 3-4, Col. 4:65-Col. 5:63, Col. 8:18-40 - here after the highlighting of the user element the user selects the user element by providing a second user input such as pressing of a mouse button or trackpad, etc); receiving, from the eye tracking sensor, in response to receiving the first user input data from the handheld controller, in response to highlighting the user interface element and in response to receiving the second user input data from the handheld controller, eye data associated with one or more eyes of the user (Figs. 3-4, Col. 4:65-Col. 5:63, Col. 8:18-40). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system, particularly, the user input means and in response to the receipt of user input means receiving eye data means of Davies with the first and second user input means and in response to the receipt of first and second user input means receiving eye data means as taught by Bonnier because such a modification would have yielded predictable results, namely, a means of calibrating the eye tracking system in which at least Davies is intended (see above). Such a modification improves the accuracy of calibration of the eye movement data (Bonnier – Col. 1:38-52).
Davies in view of Bonnier teaches the above, but lacks explicitly suggesting wherein highlighting the user interface element comprises at least one of: changing a color of the user interface element; increasing an intensity of the color; changing text of the user interface element to bold font; or underlining the text. Davies at least teaches that various modification can be applied without departing from the overall scope of the invention (see above). Furthermore, an analogous art of Nishizawa teaches wherein highlighting the user interface element comprises at least one of: changing a color of the user interface element; increasing an intensity of the color; changing text of the user interface element to bold font; or underlining the text (¶ 86). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the highlighting means of Davies in view of Bonnier with the change in color of the user interface element means of Nishizawa because such a modification would have yielded predictable results, namely, a means of highlighting a user interface element to provide a indication of selection in which at least Davies in view of Bonnier is intended (see above). Such a modification shows the user that the user interface element is in a selectable state (Nishizawa - ¶ 86); making the overall user interface experience more user friendly.
Claims 4, 12, and 18: Davies teaches wherein the user interface element is presented during gameplay of a video game on the HMD (Fig. 9, ¶ 61, 78 – “In a typical example, the stimulus event can be an event in the user interface (UI) that attracts the user's gaze (e.g., an animation or motion in a video game), or a user input event that requires the user's attention (e.g., a mouse click or typing)”).
Claims 6, 13, and 20: Davies teaches wherein the computer-executable instructions, when executed by the processor, further cause the processor to, after calibrating the eye tracking system (¶ 56-57, 75): execute an application that uses the eye tracking system (¶ 75); present a second user interface element on the display panel (Figs. 7a, c); receive, from the handheld controller, third user input data indicating that the user has provided third user input associated with the second user interface element via the handheld controller (¶ 60-67); receive, from the eye tracking sensor, and in response to receiving the third user input from the handheld controller, second eye data associated with the one or more eyes of the user (Figs. 8, ¶ 60-67, 70 – “The user interface stimulus events are denoted as circles, and the gaze projected using a trial set of eye model parameters is shown as an X. The eye tracking system can access one or more interaction observations to determine the associated eye sequence and stimulus event. The system then determines gaze information from the eye sequence, and projects the user's gaze from the gaze information according to a trial eye model (i.e., determines a projected location where the gaze would fall according to the trial eye model)”); and recalibrate the eye tracking system based at least in part on the second eye data and second location data indicating a second location on the display panel where the second user interface element is presented (Abstract, Figs. 8, ¶ 62-67, 70 – “The system then computes a distance between the projected gaze location and the stimulus event location indicated in the interaction observation. The strategy of the recalibration is to update the eye model such that the projected gaze and stimulus event are as proximate to each other as possible, thereby minimizing this distance”, ¶ 80). In this example, prior calibrations are stored and updated to reflect new interactions (see above), and another user interface can be provided that includes a second user interface element such as clickable hyperlink (¶ 61) to the user, wherein the process pertaining to at least Figs. 7a, c is repeated for another calibration of the hmd (see above).
Claim 7: Davies teaches wherein the eye tracking sensor comprises a camera (Fig. 1, ¶ 42- “The image sensor 113 may for example be a camera, such as a complementary metal oxide semiconductor (CMOS) camera or a charged coupled device (CCD) camera”); and the eye data comprises image data of the one or more eyes of the user (Fig. 1, ¶ 42- “FIG. 1 shows an eye tracking system 100 (which may also be referred to as a gaze tracking system), according to an embodiment. The system 100 comprises illuminators 111 and 112 for illuminating the eyes of a user, and an image sensor 113 for capturing images of the eyes of the user”).
Claims 24 and 27: Davies in view of Bonnier in view of Nishizawa teaches the above, but lacks explicitly suggesting wherein the highlighting the user interface element comprises the increasing of the intensity of the color or the changing of the text of the user interface element to the bold font. Davies in view of Bonnier in view of Nishizawa discloses that highlighting the user interface element comprises changing a color of the user interface element (see above). However, applicant fails to disclose that the highlighting the user interface element comprises the increasing of the intensity of the color or the changing of the text of the user interface element to the bold font solves any stated problem, provides an advantage, or is for any particular purpose. Furthermore, Applicant discloses to “highlight the user interface element 120 on the display panel(s) 114, such as by changing a color of the user interface element 120, increasing an intensity of the color of the user interface element 120, changing the text (e.g., “LOAD”) to bold, underlining the text, causing the user interface element 120 to “pop-out” from a background scene and/or to “pop-out” relative to surrounding user interface elements, or the like” (¶ 23, 33), which suggests that how it is highlighted is a matter of design as long as it is highlighted. Moreover, it appears that the highlighting means of Davies in view of Bonnier in view of Nishizawa, or applicant’s invention, would perform the same function of highlighting the user interface element, regardless of how the user interface element is highlighted. Therefore, it would have been prima facie obvious to modify Davies in view of Bonnier in view of Nishizawa to obtain the invention as specified in claims 24 & 27 because such a modification would have been considered a mere design consideration which fails to patentably distinguish over the prior art of Davies in view of Bonnier in view of Nishizawa.
Claims 25 and 28: Davies in view of Bonnier in view of Nishizawa teaches the user interface element is a button or icon (Davies - ¶ 62, Bonnier – Col. 4:65-Col. 5:10, Col. 7:10-50). The examiner understands that a virtual button is representative of a icon.
Claim(s) 14 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Davies (US 2019/0354172) in view of Bonnier (US 10,379,612 B1), and in further view of Nishizawa (US 2015/0169096 A1), and in further view of Noda (WO 2021/153577 A1)(please refer to WO2020266229A1 English Translation found at https://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval&COUNTRY=WO&ENGINE=google&FORMAT=docdb&KIND=A1&LOCALE=en_EP&NUMBER=2021153577&OPS=ops.epo.org/3.2&SRCLANG=ja&TRGLANG=en and attached as a PDF).
Claims 14 and 19: Davies in view of Bonnier in view of Nishizawa teaches the above, but lacks explicitly suggesting waiting, by the processor, to execute an application that uses the eye tracking system until the calibrating; and starting, by the processor, execution of the application after the calibrating. Davies at least teaches that various modifications can be applied without departing from the overall scope of the invention (see above). Furthermore, an analogous art of Noda teaches a similar medium and/or method that includes waiting, by the processor (¶ 59), to execute an application that uses the eye tracking system until the calibrating (¶ 20 – “The correction control unit may perform the notification process at least one of the following times: when a task performed by the user is completed, when the task is being executed, or before an application that uses gaze is executed”, ¶ 48 – “The first calibration data 21 is typically generated before executing an application or the like that utilizes the user's line of sight.”, ¶ 124 – “Also, immediately before the user next executes an application that uses the line of sight, a window 39 as shown in FIG. 13A may be displayed to inform the user that static calibration is necessary. In this manner, the abnormality determination unit 50 performs the notification process at a timing before the execution of an application that utilizes the line of sight. This makes it possible to perform the first calibration appropriately from an early stage of using the application.”); and starting, by the processor (¶ 59), execution of the application after the calibrating (¶ 20 – “The correction control unit may perform the notification process at least one of the following times: when a task performed by the user is completed, when the task is being executed, or before an application that uses gaze is executed”, ¶ 48 – “The first calibration data 21 is typically generated before executing an application or the like that utilizes the user's line of sight.”, ¶ 124 – “Also, immediately before the user next executes an application that uses the line of sight, a window 39 as shown in FIG. 13A may be displayed to inform the user that static calibration is necessary. In this manner, the abnormality determination unit 50 performs the notification process at a timing before the execution of an application that utilizes the line of sight. This makes it possible to perform the first calibration appropriately from an early stage of using the application.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the medium and/or method, as it pertains to particularly the calibration process of Davies in view of Bonnier in view of Nishizawa, with the waiting/starting means of Noda because such a modification would help prevent a decrease in the accuracy of gaze detection (Noda - ¶ 6, 127-128).
Claim(s) 21, 23, and 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Davies (US 2019/0354172) in view of Bonnier (US 10,379,612 B1) in view of Nishizawa (US 2015/0169096 A1), and in further view of Hudman (US 11,073,906).
Claim 21, 23, and 26: Davies in view of Bonnier in view of Nishizawa teaches the above, but lacks explicitly suggesting further cause the processor to, after calibrating the eye tracking system: execute a virtual reality application that uses the eye tracking system; determine, based at least in part on eye tracking data generated by the eye tracking system, a gaze point associated with the user; and use the gaze point for wireless foveated transport of pixel data generated by the virtual reality application to the HMD. Davies at least teaches that various modifications can be applied without departing from the overall scope of the invention (see above). Furthermore, an analogous art of Hudman teaches further cause the processor (125) to, after calibrating the eye tracking system (135)(Figs. 1-2, Col. 14:53-Col. 15:19, Col. 16:58-Col. 17:3, Col. 33-35): execute a virtual reality application that uses the eye tracking system (Abstract, Col. 4:39-67, Col. 5:1-29, Col. 6:50-Col. 7:11, Col. 8:43-63, Col. 12:46-Col. 13:12, Col. 15:20-32); determine, based at least in part on eye tracking data generated by the eye tracking system, a gaze point associated with the user; and use the gaze point for wireless (Col. 6:35-45, Col. 7:35-49, Col. 8:43-Col. 9:13) foveated transport of pixel data generated by the virtual reality application to the HMD (see above, Col. 4:55-67, Col. 5:30-61, Col. 17:4-18). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system, method, and/or medium of Davies in view of Bonnier in view of Nishizawa with the foveated teachings describe above of Hudman because such a modification provides VR content in such a way that conserves power, bandwidth, and/or computational resources (Hudman - Col. 4:55-67, Col. 5:30-61, Col. 17:4-18); thereby making the overall VR experience more optimal or efficient.
Response to Arguments
Applicant’s arguments with respect to claim(s) Claims 1, 4, 6-8, 12-15, and 18-28 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The prior art of at least Bonnier (US 10,379,612 B1) and Nishizawa (US 2015/0169096 A1) is used to teach the newly added limitations applied to Davies.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Please see PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRAMAR HARPER whose telephone number is (571)272-6177. The examiner can normally be reached 7:30am to 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kang Hu can be reached at (571) 270-1344. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRAMAR HARPER/Primary Examiner, Art Unit 3715