DETAILED ACTION
This is a first office action in response to application 19/203,225 filed 05/09/2025, in which claims 1-10 are presented for examination. Currently claims 1-10 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 5-7, 8-10 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Stengel et al. U.S. Patent Application Publication No. 2021/0350550 A1 hereinafter Stengel.
Consider Claim 1:
Stengel discloses an eye tracking device, comprising: (Stengel, See Abstract.)
a transceiver; and a processor, coupled to the transceiver, wherein the processor is configured to: (Stengel, [0045] In at least one embodiment, a user 102 can view digital content via a display mechanism, as may include a display 106 internal to a virtual reality (VR) headset. In at least one embodiment, such content can be presented using an alternative display mechanism, such as a monitor, touchscreen, augmented reality (AR) display, or projection. In at least one embodiment, for VR content this VR headset may be in communication with a separate client device 110 that can provide, transmit, or render content to be presented via VR headset 104. In at least one embodiment, VR headset 104 can include one or more orientation sensors that provide data for movement of a head of user 102, which can be used to control a point of view (POV) of content rendered via headset 104, in order to provide a realistic and immersive experience for user 102. In at least one embodiment, a VR application 112 executing on client device 110 can accept orientation data from headset 104 and use this information for rendering VR content to be presented via headset 104. In at least one embodiment, client device 110 may be in contact with at least one content server 130 that can stream or otherwise transmit content over at least one network 120, such as a wired or wireless communication network, to be used in rendering or presenting this VR content. In at least one embodiment, a transmission manager 132 on content server 130 can be responsible for transmitting content in an appropriate format and channel, such as in a streaming video format, over an appropriate network connection, such as an Internet or cellular connection. In at least one embodiment, VR content to be displayed can be rendered on content server 130, as may relate to orientation data received from headset 104, and transmitted over network 120 to client device 110 and ultimately VR headset 104.”)
obtain an eye image set via the transceiver; (Stengel, [0065], “In at least one embodiment, a process 400 for determining gaze of a user can be utilized as illustrated in FIG. 4. In at least one embodiment, one or more eye images can be captured 402 for a user, such as by using one or more cameras. In at least one embodiment, these can be ambient or infrared images of any appropriate resolution and format. In at least one embodiment, a determination can be made 404 as to whether there are separate images obtained for both eyes of this user.”)
input the eye image set to a machine learning model to obtain an estimated eye image; and (Stengel, [0065], “In at least one embodiment, a determination can be made 404 as to whether there are separate images obtained for both eyes of this user. If so, those images can be concatenated 406 into a single image, separated by a determined amount of padding to prevent cross-talk and other such issues. In at least one embodiment, this current image can be compared 408 against a prior image to determine whether there has been a significant change in pupil position. In at least one embodiment, this can be based on intensities or other information in those images. In at least one embodiment, if it is determined 410 there has been an actionable change, such as an amount of change meeting or exceeding a determined threshold, then that image can be analyzed 412 using a coarse CNN to obtain a coarse pupil position estimate.”)
perform eye tracking according to the estimated eye image. (Stengel, [0065], “In at least one embodiment, this image of an estimated pupil region can then be analyzed 416 using this fine CNN to obtain a fine pupil position estimate. In at least one embodiment, this fine pupil estimate can be used to determine 418 user gaze.”)
Consider Claim 2:
Stengel discloses the eye tracking device according to claim 1, wherein the processor is further configured to: perform interpolation between a first image of the eye image set and the estimated eye image to generate an interpolated eye image; and perform the eye tracking according to the interpolated eye image and the estimated eye image. (Stengel, [0050], “In at least one embodiment, a two-phase approach to pupil position determination can be utilized that takes advantage of at least two neural networks, such as convolutional neural networks (CNNs). In at least one embodiment, these networks include a course CNN 212 and a fine CNN 214. In at least one embodiment, pupil center estimation accuracy is achieved by making use of these different CNNs to process incoming frames of data from an eye tracking camera. In at least one embodiment, a coarse CNN 212 can receive as input full eye images as captured by an eye-tracker camera, and can process these images to provide a coarse estimate, or first estimate, of pupil center position. In at least one embodiment, a fine CNN 214 can then analyze portions of these input images that correspond to pupil regions, as determined by coarse CNN 212. In at least one embodiment, fine CNN 214 can provide more accurate pupil center estimates than coarse CNN 212, which analyzes entire captured images. In at least one embodiment, fine CNN 214 can further be trained to provide accurate pupil localization within approximate pupil regions of captured images.”)
Consider Claim 3:
Stengel discloses the eye tracking device according to claim 1, wherein the eye image set comprises a first eye image corresponding to a first time point and a second eye image corresponding to a second time point different from the first time point. (Stengel, [0048], “In at least one embodiment, pupil localization can be performed on a processor such as graphics processing unit (GPU), which can be optimized in such a way as to provide fast performance for localization determinations. In at least one embodiment, various approaches can also improve accuracy of determinations while still achieving such levels of performance. In at least one embodiment, such advantages can be obtained for both monocular and binocular pupil center estimation. In at least one embodiment, pupil localization for gaze estimation using deep learning can start with an uploading or providing of one or more trained neural networks to one or more GPUs of a device to perform inferencing. In at least one embodiment, at inference time, a sequence of images will be received that include views of one or more eyes of a user during a relevant session or experience. In at least one embodiment, this image data can be stored in system memory until such time as it is transferred to GPU memory for inferencing. In at least one embodiment, network inferencing can be run on this GPU to compute pupil center position in image space. In at least one embodiment, this pupil position result data can then be copied back to system memory, where this data can be accessed by an application or CPU programmed to process that data, such as to determine gaze information based on this pupil data.”)
Consider Claim 5:
Stengel discloses the eye tracking device according to claim 1, wherein the machine learning model comprises a neural network. (Stengel, [0046], “In at least one embodiment, VR headset 104 can include one or more cameras, such as eye tracking cameras 108, that can capture images of one or both eyes of a user. In at least one embodiment, VR headset 104 can include a gaze-tracking subsystem that includes one or more cameras configured to capture images of one or both eyes of a user wearing headset 104. In at least one embodiment, one or more neural networks of a gaze tracking module 114 on client device 110 can be used to determine gaze information for individual images or video frames, along with a confidence for each indication.”)
Consider Claim 6:
Stengel discloses a method for eye tracking, comprising: (Stengel, See Abstract.)
obtaining an eye image set; (Stengel, [0065], “In at least one embodiment, a process 400 for determining gaze of a user can be utilized as illustrated in FIG. 4. In at least one embodiment, one or more eye images can be captured 402 for a user, such as by using one or more cameras. In at least one embodiment, these can be ambient or infrared images of any appropriate resolution and format. In at least one embodiment, a determination can be made 404 as to whether there are separate images obtained for both eyes of this user.”)
inputting the eye image set to a machine learning model to obtain an estimated eye image; and (Stengel, [0065], “In at least one embodiment, a determination can be made 404 as to whether there are separate images obtained for both eyes of this user. If so, those images can be concatenated 406 into a single image, separated by a determined amount of padding to prevent cross-talk and other such issues. In at least one embodiment, this current image can be compared 408 against a prior image to determine whether there has been a significant change in pupil position. In at least one embodiment, this can be based on intensities or other information in those images. In at least one embodiment, if it is determined 410 there has been an actionable change, such as an amount of change meeting or exceeding a determined threshold, then that image can be analyzed 412 using a coarse CNN to obtain a coarse pupil position estimate.”)
performing eye tracking according to the estimated eye image. (Stengel, [0065], “In at least one embodiment, this image of an estimated pupil region can then be analyzed 416 using this fine CNN to obtain a fine pupil position estimate. In at least one embodiment, this fine pupil estimate can be used to determine 418 user gaze.”)
Consider Claim 7:
Stengel discloses the method according to claim 6, wherein the step of performing the eye tracking according to the estimated eye image comprising: performing interpolation between a first image of the eye image set and the estimated eye image to generate an interpolated eye image; and performing the eye tracking according to the interpolated eye image and the estimated eye image. (Stengel, [0050], “In at least one embodiment, a two-phase approach to pupil position determination can be utilized that takes advantage of at least two neural networks, such as convolutional neural networks (CNNs). In at least one embodiment, these networks include a course CNN 212 and a fine CNN 214. In at least one embodiment, pupil center estimation accuracy is achieved by making use of these different CNNs to process incoming frames of data from an eye tracking camera. In at least one embodiment, a coarse CNN 212 can receive as input full eye images as captured by an eye-tracker camera, and can process these images to provide a coarse estimate, or first estimate, of pupil center position. In at least one embodiment, a fine CNN 214 can then analyze portions of these input images that correspond to pupil regions, as determined by coarse CNN 212. In at least one embodiment, fine CNN 214 can provide more accurate pupil center estimates than coarse CNN 212, which analyzes entire captured images. In at least one embodiment, fine CNN 214 can further be trained to provide accurate pupil localization within approximate pupil regions of captured images.”)
Consider Claim 8:
Stengel discloses the method according to claim 6, wherein the eye image set comprises a first eye image corresponding to a first time point and a second eye image corresponding to a second time point different from the first time point. (Stengel, [0048], “In at least one embodiment, pupil localization can be performed on a processor such as graphics processing unit (GPU), which can be optimized in such a way as to provide fast performance for localization determinations. In at least one embodiment, various approaches can also improve accuracy of determinations while still achieving such levels of performance. In at least one embodiment, such advantages can be obtained for both monocular and binocular pupil center estimation. In at least one embodiment, pupil localization for gaze estimation using deep learning can start with an uploading or providing of one or more trained neural networks to one or more GPUs of a device to perform inferencing. In at least one embodiment, at inference time, a sequence of images will be received that include views of one or more eyes of a user during a relevant session or experience. In at least one embodiment, this image data can be stored in system memory until such time as it is transferred to GPU memory for inferencing. In at least one embodiment, network inferencing can be run on this GPU to compute pupil center position in image space. In at least one embodiment, this pupil position result data can then be copied back to system memory, where this data can be accessed by an application or CPU programmed to process that data, such as to determine gaze information based on this pupil data.”)
Consider Claim 10:
Park discloses the method according to claim 6, wherein the machine learning model comprises a neural network. (Stengel, [0046], “In at least one embodiment, VR headset 104 can include one or more cameras, such as eye tracking cameras 108, that can capture images of one or both eyes of a user. In at least one embodiment, VR headset 104 can include a gaze-tracking subsystem that includes one or more cameras configured to capture images of one or both eyes of a user wearing headset 104. In at least one embodiment, one or more neural networks of a gaze tracking module 114 on client device 110 can be used to determine gaze information for individual images or video frames, along with a confidence for each indication.”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 4 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Stengel et al. U.S. Patent Application Publication No. 2021/0350550 A1 as applied to claim 1 and 6, respectively, above, and further in view of Park et al. U.S. Patent Application Publication No. 2025/0200940 A1 hereinafter Park.
Consider Claim 4:
Stengel discloses the eye tracking device according to claim 1, model according to the historical eye image set, however does not specify wherein the preprocessing comprises translation, rotation, or shearing.
Park however teaches that it was a known technique to those of skill in the art before the effective filing date of the invention to take into consideration types of translation, rotation, or shearing. Park therefore teaches wherein the preprocessing comprises translation, rotation, or shearing. (Park, [0068], “Furthermore, non-adjacent/non-edge aligned patch relationships have additional properties that are desirable. In particular, the ability to change the location of one patch relative to another may be used to modify the ML patch classifier (during online operation) for rotations, translations, scaling, mirroring, etc. For example, rotation may be difficult with Haar-like features. In conventional object detection solutions, the images are rotated so that the Haar-like feature may be used in its normal orientation (which is important for integral image calculations) or the model must actually be trained for different rotations, etc. In contrast, a rotation of ML patch features can be approximated by changing the coordinates of the patches relative to one another. Similar effects may be observed with translations, scaling, mirroring/flipping, etc. In other words, handling image modifications by changing the ML patch feature coordinates may be much more efficient than trying to modify the image data or run a more complicated model.”)
It therefore would have bene obvious to those having ordinary skill in the art before the effective filing date of the invention to take into consideration types of translation, rotation, or shearing as taught in Park as this was a known technique and would have been utilized for the purpose of that these parameters may be extended to achieve a broad spectrum of performance, speed, and/or power efficiency. (Park, [0067])
Consider Claim 9:
Park discloses the method according to claim 6, further comprising: model according to the historical eye image set, however does not specify wherein the preprocessing comprises translation, rotation, or shearing.
Park however teaches that it was a known technique to those of skill in the art before the effective filing date of the invention to take into consideration types of translation, rotation, or shearing. Park therefore teaches wherein the preprocessing comprises translation, rotation, or shearing. (Park, [0068], “Furthermore, non-adjacent/non-edge aligned patch relationships have additional properties that are desirable. In particular, the ability to change the location of one patch relative to another may be used to modify the ML patch classifier (during online operation) for rotations, translations, scaling, mirroring, etc. For example, rotation may be difficult with Haar-like features. In conventional object detection solutions, the images are rotated so that the Haar-like feature may be used in its normal orientation (which is important for integral image calculations) or the model must actually be trained for different rotations, etc. In contrast, a rotation of ML patch features can be approximated by changing the coordinates of the patches relative to one another. Similar effects may be observed with translations, scaling, mirroring/flipping, etc. In other words, handling image modifications by changing the ML patch feature coordinates may be much more efficient than trying to modify the image data or run a more complicated model.”)
It therefore would have bene obvious to those having ordinary skill in the art before the effective filing date of the invention to take into consideration types of translation, rotation, or shearing as taught in Park as this was a known technique and would have been utilized for the purpose of that these parameters may be extended to achieve a broad spectrum of performance, speed, and/or power efficiency. (Park, [0067])
Conclusion
Prior art made of record and not relied upon which is still considered pertinent to applicant's disclosure is cited in a current or previous PTO-892. The prior art cited in a current or previous PTO-892 reads upon the applicants claims in part, in whole and/or gives a general reference to the knowledge and skill of persons having ordinary skill in the art before the effective filing date of the invention. Applicant, when responding to this Office action, should consider not only the cited references applied in the rejection but also any additional references made of record.
In the response to this office action, the Examiner respectfully requests support be shown for any new or amended claims. More precisely, indicate support for any newly added language or amendments by specifying page, line numbers, and/or figure(s). This will assist The Office in compact prosecution of this application. The Office has cited particular columns, paragraphs, and/or line numbers in the applied rejection of the claims above for the convenience of the applicant. Citations are representative of the teachings in the art and are applied to the specific limitations within each claim, however other passages and figures may apply. Applicant, in preparing a response, should fully consider the cited reference(s) in its entirety and not only the cited portions as other sections of the reference may expand on the teachings of the cited portion(s).
Applicant Representatives are reminded of CFR 1.4(d)(2)(ii) which states “A patent practitioner (§ 1.32(a)(1) ), signing pursuant to §§ 1.33(b)(1) or 1.33(b)(2), must supply his/her registration number either as part of the S-signature, or immediately below or adjacent to the S-signature. The number (#) character may be used only as part of the S-signature when appearing before a practitioner’s registration number; otherwise the number character may not be used in an S-signature.” When an unsigned or improperly signed amendment is received the amendment will be listed in the contents of the application file, but not entered. The examiner will notify applicant of the status of the application, advising him or her to furnish a duplicate amendment properly signed or to ratify the amendment already filed. In an application not under final rejection, applicant should be given a two month time period in which to ratify the previously filed amendment (37 CFR 1.135(c) ).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J JANSEN II whose telephone number is (571)272-5604. The examiner can normally be reached Normally Available Monday-Friday 9am-4pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Temesghen Ghebretinsae can be reached on 571-272-3017. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Michael J Jansen II/ Primary Examiner, Art Unit 2626