DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are the currently pending claims hereby under examination.
Claim Warning
Applicant is advised that should claims 14 and 15 be found allowable, claims 19 and 20, respectively, will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Objections
Claims 1 and 8 are objected to because of the following informalities:
In claim 1, lines 2–3: “a virtual-reality headset or augmented-reality system configured to measure eye-associated positions and gaze-associated direction” uses “direction” in singular while later recitations use “directions" (lines 8, 11, and 12) creating plurality inconsistency; please reconcile for clarity and consistency; and
In claim 8, lines 1-2: “visual-reality headset” appears to be a typographical error for “virtual-reality headset”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 7-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as failing to set forth the subject matter which the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the applicant regards as the invention.
Claim 7 recites generating the scene/object “in a virtual-reality headset or augmented-reality system” (lines 4–5), and further recites that “the virtual-reality headset or augmented-reality system [is] configured to measure eye-associated positions and gaze-associated direction” (lines 5–6), while the preamble claims “a system comprising: an analysis system” (lines 1–2). It is unclear whether the headset/system is included within the claimed combination or constitutes an external environment/device; this internal boundary ambiguity fails to particularly point out the invention. The Examiner interprets the recited “analysis system” as the sole claimed system and treats the “virtual-/augmented-reality headset/system” as either an external device not included in the claimed combination but required for operation or an internal device that is part of the overall "system".
Claims 8-12 are rejected by virtue of their dependence from claim 7.
Claim 8 recites “A system of claim 7, wherein the system is the virtual-reality headset …” (lines 1–2). This redefines “the system” of claim 7 (an analysis system) to be the headset, creating an internal contradiction since claim 7 treats the system and headset as separate and distinct from each other; claim 8 is indefinite under § 112(b). The Examiner interprets “the system” in claim 8 as limited to the virtual-reality headset itself (as recited); the irreconcilable inconsistency renders the metes and bounds not reasonably certain.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3-5, 7, 9, 11-14, and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Keun et al. (KR 20220170336 A), hereto referred as Keun.
Regarding claim 1, Keun teaches that a system comprises: a virtual-reality headset or augmented-reality system configured to measure eye-associated positions and gaze-associated direction; (Keun, ¶[0135]: “The augmented reality device (100) can obtain left eye gaze direction information based on the position of the pupil detected from the left eye, and can obtain right eye gaze direction information based on the position of the pupil detected from the right eye”, teaches measuring eye-associated positions (pupil feature points) and obtaining gaze-associated directions for both eyes; ¶[0122]: “The processor (150) can obtain pupil position information based on the position of the pupil feature point, and can obtain information about the gaze direction based on the pupil position information”, shows measuring eye-associated positions and corresponding gaze direction; FIG. 1a, 2: depict the augmented reality device as head worn glasses); an analysis system having a processor and a memory having instructions stored thereon, wherein execution of the instructions by the processor causes the processor to: (Keun, ¶[0104]: “The processor (150) can execute one or more instructions or program codes stored in the memory (160) and perform functions and/or operations corresponding to the instructions or program codes”, shows a processor executing instructions stored in memory; ¶[0108]: “Commands or program codes for performing functions or operations of the augmented reality device (100) may be stored in the memory (160) … the memory (160) may store at least one of instructions, an algorithm, a data structure, a program code, and an application program that can be read by the processor (150)”, teaches a memory storing instructions/programs for execution by the processor; ¶[0110]: “the processor (150) may be implemented by executing instructions or program codes stored in the memory (160)”, further confirms execution of stored instructions by the processor); generate a rendered scene and/or object in the virtual-reality headset or augmented-reality system; (Keun, ¶[0039]: “A typical augmented reality device has an optical engine for generating a virtual image”, shows generating a rendered image/scene for display in an AR system, where ¶[0072]: “The display engine (130) may be an optical engine"; ¶[0004]: “Augmented reality devices project virtual images onto the user's eyes through a see-through display, allowing the user to simultaneously view real-world objects and the projected virtual images”, shows rendering and presenting virtual images/objects in the headset; ¶[0088]: “The display engine (130) is configured to project a virtual image onto the wave guide (120)”, shows projecting rendered virtual images for display); receive the measured eye-associated positions and measured gaze-associated directions of a user viewing the rendered scene and/or object; (Keun, ¶[0134]: “the augmented reality device (100) can detect a left eye pupil from a left eye image acquired using a first gaze tracking sensor, and can detect a right eye pupil from a right eye image acquired using a second gaze tracking sensor”, shows receiving measurements for eye-associated positions via sensors; ¶[0135]: “The augmented reality device (100) can obtain left eye gaze direction information … and can obtain right eye gaze direction information …”, shows receiving/deriving measured gaze-associated directions); determine a gaze control point from the measured eye-associated positions and measured gaze-associated directions, wherein the gaze control point is a binocular convergence in the rendered scene and/or on the object using the measured gaze-associated directions from both eyes to reflect a combined eye movement pattern (Keun, ¶[0122]: “The processor (150) can obtain pupil position information based on the position of the pupil feature point”, shows measured eye positions; ¶[0121]: “detecting the gaze point, which is the point where the gaze directions of the user's two eyes converge”, teaches determining a binocular convergence point from both eyes’ gaze directions; ¶[0134]: “the augmented reality device (100) obtains a gaze point where the gaze directions of the user's two eyes converge”, shows binocular convergence determined using both eyes’ gaze directions; ¶[0135]: “The augmented reality device (100) can estimate the position coordinates of the gaze point by using gaze information regarding binocular disparity, the gaze direction of the left eye, and the gaze direction of the right eye”, explicitly recites binocular disparity and shows the gaze point derived from both eyes’ gaze directions, i.e., a binocular convergence in the rendered scene/object; ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G, see FIGS. 1A and 1B) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, same express “binocular disparity” language tied to calculating the binocular convergence point G; ¶[0219], “the distance d between the user's eyes and the virtual screen … As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, ties the gaze point to coordinates with respect to the rendered virtual screen; ¶[0205], “the augmented reality device (100) can calculate the two-dimensional position coordinate value of the gaze direction of the user's eye (E) on the virtual screen (1300) using the degree of rotation (α and β) of the user's eye (E)”, places gaze coordinates on the rendered screen; ¶[0211], “The processor (150)… can determine the gaze direction of the left and right eyes using gaze information output from the gaze tracking sensor (140a, 140b). In one embodiment… can calculate a first gaze vector … and a second gaze vector …”, shows explicit per-eye vectors that are then used together, as shown in ¶[0212], to track the combined eye movement patterns and 3D gaze coordinates; see also ¶[0216]).
Regarding claim 3, Keun teaches determining one or more statistical parameters, or associated values, from the determined gaze control point (Keun, ¶[0216]: “When calculating the focal length to the point of gaze, the visual axes of the two eyes may not meet… the coordinates of the vertical axis (y-axis) can be calculated as the average of the vertical axis (y-axis) coordinates of the two eyes”, teaches determining a statistical parameter (an average/mean) derived from binocular gaze geometry tied to the gaze point G; ¶[0219]: “As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, shows calculating associated values (distances l and z) from the determined gaze point G on the rendered screen; ¶[0136]: “the augmented reality device (100) can obtain the position coordinates of the center of focus based on the lens-eye distance, the convergence distance which is the distance between the gaze point and the user's eye, and the interpupillary distance”, shows further derived coordinates/values based on the convergence distance to the gaze point G).
Regarding claim 4, Keun teaches that the determined one or more statistical parameters includes at least one of: (i) a variance measure of the gaze control point, (ii) a mean location of the gaze control point (Keun, ¶[0216]: “the coordinates of the vertical axis (y-axis) can be calculated as the average of the vertical axis (y-axis) coordinates of the two eyes”, teaches a mean used in computing the binocular gaze point location along the y-axis (Keun, ¶[0212]: “estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows the 3D gaze point coordinates to which a mean can pertain);(iii) a latency measure of the gaze control point, (iv) a change in the variance of the gaze control point over time, (v) a change in the latency of the gaze control point over time, (vi) an instantaneous location of the gaze control point (Keun, ¶[0212]: “estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, teaches determining the instantaneous 3D location of the gaze control point G (Keun, ¶[0214]: “map a point (gaze point, G)… to a three-dimensional position coordinate value… or may store the three-dimensional position coordinate value of the gaze point (G) in a memory (160)”, further shows the specific 3D coordinates of G; ¶[0211]–[0212]: “The processor (150)… can calculate a first gaze vector indicating a gaze direction of the left eye and a second gaze vector indicating a gaze direction of the right eye using gaze information output from the gaze tracking sensor… [and] can estimate the position coordinates of the gaze point (G)…,” teaches determining the gaze point in real-time from live sensor data, thereby producing an instantaneous 3-D location of the gaze control point; Keun, ¶[0221]: “The convergence distance l may be adjusted depending on the movement of the gaze point or the user’s focus”, further supports instantaneous velocity and acceleration measures (items (vii) and (viii)) as changes in gaze distance over time based on user motion and focus);(vii) an instantaneous velocity measure of the gaze control point, and (viii) an instantaneous acceleration measure of the gaze control point.
Regarding claim 5, Keun teaches that the determined gaze control point, or statistics derived therefrom, are subsequently employed to assess at least one of: vergence measure (Keun, ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows binocular, user-based convergence geometry from which a vergence measure is derived) (Keun, ¶[0219]: “As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, discloses a concrete vergence distance tied to the user’s binocular gaze that can be assessed) (Keun, ¶[0058]: “the augmented reality device (100) includes a lens-to-eye distance (ER), a convergence distance l which is a distance between a gaze point (G) and a lens, and a gaze vector of both eyes of the user toward the gaze point (G)”, further ties l to per-eye vectors and binocular geometry) (Keun, ¶[0136]: “the augmented reality device (100) can obtain the position coordinates of the center of focus based on the lens-eye distance, the convergence distance which is the distance between the gaze point and the user’s eye, and the interpupillary distance”, shows derived coordinates based on convergence distance usable for assessment; ¶[0136]: “obtain the position coordinates of the center of focus based on… the convergence distance which is the distance between the gaze point and the user’s eye”, shows an explicit vergence-related distance derived from the binocular gaze point that can be assessed; ¶[0281]: “the focal length… can be adjusted to be equal to the convergence distance”, further ties processing to a measurable convergence distance; see also FIG. 13-16 and 18), visual fixation measure, saccades measure, smooth pursuit measure, eye dominance measure, vestibular measure, optokinetic measure, nystagmus quick phase measure, or a combination thereof.
Regarding claim 7, Keun teaches that a system comprising: an analysis system having a processor and a memory having instructions stored thereon, wherein execution of the instructions by the processor causes the processor to: (Keun, ¶[0104]: “The processor (150) can execute one or more instructions or program codes stored in the memory (160) and perform functions and/or operations corresponding to the instructions or program codes”, shows a processor executing instructions stored in memory; ¶[0108]: “Commands or program codes for performing functions or operations of the augmented reality device (100) may be stored in the memory (160) … the memory (160) may store at least one of instructions, an algorithm, a data structure, a program code, and an application program that can be read by the processor (150)”, teaches a memory storing instructions/programs for execution by the processor; ¶[0110]: “the processor (150) may be implemented by executing instructions or program codes stored in the memory (160)”, further confirms execution of stored instructions by the processor); generate a rendered scene and/or object in a virtual-reality headset or augmented-reality system, wherein the virtual-reality headset or augmented-reality system is configured to measure eye-associated positions and gaze-associated direction; (Keun, ¶[0039]: “A typical augmented reality device has an optical engine for generating a virtual image”, shows generating a rendered image/scene for display in an AR system, where ¶[0072]: “The display engine (130) may be an optical engine"; ¶[0004]: “Augmented reality devices project virtual images onto the user's eyes through a see-through display, allowing the user to simultaneously view real-world objects and the projected virtual images”, shows rendering and presenting virtual images/objects in the headset; ¶[0088]: “The display engine (130) is configured to project a virtual image onto the wave guide (120)”, shows projecting rendered virtual images for display; ¶[0134]: “the augmented reality device (100) can detect a left eye pupil from a left eye image acquired using a first gaze tracking sensor, and can detect a right eye pupil from a right eye image acquired using a second gaze tracking sensor”, shows configuration to measure eye-associated positions; ¶[0122]: “The processor (150) can obtain pupil position information based on the position of the pupil feature point”, shows measured positions ¶[0135]: “The augmented reality device (100) can obtain left eye gaze direction information based on the position of the pupil detected from the left eye, and can obtain right eye gaze direction information based on the position of the pupil detected from the right eye”, shows configuration to obtain gaze-associated directions); receive the measured eye-associated positions and measured gaze-associated directions of a user viewing the rendered scene and/or object; (Keun, ¶[0134]: “the augmented reality device (100) can detect a left eye pupil from a left eye image acquired using a first gaze tracking sensor, and can detect a right eye pupil from a right eye image acquired using a second gaze tracking sensor”, shows receiving measurements for eye-associated positions via sensors; ¶[0135]: “The augmented reality device (100) can obtain left eye gaze direction information … and can obtain right eye gaze direction information …”, shows receiving/deriving measured gaze-associated directions); determine a gaze control point from the measured eye-associated positions and measured gaze-associated directions, wherein the gaze control point is a binocular convergence in the rendered scene and/or on the object using the measured gaze-associated directions from both eyes to reflect a combined eye movement pattern (Keun, ¶[0121]: “detecting the gaze point, which is the point where the gaze directions of the user's two eyes converge”, teaches determining a binocular convergence point from both eyes’ gaze directions; ¶[0134]: “the augmented reality device (100) obtains a gaze point where the gaze directions of the user's two eyes converge”, shows binocular convergence determined using both eyes’ gaze directions; ¶[0135]: “The augmented reality device (100) can estimate the position coordinates of the gaze point by using gaze information regarding binocular disparity, the gaze direction of the left eye, and the gaze direction of the right eye”, explicitly recites binocular disparity and shows the gaze point derived from both eyes’ gaze directions, i.e., a binocular convergence in the rendered scene/object; ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G, see FIGS. 1A and 1B) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, same express “binocular disparity” language tied to calculating the binocular convergence point G; ¶[0219]: “the distance d between the user's eyes and the virtual screen … As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, ties the gaze point to coordinates with respect to the rendered virtual screen; ¶[0205]: “the augmented reality device (100) can calculate the two-dimensional position coordinate value of the gaze direction of the user's eye (E) on the virtual screen (1300) using the degree of rotation (α and β) of the user's eye (E)”, places gaze coordinates on the rendered screen; ¶[0211]: “The processor (150)… can determine the gaze direction of the left and right eyes using gaze information output from the gaze tracking sensor (140a, 140b). In one embodiment… can calculate a first gaze vector … and a second gaze vector …”, shows explicit per-eye vectors that are then used together, as shown in ¶[0212], to track the combined eye movement patterns and 3D gaze coordinates; see also ¶[0216]).
Regarding claim 9, Keun teaches that the determined gaze control point, or statistics derived therefrom, are subsequently employed to assess at least one of: vergence measure (Keun, ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows binocular, user-based convergence geometry from which a vergence measure is derived) (Keun, ¶[0219]: “As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, discloses a concrete vergence distance tied to the user’s binocular gaze that can be assessed) (Keun, ¶[0058]: “the augmented reality device (100) includes a lens-to-eye distance (ER), a convergence distance l which is a distance between a gaze point (G) and a lens, and a gaze vector of both eyes of the user toward the gaze point (G)”, further ties l to per-eye vectors and binocular geometry) (Keun, ¶[0136]: “the augmented reality device (100) can obtain the position coordinates of the center of focus based on the lens-eye distance, the convergence distance which is the distance between the gaze point and the user’s eye, and the interpupillary distance”, shows derived coordinates based on convergence distance usable for assessment; ¶[0136]: “obtain the position coordinates of the center of focus based on… the convergence distance which is the distance between the gaze point and the user’s eye”, shows an explicit vergence-related distance derived from the binocular gaze point that can be assessed; ¶[0281]: “the focal length… can be adjusted to be equal to the convergence distance”, further ties processing to a measurable convergence distance; see also FIG. 13-16 and 18), but does not teach: visual fixation measure, saccades measure, smooth pursuit measure, eye dominance measure, vestibular measure, optokinetic measure, or nystagmus quick phase measure.
Regarding claim 11, Keun teaches that the determined one or more statistical parameters includes at least one of: (i) a variance measure of the gaze control point, (ii) a mean location of the gaze control point (Keun, ¶[0216]: “the coordinates of the vertical axis (y-axis) can be calculated as the average of the vertical axis (y-axis) coordinates of the two eyes”, teaches a mean used in computing the binocular gaze point location along the y-axis (Keun, ¶[0212]: “estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows the 3D gaze point coordinates to which a mean can pertain);(iii) a latency measure of the gaze control point, (iv) a change in the variance of the gaze control point over time, (v) a change in the latency of the gaze control point over time, (vi) an instantaneous location of the gaze control point (Keun, ¶[0212]: “estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, teaches determining the instantaneous 3D location of the gaze control point G (Keun, ¶[0214]: “map a point (gaze point, G)… to a three-dimensional position coordinate value… or may store the three-dimensional position coordinate value of the gaze point (G) in a memory (160)”, further shows the specific 3D coordinates of G; ¶[0211]–[0212]: “The processor (150)… can calculate a first gaze vector indicating a gaze direction of the left eye and a second gaze vector indicating a gaze direction of the right eye using gaze information output from the gaze tracking sensor… [and] can estimate the position coordinates of the gaze point (G)…,” teaches determining the gaze point in real-time from live sensor data, thereby producing an instantaneous 3-D location of the gaze control point; Keun, ¶[0221]: “The convergence distance l may be adjusted depending on the movement of the gaze point or the user’s focus”, further supports instantaneous velocity and acceleration measures (items (vii) and (viii)) as changes in gaze distance over time based on user motion and focus);(vii) an instantaneous velocity measure of the gaze control point, and (viii) an instantaneous acceleration measure of the gaze control point.
Regarding claim 12, Keun teaches that the determined gaze control point, or statistics derived therefrom, are subsequently employed to assess at least one of: vergence measure (Keun, ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows binocular, user-based convergence geometry from which a vergence measure is derived) (Keun, ¶[0219]: “As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, discloses a concrete vergence distance tied to the user’s binocular gaze that can be assessed) (Keun, ¶[0058]: “the augmented reality device (100) includes a lens-to-eye distance (ER), a convergence distance l which is a distance between a gaze point (G) and a lens, and a gaze vector of both eyes of the user toward the gaze point (G)”, further ties l to per-eye vectors and binocular geometry) (Keun, ¶[0136]: “the augmented reality device (100) can obtain the position coordinates of the center of focus based on the lens-eye distance, the convergence distance which is the distance between the gaze point and the user’s eye, and the interpupillary distance”, shows derived coordinates based on convergence distance usable for assessment; ¶[0136]: “obtain the position coordinates of the center of focus based on… the convergence distance which is the distance between the gaze point and the user’s eye”, shows an explicit vergence-related distance derived from the binocular gaze point that can be assessed; ¶[0281]: “the focal length… can be adjusted to be equal to the convergence distance”, further ties processing to a measurable convergence distance; see also FIG. 13-16 and 18), visual fixation measure, saccades measure, smooth pursuit measure, eye dominance measure, vestibular measure, optokinetic measure, nystagmus quick phase measure, or a combination thereof.
Regarding claim 13, Keun teaches that a non-transitory computer-readable medium having instructions stored thereon, wherein execution of the instructions by a processor causes the processor to: (Keun, ¶[0108]: “Commands or program codes for performing functions or operations of the augmented reality device (100) may be stored in the memory (160)”, teaches a computer-readable memory storing program instructions; ¶[0104]: “The processor (150) can execute one or more instructions or program codes stored in the memory (160) and perform functions and/or operations corresponding to the instructions or program codes”, shows execution of stored instructions by a processor; ¶[0110]: “the processor (150) may be implemented by executing instructions or program codes stored in the memory (160)”, further confirms execution of instructions stored on a memory medium); generate a rendered scene and/or object in the virtual-reality headset or augmented-reality system (Keun, ¶[0039]: “A typical augmented reality device has an optical engine for generating a virtual image”, shows generating a rendered image/scene in a head-mounted AR/VR system, where ¶[0072]: “The display engine (130) may be an optical engine"; ¶[0088]: “The display engine (130) is configured to project a virtual image onto the wave guide (120)”, shows projecting rendered virtual images for display); receive, from a virtual-reality headset or augmented-reality system, measured eye-associated positions and measured gaze-associated directions of a user viewing the rendered scene and/or object (Keun, ¶[0134]: “the augmented reality device (100) can detect a left eye pupil from a left eye image acquired using a first gaze tracking sensor, and can detect a right eye pupil from a right eye image acquired using a second gaze tracking sensor”, teaches receiving/measuring eye-associated positions; ¶[0135]: “The augmented reality device (100) can obtain left eye gaze direction information based on the position of the pupil detected from the left eye, and can obtain right eye gaze direction information based on the position of the pupil detected from the right eye”, shows obtaining measured gaze-associated directions); determine a gaze control point from the measured eye associated positions and measured gaze-associated directions, wherein the gaze control point is a binocular convergence in the rendered scene and/or on the object using the measured gaze-associated directions from both eyes to reflect a combined eye movement pattern (Keun, ¶[0121]: “detecting the gaze point, which is the point where the gaze directions of the user's two eyes converge”, teaches determining a binocular convergence point from both eyes’ gaze directions; ¶[0134]: “the augmented reality device (100) obtains a gaze point where the gaze directions of the user's two eyes converge”, shows binocular convergence determined using both eyes’ gaze directions; ¶[0135]: “The augmented reality device (100) can estimate the position coordinates of the gaze point by using gaze information regarding binocular disparity, the gaze direction of the left eye, and the gaze direction of the right eye”, explicitly recites binocular disparity and shows the gaze point derived from both eyes’ gaze directions, i.e., a binocular convergence in the rendered scene/object; ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G, see FIGS. 1A and 1B) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, same express “binocular disparity” language tied to calculating the binocular convergence point G; ¶[0219], “the distance d between the user's eyes and the virtual screen … As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, ties the gaze point to coordinates with respect to the rendered virtual screen; ¶[0205], “the augmented reality device (100) can calculate the two-dimensional position coordinate value of the gaze direction of the user's eye (E) on the virtual screen (1300) using the degree of rotation (α and β) of the user's eye (E)”, places gaze coordinates on the rendered screen; ¶[0211], “The processor (150)… can determine the gaze direction of the left and right eyes using gaze information output from the gaze tracking sensor (140a, 140b). In one embodiment… can calculate a first gaze vector … and a second gaze vector …”, shows explicit per-eye vectors that are then used together, as shown in ¶[0212], to track the combined eye movement patterns and 3D gaze coordinates; see also ¶[0216]).
Regarding claim 14, Keun teaches that the determined gaze control point, or statistics derived therefrom, are subsequently employed to assess at least one of: vergence measure (Keun, ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows binocular, user-based convergence geometry from which a vergence measure is derived) (Keun, ¶[0219]: “As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, discloses a concrete vergence distance tied to the user’s binocular gaze that can be assessed) (Keun, ¶[0058]: “the augmented reality device (100) includes a lens-to-eye distance (ER), a convergence distance l which is a distance between a gaze point (G) and a lens, and a gaze vector of both eyes of the user toward the gaze point (G)”, further ties l to per-eye vectors and binocular geometry) (Keun, ¶[0136]: “the augmented reality device (100) can obtain the position coordinates of the center of focus based on the lens-eye distance, the convergence distance which is the distance between the gaze point and the user’s eye, and the interpupillary distance”, shows derived coordinates based on convergence distance usable for assessment; ¶[0136]: “obtain the position coordinates of the center of focus based on… the convergence distance which is the distance between the gaze point and the user’s eye”, shows an explicit vergence-related distance derived from the binocular gaze point that can be assessed; ¶[0281]: “the focal length… can be adjusted to be equal to the convergence distance”, further ties processing to a measurable convergence distance; see also FIG. 13-16 and 18), visual fixation measure, saccades measure, smooth pursuit measure, eye dominance measure, vestibular measure, optokinetic measure, nystagmus quick phase measure, or a combination thereof.
Regarding claim 19, Keun teaches that the determined gaze control point, or statistics derived therefrom, are subsequently employed to assess at least one of: vergence measure (Keun, ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows binocular, user-based convergence geometry from which a vergence measure is derived) (Keun, ¶[0219]: “As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, discloses a concrete vergence distance tied to the user’s binocular gaze that can be assessed) (Keun, ¶[0058]: “the augmented reality device (100) includes a lens-to-eye distance (ER), a convergence distance l which is a distance between a gaze point (G) and a lens, and a gaze vector of both eyes of the user toward the gaze point (G)”, further ties l to per-eye vectors and binocular geometry) (Keun, ¶[0136]: “the augmented reality device (100) can obtain the position coordinates of the center of focus based on the lens-eye distance, the convergence distance which is the distance between the gaze point and the user’s eye, and the interpupillary distance”, shows derived coordinates based on convergence distance usable for assessment; ¶[0136]: “obtain the position coordinates of the center of focus based on… the convergence distance which is the distance between the gaze point and the user’s eye”, shows an explicit vergence-related distance derived from the binocular gaze point that can be assessed; ¶[0281]: “the focal length… can be adjusted to be equal to the convergence distance”, further ties processing to a measurable convergence distance; see also FIG. 13-16 and 18), visual fixation measure, saccades measure, smooth pursuit measure, eye dominance measure, vestibular measure, optokinetic measure, nystagmus quick phase measure, or a combination thereof.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Keun et al. (KR 20220170336 A), hereto referred as Keun, and further in view of Young et al. (US 20190354173 A1), hereto referred as Young.
Keun teaches claim 1 as described above.
Regarding claim 2, Keun does not fully teach that execution of the instructions by the processor further cause the processor to: generate a rendered gaze ray associated with the binocular convergence. Rather, Keun teaches computing binocular gaze directions and the convergence point G, and placing gaze coordinates relative to the virtual screen, but Keun does not disclose rendering a gaze ray on the display associated with that convergence. For example: (Keun, ¶[0211]: “The processor (150… ) can determine the gaze direction of the left and right eyes… In one embodiment… can calculate a first gaze vector… and a second gaze vector…”, shows per-eye vectors computed internally for both eyes; ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows estimating the binocular convergence point coordinates; ¶[0219]: “the distance d between the user’s eyes and the virtual screen… z represents the distance between the virtual screen and the gaze point”, locates the gaze point relative to the rendered virtual screen but still does not disclose drawing/rendering a gaze ray; ¶[0042]: “'gaze' means an imaginary line from the user’s pupil to the gaze direction”, defines gaze as an imaginary line rather than a rendered on-screen line).
Young explicitly teaches that the headset’s display renders gaze-related visual indicators corresponding to eye movement vectors (Young, ¶[0094]: “The saccade path 510 is superimposed onto display 810 and shows fixation point A (e.g., direction 506 and vector XF-o)”, teaches a rendered, on-screen path/vector corresponding to gaze movement; ¶[0096]: “predicted landing points for fixation point B are superimposed onto display 810”, further shows rendered gaze-related indicators superimposed on the HMD display.) These passages demonstrate that Young visualizes where the user is looking by overlaying a rendered line or point on the virtual display. This fills the gap left by Keun, which performs gaze computations internally but does not visually depict them. One of ordinary skill in the art would recognize that Young’s display technique could be integrated into Keun’s existing binocular gaze system to represent the convergence point G visually on the virtual screen, thereby creating the claimed rendered gaze ray associated with binocular convergence.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Keun in view of Young to generate a rendered gaze ray associated with the binocular convergence. The combination is feasible because Keun already computes left/right gaze vectors and the 3-D gaze point G (including coordinates relative to the virtual screen), and Young demonstrates on-display overlays of gaze-related vectors/paths and landing points; adding a simple graphics overlay to depict a line (ray) from the eye or display toward G uses the same rendering pipeline that presents AR content. The benefit of the combination includes clear visual feedback for calibration and interaction, improved alignment between gaze estimation and displayed content, and enhanced usability by exposing the system’s internal gaze solution as an on-screen indicator.
Claims 6, 10, 15, 17-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Keun et al. (KR 20220170336 A), hereto referred as Keun, and further in view of Monti (GB 2613084 A), hereto referred as Monti.
Keun teaches claim 1, claim 7, and claim 13 as described above.
Regarding claim 6, Keun does not teach that the determined gaze control point, or statistics derived therefrom, are subsequently employed in a therapy to address an eye-tracking problem or a disease. Monti teaches gaze-based head-mounted displays configured to detect user eye conditions such as macular degeneration, glaucoma, cataracts, or other vision impairments and to adaptively generate images that compensate for regions of vision loss. The system tracks eye gaze and dynamically modifies displayed objects or viewpoints to redirect images toward functional regions of the retina (Monti, p. 17, ll. 28-35: “The processor 1240 can therefore generate images for display to the user according to the type of eye condition so as to compensate for a region of vision loss for the eye of the user”; p. 20-21, ll. 33-20: “In response to detecting that the object is within the predetermined distance…the object is moved in a direction so as to increase the separation distance…to allow the object to be more easily observed by the user”). Monti further discloses calibration procedures using gaze detection and saccadic eye movement tracking to determine parameters related to preferred retinal locus (PRL) and field-of-view limitations, thereby improving user adaptation and restoring visual function; p. 28-29, ll. 35-20: “The calibration circuitry 1250 is configured to calculate one or more offset parameters for the eye indicative of an offset for the detected gaze direction”; p. 30-31, ll. 30-26: “the processor 1240 is configured to calculate a modified gaze direction for the eye in dependence upon one or more of the offset parameters… the modified gaze direction being offset with respect to the detected gaze direction”, explains how the modified gaze direction is computed from offsets, “the detected gaze direction of the eye is arranged to intersect a fovea portion of the retina of the eye and the calculated modified gaze direction for the eye is arranged to intersect a different portion of the retina”, shows the modified gaze direction intentionally targets a non-foveal retinal region, “Specifically, for a user having a preferred retinal locus the different portion of the retina corresponds to the preferred retinal locus of the retina of the eye”, expressly ties that “different portion” to the PRL).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Keun in view of Monti to employ the determined gaze control point, or statistics derived therefrom, in a therapy or rehabilitation process for correcting gaze or vision deficiencies. The combination is feasible because both references employ gaze detection and convergence data in head-mounted or AR display systems to generate user-specific visual feedback. One of ordinary skill in the art would have recognized that Monti’s compensatory rendering and calibration for pathological vision loss naturally extend Keun’s gaze control data toward therapeutic use, leveraging gaze convergence information to adapt images for restoring visual function. The motivation for the combination would have been to enhance visual training and rehabilitation for patients with oculomotor or retinal deficiencies, thereby improving accuracy, adaptation, and comfort in visual interactions.
Regarding claim 10, Keun does not teach that the determined gaze control point, or statistics derived therefrom, are subsequently employed in a therapy to address an eye-tracking problem or a disease. Monti teaches gaze-based head-mounted displays configured to detect user eye conditions such as macular degeneration, glaucoma, cataracts, or other vision impairments and to adaptively generate images that compensate for regions of vision loss. The system tracks eye gaze and dynamically modifies displayed objects or viewpoints to redirect images toward functional regions of the retina (Monti, p. 17, ll. 28-35: “The processor 1240 can therefore generate images for display to the user according to the type of eye condition so as to compensate for a region of vision loss for the eye of the user”; p. 20-21, ll. 33-20: “In response to detecting that the object is within the predetermined distance…the object is moved in a direction so as to increase the separation distance…to allow the object to be more easily observed by the user”). Monti further discloses calibration procedures using gaze detection and saccadic eye movement tracking to determine parameters related to preferred retinal locus (PRL) and field-of-view limitations, thereby improving user adaptation and restoring visual function; p. 28-29, ll. 35-20: “The calibration circuitry 1250 is configured to calculate one or more offset parameters for the eye indicative of an offset for the detected gaze direction”; p. 30-31, ll. 30-26: “the processor 1240 is configured to calculate a modified gaze direction for the eye in dependence upon one or more of the offset parameters… the modified gaze direction being offset with respect to the detected gaze direction”, explains how the modified gaze direction is computed from offsets, “the detected gaze direction of the eye is arranged to intersect a fovea portion of the retina of the eye and the calculated modified gaze direction for the eye is arranged to intersect a different portion of the retina”, shows the modified gaze direction intentionally targets a non-foveal retinal region, “Specifically, for a user having a preferred retinal locus the different portion of the retina corresponds to the preferred retinal locus of the retina of the eye”, expressly ties that “different portion” to the PRL).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Keun in view of Monti to employ the determined gaze control point, or statistics derived therefrom, in a therapy or rehabilitation process for correcting gaze or vision deficiencies. The combination is feasible because both references employ gaze detection and convergence data in head-mounted or AR display systems to generate user-specific visual feedback. One of ordinary skill in the art would have recognized that Monti’s compensatory rendering and calibration for pathological vision loss naturally extend Keun’s gaze control data toward therapeutic use, leveraging gaze convergence information to adapt images for restoring visual function. The motivation for the combination would have been to enhance visual training and rehabilitation for patients with oculomotor or retinal deficiencies, thereby improving accuracy, adaptation, and comfort in visual interactions.
Regarding claim 15, Keun does not teach that the determined gaze control point, or statistics derived therefrom, are subsequently employed in a therapy to address an eye-tracking problem or a disease. Monti teaches gaze-based head-mounted displays configured to detect user eye conditions such as macular degeneration, glaucoma, cataracts, or other vision impairments and to adaptively generate images that compensate for regions of vision loss. The system tracks eye gaze and dynamically modifies displayed objects or viewpoints to redirect images toward functional regions of the retina (Monti, p. 17, ll. 28-35: “The processor 1240 can therefore generate images for display to the user according to the type of eye condition so as to compensate for a region of vision loss for the eye of the user”; p. 20-21, ll. 33-20: “In response to detecting that the object is within the predetermined distance…the object is moved in a direction so as to increase the separation distance…to allow the object to be more easily observed by the user”). Monti further discloses calibration procedures using gaze detection and saccadic eye movement tracking to determine parameters related to preferred retinal locus (PRL) and field-of-view limitations, thereby improving user adaptation and restoring visual function; p. 28-29, ll. 35-20: “The calibration circuitry 1250 is configured to calculate one or more offset parameters for the eye indicative of an offset for the detected gaze direction”; p. 30-31, ll. 30-26: “the processor 1240 is configured to calculate a modified gaze direction for the eye in dependence upon one or more of the offset parameters… the modified gaze direction being offset with respect to the detected gaze direction”, explains how the modified gaze direction is computed from offsets, “the detected gaze direction of the eye is arranged to intersect a fovea portion of the retina of the eye and the calculated modified gaze direction for the eye is arranged to intersect a different portion of the retina”, shows the modified gaze direction intentionally targets a non-foveal retinal region, “Specifically, for a user having a preferred retinal locus the different portion of the retina corresponds to the preferred retinal locus of the retina of the eye”, expressly ties that “different portion” to the PRL).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Keun in view of Monti to employ the determined gaze control point, or statistics derived therefrom, in a therapy or rehabilitation process for correcting gaze or vision deficiencies. The combination is feasible because both references employ gaze detection and convergence data in head-mounted or AR display systems to generate user-specific visual feedback. One of ordinary skill in the art would have recognized that Monti’s compensatory rendering and calibration for pathological vision loss naturally extend Keun’s gaze control data toward therapeutic use, leveraging gaze convergence information to adapt images for restoring visual function. The motivation for the combination would have been to enhance visual training and rehabilitation for patients with oculomotor or retinal deficiencies, thereby improving accuracy, adaptation, and comfort in visual interactions.
Regarding claim 17, Keun teaches that the instructions when executed by the processor further cause the processor to determining one or more statistical parameters, or associated values, from the determined gaze control point (Keun, ¶[0216]: “When calculating the focal length to the point of gaze, the visual axes of the two eyes may not meet… the coordinates of the vertical axis (y-axis) can be calculated as the average of the vertical axis (y-axis) coordinates of the two eyes”, teaches determining a statistical parameter (an average/mean) derived from binocular gaze geometry tied to the gaze point G; ¶[0219]: “As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, shows calculating associated values (distances l and z) from the determined gaze point G on the rendered screen; ¶[0136]: “the augmented reality device (100) can obtain the position coordinates of the center of focus based on the lens-eye distance, the convergence distance which is the distance between the gaze point and the user's eye, and the interpupillary distance”, shows further derived coordinates/values based on the convergence distance to the gaze point G).
Regarding claim 18, Keun teaches that the determined one or more statistical parameters includes at least one of: (i) a variance measure of the gaze control point, (ii) a mean location of the gaze control point (Keun, ¶[0216]: “the coordinates of the vertical axis (y-axis) can be calculated as the average of the vertical axis (y-axis) coordinates of the two eyes”, teaches a mean used in computing the binocular gaze point location along the y-axis (Keun, ¶[0212]: “estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows the 3D gaze point coordinates to which a mean can pertain);(iii) a latency measure of the gaze control point, (iv) a change in the variance of the gaze control point over time, (v) a change in the latency of the gaze control point over time, (vi) an instantaneous location of the gaze control point (Keun, ¶[0212]: “estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, teaches determining the instantaneous 3D location of the gaze control point G (Keun, ¶[0214]: “map a point (gaze point, G)… to a three-dimensional position coordinate value… or may store the three-dimensional position coordinate value of the gaze point (G) in a memory (160)”, further shows the specific 3D coordinates of G; ¶[0211]–[0212]: “The processor (150)… can calculate a first gaze vector indicating a gaze direction of the left eye and a second gaze vector indicating a gaze direction of the right eye using gaze information output from the gaze tracking sensor… [and] can estimate the position coordinates of the gaze point (G)…,” teaches determining the gaze point in real-time from live sensor data, thereby producing an instantaneous 3-D location of the gaze control point; Keun, ¶[0221]: “The convergence distance l may be adjusted depending on the movement of the gaze point or the user’s focus”, further supports instantaneous velocity and acceleration measures (items (vii) and (viii)) as changes in gaze distance over time based on user motion and focus); (vii) an instantaneous velocity measure of the gaze control point, and (viii) an instantaneous acceleration measure of the gaze control point.
Regarding claim 20, Keun does not teach that the determined gaze control point, or statistics derived therefrom, are subsequently employed in a therapy to address an eye-tracking problem or a disease. Monti teaches gaze-based head-mounted displays configured to detect user eye conditions such as macular degeneration, glaucoma, cataracts, or other vision impairments and to adaptively generate images that compensate for regions of vision loss. The system tracks eye gaze and dynamically modifies displayed objects or viewpoints to redirect images toward functional regions of the retina (Monti, p. 17, ll. 28-35: “The processor 1240 can therefore generate images for display to the user according to the type of eye condition so as to compensate for a region of vision loss for the eye of the user”; p. 20-21, ll. 33-20: “In response to detecting that the object is within the predetermined distance…the object is moved in a direction so as to increase the separation distance…to allow the object to be more easily observed by the user”). Monti further discloses calibration procedures using gaze detection and saccadic eye movement tracking to determine parameters related to preferred retinal locus (PRL) and field-of-view limitations, thereby improving user adaptation and restoring visual function; p. 28-29, ll. 35-20: “The calibration circuitry 1250 is configured to calculate one or more offset parameters for the eye indicative of an offset for the detected gaze direction”; p. 30-31, ll. 30-26: “the processor 1240 is configured to calculate a modified gaze direction for the eye in dependence upon one or more of the offset parameters… the modified gaze direction being offset with respect to the detected gaze direction”, explains how the modified gaze direction is computed from offsets, “the detected gaze direction of the eye is arranged to intersect a fovea portion of the retina of the eye and the calculated modified gaze direction for the eye is arranged to intersect a different portion of the retina”, shows the modified gaze direction intentionally targets a non-foveal retinal region, “Specifically, for a user having a preferred retinal locus the different portion of the retina corresponds to the preferred retinal locus of the retina of the eye”, expressly ties that “different portion” to the PRL).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Keun in view of Monti to employ the determined gaze control point, or statistics derived therefrom, in a therapy or rehabilitation process for correcting gaze or vision deficiencies. The combination is feasible because both references employ gaze detection and convergence data in head-mounted or AR display systems to generate user-specific visual feedback. One of ordinary skill in the art would have recognized that Monti’s compensatory rendering and calibration for pathological vision loss naturally extend Keun’s gaze control data toward therapeutic use, leveraging gaze convergence information to adapt images for restoring visual function. The motivation for the combination would have been to enhance visual training and rehabilitation for patients with oculomotor or retinal deficiencies, thereby improving accuracy, adaptation, and comfort in visual interactions
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Keun et al. (KR 20220170336 A), hereto referred as Keun, and further in view of Ning (US 11431955 B1), hereto referred as Ning.
Keun teaches claim 7 as described above.
Regarding claim 8, with regards to the system is the virtual-reality headset, Keun primarily teaches an augmented-reality headset as shown above in claim 7, but it does not expressly teach a full virtual-reality headset.
Ning, however, explicitly discloses that its technology applies to both augmented-reality (AR) and virtual-reality (VR) systems, stating: “The present disclosure relates generally to augmented reality (AR) and/or virtual reality (VR) systems” (Ning, Col. 1, ll. 13-25). Ning demonstrates that eye tracking systems are equally compatible with both AR and VR headsets: "the AR/VR systems can use eye tracking and/or head tracking to track the user's eyes or head and accordingly present images" (Ning, Col. 1, ll. 13-25).
It would have been prima facie obvious before the effective filing date of the claimed invention to modify Keun’s augmented-reality headset to function as a virtual-reality headset as taught by Ning, because both references address gaze-based visual scene rendering in head-mounted displays. A person of ordinary skill in the art would have recognized that Keun’s optical and processing architecture could be implemented in the VR configuration of Ning, since both AR and VR HMDs use the same classes of sensors, processors, and gaze-based rendering pipelines, with merely the see-through optics removed, yielding predictable results. This has the benefit of providing a fully controlled non-see-through visual environment that (i) simplifies the optical stack by removing see through combiners and associated alignment tolerances, (ii) reduces stray reflections/real-world background interference to improve stability of gaze calibration and binocular convergence estimation, (iii) enables consistent full field rendered stimuli and predictable vergence-accommodation conditions that enhance rendering fidelity and user comfort, and (iv) reuses the same eye tracking sensors, processors and gaze based rendering pipeline for predictable equivalent operation without undue experimentation.
Also regarding claim 8, the combined Keun and Ning (Ning teaches AR and/or VR head-mounted systems with eye tracking (Ning, Col. 1, ll. 13–25), Keun’s AR headset is implemented as a VR headset for Claim 8; the remaining Keun teachings (processor/memory, rendering, eye/gaze measurements, binocular convergence) apply in the VR HMD context) teach that the virtual-reality headset comprising a processor; and a memory having instructions stored thereon (Keun, ¶[0104]: “The processor (150) can execute one or more instructions or program codes stored in the memory (160) and perform functions and/or operations corresponding to the instructions or program codes”, shows a processor executing instructions stored in memory within the headset; ¶[0108]: “Commands or program codes for performing functions or operations of the augmented reality device (100) may be stored in the memory (160) … the memory (160) may store at least one of instructions, an algorithm, a data structure, a program code, and an application program that can be read by the processor (150)”, teaches a memory storing instructions/programs executed by the onboard processor; Ning, ¶[0002]: “The present disclosure relates generally to augmented reality (AR) and/or virtual reality (VR) systems,” confirms that such processors and memories are applicable to both AR and VR systems); wherein execution of the instructions by the processor causes the processor to: (i) display a rendered scene and/or object (Keun, ¶[0039]: “A typical augmented reality device has an optical engine for generating a virtual image”, teaches generating a rendered image/scene for display in the headset; ¶[0004]: “Augmented reality devices project virtual images onto the user's eyes through a see-through display, allowing the user to simultaneously view real-world objects and the projected virtual images”, shows rendering and presenting virtual images/objects in the headset. When implemented as in Ning’s VR context, these rendered images are fully immersive virtual scenes displayed within the VR headset); (ii) measure eye-associated positions of a user wearing the virtual-reality headset (Keun, ¶[0134]: “the augmented reality device (100) can detect a left eye pupil from a left eye image acquired using a first gaze tracking sensor, and can detect a right eye pupil from a right eye image acquired using a second gaze tracking sensor”, teaches detecting pupil feature points (eye-associated positions) from each eye using onboard sensors; ¶[0122]: “The processor (150) can obtain pupil position information based on the position of the pupil feature point”, shows measured eye positions); (iii) measure gaze-associated directions of the user (Keun, ¶[0135]: “The augmented reality device (100) can obtain left eye gaze direction information based on the position of the pupil detected from the left eye, and can obtain right eye gaze direction information based on the position of the pupil detected from the right eye”, teaches per-eye gaze direction; ¶[0122]: “The processor (150) can obtain pupil position information based on the position of the pupil feature point, and can obtain information about the gaze direction based on the pupil position information”, shows deriving gaze direction from measured positions); (iv) determine a gaze control point from the measured eye-associated positions and measured gaze-associated directions, wherein the gaze control point is a binocular convergence in the rendered scene and/or on the object using the measured gaze-associated directions from both eyes to reflect a combined eye movement pattern (Keun, ¶[0121]: “detecting the gaze point, which is the point where the gaze directions of the user's two eyes converge”, teaches determining a binocular convergence point from both eyes’ gaze directions; ¶[0134]: “the augmented reality device (100) obtains a gaze point where the gaze directions of the user's two eyes converge”, shows binocular convergence determined using both eyes’ gaze directions; ¶[0135]: “The augmented reality device (100) can estimate the position coordinates of the gaze point by using gaze information regarding binocular disparity, the gaze direction of the left eye, and the gaze direction of the right eye”, explicitly recites binocular disparity and shows the gaze point derived from both eyes’ gaze directions, i.e., a binocular convergence in the rendered scene/object; ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G, see FIGS. 1A and 1B) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, same express “binocular disparity” language tied to calculating the binocular convergence point G; ¶[0219], “the distance d between the user's eyes and the virtual screen … As a result, the vergence distance l, which is the distance to the gaze point, is given by the following mathematical formula. z represents the distance between the virtual screen and the gaze point”, ties the gaze point to coordinates with respect to the rendered virtual screen; ¶[0205], “the augmented reality device (100) can calculate the two-dimensional position coordinate value of the gaze direction of the user's eye (E) on the virtual screen (1300) using the degree of rotation (α and β) of the user's eye (E)”, places gaze coordinates on the rendered screen; ¶[0211], “The processor (150)… can determine the gaze direction of the left and right eyes using gaze information output from the gaze tracking sensor (140a, 140b). In one embodiment… can calculate a first gaze vector … and a second gaze vector …”, shows explicit per-eye vectors that are then used together, as shown in ¶[0212], to track the combined eye movement patterns and 3D gaze coordinates; see also ¶[0216]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Keun et al. (KR 20220170336 A), hereto referred as Keun, and further in view of Monti (GB 2613084 A), hereto referred as Monti, and further in view of Young et al. (US 20190354173 A1), hereto referred as Young.
Keun teaches claim 13 as described above. The combined Keun and Monti teaches claim 15 as described above.
Regarding claim 16, Keun does not fully teach that execution of the instructions by the processor further cause the processor to: generate a rendered gaze ray associated with the binocular convergence. Rather, Keun teaches computing binocular gaze directions and the convergence point G, and placing gaze coordinates relative to the virtual screen, but Keun does not disclose rendering a gaze ray on the display associated with that convergence. For example: (Keun, ¶[0211]: “The processor (150… ) can determine the gaze direction of the left and right eyes… In one embodiment… can calculate a first gaze vector… and a second gaze vector…”, shows per-eye vectors computed internally for both eyes; ¶[0212]: “The augmented reality device (100) can estimate the position coordinates of the gaze point (G… ) by using gaze information about binocular disparity and the gaze direction of the left eye and the gaze direction of the right eye”, shows estimating the binocular convergence point coordinates; ¶[0219]: “the distance d between the user’s eyes and the virtual screen… z represents the distance between the virtual screen and the gaze point”, locates the gaze point relative to the rendered virtual screen but still does not disclose drawing/rendering a gaze ray; ¶[0042]: “'gaze' means an imaginary line from the user’s pupil to the gaze direction”, defines gaze as an imaginary line rather than a rendered on-screen line).
Young explicitly teaches that the headset’s display renders gaze-related visual indicators corresponding to eye movement vectors (Young, ¶[0094]: “The saccade path 510 is superimposed onto display 810 and shows fixation point A (e.g., direction 506 and vector XF-o)”, teaches a rendered, on-screen path/vector corresponding to gaze movement; ¶[0096]: “predicted landing points for fixation point B are superimposed onto display 810”, further shows rendered gaze-related indicators superimposed on the HMD display.) These passages demonstrate that Young visualizes where the user is looking by overlaying a rendered line or point on the virtual display. This fills the gap left by Keun, which performs gaze computations internally but does not visually depict them. One of ordinary skill in the art would recognize that Young’s display technique could be integrated into Keun’s existing binocular gaze system to represent the convergence point G visually on the virtual screen, thereby creating the claimed rendered gaze ray associated with binocular convergence.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Keun in view of Young to generate a rendered gaze ray associated with the binocular convergence. The combination is feasible because Keun already computes left/right gaze vectors and the 3-D gaze point G (including coordinates relative to the virtual screen), and Young demonstrates on-display overlays of gaze-related vectors/paths and landing points; adding a simple graphics overlay to depict a line (ray) from the eye or display toward G uses the same rendering pipeline that presents AR content. The benefit of the combination includes clear visual feedback for calibration and interaction, improved alignment between gaze estimation and displayed content, and enhanced usability by exposing the system’s internal gaze solution as an on-screen indicator.
Conclusion
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to AARON MERRIAM whose telephone number is (703) 756-
5938. The examiner can normally be reached M-F 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is
encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Jason Sims can be reached on (571)272-4867. The fax phone number for the
organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained
from Patent Center. Unpublished application information in Patent Center is available to
registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for
more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON MERRIAM/Examiner, Art Unit 3791
/MATTHEW KREMER/Primary Examiner, Art Unit 3791
Prosecution Insights