DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Independent claims:
Claim(s) 1-15, 17, and 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gibson et al. (US 20210106219) in view of Pohl et al. (US 20190180723).
Regarding claim 1.
Gibson teaches:
An apparatus comprising: a light source to generate a light signal (Gibson [0061] Moreover, in some examples, the eye tracking laser light source may be illuminated only for sufficient time to obtain each sample and turned off between samples. This may further help to reduce power consumption by the laser and current-to-voltage converters.);
a scanner to steer the light signal in a two-dimensional scan pattern that defines a scan region of an eye of a user (Gibson [0003] In an example approach, light from multiple laser light sources is scanned across a region that includes an eye of a user.);
a non-imaging photodetector to generate one or more output signals based on one or more reflected signals from the scan region ([0055] FIG. 9 is a block diagram of example processing pipelines 900 in another eye tracking system. The processing pipelines 900 are described and illustrated in the context of a single photodetector 902 for non-limiting, illustrative purposes. It will be understood that the system and processes described may apply to each of a plurality of photodetectors in the eye tracking system. As shown in FIG. 9, the pipelines 900 include the photodetector 902, an infrared (IR) laser drive and scanning optics system 904, a current-to-voltage converter 908, comparators 910, a digital-to-analog converter (DAC) 912, a serializer 914, position logic 916, a summing junction 920, an analog-to-digital converter (ADC) 922, gamma correction logic 924, and a grayscale image frame buffer 926. The photodetector 902 detects light that is reflected from an eye that is illuminated by an infrared laser light source controlled by the infrared laser drive and scanning optics system 904. The photodetector 902 generates a current based on the detected light.);
and a processing system to: estimate the position of the center of the eye at a first time step (Figure 4: 412: region of interests (center of eye) [0036] The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.);
estimate the gaze angle of the eye at the first time step (Gibson [0001] Angles associated with the eye that are measured with reference to the head are referred to as “eye-in-head angles.” Information regarding a direction in which the head is facing in a coordinate system (e.g., a three-dimensional coordinate system) may be combined with the eye-in-head angles to determine a direction of gaze of a user (i.e., the direction in which the user looks) in the coordinate system and/or a point of gaze of the user (i.e., a location at which the user looks) in the coordinate system.);
determine a subregion of the scan region based on the feature positions (Gibson [0060] Additionally, with such a harmonically oscillating mirror system, the mirror motion has a greater speed in the center of the motion than at the edges of the motion. As such, if a constant sample rate is utilized for gaze tracking, more gaze signals are sampled at the edges of the image than in the center, resulting in variable resolution across the image. Accordingly, in some examples, the system may be configured to utilize a variable sample rate to compensate for the variable mirror speed and thus to achieve a more even resolution across the image.);
and transmit, during the second time step, one or more signals to (1) disable at least one of the non-imaging photodetector or the light source while scanning portions of the scan region outside of the subregion, and (2) enable the non- imaging photodetector and the light source while the scanner scans the subregion (Gibson [0059] In the above examples, the scanning optics may include in a sine-wave mirror system that scans in a first direction (e.g. an x-direction) faster than in a second orthogonal direction (e.g. a y-direction). Due to the harmonic oscillation of the mirror, the speed of the mirror slows to a stop at the vertical and horizontal edges of the mirror motion at respective points in time, resulting in unnecessarily higher power density at the edges. As such, in some examples, sinusoidal correction may be applied to the system by turning off the infrared light source when the scanning mirror is scanning at the edges, and turning the infrared light source on when the scanning mirror is scanning between the edges. Such a correction function may further help to conserve power.).
Gibson fails to teach:
estimate, based on the eye center position and the gaze angle, respective positions of a plurality of features of the eye at a second time step, the second time step occurring after the first time step (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.);
Pohl teaches:
estimate, based on the eye center position and the gaze angle, respective positions of a plurality of features of the eye at a second time step, the second time step occurring after the first time step (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 7.
Gibson teaches:
An apparatus comprising: a light source to generate a light signal (Gibson [0061] Moreover, in some examples, the eye tracking laser light source may be illuminated only for sufficient time to obtain each sample and turned off between samples. This may further help to reduce power consumption by the laser and current-to-voltage converters.);
a scanner to steer the light signal in a two-dimensional scan pattern that defines a scan region of an eye of a user (Gibson [0003] In an example approach, light from multiple laser light sources is scanned across a region that includes an eye of a user.);
a non-imaging photodetector to generate one or more output signals based on one or more reflected signals from the scan region ([0055] FIG. 9 is a block diagram of example processing pipelines 900 in another eye tracking system. The processing pipelines 900 are described and illustrated in the context of a single photodetector 902 for non-limiting, illustrative purposes. It will be understood that the system and processes described may apply to each of a plurality of photodetectors in the eye tracking system. As shown in FIG. 9, the pipelines 900 include the photodetector 902, an infrared (IR) laser drive and scanning optics system 904, a current-to-voltage converter 908, comparators 910, a digital-to-analog converter (DAC) 912, a serializer 914, position logic 916, a summing junction 920, an analog-to-digital converter (ADC) 922, gamma correction logic 924, and a grayscale image frame buffer 926. The photodetector 902 detects light that is reflected from an eye that is illuminated by an infrared laser light source controlled by the infrared laser drive and scanning optics system 904. The photodetector 902 generates a current based on the detected light.);
and a processing system to: estimate the position of the center of the eye at a first time step (Figure 4: 412: region of interests (center of eye) [0036] The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.);
estimate the gaze angle of the eye at the first time step (Gibson [0001] Angles associated with the eye that are measured with reference to the head are referred to as “eye-in-head angles.” Information regarding a direction in which the head is facing in a coordinate system (e.g., a three-dimensional coordinate system) may be combined with the eye-in-head angles to determine a direction of gaze of a user (i.e., the direction in which the user looks) in the coordinate system and/or a point of gaze of the user (i.e., a location at which the user looks) in the coordinate system.);
determine a subregion of the scan region based on the feature positions (Gibson [0060] Additionally, with such a harmonically oscillating mirror system, the mirror motion has a greater speed in the center of the motion than at the edges of the motion. As such, if a constant sample rate is utilized for gaze tracking, more gaze signals are sampled at the edges of the image than in the center, resulting in variable resolution across the image. Accordingly, in some examples, the system may be configured to utilize a variable sample rate to compensate for the variable mirror speed and thus to achieve a more even resolution across the image.);
and transmit, during the second time step, one or more signals to (1) disable at least one of the non-imaging photodetector or the light source while scanning portions of the scan region outside of the subregion, and (2) enable the non- imaging photodetector and the light source while the scanner scans the subregion (Gibson [0059] In the above examples, the scanning optics may include in a sine-wave mirror system that scans in a first direction (e.g. an x-direction) faster than in a second orthogonal direction (e.g. a y-direction). Due to the harmonic oscillation of the mirror, the speed of the mirror slows to a stop at the vertical and horizontal edges of the mirror motion at respective points in time, resulting in unnecessarily higher power density at the edges. As such, in some examples, sinusoidal correction may be applied to the system by turning off the infrared light source when the scanning mirror is scanning at the edges, and turning the infrared light source on when the scanning mirror is scanning between the edges. Such a correction function may further help to conserve power.).
Gibson fails to teach:
estimate, based on the eye center position and the gaze angle, the position of a feature of the eye at a second time step, the second time step occurring after the first time step (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.);
Pohl teaches:
estimate, based on the eye center position and the gaze angle, the position of a feature of the eye at a second time step, the second time step occurring after the first time step (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 11.
Gibson teaches:
An apparatus comprising: a light source to generate a light signal (Gibson [0061] Moreover, in some examples, the eye tracking laser light source may be illuminated only for sufficient time to obtain each sample and turned off between samples. This may further help to reduce power consumption by the laser and current-to-voltage converters.);
a scanner to steer the light signal in a two-dimensional scan pattern that defines a scan region of an eye of a user (Gibson [0003] In an example approach, light from multiple laser light sources is scanned across a region that includes an eye of a user.);
a non-imaging photodetector to generate one or more output signals based on one or more reflected signals from the scan region ([0055] FIG. 9 is a block diagram of example processing pipelines 900 in another eye tracking system. The processing pipelines 900 are described and illustrated in the context of a single photodetector 902 for non-limiting, illustrative purposes. It will be understood that the system and processes described may apply to each of a plurality of photodetectors in the eye tracking system. As shown in FIG. 9, the pipelines 900 include the photodetector 902, an infrared (IR) laser drive and scanning optics system 904, a current-to-voltage converter 908, comparators 910, a digital-to-analog converter (DAC) 912, a serializer 914, position logic 916, a summing junction 920, an analog-to-digital converter (ADC) 922, gamma correction logic 924, and a grayscale image frame buffer 926. The photodetector 902 detects light that is reflected from an eye that is illuminated by an infrared laser light source controlled by the infrared laser drive and scanning optics system 904. The photodetector 902 generates a current based on the detected light.);
and a processing system to: estimate the position of the center of the eye at a first time step (Figure 4: 412: region of interests (center of eye) [0036] The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.);
estimate the gaze angle of the eye at the first time step (Gibson [0001] Angles associated with the eye that are measured with reference to the head are referred to as “eye-in-head angles.” Information regarding a direction in which the head is facing in a coordinate system (e.g., a three-dimensional coordinate system) may be combined with the eye-in-head angles to determine a direction of gaze of a user (i.e., the direction in which the user looks) in the coordinate system and/or a point of gaze of the user (i.e., a location at which the user looks) in the coordinate system.);
determine a subregion of the scan region based on the eye center position and the gaze angle range (Gibson [0060] Additionally, with such a harmonically oscillating mirror system, the mirror motion has a greater speed in the center of the motion than at the edges of the motion. As such, if a constant sample rate is utilized for gaze tracking, more gaze signals are sampled at the edges of the image than in the center, resulting in variable resolution across the image. Accordingly, in some examples, the system may be configured to utilize a variable sample rate to compensate for the variable mirror speed and thus to achieve a more even resolution across the image.);
and region, and (2) enable the non- imaging photodetector and the light source while the scanner scans the subregion (Gibson [0059] In the above examples, the scanning optics may include in a sine-wave mirror system that scans in a first direction (e.g. an x-direction) faster than in a second orthogonal direction (e.g. a y-direction). Due to the harmonic oscillation of the mirror, the speed of the mirror slows to a stop at the vertical and horizontal edges of the mirror motion at respective points in time, resulting in unnecessarily higher power density at the edges. As such, in some examples, sinusoidal correction may be applied to the system by turning off the infrared light source when the scanning mirror is scanning at the edges, and turning the infrared light source on when the scanning mirror is scanning between the edges. Such a correction function may further help to conserve power.).
Gibson fails to teach:
determine a gaze angle range that includes the gaze angle (Gibson [0051] The gaze direction logic 828 may then calculate the location of a glint using the known mirror scan angle at the time that the glint was received by a corresponding photodetector. Thus, glint locations may be determined using comparator outputs without performing image analysis, which may allow glint tracking to be performed in a power-efficient manner. The gaze direction logic 828 may use an eye tracking algorithm to determine an eye gaze direction based on the glint locations and the pupil location, as determined from the greyscale image.);
Pohl teaches:
determine a gaze angle range that includes the gaze angle (Gibson [0051] The gaze direction logic 828 may then calculate the location of a glint using the known mirror scan angle at the time that the glint was received by a corresponding photodetector. Thus, glint locations may be determined using comparator outputs without performing image analysis, which may allow glint tracking to be performed in a power-efficient manner. The gaze direction logic 828 may use an eye tracking algorithm to determine an eye gaze direction based on the glint locations and the pupil location, as determined from the greyscale image.);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 17.
Gibson teaches:
An apparatus comprising: a light source to generate a light signal (Gibson [0061] Moreover, in some examples, the eye tracking laser light source may be illuminated only for sufficient time to obtain each sample and turned off between samples. This may further help to reduce power consumption by the laser and current-to-voltage converters.);
a scanner to steer the light signal in a two-dimensional scan pattern that defines a scan region of an eye of a user (Gibson [0003] In an example approach, light from multiple laser light sources is scanned across a region that includes an eye of a user.);
a non-imaging photodetector to generate one or more output signals based on one or more reflected signals from the scan region ([0055] FIG. 9 is a block diagram of example processing pipelines 900 in another eye tracking system. The processing pipelines 900 are described and illustrated in the context of a single photodetector 902 for non-limiting, illustrative purposes. It will be understood that the system and processes described may apply to each of a plurality of photodetectors in the eye tracking system. As shown in FIG. 9, the pipelines 900 include the photodetector 902, an infrared (IR) laser drive and scanning optics system 904, a current-to-voltage converter 908, comparators 910, a digital-to-analog converter (DAC) 912, a serializer 914, position logic 916, a summing junction 920, an analog-to-digital converter (ADC) 922, gamma correction logic 924, and a grayscale image frame buffer 926. The photodetector 902 detects light that is reflected from an eye that is illuminated by an infrared laser light source controlled by the infrared laser drive and scanning optics system 904. The photodetector 902 generates a current based on the detected light.);
and a processing system to: estimate the position of the center of the eye at a first time step (Figure 4: 412: region of interests (center of eye) [0036] The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.);
estimate the position of the center of the eye at a first time step (Gibson [0001] Angles associated with the eye that are measured with reference to the head are referred to as “eye-in-head angles.” Information regarding a direction in which the head is facing in a coordinate system (e.g., a three-dimensional coordinate system) may be combined with the eye-in-head angles to determine a direction of gaze of a user (i.e., the direction in which the user looks) in the coordinate system and/or a point of gaze of the user (i.e., a location at which the user looks) in the coordinate system.);
determine a subregion of the scan region based on the eye center range and the gaze angle (Gibson [0060] Additionally, with such a harmonically oscillating mirror system, the mirror motion has a greater speed in the center of the motion than at the edges of the motion. As such, if a constant sample rate is utilized for gaze tracking, more gaze signals are sampled at the edges of the image than in the center, resulting in variable resolution across the image. Accordingly, in some examples, the system may be configured to utilize a variable sample rate to compensate for the variable mirror speed and thus to achieve a more even resolution across the image.);
and transmit, during the second time step, one or more signals to (1) disable at least one of the non-imaging photodetector or the light source while scanning portions of the scan region outside of the subregion, and (2) enable the non- imaging photodetector and the light source while the scanner scans the subregion (Gibson [0059] In the above examples, the scanning optics may include in a sine-wave mirror system that scans in a first direction (e.g. an x-direction) faster than in a second orthogonal direction (e.g. a y-direction). Due to the harmonic oscillation of the mirror, the speed of the mirror slows to a stop at the vertical and horizontal edges of the mirror motion at respective points in time, resulting in unnecessarily higher power density at the edges. As such, in some examples, sinusoidal correction may be applied to the system by turning off the infrared light source when the scanning mirror is scanning at the edges, and turning the infrared light source on when the scanning mirror is scanning between the edges. Such a correction function may further help to conserve power.).
Gibson fails to teach:
estimate the gaze angle of the eye at the first time step;determine an eye center range that includes the eye center position; (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.);
Pohl teaches:
estimate the gaze angle of the eye at the first time step;determine an eye center range that includes the eye center position; (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 21.
Gibson and Pohl teach:
An apparatus comprising: a light source to generate a light signal (Gibson [0061] Moreover, in some examples, the eye tracking laser light source may be illuminated only for sufficient time to obtain each sample and turned off between samples. This may further help to reduce power consumption by the laser and current-to-voltage converters.);
a scanner to steer the light signal in a two-dimensional scan pattern that defines a scan region of an eye of a user (Gibson [0003] In an example approach, light from multiple laser light sources is scanned across a region that includes an eye of a user.);
a non-imaging photodetector to generate one or more output signals based on one or more reflected signals from the scan region ([0055] FIG. 9 is a block diagram of example processing pipelines 900 in another eye tracking system. The processing pipelines 900 are described and illustrated in the context of a single photodetector 902 for non-limiting, illustrative purposes. It will be understood that the system and processes described may apply to each of a plurality of photodetectors in the eye tracking system. As shown in FIG. 9, the pipelines 900 include the photodetector 902, an infrared (IR) laser drive and scanning optics system 904, a current-to-voltage converter 908, comparators 910, a digital-to-analog converter (DAC) 912, a serializer 914, position logic 916, a summing junction 920, an analog-to-digital converter (ADC) 922, gamma correction logic 924, and a grayscale image frame buffer 926. The photodetector 902 detects light that is reflected from an eye that is illuminated by an infrared laser light source controlled by the infrared laser drive and scanning optics system 904. The photodetector 902 generates a current based on the detected light.);
and a processing system to: detect, at a first time step, a plurality of features of the eye (Figure 4: 412: region of interests (center of eye) [0036] The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.);
estimate respective positions of the features at the first time step (Figure 4: 412: region of interests (center of eye) [0036] The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.);
determine a subregion of the scan region based on the feature positions (Gibson [0060] Additionally, with such a harmonically oscillating mirror system, the mirror motion has a greater speed in the center of the motion than at the edges of the motion. As such, if a constant sample rate is utilized for gaze tracking, more gaze signals are sampled at the edges of the image than in the center, resulting in variable resolution across the image. Accordingly, in some examples, the system may be configured to utilize a variable sample rate to compensate for the variable mirror speed and thus to achieve a more even resolution across the image.);
and transmit, during the second time step, one or more signals to (1) disable at least one of the non-imaging photodetector or the light source while scanning portions of the scan region outside of the subregion, and (2) enable the non- imaging photodetector and the light source while the scanner scans the subregion (Gibson [0059] In the above examples, the scanning optics may include in a sine-wave mirror system that scans in a first direction (e.g. an x-direction) faster than in a second orthogonal direction (e.g. a y-direction). Due to the harmonic oscillation of the mirror, the speed of the mirror slows to a stop at the vertical and horizontal edges of the mirror motion at respective points in time, resulting in unnecessarily higher power density at the edges. As such, in some examples, sinusoidal correction may be applied to the system by turning off the infrared light source when the scanning mirror is scanning at the edges, and turning the infrared light source on when the scanning mirror is scanning between the edges. Such a correction function may further help to conserve power.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Dependent claims:
Regarding claim 2.
Gibson and Pohl teach:
The apparatus of claim 1
wherein the estimating of the feature positions is further based on one or more geometric parameters of the eye (Gibson [0036] FIG. 4 illustrates a multi-laser scan 400 in a region that includes an eye 410 of a user in accordance with an embodiment. As shown in FIG. 4, light that is emitted from a first laser is scanned across a first portion of the region along a first path 408a from a first starting point (labeled “START1”) to a first stopping point (labeled “STOP1”) for each frame. Light that is emitted from a second laser is scanned across a second portion of the region along a second path 408b from a second starting point (labeled “START2”) to a second stopping point (labeled “STOP2”) for each frame. Each of the first and second paths 408a and 408b is shown to form a respective raster pattern for non-limiting, illustrative purposes. For instance, the first path 408a is shown to form a raster pattern that includes ten horizontal lines for non-limiting, illustrative purposes, and the second path 408b is shown to form a raster pattern that includes six horizontal lines for non-limiting, illustrative purposes. It will be recognized that each of the raster patterns may include any suitable number of horizontal lines. The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 3.
Gibson and Pohl teach:
The apparatus of claim 1
wherein the estimating of the feature positions is based on a gaze angle range that includes the gaze angle (Gibson [0051] The gaze direction logic 828 may then calculate the location of a glint using the known mirror scan angle at the time that the glint was received by a corresponding photodetector. Thus, glint locations may be determined using comparator outputs without performing image analysis, which may allow glint tracking to be performed in a power-efficient manner. The gaze direction logic 828 may use an eye tracking algorithm to determine an eye gaze direction based on the glint locations and the pupil location, as determined from the greyscale image.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 4.
Gibson and Pohl teach:
The apparatus of claim 1
wherein the estimating of the feature positions is based on an eye center range that includes the eye center position (Gibson: Figure 3 and Figure 4: the eye is split into different regions and scanned, based on the knowledge of the anatomy of an eye it can see what each part of an eye. You can see that the center would include the iris and other portions based on the center of the eye).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 5.
Gibson and Pohl teach:
The apparatus of claim 1
wherein the estimating of the gaze angle is based on respective positions of at least one of the features at the first time step (Gibson: Figure 3 and Figure 4: the eye is split into different regions and scanned, based on the knowledge of the anatomy of an eye it can see what each part of an eye. You can see that the center would include the iris and other portions based on the center of the eye).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 6.
Gibson and Pohl teach:
The apparatus of claim 1
wherein the processing system is further to determine a bounding region based on the feature positions, and wherein the determining of the subregion is based on the bounding region (Gibson [0073] In further accordance with this embodiment, the method of flowchart 1000 further includes stopping the scanning of the first light from the first laser light source across the first portion of the region and beginning the scanning of the second light from the second laser light source across the second portion of the region based at least in part on a scan of the first light traversing the region of interest and reaching an outer boundary of the region of interest. For example, the first portion of the region and the second portion of the region may include the region of interest. In accordance with this example, stopping the scanning of the first light from the first laser light source across the first portion of the region and beginning the scanning of the second light from the second laser light source across the second portion of the region may cause the first light from the first laser light source and the second light from the second laser light source to be scanned across the region of interest. In an example implementation, the sequential scan control logic 712 may stop the scanning of the first light from the first laser diode D1 across the first portion of the region and begin the scanning of the second light from the second laser diode D2 across the second portion of the region based at least in part on the scan of the first light traversing the region of interest and reaching the outer boundary of the region of interest.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 8.
Gibson and Pohl teach:
The apparatus of claim 7
wherein the estimating of the feature position is further based on one or more geometric parameters of the eye (Gibson [0036] FIG. 4 illustrates a multi-laser scan 400 in a region that includes an eye 410 of a user in accordance with an embodiment. As shown in FIG. 4, light that is emitted from a first laser is scanned across a first portion of the region along a first path 408a from a first starting point (labeled “START1”) to a first stopping point (labeled “STOP1”) for each frame. Light that is emitted from a second laser is scanned across a second portion of the region along a second path 408b from a second starting point (labeled “START2”) to a second stopping point (labeled “STOP2”) for each frame. Each of the first and second paths 408a and 408b is shown to form a respective raster pattern for non-limiting, illustrative purposes. For instance, the first path 408a is shown to form a raster pattern that includes ten horizontal lines for non-limiting, illustrative purposes, and the second path 408b is shown to form a raster pattern that includes six horizontal lines for non-limiting, illustrative purposes. It will be recognized that each of the raster patterns may include any suitable number of horizontal lines. The region includes a region of interest 412, which includes the eye 410. The eye 410 is shown to include a pupil 402, an iris 404, and a cornea 406. It will be recognized that the region of interest 412 need not necessarily include an entirety of the eye 410.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 9.
Gibson and Pohl teach:
The apparatus of claim 7
wherein the second time step is the next time step after the first time step (Gibson [0003] In an example approach, light from multiple laser light sources is scanned across a region that includes an eye of a user. The laser light sources include at least a first laser light source and a second laser light source. First light from the first laser light source is scanned across a first portion of the region during a first period of time. Second light from the second laser light source is scanned across a second portion of the region during a second period of time that is different from the first period of time. The second portion of the region at least partially overlaps the first portion of the region. Portion(s) of the light that are reflected from an iris of the eye are detected by one or more respective photodetectors. Analog signal(s) are generated by the respective photodetector(s) based at least in part on the respective detected portion(s) of the light. A sum of the analog signal(s) that are generated by the respective photodetector(s) is converted to a digital signal. A current mirror scan angle of a scanning mirror that is used to scan the light from the laser light sources across the region is calculated. The digital signal is provided in a pixel of a frame buffer based at least in part on the current mirror scan angle.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 10.
Gibson and Pohl teach:
The apparatus of claim 7
wherein at least one time step occurs between the first time step and the second time step (Gibson [0003] In an example approach, light from multiple laser light sources is scanned across a region that includes an eye of a user. The laser light sources include at least a first laser light source and a second laser light source. First light from the first laser light source is scanned across a first portion of the region during a first period of time. Second light from the second laser light source is scanned across a second portion of the region during a second period of time that is different from the first period of time. The second portion of the region at least partially overlaps the first portion of the region. Portion(s) of the light that are reflected from an iris of the eye are detected by one or more respective photodetectors. Analog signal(s) are generated by the respective photodetector(s) based at least in part on the respective detected portion(s) of the light. A sum of the analog signal(s) that are generated by the respective photodetector(s) is converted to a digital signal. A current mirror scan angle of a scanning mirror that is used to scan the light from the laser light sources across the region is calculated. The digital signal is provided in a pixel of a frame buffer based at least in part on the current mirror scan angle.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 12.
Gibson and Pohl teach:
The apparatus of claim 11
wherein the processing system is further to estimate the velocity of rotation of the eye at the first time step, and wherein the determining of the gaze angle range is based on the eye rotation velocity (Gibson [0001] An eye tracking system is a system that is configured to track an eye with respect to a frame of reference. Tracking the eye typically includes determining location and/or movement (e.g., rotation) of the eye with respect to the frame of reference. For instance, the frame of reference may be a head in which the eye is located. Angles associated with the eye that are measured with reference to the head are referred to as “eye-in-head angles.” Information regarding a direction in which the head is facing in a coordinate system (e.g., a three-dimensional coordinate system) may be combined with the eye-in-head angles to determine a direction of gaze of a user (i.e., the direction in which the user looks) in the coordinate system and/or a point of gaze of the user (i.e., a location at which the user looks) in the coordinate system. [0034] As the light scans across the eye, each of the photodetectors 208a and 208b receives light that is scattered by the iris of the eye 202 and light that is specularly reflected from the cornea of the eye 202 at specific scanning system angles based upon the locations of the photodetectors 204a and 204b and the rotational position of the eye 202. Lower-intensity scattered light (e.g., reflected from the iris) is used to form a greyscale image of the scanned region of the eye 202 in a pupil location processing system, and higher-intensity specular reflections (e.g., reflected from the cornea) are utilized to determine glint locations in a glint location processing system. For pupil location processing, the signals from the photodetectors 208a and 208b may be sampled and computationally combined (e.g., summed) at each angular position of the scanning mirror system 206 to form a bitmap image for use in identifying a location of the pupil. Summing the signals from the photodetectors 208a and 208b may provide a higher signal-to-noise ratio than using a single sensor for detecting scattered light. Specular reflections may be processed by recording a time stamp and an angular position at which the higher intensity of the specular reflection is received at that photodetector, and changes in the relative locations of the glints may be used to provide information regarding eye rotation. Eye rotation information from glint tracking may be used to control a frame rate of the pupil location processing system in some examples.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 13.
Gibson and Pohl teach:
The apparatus of claim 11
wherein the determining of the gaze angle range is based on a maximum eye rotation velocity (Gibson [0053] Pupil location processing may consume more power than glint location processing at a same frame rate. As such, the pupil location processing system 810 may be configured to be inactive or operate at a lower frame rate until a threshold magnitude of eye rotation is detected via the glint location processing system 812 system. The eye tracking algorithm may use a most recent pupil image stored in the frame buffer 820 for gaze determination until eye rotation of sufficient magnitude is determined from the glint location processing system 812 to trigger operation of (or a higher frame rate of operation for) the pupil location processing system 812. This may help the eye tracking system to conserve power.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 14.
Gibson and Pohl teach:
The apparatus of claim 11
wherein the determining of the subregion is based on an eye center range that includes the eye center position (Gibson: Figure 3 and Figure 4: the eye is split into different regions and scanned, based on the knowledge of the anatomy of an eye it can see what each part of an eye. You can see that the center would include the iris and other portions based on the center of the eye) (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 15.
Gibson and Pohl teach:
The apparatus of claim 11
wherein the estimating of the gaze angle is based on a position of a feature at the first time step (Pohl [0047] FIG. 3 illustrates an eye 8 where the centre of the pupil 8.2 is detected and illustrated by a cross. The position of that centre relative to the global position of the eye indicates the eye's gaze.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Regarding claim 22.
Gibson and Pohl teach:
The apparatus of claim 21
wherein the processing system is further to determine a bounding region based on the feature positions, and wherein the determining of the subregion is based on the bounding region (Gibson [0073] In further accordance with this embodiment, the method of flowchart 1000 further includes stopping the scanning of the first light from the first laser light source across the first portion of the region and beginning the scanning of the second light from the second laser light source across the second portion of the region based at least in part on a scan of the first light traversing the region of interest and reaching an outer boundary of the region of interest. For example, the first portion of the region and the second portion of the region may include the region of interest. In accordance with this example, stopping the scanning of the first light from the first laser light source across the first portion of the region and beginning the scanning of the second light from the second laser light source across the second portion of the region may cause the first light from the first laser light source and the second light from the second laser light source to be scanned across the region of interest. In an example implementation, the sequential scan control logic 712 may stop the scanning of the first light from the first laser diode D1 across the first portion of the region and begin the scanning of the second light from the second laser diode D2 across the second portion of the region based at least in part on the scan of the first light traversing the region of interest and reaching the outer boundary of the region of interest.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson with Pohl. Estimating the gaze and the eye features, as in Pohl, would benefit the Gibson teachings by allowing a way to find out the features of the eye. Additionally, this is the application of a known technique, Estimating the gaze and the eye features, to yield predictable results.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gibson et al. (US 20210106219) in view of Pohl et al. (US 20190180723) and Hong et al. (KR 20160002258).
Regarding claim 16.
Gibson and Pohl teach:
The apparatus of claim 11
Gibson and Pohl fail to teach:
wherein the gaze angle comprises an azimuth angle and an elevation angle, and wherein the gaze angle range comprises an azimuth angle range and an elevation angle range (Hong [Pg 6 Par 3] The azimuth angle and elevation change value 'of the first mode, and the' gaze angle change value 'in the second mode. This is because a visual field that can be viewed according to a posture that a person can take is limited, so that a more realistic simulation image can be provided when the viewing angle and elevation angle are restricted to vary within a predetermined range. [Pg 6 Par 3] And limits the range of the gaze angle change value or the reference azimuth angle and elevation angle change value that is calculated differently depending on the distance and the direction deviating from the boundary of the virtual figure.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson and Pohl with Hong. Having an azimuth angle and elevation angle, as in Hong, would benefit the Gibson and Pohl teachings by allowing a way to find out the features of the eye as well as other ways to scan. Additionally, this is the application of a known technique, Having an azimuth angle and elevation angle, to yield predictable results.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gibson et al. (US 20210106219) in view of Pohl et al. (US 20190180723) and Baier et al. (US 20210248827).
Regarding claim 20.
Gibson and Pohl teach:
The apparatus of claim 16
Gibson and Pohl fail teach:
wherein the eye center position comprises a first coordinate offset, a second coordinate offset, and a third coordinate offset, and wherein the eye center range comprises a first range that includes the first coordinate offset, a second range that includes the second coordinate offset, and a third range that includes the third coordinate offset (Baier [0052] In the example, the 6DOF totem subsystem 404A and the 6DOF subsystem 404B cooperate to determine six coordinates (e.g., offsets in three translation directions and rotation along three axes) of the handheld controller 400B relative to the wearable head device 400A. The six degrees of freedom may be expressed relative to a coordinate system of the wearable head device 400A. The three translation offsets may be expressed as X, Y, and Z offsets in such a coordinate system, as a translation matrix, or as some other representation. The rotation degrees of freedom may be expressed as sequence of yaw, pitch and roll rotations, as a rotation matrix, as a quaternion, or as some other representation.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson and Pohl with Baier. Having coordinate offsets, as in Baier, would benefit the Gibson and Pohl teachings by allowing a way to adjust coordinates to a fixed range. Additionally, this is the application of a known technique, Having coordinate offsets, to yield predictable results.
Claim(s) 23 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gibson et al. (US 20210106219) in view of Pohl et al. (US 20190180723) and Edwin et al. (US 20190222830).
Regarding claim 23.
Gibson and Pohl teach:
The apparatus of claim 21
Gibson and Pohl fail to teach:
wherein the estimating of the feature positions comprises determining a center of mass (Edwin [0226] Pupil identification module 712 may receive preprocessed images from the image preprocessing module 710 and may identify regions of those images that include the user's pupil. The pupil identification module 712 may, in some embodiments, determine the coordinates of the position, or coordinates, of the center, or centroid, of the user's pupil in the eye tracking images from camera 324. In at least some embodiments, pupil identification module 712 may identify contours in eye tracking images (e.g., contours of pupil iris boundary), identify contour moments (i.e., centers of mass), apply a starburst pupil detection and/or a canny edge detection algorithm, reject outliers based on intensity values, identify sub-pixel boundary points, correct for eye-camera distortion (i.e., distortion in images captured by eye camera 324), apply a random sample consensus (RANSAC) iterative algorithm to fit an ellipse to boundaries in the eye tracking images, apply a tracking filter to the images, and identify sub-pixel image coordinates of the user's pupil centroid. The pupil identification module 712 may output pupil identification data, which may indicate which regions of the preprocessing images module 712 identified as showing the user's pupil, to glint detection and labeling module 714. The pupil identification module 712 may provide the 2D coordinates of the user's pupil (i.e., the 2D coordinates of the centroid of the user's pupil) within each eye tracking image to glint detection module 714. In at least some embodiments, pupil identification module 712 may also provide pupil identification data of the same sort to coordinate system normalization module 718.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson and Pohl with Edwin. Determining centers of mass, as in Edwin, would benefit the Gibson and Pohl teachings by allowing a way to see the center of mass and using that data to further make more accurate. Additionally, this is the application of a known technique, Determining centers of mass, to yield predictable results.
Regarding claim 24.
The apparatus of claim 21 wherein the estimating of the feature positions comprises fitting an ellipse (Edwin [0226] Pupil identification module 712 may receive preprocessed images from the image preprocessing module 710 and may identify regions of those images that include the user's pupil. The pupil identification module 712 may, in some embodiments, determine the coordinates of the position, or coordinates, of the center, or centroid, of the user's pupil in the eye tracking images from camera 324. In at least some embodiments, pupil identification module 712 may identify contours in eye tracking images (e.g., contours of pupil iris boundary), identify contour moments (i.e., centers of mass), apply a starburst pupil detection and/or a canny edge detection algorithm, reject outliers based on intensity values, identify sub-pixel boundary points, correct for eye-camera distortion (i.e., distortion in images captured by eye camera 324), apply a random sample consensus (RANSAC) iterative algorithm to fit an ellipse to boundaries in the eye tracking images, apply a tracking filter to the images, and identify sub-pixel image coordinates of the user's pupil centroid. The pupil identification module 712 may output pupil identification data, which may indicate which regions of the preprocessing images module 712 identified as showing the user's pupil, to glint detection and labeling module 714. The pupil identification module 712 may provide the 2D coordinates of the user's pupil (i.e., the 2D coordinates of the centroid of the user's pupil) within each eye tracking image to glint detection module 714. In at least some embodiments, pupil identification module 712 may also provide pupil identification data of the same sort to coordinate system normalization module 718.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Gibson and Pohl with Edwin. Determining centers of mass, as in Edwin, would benefit the Gibson and Pohl teachings by allowing a way to see the center of mass and using that data to further make more accurate. Additionally, this is the application of a known technique, Determining centers of mass, to yield predictable results.
Allowable Subject Matter
Claims 18 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENIS VASILIY MINKO whose telephone number is (571)270-5226. The examiner can normally be reached Monday-Thursday 8:30-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENIS VASILIY MINKO/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights