Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is in response to the communication filed 7/27/2026.
Amendments to claims 1, 14, 15, 18-20, and, filed 7/27/2026, are acknowledged and accepted.
Cancellation of claim 12, filed 7/27/2026, is acknowledged and accepted.
Newly submitted claim 21, filed 7/27/2026, is acknowledged and accepted.
Due to the amendments, the previous rejections under 35 U.S.C. 112(b) are now withdrawn.
Response to Arguments
On pgs. 14-17 of the Remarks, filed 7/27/2026, Applicant's arguments with respect to claims 1, 14, and 18 have been fully considered but are moot because the Applicant is arguing newly amended claims, filed 7/27/2026, not the Non-Final Rejection, filed 4/29/2026. Newly amended claims are argued below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zakharov et al (EP 3760116 A1, hereinafter “Zakharov”) in view of Matthies et al (NPL entitled CapGlasses: Untethered Capacitive Sensing with Smart Glasses, hereinafter “Matthies”), Komaki (US 20170150897), and Hayashibara (JP 2001154794 A).
Regarding claim 1, Zakharov discloses (see FIGs.1-2, ¶s 25-33) a method for detecting an activity of a first eye (eye 4) of a user using glasses, the method comprising:
acquiring a first electrostatic charge variation signal (“detection signal”) through a first electrode (2) and a second electrode (3) of a first electrostatic charge variation sensor (eye blink sensor 1) of the glasses, the first electrostatic charge variation signal (“detection signal”) indicative of a difference (“voltage drop”) between electrostatic charge variations (i.e. varying electric potentials) detected by the first and the second electrodes (2 and 3), the first electrode (2) and the second electrode (3) being spaced from each other, each facing the first eye (eye 4), the first electrode (2) being configured to be electrostatically coupled to the first eye (eye 4) without physical contact with the user to detect respective electrostatic charge variations (i.e. varying electric potentials) generated by a blink of the first eye (eye 4);
obtaining, at a controller (measurement unit 12 with processing unit 13), the first electrostatic charge variation signal (“detection signal”) from the first electrostatic charge variation sensor (eye blink sensor 1);
(see ¶ 32: “The measurement unit 12 is configured to measure an electrical voltage drop between first electrode 2 and the second electrode 3 and to generate a detection signal based on the voltage drop… such as a time-dependent function of a voltage drop”)
verifying, by the controller (measurement unit 12 with processing unit 13), a presence of one or more blink patterns (“blinking parameters”) in the first electrostatic charge variation signal (“detection signal”), each blink pattern (“blinking parameter”) being indicative of a respective click or of a respective blink, the click being a voluntary blink of the first eye (eye 4) and the blink being an involuntary blink of the first eye (eye 4) (see ¶ 33: “The processing unit [13] is configured to determine a blinking parameter associated with the eye 4, based on the detection signal(s). The blinking parameter is one of, or statistics of, a blinking frequency, a blinking pattern… a detection of a voluntary blink, and a detection of a spontaneous [i.e. involuntary] blink”);
when the first electrostatic charge variation signal (“detection signal”) has the one or more blink patterns (“blinking parameters”), determining, by the controller (measurement unit 12 with processing unit 13) for each blink pattern (“blinking parameter”), whether a first condition is verified;
when the first condition is not verified, detecting, by the controller (measurement unit 12 with processing unit 13), a respective blink; and
when the first condition is verified, detecting, by the controller (measurement unit 12 with processing unit 13), a respective click.
(Examiner notes that whether a “first condition” is verified or not simply corresponds to whether voluntary blinks (clicks) have been detected or not. Zakharov thus discloses steps D-F above, as they separately disclose detection of both voluntary and incomplete/involuntary/spontaneous blinks in ¶ 33, as cited above. Zakharov further discusses how (electrodynamic) measurements and signals may be analyzed to discern between voluntary and involuntary blinks – see ¶s 47-55 and FIGs. 6-8.)
Zakharov does not disclose:
contactlessly detecting an activity of a first eye
the first electrode and the second electrode being fixed to a frame of the glasses
each of the first and second electrodes being configured to be electrostatically coupled to the first eye without physical contact with the user
acquiring, by the controller and through a respective first electrode and a respective second electrode of a further first electrostatic charge variation sensor of the glasses, the respective first and second electrodes being fixed to the frame and each facing a second eye of the user, a further first electrostatic charge variation signal indicative of a difference between electrostatic charge variations detected by the respective first and second electrodes, each of the respective first and second electrodes being configured to be electrostatically coupled to the second eye without physical contact with the user to detect respective electrostatic charge variations generated by a blink of the second eye;
the determining comprising: determining whether the further first electrostatic charge variation signal has a blink pattern within a first threshold period of the blink pattern of the first electrostatic charge variation signal; determining that the first condition is not verified when the further first electrostatic charge variation signal has the blink pattern within the first threshold period; and determining that the first condition is verified when the further first electrostatic charge variation signal does not have the blink pattern within the first threshold period;
Zakharov and Matthies commonly relate to electrooculography.
Matthies discloses (see FIGs. 2-4, secs. 3-4):
contactlessly detecting an activity of a first eye (sec. 3: “Capacitive Sensing (CapSense) can be useful for proximity sensing”. Note also sec. 1: “we demonstrate a wearable solution to sense facial expressions and head gestures by untethered [i.e. contactless] CapSense technology”. See also FIG. 1 – among the Detectable Facial Expressions are eye-winks.)
the first electrode and the second electrode being fixed to a frame of the glasses (sec. 3.2: “Electrodes can be incorporated into glass frames”)
each of the first and second electrodes being configured to be electrostatically coupled (i.e. capacitively coupled) to the first eye without physical contact with the user (sec. 3.4 – “our interest to investigate a hardware configuration that can be truly mobile without being tethered to an external ground and thus would be hardware-wise self-contained”; sec. 4.1.2: “we do not need to attach a reference electrode”)
Zakharov and Komaki commonly relate to blink-detecting ocular devices.
Komaki, in combination with references already cited above, further supports acquiring, by the controller and through a respective first electrode and a respective second electrode of a further first electrostatic charge variation sensor of the glasses, the respective first and second electrodes being fixed to the frame and each facing a second eye of the user, a further first electrostatic charge variation signal indicative of a difference between electrostatic charge variations detected by the respective first and second electrodes, each of the respective first and second electrodes being configured to be electrostatically coupled to the second eye without physical contact with the user to detect respective electrostatic charge variations generated by a blink of the second eye;
(Examiner notes that this limitation simply duplicates prior recited/addressed limitations regarding the first eye with some corresponding first electrode, second electrode, electrostatic charge variation sensor, etc. – applying substantially similar language to a second eye with some analogous respective first electrode, respective second electrode, further first electrostatic charge variation sensor, etc.
Examiner notes also that Komaki similarly discloses ocular action/eye-motion detection apparatus accommodating both eyes, including symmetrized electrode arrangements and detection channels corresponding to the left and right eyes; see FIGs. 1-2 and ¶s 38-43.)
Zakharov and Hayashibara commonly relate to blink-detecting ocular devices.
Hayashibara, in combination with references already cited above, further supports the determining comprising: determining whether the further first electrostatic charge variation signal has a blink pattern within a first threshold period of the blink pattern of the first electrostatic charge variation signal; determining that the first condition is not verified when the further first electrostatic charge variation signal has the blink pattern within the first threshold period; and determining that the first condition is verified when the further first electrostatic charge variation signal does not have the blink pattern within the first threshold period. (As is standard knowledge regarding basic human physiology, ordinary/bilateral blinking occurs when both eyes open and close at (roughly) the same time. Hayashibara takes into account this basic behavior to distinguish single-eye blinks (i.e. for blink-induced click signals in pointing device 10, corresponding to when the claimed first condition is verified) from ordinary blinking (first condition is not verified) – implementing a simple logic/circuit that suppresses click signals for a certain period of time once the first eye closes, thereby allowing sufficient time to determine whether the other eye is also closing as part of an ordinary blink. See ¶s 33-37)
It would have therefore been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine teachings of Zakharov and Matthies, in order to achieve untethered, self-contained capacitive sensing to overcome limitations of more traditional methods that are prone to interference, limited for mobile use, etc. (Matthies sec. 1, 3.3-3.34).
It would have also been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to further combine Zakharov with aspects of Komaki’s electrooculographic device/methods, in order to provide bilateral detection for both eyes and support detection of a broad range of ocular activity – including the tracking of user eye movement in addition to blinking activity (Komaki ¶s 41-43).
It would have then been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to also implement basic timing-tolerance logic as taught by Hayashibara, in order to reliably identify when bilateral blinking occurs despite natural variations in relative blinking times between left and right eyes (Hayashibara ¶s 33-34, 36-37).
Regarding claim 2, modified Zakharov discloses the method according to claim 1.
Zakharov also discloses the method further comprising, when the first condition is not verified (i.e. when voluntary blinking is not detected):
determining, by the controller (measurement unit 12 with processing unit 13) and as a function of the one or more blink patterns (“blinking parameters”) of the first electrostatic charge variation signal (“detection signal”), an Inter-Eye blink interval (IEBI) parameter indicative of a blink frequency (refer again to ¶ 33: “The processing unit [13] is configured to determine a blinking parameter… The blinking parameter is one of, or statistics of, a blinking frequency, … an inter-blink interval, …”; thus Zakharov’s blinking parameter encompasses both blinking frequencies and inter-(eye )blink intervals. Examiner notes further that since frequencies and intervals (of time, i.e. periods) are simply inverse to one another, methods which employ either metric and differing only in this manner are equivalent to one another);
verifying, by the controller (measurement unit 12 with processing unit 13), whether the IEBI parameter (“blinking parameter”) is greater than an IEBI threshold value (“pre- blinking parameter”) (see ¶ 22: “The determined blinking parameter may be compared with predetermined blinking parameters…”; see also ¶ 56 for exemplary discussion on comparing determined and predetermined blinking frequencies (for spontaneous/involuntary blinks));
when the IEBI parameter (“blinking parameter”) is greater than the IEBI threshold value (“predetermined blinking parameter”), detecting a first activity of the first eye (eye 4), correlated to a lower attention level of the user; and
when the IEBI parameter (“blinking parameter”) is not greater than the IEBI threshold value (“predetermined blinking parameter”), detecting a second activity of the first eye (eye 4), correlated to a higher attention level of the user.
(Regarding steps C and D above, Examiner again refers to ¶ 56; here the blinking parameter is taken to be a blinking frequency. Zakharov discusses how frequencies may be determined and compared with other predetermined frequencies in order to detect activity associated with certain eye events/states, and that action may be taken based on such eye states (e.g. by outputting warning/trigger signals when detecting activity related to dry eye, tiredness, …). Note here that while Zakharov mentions a case of “low blinking frequency” – which establishes the general notion of threshold values – Zakharov also discusses, in ¶ 20, how detection signals may be compared to “reference signals describing a threshold value…”. Examiner lastly reemphasizes that, per ¶ 33, Zakharov’s blinking parameter broadly encompasses other metrics of eye activity (e.g. inter-blink intervals) which may be determined/detected/analyzed instead of blinking frequencies.)
Regarding claim 3, modified Zakharov discloses the method according claim 2.
Zakharov further discloses wherein, when the first condition is not verified (i.e. when voluntary blinking is not detected), determining the IEBI parameter (“blinking parameter”) comprises calculating a blink time distance ( “inter-blink interval”) between two consecutive blink patterns of the first electrostatic charge variation signal (“detection signal”), each blink pattern defining a respective blink. (Examiner again notes that Zakharov’s inter-blink interval simply corresponds to the blinking frequency’s inverse, and that both are encompassed by Zakharov’s blinking parameter; see ¶ 33. Refer also to ¶ 56 regarding spontaneous/involuntary blink frequencies.)
Regarding claim 4, modified Zakharov discloses the method according to claim 2.
Zakharov further discloses (see FIGs.1-2, ¶s 25-33) wherein, when the first condition is not verified (i.e. when detected blinking is not voluntary), determining the IEBI parameter (“blinking parameter”) comprises calculating a blink time distance (“inter-blink interval”) between two consecutive pairs of blink patterns of the first electrostatic charge variation signal (“detection signal”).
Komaki further supports a further first electrostatic charge variation signal, each pair of blink patterns of the first electrostatic charge variation signal and of the further first electrostatic charge variation signal defining a respective blink. (As noted with respect to claim 1 above, Komaki discloses ocular action/eye-motion detection for activity involving both eyes, see FIGs. 1-2 and ¶s 38-43; see also FIG. 11 and ¶s 79-105 showing signal waveforms for different types of eye activity, including eye blinks.)
Regarding claim 5, modified Zakharov discloses the method according to claim 2.
Zakharov further discloses (see FIGs. 1-2, FIG. 1 annotated below, and ¶s 25-33):
wherein the glasses comprise a first lens (lens 10) facing the first eye (eye 4),
wherein the glasses comprise a second electrostatic charge variation sensor (eye blink sensor 1) including a third electrode and a fourth electrode (third electrodes 9, of which three are shown in the annotated FIG. 1 above) spaced from each other, facing the first eye (eye 4) and configured to detect the respective electrostatic charge variations (i.e. varying electric potentials) generated by a movement of an eyeball of the first eye (eye 4), the first and second electrodes (2 and 3) being radially external with respect to the third and fourth electrodes (third electrodes 9) and with respect to a center of the first lens (lens 10), and
wherein the controller (measurement unit 12 with processing unit 13) is further coupled to the third and fourth electrodes (third electrodes 9), and
wherein the method further comprises:
acquiring, by the controller (measurement unit 12 with processing unit 13) and through the third and fourth electrodes (third electrodes 9), a second electrostatic charge variation signal (“detection signal”);
PNG
media_image1.png
680
1376
media_image1.png
Greyscale
[AltContent: textbox (FIG. 1 of Zakharov is annotated to highlight various features. In the left panel, the original figure, Zakharov’s reference numerals and labels are used. In the right panel, Zakharov’s three third electrodes 9 are identified with Applicant’s third, fourth, fifth, and sixth electrodes.)]when the one or more single peaks are not present in the second electrostatic charge variation signal (“detection signal”), determining the IEBI parameter (“blinking parameter”) (Examiner notes that the single peaks correspond to non-blinking motion, as discussed more below; since Zakharov discloses only on blinking motion, determination of blinking parameters will always occur in the absence of single peaks).
Komaki further supports (see FIGs. 1-2 and ¶s 38-43):
movement of an eyeball of the first eye in an absence of the blink of the first eye (Note: using similar electrooculographic principles as Zakharov, Komaki arranges electrodes in proximity to the user’s eyes in order to monitor eye activity by measuring induced electric potentials, this activity is not only limited to blinking as in Zakharov disclosure, however, but also extends to general eye movement including gaze direction. Komaki further discloses how signals generated by such general eye movements can be distinguished from signals caused by blinking)
wherein the method further comprises (see also the annotated FIG. 11 below):
the second electrostatic charge variation signal indicative of a difference between the electrostatic charge variations detected by the third and fourth electrodes (151a and 151b, their difference corresponding to Ch1 in FIG. 11) (Note: unlike Zakharov, who only discloses the measuring of electric potentials of each electrode with respect to that of the second electrode 3 (see Zakharov ¶ 32), Komaki measures potential drops between different pairs of electrodes to detect different types of eye movement besides blinking);
when the first condition is verified (i.e. when voluntary blinking is detected), verifying, by the controller (data processor 11), a presence of one or more single peaks in the second electrostatic charge variation signal (in Ch1), each single peak being a respective peak of the second electrostatic charge variation signal (in Ch1) not forming part of any blink pattern (as shown in the annotated FIG. 11 below) and being indicative of a respective movement (“Leftward eye movement”, “Rightward eye movement”) of the eyeball of the first eye in the absence of blink of the first eye;
when the one or more single peaks are present in the second electrostatic charge variation signal (in Ch1), determining an orientation (i.e. “signal waveforms”) of each of the single peaks with respect to a baseline (i.e. waveforms for periods devoid of eye activity) of the second electrostatic charge variation signal (in Ch1);
for each of the one or more single peaks in the second electrostatic charge variation signal (in Ch1), verifying whether the orientation (“signal waveforms”) is a predefined orientation (“signal waveform”, predefined in that each type of eye activity is associated with a distinguishing waveform, though that which corresponds to “Leftward eye movement” in FIG. 11 may freely be chosen as a particular predefinition);
for each of the one or more single peaks in the second electrostatic charge variation signal (in Ch1), when the respective orientation (“signal waveform”) is the predefined orientation (“signal waveform”), detecting a fifth activity of the first eye, indicative of a first movement (“Leftward movement”) of the first eye; and
for each of the one or more single peaks in the second electrostatic charge variation signal (in Ch1), when the respective orientation (“signal waveform”) is not the predefined orientation (“signal waveform”), detecting a sixth activity of the first eye, indicative of a second movement (“Rightward movement”) of the first eye opposite to the first movement (“Leftward movement”).
(As noted in ¶s 40-43 different types of eye movements may be distinguished based on their associated signal waveforms. See also FIG. 19 and ¶s 154-163 regarding data processor 11 which receives the signals measured from electrodes 151a and 151b and is also responsible for processing and interpreting these waveforms, as well as executing instructions based on its
PNG
media_image3.png
721
1270
media_image3.png
Greyscale
[AltContent: textbox (FIG. 11 of Komaki is annotated to highlight various areas.)]analysis.)
Regarding claim 6, modified Zakharov discloses the method according to claim 1.
Zakharov also discloses (see FIGs.1-2, ¶s 25-33) the method further comprising when the first condition is verified (i.e. when voluntary blinking is detected):
determining, by the controller (measurement unit 12 with processing unit 13), as a function of the blink patterns (“blinking parameters”) of the first electrostatic charge variation signal (“detection signal”) and for each pair of detected and consecutive clicks, a respective click time distance (“inter-blink interval”) between the clicks of the pair of clicks;
verifying, by the controller (measurement unit 12 with processing unit 13) and for each click time distance, whether the click time distance (“inter-blink interval”) is less than a click threshold distance. (See also ¶ 20, where Zakharov discusses how detection signals may be compared to “reference signals describing a threshold value…”.)
Komaki further supports:
when the click time distance is less than the click threshold distance, detecting a third activity of the first eye indicative of a double click of the first eye;
when the click time distance is not less than the click threshold distance, detecting a fourth activity of the first eye indicative of a single click of the first eye.
(See ¶s 142-143, describing different instances of users issuing instructions to data processor 11 to control external devices such as cameras. Different instructions may be given by performing, e.g., two or three blinks within one second, which establishes a (click) threshold (distance) of 0.5 or 0.333… seconds, respectively. See also FIG. 11, annotated above, where waveforms of signals produced by different types of eye activity are identified/detected, including those distinguished for one, two, and three blinks/clicks.)
Regarding claim 7, modified Zakharov discloses the method according to claim 1.
Zakharov further discloses (see FIG. 5, annotated below) wherein verifying the presence of the one or more blink patterns (“blinking parameters”) in the first electrostatic charge variation signal (“detection signal”) comprises identifying, for each blink pattern (“blinking parameter”), a first peak and a second peak in a blink period, the first peak and the second peak having signs opposite to each other with respect to a baseline of the first electrostatic charge variation signal (“detection signal”) and forming the blink pattern (“blinking parameter”)
PNG
media_image5.png
513
893
media_image5.png
Greyscale
[AltContent: textbox (FIG. 5 of Zakharov is annotated to highlight the peaks and baseline)]
Regarding claim 8, modified Zakharov discloses the method according to claim 1.
Zakharov further discloses (see ¶s 49-50, annotated FIG. 7 below) wherein detecting the blink comprises generating a blink signal with a first value indicative of the blink detection, and wherein detecting the click comprises generating the blink signal with a second value indicative of the click detection.
[AltContent: textbox (FIG. 7 of Zakharov is annotated to highlight various features)]
PNG
media_image7.png
736
1427
media_image7.png
Greyscale
Regarding claim 9, modified Zakharov discloses the method according to claim 1.
Zakharov further discloses wherein acquiring the first electrostatic charge variation signal (“detection signal”) comprises:
acquiring respective detection signals through the first and second electrodes (2 and 3), each detection signal being indicative of the electrostatic charge variations (i.e. varying electric potentials) on the respective first electrode (2) or the respective second electrode (3) (see again ¶ 32: “The detection signal may correspond to a time-dependent measurement… such as a time-dependent function of a voltage drop between two electrodes”); and
calculating a difference between the detection signals (“voltage drop”), and comprises at least one of the following:
filtering the difference (“voltage drop”) between the detection signals (see FIGs. 6-7 and ¶s 47-48 on the filtering of noise from Zakharov’s detection signals –corresponding to Applicant’s first electrostatic charge variation signal and plotting voltage drops as a function of time. The 50 Hz frequency component is removed); and
removing an offset of the difference (“voltage drop”) between the detection signals (see also ¶ 34 regarding offset unit 14 for offsetting detection signals).
Regarding claim 10, modified Zakharov discloses the method according to claim 1.
Komaki further supports (see FIG. 19, ¶s 156-159) wherein the glasses (eyewear 100) further comprise at least one accelerometer and at least one gyroscope coupled to the controller (data processor 11) (¶s 157-158: “the data processor [11] includes, for example, … sensor lie … The sensor lie includes a sensor group [11e] … Specifically, the sensor group [11e] includes an acceleration sensor …, gyroscope …”), and wherein the method further comprises:
acquiring, by the controller (data processor 11) and through the at least one gyroscope, one or more angular velocity signals indicative of respective angular velocities correlated to a movement of a user's head (to which eyewear 100 is mounted) (Examiner notes that gyroscopes, such as that included in the integral circuit of Zakharov’s data processor 11, are generally instruments which measure angular velocity);
acquiring, by the controller (data processor 11) and through the at least one accelerometer (“acceleration sensor which detects movement along three-axes (x, y, z)”), one or more linear acceleration signals indicative of respective linear accelerations correlated to the movement of the user's head (to which eyewear 100 is mounted);
verifying, by the controller (data processor 11) and as a function of the one or more angular velocity signals and of the one or more linear acceleration signals, whether the movement of the user's head is present (“The sensor lie includes a sensor group [11e] to detect position and/or direction of the eyewear 100”, eyewear 100 being mounted to the user’s head).
(See also ¶ 159 reciting the various components of data processor 11, including the sensor lie’s sensor group 11e as well as an ADC (analog-to-digital converter) for processing of sensor signals.)
Regarding claim 11, modified Zakharov discloses the method according to claim 1.
Zakharov further discloses wherein the first condition is verified (i.e. wherein voluntary blinking is detected) when the respective blink pattern (“blinking parameters”) has a time duration greater than a click threshold period. (See ¶ 55; the timing/duration of peaks detected in detection signals may be used to obtain blinking speeds and determine whether a blink is spontaneous (“fast movement”) or voluntary (“slower movement”), which establishes a click threshold period. See also ¶ 20, where Zakharov discusses how detection signals may be compared to “reference signals describing a threshold value…”.)
Regarding claim 13, modified Zakharov discloses the method according to claim 1.
Zakharov also discloses the method further comprising controlling, as a function of the detected activity of the first eye (eye 4), one or more functionalities of the glasses or of an apparatus operatively coupled to the glasses (see ¶ 56 describing how if activity associated with, e.g., dry eye or tiredness is detected, warning/trigger signals may be output).
Regarding claim 21, modified Zakharov discloses the method according to claim 1.
Hayashibara further discloses (see ¶s 33-37, FIG. 9(a,b)) wherein the first threshold period (LS,RS steps) is smaller than a time window period (e.g. of FIGs. 9(a,b)), in which the first and further first electrostatic charge variation signals are acquired.
Zakharov further discloses wherein the first threshold period is greater than a blink period corresponding to a duration of a blink pattern (“blinking parameters”). (See ¶ 55; the timing/duration of peaks detected in detection signals may be used to obtain blinking speeds and determine whether a blink is spontaneous (“fast movement”) or voluntary (“slower movement”), which establishes a click threshold period. See also ¶ 20, where Zakharov discusses how detection signals may be compared to “reference signals describing a threshold value…”.)
Claim 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Zakharov et al (EP 3760116 A1, hereinafter “Zakharov”) in view of Matthies et al (NPL entitled CapGlasses: Untethered Capacitive Sensing with Smart Glasses, hereinafter “Matthies”) and Komaki (US 20170150897).
Regarding claim 14, Zakharov discloses (see FIGs.1-2, ¶s 25-33) glasses comprising:
a frame (stem 19);
lenses (lens 10) attached to the frame (stem 19);
a first electrostatic charge variation sensor (eye blink sensor 1) including a first electrode (2) and a second electrode (3) spaced from each other, the first and the second electrodes (2 and 3) facing a first eye (eye 4) of a user, the first electrode (2) being configured to be electrostatically coupled to the first eye (eye 4) without physical contact with the user to detect respective electrostatic charge variations (i.e. varying electric potentials) generated by a blink of the first eye (eye 4); and
a controller (measurement unit 12 with processing unit 13) coupled to the first and second electrodes (2 and 3) and configured to:
contactlessly acquire, through the first and second electrodes (2 and 3) of the first electrostatic charge variation sensor (eye blink sensor 1), a first electrostatic charge variation signal (“detection signal”) indicative of a difference (“voltage drop”) between the electrostatic charge variations (i.e. varying electric potentials) detected by the first and the second electrodes (2 and 3) (see ¶ 32: “The measurement unit 12 is configured to measure an electrical voltage drop between first electrode 2 and the second electrode 3 and to generate a detection signal based on the voltage drop… such as a time-dependent function of a voltage drop”);
verify a presence of one or more blink patterns (“blinking parameters”) in the first electrostatic charge variation signal (“detection signal”), each blink pattern (“blinking parameter”) being indicative of a respective click or of a respective blink, the click being a voluntary blink of the first eye and the blink being an involuntary blink of the first eye (eye 4) (see ¶ 33: “The processing unit [13] is configured to determine a blinking parameter associated with the eye 4, based on the detection signal(s). The blinking parameter is one of, or statistics of, a blinking frequency, a blinking pattern… a detection of a voluntary blink, and a detection of a spontaneous [i.e. involuntary] blink”);
when the first electrostatic charge variation signal (“detection signal”) has the one or more blink patterns (“blinking parameters”), determine, for each blink pattern (“blinking parameters”), whether a first condition is verified;
when the first condition is not verified, detect a respective blink; and
when the first condition is verified, detect a respective click.
(Examiner notes that whether a “first condition” is verified or not simply corresponds to whether voluntary blinks (clicks) have been detected or not. Zakharov thus discloses steps iii-v above, as they separately disclose detection of both voluntary and incomplete/involuntary/spontaneous blinks in ¶ 33, as cited above. Zakharov further discusses how (electrodynamic) measurements and signals may be analyzed to discern between voluntary and involuntary blinks – see ¶s 47-55 and FIGs. 6-8.)
Zakharov does not disclose
the first and the second electrodes being fixed to the frame
the first and second electrodes being arranged on opposite ends of a support portion of the frame that supports a first lens of the lenses facing the first eye, and being aligned with each other along an axis joining a center of the first lens and a center of a second lens of the lenses so as to face opposite sides of a cornea of the first eye,
each of the first and second electrodes being configured to be electrostatically coupled to the first eye without physical contact with the user
Zakharov and Matthies commonly relate to electrooculography.
Matthies discloses (see FIGs. 2-4, secs. 3-4):
the first and the second electrodes being fixed to the frame (sec. 3.2: “Electrodes can be incorporated into glass frames”)
each of the first and second electrodes being configured to be electrostatically coupled (i.e. capacitively coupled) to the first eye without physical contact with the user (sec. 3.4 – “our interest to investigate a hardware configuration that can be truly mobile without being tethered to an external ground and thus would be hardware-wise self-contained”; sec. 4.1.2: “we do not need to attach a reference electrode”)
Zakharov and Komaki commonly relate to blink-detecting ocular devices.
Komaki, in combination with references already cited above, further supports (see FIG. 18, ¶s 144-153) the first and second electrodes (151(a,b)) being arranged on opposite ends of a support portion (right lens frame 101) of the frame that supports a first lens (right display 12R) of the lenses (12(R,L)) facing the first eye, so as to face opposite sides of a cornea of the first eye (¶ 152: “An EOG electrode 151a is arranged on the lens frame 101 […] Across the right display 12R, an EOG electrode 151b is provided with a position which is symmetrical with respect to the center of the right eye.”)
The combination of references cited above thus discloses the invention substantially as claimed, except for the first and second electrodes being aligned with each other along an axis joining a center of the first lens and a center of a second lens of the lenses. However, Examiner finds this limitation may only distinguish the claimed invention over the prior art by a mere rearrangement of parts – i.e. of the electrodes so as to have some horizontal alignment along an axis joining lens centers. It is noted also that Komaki cited above directly states that “EOG electrodes can be arranged with greater degree of freedom, and the position and the number of electrodes can be optimized” – ¶ 182.
It would have therefore been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine teachings of Zakharov and Matthies, in order to achieve untethered, self-contained capacitive sensing to overcome limitations of more traditional methods that are prone to interference, limited for mobile use, etc. (Matthies sec. 1, 3.3-3.34).
It would have also been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to further combine Zakharov with aspects of Komaki’s electrooculographic device/methods, in order to provide bilateral detection for both eyes and support detection of a broad range of ocular activity – including the tracking of user eye movement in addition to blinking activity (Komaki ¶s 41-43).
It would have then been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to further rearrange/realign electrodes in the modified Zakharov, in order to achieve some suitable accuracy/reliability for ocular action detection (Komaki ¶ 182) – since it has been held that rearranging parts of an invention involves only routine skill in the art. In re Japikse, 181 F.2d 1019, 86 USPQ 70 (CCPA 1950).
Regarding claim 15, modified Zakharov discloses the glasses according to claim 14.
Zakharov also discloses the glasses further comprising (see FIGs.1-2, FIG. 1 annotated above, and ¶s 25-33):
a second electrostatic charge variation sensor (eye blink sensor 1) including a third electrode and a fourth electrode (third electrodes 9, of which three are shown in the annotated FIG. 1 above) spaced from each other and with respect to the first and second electrodes (2 and 3), facing the first eye (eye 4) of the user and configured to detect respective electrostatic charge variations (i.e. varying electric potentials) generated by a movement of an eyeball of the first eye (eye 4), the first and second electrodes (2 and 3) of the first electrostatic charge variation sensor (eye blink sensor 1) being radially external with respect to the third and fourth electrodes (third electrodes 9) of the second electrostatic charge variation sensor (eye blink sensor 1) and with respect to a center of the first lens (lens 10),
wherein the controller (measurement unit 12 with processing unit 13) is further coupled to the third and fourth electrodes (third electrodes 9) of the second electrostatic charge variation sensor (eye blink sensor 1).
Komaki further supports (see FIGs. 1-2 and ¶s 38-43 detailing a base embodiment which overlaps heavily and shares much of the same features as that of FIG. 18 cited above) movement of an eyeball of the first eye in the absence of blink of the first eye. (Using similar electrooculographic principles as Zakharov, Komaki arranges electrodes in proximity to the user’s eyes in order to monitor eye activity by measuring induced electric potentials, this activity is not only limited to blinking as in Zakharov disclosure, however, but also extends to general eye movement including gaze direction. Komaki further discloses how signals generated by such general eye movements can be distinguished from signals caused by blinking)
Regarding claim 16, modified Zakharov discloses the glasses according to claim 15.
Zakharov also discloses the glasses further comprising (see FIGs.1-2, FIG. 1 annotated above, and ¶s 25-33):
a third electrostatic charge variation sensor (eye blink sensor 1) including a fifth electrode and a sixth electrode (third electrodes 9, of which three are shown in annotated FIG. 1 above) which are spaced from each other and with respect to the first (2), second (3), third and fourth electrodes (third electrodes 9), facing the first eye (eye 4) and configured to detect respective electrostatic charge variations (i.e. varying electric potentials) generated by the movement of the eyeball of the first eye (eye 4), the fifth and sixth electrodes (third electrodes 9) being radially internal with respect to the first and second electrodes (2 and 3) and with respect to the center of the first lens (lens 10),
wherein the controller (measurement unit 12 with processing unit 13) is further coupled to the fifth and sixth electrodes (third electrodes 9) of the third electrostatic charge variation sensor (eye blink sensor 1).
(Note that the three third electrodes 9 shown in annotated FIG. 1 are merely exemplary, and Zakharov more generally recites a plurality of third electrodes in vertical alignment/displacement with/from one another (¶ 29) which may admit additional third electrodes. Thus, besides the two outermost electrodes identified with Applicant’s third and fourth electrodes in annotated FIG. 1 above, those which may further be added and are located in a radially internal region are also identified and correspond to Applicant’s fifth and sixth electrodes.)
Komaki further supports:
movement of the eyeball of the first eye in the absence of blink of the first eye (as established above regarding claim 15, Komaki’s discloses detection of general eye movements, or gaze direction, in addition to blinking activity; see FIGs. 1-2 and ¶s 38-43)
the fifth (151a) and sixth electrodes (151b) being angularly rotated with respect to the third and fourth electrodes around the center of the first lens (see any of FIGs. 1-2 or 18, Komaki already provides electrodes 151(a,b) and 152(a,b) distributed around the periphery of each eye, as if angularly rotated about the eye center relative to some horizontal axis. See also any one of FIGs. 3-7 for other electrode arrangements).
Regarding claim 17, modified Zakharov discloses the glasses according to claim 14.
Zakharov further discloses (see FIGs.1-2, ¶s 25-33) the glasses comprising: a first electrostatic charge variation sensor (eye blink sensor 1) including a first electrode (2) and a second electrode (3) spaced from each other, facing a first eye (eye 4) of the user and configured to detect respective electrostatic charge variations (i.e. varying electric potentials) generated by a blink of the first eye (eye 4), the controller (measurement unit 12 with processing unit 13) being coupled to the first and second electrodes (2 and 3) of the first electrostatic charge variation sensor (eye blink sensor 1).
Komaki further supports:
a further first electrostatic charge variation sensor
a respective first electrode and a respective second electrode
a second eye of the user
(Komaki discloses ocular action/eye-motion detection apparatus accommodating both eyes, including symmetrized electrode arrangements and detection channels corresponding to the left and right eyes; see again FIGs. 1-2 and ¶s 38-43, FIG. 18 and ¶s 144-153.)
Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Zakharov et al (EP 3760116 A1, hereinafter “Zakharov”) in view of Matthies et al (NPL entitled CapGlasses: Untethered Capacitive Sensing with Smart Glasses, hereinafter “Matthies”) and The Linux Information Project (NPL entitled Buffer Definition, hereinafter “Linfo”).
Regarding claim 18, Zakharov discloses (see FIGs.1-2, ¶s 25-33) a non-transitory computer readable media (“program storage medium”) loadable into a controller (measurement unit 12 with processing unit 13, see ¶ 23) of glasses wearable by a user, wherein the glasses comprise a first electrostatic charge variation sensor (eye blink sensor 1) including a first electrode (2) and a second electrode (3) spaced from each other, facing a first eye (eye 4 )of the user, the first electrode (2) being configured to be electrostatically coupled to the first eye (eye 4) without physical contact with the user to detect respective electrostatic charge variations (i.e. varying electric potentials) generated by a blink of the first eye (eye 4), the controller (measurement unit 12 with processing unit 13) being coupled to the first and second electrodes (2 and 3) of the first electrostatic charge variation sensor (eye blink sensor 1), wherein the computer readable media includes instructions, when executed by the controller (measurement unit 12 with processing unit 13), that perform the following method:
contactlessly acquiring, by the first and second electrodes (2 and 3), a first electrostatic charge variation signal (“detection signal”) indicative of a difference (“voltage drop”) between the electrostatic charge variations (i.e. varying electric potentials) detected by the first and the second electrodes (2 and 3), the first electrostatic charge variation signal (“detection signal”) being acquired in a time window having a time window period greater than a duration of a blink pattern (see any of FIGs. 5-8);
obtaining, at the controller (measurement unit 12 with processing unit 13), the first electrostatic charge variation signal (“detection signal”) from the first electrostatic charge variation sensor (eye blink sensor 1);
(see ¶ 32: “The measurement unit 12 is configured to measure an electrical voltage drop between first electrode 2 and the second electrode 3 and to generate a detection signal based on the voltage drop… such as a time-dependent function of a voltage drop”)
verifying, by the controller (measurement unit 12 with processing unit 13), a presence of one or more blink patterns (“blinking parameters”) in the first electrostatic charge variation signal (“detection signal”) acquired in the time window, each blink pattern (“blinking parameters”) being indicative of a respective click or of a respective blink, the click being a voluntary blink of the first eye (eye 4) and the blink being an involuntary blink of the first eye (eye 4) (see ¶ 33: “The processing unit [13] is configured to determine a blinking parameter associated with the eye 4, based on the detection signal(s). The blinking parameter is one of, or statistics of, a blinking frequency, a blinking pattern… a detection of a voluntary blink, and a detection of a spontaneous [i.e. involuntary] blink”);
when the first electrostatic charge variation signal (“detection signal”) has the one or more blink patterns (“blinking parameters”), determining, by the controller (measurement unit 12 with processing unit 13) for each blink pattern (“blinking parameter”), whether a first condition is verified;
when the first condition is not verified, detecting, by the controller (measurement unit 12 with processing unit 13), a respective blink; and
when the first condition is verified, detecting, by the controller (measurement unit 12 with processing unit 13), a respective click.
Zakharov does not disclose:
a first electrode and a second electrode fixed to a frame of the glasses
each of the first and second electrodes being configured to be electrostatically coupled to the first eye without physical contact with the user
the first electrostatic charge variation signal being acquired iteratively through a scrolling buffer, wherein at each iteration the first electrostatic charge variation signal is acquired in a time window having a time window period greater than a duration of a blink pattern, an oldest sample of the first electrostatic charge variation signal is deleted from the scrolling buffer, and a new sample of the first electrostatic charge variation signal is stored in the scrolling buffer;
Zakharov and Matthies commonly relate to electrooculography.
Matthies discloses (see FIGs. 2-4, secs. 3-4):
a first electrode and a second electrode fixed to a frame of the glasses (sec. 3.2: “Electrodes can be incorporated into glass frames”)
each of the first and second electrodes being configured to be electrostatically coupled (i.e. capacitively coupled) to the first eye without physical contact with the user (sec. 3.4 – “our interest to investigate a hardware configuration that can be truly mobile without being tethered to an external ground and thus would be hardware-wise self-contained”; sec. 4.1.2: “we do not need to attach a reference electrode”)
Zakharov and Linfo commonly relate to the acquisition and handling of electronic data.
Linfo, in combination with references already cited above, further supports the first electrostatic charge variation signal being acquired iteratively through a scrolling buffer, wherein at each iteration the first electrostatic charge variation signal is acquired in a time window having a time window period greater than a duration of a blink pattern, an oldest sample of the first electrostatic charge variation signal is deleted from the scrolling buffer, and a new sample of the first electrostatic charge variation signal is stored in the scrolling buffer (Examiner notes the current limitation merely describes a basic and conventional memory management structure, commonly referred to as a circular or ring buffer, and applies it to the claimed first electrostatic charge variation signal. Examiner notes also that the basic approach is rather simply summarized by Linfo on pg. 1: “A ring buffer is a buffer of fixed size into which new data is placed back at its start when it is full, thereby overwriting the data that was first placed in it”)
It would have therefore been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine teachings of Zakharov and Matthies, in order to achieve untethered, self-contained capacitive sensing to overcome limitations of more traditional methods that are prone to interference, limited for mobile use, etc. (Matthies sec. 1, 3.3-3.34).
It would have also been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to implement a basic ring buffer as taught by Linfo, in order to reliably handle a continuous stream of data with only a fixed allocation of memory.
Regarding claim 19, modified Zakharov discloses the non-transitory computer readable media according to claim 18.
Zakharov also discloses wherein the instructions, when executed by the controller (measurement unit 12 with processing unit 13), perform the method that further comprises: when the first condition is not verified (i.e. when voluntary blinking is not detected):
determining, by the controller (measurement unit 12 with processing unit 13) and as a function of the one or more blink patterns (“blinking parameters”) of the first electrostatic charge variation signal (“detection signal”), an Inter-Eye blink interval (IEBI) parameter indicative of a blink frequency (refer again to ¶ 33: “The processing unit [13] is configured to determine a blinking parameter… The blinking parameter is one of, or statistics of, a blinking frequency, … an inter-blink interval, …”; thus Zakharov’s blinking parameter encompasses both blinking frequencies and inter-(eye )blink intervals. Examiner notes further that since frequencies and intervals (of time, i.e. periods) are simply inverse to one another, methods which employ either metric and differing only in this manner are equivalent to one another);
verifying, by the controller (measurement unit 12 with processing unit 13), whether the IEBI parameter (“blinking parameter”) is greater than an IEBI threshold value (“pre- blinking parameter”) (see ¶ 22: “The determined blinking parameter may be compared with predetermined blinking parameters…”; see also ¶ 56 for exemplary discussion on comparing determined and predetermined blinking frequencies (for spontaneous/involuntary blinks));
when the IEBI parameter (“blinking parameter”) is greater than the IEBI threshold value (“predetermined blinking parameter”), detecting a first activity of the first eye (eye 4), correlated to a lower attention level of the user; and
when the IEBI parameter (“blinking parameter”) is not greater than the IEBI threshold value (“predetermined blinking parameter”), detecting a second activity of the first eye (eye 4), correlated to a higher attention level of the user.
(Regarding steps C and D above, Examiner again refers to ¶ 56; here the blinking parameter is taken to be a blinking frequency. Zakharov discusses how frequencies may be determined and compared with other predetermined frequencies in order to detect activity associated with certain eye events/states, and that action may be taken based on such eye states (e.g. by outputting warning/trigger signals when detecting activity related to dry eye, tiredness, …). Note here that while Zakharov mentions a case of “low blinking frequency” – which establishes the general notion of threshold values – Zakharov also discusses, in ¶ 20, how detection signals may be compared to “reference signals describing a threshold value…”. Examiner lastly reemphasizes that, per ¶ 33, Zakharov’s blinking parameter broadly encompasses other metrics of eye activity (e.g. inter-blink intervals) which may be determined/detected/analyzed instead of blinking frequencies.)
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Zakharov in view of Matthies and Linfo, as applied to claim 18 above, and in further view of Komaki (US 20170150897).
Regarding claim 20, modified Zakharov discloses the non-transitory computer readable media according to claim 18.
Zakharov also discloses (see FIGs.1-2, ¶s 25-33) wherein the instructions, when executed by the controller, perform the method that further comprises: when the first condition is verified (i.e. when voluntary blinking is detected):
determining, by the controller (measurement unit 12 with processing unit 13), as a function of the blink patterns (“blinking parameters”) of the first electrostatic charge variation signal (“detection signal”) and for each pair of detected and consecutive clicks, a respective click time distance (“inter-blink interval”) between the clicks of the pair of clicks;
verifying, by the controller (measurement unit 12 with processing unit 13) and for each click time distance, whether the click time distance (“inter-blink interval”) is less than a click threshold distance. (See also ¶ 20, where Zakharov discusses how detection signals may be compared to “reference signals describing a threshold value…”.)
Modified Zakharov does not disclose:
when the click time distance is less than the click threshold distance, detecting a third activity of the first eye indicative of a double click of the first eye; and
when the click time distance is not less than the click threshold distance, detecting a fourth activity of the first eye indicative of a single click of the first eye.
Zakharov and Komaki commonly relate to blink-detecting ocular devices.
Komaki discloses:
when the click time distance is less than the click threshold distance, detecting a third activity of the first eye indicative of a double click of the first eye; and
when the click time distance is not less than the click threshold distance, detecting a fourth activity of the first eye indicative of a single click of the first eye.
(See ¶s 142-143, describing different instances of users issuing instructions to data processor 11 to control external devices such as cameras. Different instructions may be given by performing, e.g., two or three blinks within one second, which establishes a (click) threshold (distance) of 0.5 or 0.333… seconds, respectively. See also FIG. 11, annotated above, where waveforms of signals produced by different types of eye activity are identified/detected, including those distinguished for one, two, and three blinks/clicks.)
It would have also been obvious for one of ordinary skill in the art, before the effective filing date of the claimed invention, to further combine Zakharov with aspects of Komaki’s electrooculographic device/methods, in order to provide bilateral detection for both eyes and support detection of a broad range of ocular activity – including the tracking of user eye movement in addition to blinking activity (Komaki ¶s 41-43).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WAI-GA D. HO whose telephone number is (571)270-1624. The examiner can normally be reached Monday through Friday, 10AM - 6PM E.T..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephone Allen can be reached at (571) 272-2434. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/W.D.H./Examiner, Art Unit 2872
/STEPHONE B ALLEN/Supervisory Patent Examiner, Art Unit 2872
Prosecution Insights