Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on May 25, 2026 has been entered.
Response to Amendment
In response to amendments, filed May 25, 2026, claims 1, 10-15, and 26-27 have been amended. No claims have been cancelled or added. Claims 1-27 are pending.
Response to Arguments
Applicant’s arguments, see Remarks, filed May 25, 2026, with respect to the prior art rejections have been considered but are moot because the new ground of rejection does not rely on the same reference combination applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. A new ground(s) of rejection is made in view of the combinations of Publicover (US 20150338915 A1)/Rau (US 20160022193 A1)/Stack (US 20140154651 A1)/Palti-Wasserman (US 20110109879 A1)/Aimone (US 20160077547 A1)/ Sakamoto (US 8790280 B2)/Marci (US 20100004977 A1)/Najarian (US 20120123232 A1)/Proença (doi: 10.1109/TPAMI.2009.140.). Any arguments still relevant based on the new grounds of rejection are addressed below.
Regarding Applicant’s argument that Publicover and Rau individually fail to disclose “each task configured to elicit a predetermined expected cognitive and/or emotional response from the user; varying the visual stimuli configured to elicit the predetermined expected cognitive and/or emotional response from the user, wherein the visual stimuli are selected or designed based on the predetermined expected cognitive and/or emotional response,” Examiner respectfully disagrees, and additionally relies on new 35 USC 103 rejections based on the combination of the two references in place of the 35 USC 102 rejections with. Examiner wishes to highlight that one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Publicover describes in [0099] how the degree of pupil dilation and anti-saccadic movements, when combined with information about the real or virtual objects being viewed by a user, can be used to discern what objects elicit fright, attraction, or interest, and how such information can be used to tailor subsequent displays of information. This tailoring of subsequent displays of information based on expected user reaction of fright vs attraction vs interest is, therefore, selecting various visual stimuli based on predetermined expected cognitive and/or emotional responses to elicit predetermined expected cognitive and/or emotional response from the user. More explicit recitations of selecting stimuli to elicit predetermined expected responses can be found in Rau. Rau [0126] describes standardized clinical tests that elicit expected responses based on a patient's illness, Rau [0128] describes that “standardized clinical test questions are matched with the expected emotion for that question,” and Rau [0135] describes that the standardized tests use different types of stimuli, including visual, to as pertinent to the patient condition. More details are provided in the 35 USC 103 section below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-12, 17, 22-23, and 25-27 are rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1).
Regarding claim 1, Publicover teaches a method of discovering relationships between eye movements and cognitive and/or emotional responses of a user ([0099] tracked eye movements and geometry can be used to discern the physiological and/or emotional states of an individual in a continuous fashion), the method comprising the steps of: engaging the user in at least one task, each task comprising a visual stimuli via an electronic display ([0267] Actions may be taken by the interaction model and game when such graphics are viewed, not viewed, or viewed in sequence. Points may be awarded or game play altered based upon defined gaze activity; [0293] The system may comprise a display device that is viewable by the user.).
While one may interpret Publicover [0099] to disclose selecting stimulate to elicit predetermined expected responses through tailoring subsequent displays of information based on the indication that particular objects elicit fright, attraction, or interest, Rau is introduced for more explicit recitation of eliciting a predetermined expected response. Rau teaches systems and methods for real-time measurement of objective, autonomic physiological parameters that allow for monitoring of mental health illnesses and emotional health changes.
Rau discloses and each task configured to elicit a predetermined expected cognitive and/or emotional response from the user ([0125] Types of Stimuli: [0126] Questions in the standard clinical tests and respective patient responses to those questions are grouped into various categories. Clinicians administering and interpreting these tests use these categories, compare with anticipated or expected responses for standardized clinical tests based on a patient's illness and combine the information of significant biometric changes and thresholds provided by this system through the real-time data analytics. [0127] Emotions Grouping: [0128] six primary emotions (anger, happiness, surprise, disgust, sadness, and fear as described by Paul Ekman) are classified, and further divided into secondary and tertiary emotions (as described in Shaver et al, 2001), if the clinician feels the need. Standardized clinical test questions are matched with the expected emotion for that question.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Publicover to include eliciting a predetermined response from a user as disclosed in Rau to compare the patient's various autonomic physiological parameters with that of the patient's previous sessions and other patients with comparable illness conditions to improve diagnosis, treatment plan design, and monitoring (Rau [0136, 0139]).
The combination of Publicover/Rau discloses:
varying the visual stimuli configured to elicit the predetermined expected cognitive and/or emotional response from the user, wherein the visual stimuli are selected or designed based on the predetermined expected cognitive and/or emotional response (Publicover: [0099] the degree of pupil dilation can be used to discern emotional states such as fear, interest, or cognitive load. Anti-saccadic movements can be an indication of viewing something distasteful. When combined with information about the real or virtual objects being viewed by a user, an indication of the classes of objects that, for example, elicit fright, attraction, or interest can be discerned. Such information can, for example, be used to tailor subsequent displays of information. Rau: [0135] Different types of stimuli 1002 can be given to the patient and processed through the different sensory organs of the patient 1004 such as visual, aural with patient oral responses, kinesthetic interactions etc… Standardized Stimuli are defined hereafter as the questions from standardized psychometric tests, physical tasks to test dexterities, cognition etc. and selected by the clinicians pertinent to patient condition.");
providing a camera filming at least one eye of the user (Publicover: [0019] one or more cameras or at least one sensor to monitor changes in the reflection of light … project reference light onto one or both eyes);
recording a first time series of eye movements by the user with the camera (Publicover: [0221] Using eye position data determined from a series of eye images, algorithmic “filters” can be constructed to identify and distinguish, in real-time, the presence of a saccadic or smooth pursuit eye movements; [0583] The use of deep learning approaches to eye signals can be classified as a “time series” data analysis. In other words, eye signals can be recognized from a series of (X, Y) positions of one or both eyes measured over time.);
recording each task corresponding to the first time series of eye movements by the user; wherein the first time series of eye movements and the task are taken at the same time; sending the first time series of eye movements and the task to a computing device; comparing, by the computing device, the eye movements from the first time series and the task (Publicover: [0477] FIG. 16 shows an overall sequence of algorithmic steps used to detect eye signals that may lead up to the performance of an action. Images of an eye are acquired and analyzed at 1630 to determine gaze pathways. If an eye cannot be found because it is covered by an eyelid or otherwise obscured at 1631, timing registers are simply updated and analysis proceeds to the next camera frame. Filters at 1632 are applied to all series of eye positions to determine whether primarily saccadic or smooth pursuit [or vergence or vestibulo-ocular eye movements are present. If matches to these forms of voluntary eye movement are found, then gaze pathways are further compared with screen positions at 1633 to determine if regions corresponding to interactables or real objects are being viewed. Gaze patterns are then further analyzed to determine if they generally correspond to activation interactables displayed on the screen at 1634. If a match is found, then actions corresponding to the selected interactable[s] are performed at 1635.);
and identifying, by the computing device, at least one relationship between eye movements that correlate to the actual specific cognitive and/or emotional response (Publicover: [0618] The output of the process can be a null indicating “no action” or a set of one or more intents and/or conditions 2739 of the device wearer. Intents can, for example, include the activation of an intended action [i.e., a binary classification]. Simultaneously determined conditions can include a user state [i.e., generally classified over a continuous range] such as cognitive load or a degree of fatigue; Fig. 27; Rau: [0136] The patient's various autonomic physiological parameters [APPs] will respond and react to the induced stimuli. … Some of the APPs captured by the biometric devices include … eye movements… These biometric parameters can capture changes and severity of individual patient's feelings, emotions 1014 and theft innate resiliencies, coping skills, [healthy] behavior, function and responses 1012. From the biometric devices, the data sets 1020 created will be provided in near real time to the clinician that will be compared to a master database of the patient's previous sessions and other patients with comparable illness conditions; data analytics 7004; [0131] The present invention can be used to diagnose and treat many mental health disorders and illnesses, such as, but not limited to: anxiety [e.g., generalized anxiety disorder, panic disorders, phobias, obsessive-compulsive disorders [OCD], post-traumatic stress disorder [PTSD]; attention deficient disorder [ADD], and attention deficit hyperactivity disorder [ADHD]; depressive disorder [e.g., dysthymia, depression in the elderly, postpartum depression]; stress or mild depression).
Regarding claim 2, the combination of Publicover/Rau discloses the method of claim 1, wherein the camera is physically attached to the user (Publicover: [0020] one or more illumination sources, cameras, or other sensors disposed onto, or embedded within one or more portable devices [phone, tablet, web camera, laptop, camera, video camera, nomadic device, electronic accessory components etc.] or wearable devices [e.g., glasses, watch, hat, jewelry, clothing, personal accessories, etc.]).
Regarding claim 3, the combination of Publicover/Rau discloses the method of claim 2, wherein the camera is a pair of eyeglasses worn by the user (Publicover: [0020] one or more illumination sources, cameras, or other sensors disposed onto, or embedded within one or more portable devices [phone, tablet, web camera, laptop, camera, video camera, nomadic device, electronic accessory components etc.] or wearable devices [e.g., glasses, watch, hat, jewelry, clothing, personal accessories, etc.]).
Regarding claim 4, the combination of Publicover/Rau discloses the method of claim 1, wherein the camera is not physically attached to the user (Publicover: [0020] one or more illumination sources, cameras, or other sensors disposed onto, or embedded within one or more portable devices [phone, tablet, web camera, laptop, camera, video camera, nomadic device, electronic accessory components etc.]).
Regarding claim 5, the combination of Publicover/Rau discloses the method of claim 4, wherein the electronic display and the camera are part of a smartphone or a tablet (Publicover: [0020] one or more illumination sources, cameras, or other sensors disposed onto, or embedded within one or more portable devices [phone, tablet, web camera, laptop, camera, video camera, nomadic device, electronic accessory components etc]; display 530).
Regarding claim 6, the combination of Publicover/Rau discloses the method of claim 4, wherein the computing device comprises a smartphone, a tablet, a laptop computer or a desktop computer, wherein the computing device comprises the electronic display and the camera (Publicover: [0020] one or more illumination sources, cameras, or other sensors disposed onto, or embedded within one or more portable devices [phone, tablet, web camera, laptop, camera, video camera, nomadic device, electronic accessory components etc.]; display 530).
Regarding claim 7, the combination of Publicover/Rau discloses the method of claim 1, wherein the eye movements comprise:
X gaze location, Y gaze location (Publicover: [0554] FIG. 20 illustrates the time at which a new object is introduced via a step change in opacity during a saccade. The upper traces represent measured X [i.e., horizontal] 2030 and Y [i.e., vertical] 2031 gaze locations. When the initiation of a saccadic eye movement is detected at 2032, opacity is changed to a desired level during the time of the saccade at 2033),
saccade rate, saccade peak velocity, fixation duration, fixation entropy, gaze deviation of polar angle, gaze deviation of eccentricity, re-fixations, smooth pursuits and/or scan path (Publicover: [0054] FIG. 3 is a flowchart illustrating the classification of saccades, micro-saccades, smooth pursuit eye movements, and fixations; [0227] If the eye velocity is greater than a minimum threshold for saccadic movements [typically 30° per second] at 334, then the system signifies that some form of saccadic movement has occurred at 335. If the saccadic movement occurred over a distance that is generally within the foveal view [i.e., within approximately 1° to 3°] at 335, then the eye movement is recorded as a micro-saccade at 336. If, on the other hand, the angular distance traveled by the eye is greater than this range at 335, the event is registered as a saccade at 337; [0228].).
Regarding claim 8, the combination of Publicover/Rau discloses the method of claim 1, wherein the eye movements comprise a change in the pupillary system which includes pupil diameter, velocity of the change in the pupil diameter, acceleration of the change in the pupil diameter, constriction latency, dilation duration, spectral features and/or iris muscle features (Publicover: [0038] tracked eye movements and geometries [e.g. pupil dilation, anti-saccades] may be combined with information about the real or virtual objects being viewed by a user to discern the physiological and/or emotional states of an individual in a substantially continuous fashion – [one of ordinary skill in the art with the continuous pupil geometry data can determine the velocity/acceleration/latency/duration of the dilation and constriction to discern the physiological and/or emotional states of an individual in a substantially continuous fashion]; [0366] The positions of features on the eye, such as the iris, sclera or even pupil viewed edge-on are tracked).
Regarding claim 9, the combination of Publicover/Rau discloses the method of claim 1, wherein the eye movements comprise a change in blinking which includes blink rate, blink duration, blink latency, partial blinks, blink entropy and/or squinting (Publicover: [0015] Added to this, viewing of the eye is often obscured during normal function by eyelids and lashes. Furthermore, eye blinks in which the eyelid completely blocks viewing of the position of the eye must occur periodically for sustained function in order to maintain lubrication and the aqueous environment of the surface of the eye. Blink durations [normally lasting from 0.3 to 0.4 seconds] and velocities can be affected by fatigue, attentions, injury, medications, drugs, alcohol, and disease; [0238] prolonged eye blinks to discern intern… Blinks take even longer periods of time, requiring a complex series of muscle contractions. The minimum time for a blink is about 0.3 to 0.4 seconds; [0592] For example, movement of the eye lids and/or eye lashes can be used to anticipate that a blink is about to occur. As a blink is initiated, the system can anticipate that the user will be functionally blind for the duration of a blink).
Regarding claim 10, the combination of Publicover/Rau discloses the method of claim 1, wherein the task comprises a task configured to deliver a large, unexpected reward or penalty (Publicover: [0267] Actions may be taken by the interaction model and game when such graphics are viewed, not viewed, or viewed in sequence. Points may be awarded or game play altered based upon defined gaze activity.), wherein the predetermined expected cognitive and/or emotional response comprises surprise (Publicover: [0214] eye signals may be combined with other input modalities to control device actions. These modalities may include head movements such as shakes, tilts, or leans [e.g., indicating “yes,” “no,” interest, surprise, reflection]; Rau: [0127] Emotions Grouping: [0128] six primary emotions (anger, happiness, surprise, disgust, sadness, and fear as described by Paul Ekman) are classified, and further divided into secondary and tertiary emotions (as described in Shaver et al, 2001), if the clinician feels the need. Standardized clinical test questions are matched with the expected emotion for that question.).
Regarding claim 11, the combination of Publicover/Rau discloses the method of claim 1, wherein the task comprises a task configured to alternate between highly focused attention or carefree distributed attention (Publicover: [0380] Switching or weighting different algorithms can be based upon environmental conditions (e.g. lighting) or even physiological factors such as cognitive load; [0536] steps can be taken to control whether objects that are introduced or modified within a display are presented in a fashion to either: 1] attract attention or 2] avoid distraction.), wherein the predetermined expected cognitive and/or emotional response comprises vigilance (Publicover: [0099] classes of objects that, for example, elicit fright, attraction, or interest; [0245] By recognizing stereotypic eye movements during the reading process, rates of reading, any text that might have been skipped and/or conversely, text that attracted prolonged attention can also be identified. The number of regressions, sub-vocalizations [using additional sensors], saccade distances, and fixation times can be used as indicators of both interest in and complexity of materials; Rau: [0128] Standardized clinical test questions are matched with the expected emotion for that question.).
Regarding claim 12, the combination of Publicover/Rau discloses the method of claim 1. While Publicover fails to specifically disclose a task configured to randomly disable a mouse click response or a screen touch response when the user was interacting with the display screen, computer malfunctions are well known for causing frustration and one of ordinary skill in the art would know how to incorporate such a task. Publicover discloses wherein the predetermined expected cognitive and/or emotional response comprises frustration (Publicover: [0099] an indication of the classes of objects that, for example, elicit fright, attraction, or interest can be discerned. Such information can, for example, be used to tailor subsequent displays of information; [0244] intentional eye-movements [rolling of eyes in frustration]; Rau: [0128] Standardized clinical test questions are matched with the expected emotion for that question.).
Regarding claim 17, the combination of Publicover/Rau discloses the method of claim 1, wherein the task comprises a computer game utilizing a computer mouse, joystick, keyboard and/or touch screen (Publicover: [0214] In other embodiments, eye signals may be combined with other input modalities to control device actions. These modalities may include … traditional computer input devices such as keyboards, mice, and touch screens; [0267] In a gaming environment).
Regarding claim 22, the combination of Publicover/Rau discloses the method of claim 1, wherein the first camera is an infrared camera (Rau: [0143] These systems have pairs of video [visible spectrum] and thermal infrared [IR] video camera subsystems).
Regarding claim 23, the combination of Publicover/Rau discloses the method of claim 1, wherein the first camera is a full-color camera (Rau: [0143] These systems have pairs of video [visible spectrum] and thermal infrared [IR] video camera subsystems).
Regarding claim 25, the combination of Publicover/Rau discloses the method of claim 1, wherein the first camera is both an infrared camera and a full-color camera (Rau: [0143] These systems have pairs of video [visible spectrum] and thermal infrared [IR] video camera subsystems).
Regarding claim 26, Publicover teaches a method of discovering relationships between eye movements and cognitive and/or emotional responses of a user ([0099] tracked eye movements and geometry can be used to discern the physiological and/or emotional states of an individual in a continuous fashion), the method comprising the steps of: engaging the user in at least one task, each task comprising a visual stimuli via an electronic display ([0267] Actions may be taken by the interaction model and game when such graphics are viewed, not viewed, or viewed in sequence. Points may be awarded or game play altered based upon defined gaze activity; [0293] The system may comprise a display device that is viewable by the user.).
While one may interpret Publicover [0099] to disclose selecting stimulate to elicit predetermined expected responses through tailoring subsequent displays of information based on the indication that particular objects elicit fright, attraction, or interest, Rau is introduced for more explicit recitation of eliciting a predetermined expected response.
Rau discloses and each task configured to elicit a predetermined expected cognitive and/or emotional response from the user ([0125] Types of Stimuli: [0126] Questions in the standard clinical tests and respective patient responses to those questions are grouped into various categories. Clinicians administering and interpreting these tests use these categories, compare with anticipated or expected responses for standardized clinical tests based on a patient's illness and combine the information of significant biometric changes and thresholds provided by this system through the real-time data analytics. [0127] Emotions Grouping: [0128] six primary emotions (anger, happiness, surprise, disgust, sadness, and fear as described by Paul Ekman) are classified, and further divided into secondary and tertiary emotions (as described in Shaver et al, 2001), if the clinician feels the need. Standardized clinical test questions are matched with the expected emotion for that question.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Publicover to include eliciting a predetermined response from a user as disclosed in Rau to compare the patient's various autonomic physiological parameters with that of the patient's previous sessions and other patients with comparable illness conditions to improve diagnosis, treatment plan design, and monitoring (Rau [0136, 0139]).
The combination of Publicover/Rau discloses:
varying the visual stimuli configured to elicit the predetermined expected cognitive and/or emotional response from the user, wherein the visual stimuli are selected or designed based on the predetermined expected cognitive and/or emotional response (Publicover: [0099] the degree of pupil dilation can be used to discern emotional states such as fear, interest, or cognitive load. Anti-saccadic movements can be an indication of viewing something distasteful. When combined with information about the real or virtual objects being viewed by a user, an indication of the classes of objects that, for example, elicit fright, attraction, or interest can be discerned. Such information can, for example, be used to tailor subsequent displays of information. Rau: [0135] Different types of stimuli 1002 can be given to the patient and processed through the different sensory organs of the patient 1004 such as visual, aural with patient oral responses, kinesthetic interactions etc… Standardized Stimuli are defined hereafter as the questions from standardized psychometric tests, physical tasks to test dexterities, cognition etc. and selected by the clinicians pertinent to patient condition.");
providing a camera filming at least one eye of the user (Publicover: [0019] one or more cameras or at least one sensor to monitor changes in the reflection of light … project reference light onto one or both eyes);
recording a first time series of eye movements by the user with the camera (Publicover: [0221] Using eye position data determined from a series of eye images, algorithmic “filters” can be constructed to identify and distinguish, in real-time, the presence of a saccadic or smooth pursuit eye movements; [0583] The use of deep learning approaches to eye signals can be classified as a “time series” data analysis. In other words, eye signals can be recognized from a series of (X, Y) positions of one or both eyes measured over time.);
recording each task corresponding to the first time series of eye movements by the user; wherein the first time series of eye movements and the task are taken at the same time; sending the first time series of eye movements and the task to a computing device; comparing, by the computing device, the eye movements from the first time series and the task (Publicover: [0477] FIG. 16 shows an overall sequence of algorithmic steps used to detect eye signals that may lead up to the performance of an action. Images of an eye are acquired and analyzed at 1630 to determine gaze pathways. If an eye cannot be found because it is covered by an eyelid or otherwise obscured at 1631, timing registers are simply updated and analysis proceeds to the next camera frame. Filters at 1632 are applied to all series of eye positions to determine whether primarily saccadic or smooth pursuit [or vergence or vestibulo-ocular eye movements are present. If matches to these forms of voluntary eye movement are found, then gaze pathways are further compared with screen positions at 1633 to determine if regions corresponding to interactables or real objects are being viewed. Gaze patterns are then further analyzed to determine if they generally correspond to activation interactables displayed on the screen at 1634. If a match is found, then actions corresponding to the selected interactable[s] are performed at 1635.); and
identifying, by the computing device, at least one relationship between eye movements that correlate to a diagnosis of a mental health condition (Publicover: [0618] The output of the process can be a null indicating “no action” or a set of one or more intents and/or conditions 2739 of the device wearer. Intents can, for example, include the activation of an intended action [i.e., a binary classification]. Simultaneously determined conditions can include a user state [i.e., generally classified over a continuous range] such as cognitive load or a degree of fatigue; Rau: [0136] The patient's various autonomic physiological parameters [APPs] will respond and react to the induced stimuli. By utilizing biometric devices 1018, these APPs can be detected with far more precision and accuracy than is feasible through a clinician's simultaneous visual observations. Some of the APPs captured by the biometric devices include … eye movements, facial changes including color & texture changes, posture changes, muscle movements [voluntary and involuntary], and speech and tonal changes, as applicable. These biometric parameters can capture changes and severity of individual patient's feelings, emotions 1014 and theft innate resiliencies, coping skills, [healthy] behavior, function and responses 1012. From the biometric devices, the data sets 1020 created will be provided in near real time to the clinician that will be compared to a master database of the patient's previous sessions and other patients with comparable illness conditions; data analytics 7004; [0131] The present invention can be used to diagnose and treat many mental health disorders and illnesses, such as, but not limited to: anxiety [e.g., generalized anxiety disorder, panic disorders, phobias, obsessive-compulsive disorders [OCD], post-traumatic stress disorder [PTSD]; attention deficient disorder [ADD], and attention deficit hyperactivity disorder [ADHD]]; depressive disorder [e.g., dysthymia, depression in the elderly, postpartum depression]; stress or mild depression).
Regarding claim 27, Publicover teaches a method of discovering relationships between eye movements and cognitive and/or emotional responses of a user ([0099] tracked eye movements and geometry can be used to discern the physiological and/or emotional states of an individual in a continuous fashion), the method comprising the steps of: engaging the user in at least one task, each task comprising a visual stimuli via an electronic display ([0267] Actions may be taken by the interaction model and game when such graphics are viewed, not viewed, or viewed in sequence. Points may be awarded or game play altered based upon defined gaze activity; [0293] The system may comprise a display device that is viewable by the user.).
While one may interpret Publicover [0099] to disclose selecting stimulate to elicit predetermined expected responses through tailoring subsequent displays of information based on the indication that particular objects elicit fright, attraction, or interest, Rau is introduced for more explicit recitation of eliciting a predetermined expected response.
Rau discloses and each task configured to elicit a predetermined expected cognitive and/or emotional response from the user ([0125] Types of Stimuli: [0126] Questions in the standard clinical tests and respective patient responses to those questions are grouped into various categories. Clinicians administering and interpreting these tests use these categories, compare with anticipated or expected responses for standardized clinical tests based on a patient's illness and combine the information of significant biometric changes and thresholds provided by this system through the real-time data analytics. [0127] Emotions Grouping: [0128] six primary emotions (anger, happiness, surprise, disgust, sadness, and fear as described by Paul Ekman) are classified, and further divided into secondary and tertiary emotions (as described in Shaver et al, 2001), if the clinician feels the need. Standardized clinical test questions are matched with the expected emotion for that question.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Publicover to include eliciting a predetermined response from a user as disclosed in Rau to compare the patient's various autonomic physiological parameters with that of the patient's previous sessions and other patients with comparable illness conditions to improve diagnosis, treatment plan design, and monitoring (Rau [0136, 0139]).
The combination of Publicover/Rau discloses:
varying the visual stimuli configured to elicit the predetermined expected cognitive and/or emotional response from the user; wherein the visual stimuli are selected or designed based on the predetermined expected cognitive and/or emotional response (Publicover: [0099] the degree of pupil dilation can be used to discern emotional states such as fear, interest, or cognitive load. Anti-saccadic movements can be an indication of viewing something distasteful. When combined with information about the real or virtual objects being viewed by a user, an indication of the classes of objects that, for example, elicit fright, attraction, or interest can be discerned. Such information can, for example, be used to tailor subsequent displays of information. Rau: [0135] Different types of stimuli 1002 can be given to the patient and processed through the different sensory organs of the patient 1004 such as visual, aural with patient oral responses, kinesthetic interactions etc… Standardized Stimuli are defined hereafter as the questions from standardized psychometric tests, physical tasks to test dexterities, cognition etc. and selected by the clinicians pertinent to patient condition.");
providing a camera filming at least one eye of the user (Publicover: [0019] one or more cameras or at least one sensor to monitor changes in the reflection of light … project reference light onto one or both eyes);
recording a first time series of eye movements by the user with the camera (Publicover: [0221] Using eye position data determined from a series of eye images, algorithmic “filters” can be constructed to identify and distinguish, in real-time, the presence of a saccadic or smooth pursuit eye movements; [0583] The use of deep learning approaches to eye signals can be classified as a “time series” data analysis. In other words, eye signals can be recognized from a series of (X, Y) positions of one or both eyes measured over time.);
recording each task corresponding to the first time series of eye movements by the user; wherein the first time series of eye movements and the task are taken at the same time; sending the first time series of eye movements and the task to a computing device; comparing, by the computing device, the eye movements from the first time series and the task (Publicover: [0477] FIG. 16 shows an overall sequence of algorithmic steps used to detect eye signals that may lead up to the performance of an action. Images of an eye are acquired and analyzed at 1630 to determine gaze pathways. If an eye cannot be found because it is covered by an eyelid or otherwise obscured at 1631, timing registers are simply updated and analysis proceeds to the next camera frame. Filters at 1632 are applied to all series of eye positions to determine whether primarily saccadic or smooth pursuit [or vergence or vestibulo-ocular eye movements are present. If matches to these forms of voluntary eye movement are found, then gaze pathways are further compared with screen positions at 1633 to determine if regions corresponding to interactables or real objects are being viewed. Gaze patterns are then further analyzed to determine if they generally correspond to activation interactables displayed on the screen at 1634. If a match is found, then actions corresponding to the selected interactable[s] are performed at 1635.); and
identifying, by the computing device, at least one relationship between eye movements that correlate to a measurement of a sympathetic nervous system of the user (Publicover: [0618] Simultaneously determined conditions can include a user state [i.e., generally classified over a continuous range] such as cognitive load or a degree of fatigue; Rau: [0136] The patient's various autonomic physiological parameters [APPs] will respond and react to the induced stimuli. By utilizing biometric devices 1018, these APPs can be detected with far more precision and accuracy than is feasible through a clinician's simultaneous visual observations. Some of the APPs captured by the biometric devices include … eye movements, facial changes including color & texture changes, posture changes, muscle movements [voluntary and involuntary], and speech and tonal changes, as applicable. These biometric parameters can capture changes and severity of individual patient's feelings, emotions 1014 and theft innate resiliencies, coping skills, [healthy] behavior, function and responses 1012; data analytics 7004; [0111] FIG. 1 is a block diagram depicting the relationship between central nervous system [CNS] stimulus and the biometric reactions; [0122] The present invention links the types and degrees of intensity of stimuli [e.g., discriminating, eliciting, emotional, reinforcement, nominal, functional, or pseudo-reflex] to the nervous system's reactions, validates the diagnosis and patient progress via evaluation by experienced and trained clinicians).
Claim(s) 13 are rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1), and in further view of Stack (US 20140154651 A1).
Regarding claim 13, the combination of Publicover/Rau discloses the method of claim 1. However, the combination of Publicover/Rau fails to disclose varying difficulty.
Stack teaches a method to determine the subject's peak cognitive performance using smooth pursuit tracking tests with varying difficulty. Stack discloses wherein the task comprises a task configured to vary the difficulty of puzzle between easy and hard ([0050] the test taker is slowly pushed to difficulty levels higher incrementally in order to get further confirmation that the difficulty level at this stage is really a representation of the most difficult stage level the patient can endure).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to include cognitive load corresponding to varying difficulty of a task as disclosed in Stack to evaluate cognitive abilities without overloading the patient (Stack [0054]).
The combination of Publicover/Rau/Stack discloses wherein the predetermined expected cognitive and/or emotional response comprises a corresponding low to high degree of cognitive load (Stack: [0054] this test is an ideal type of cognitive test because the upper bound cognitive load adapts in a relatively dynamic and variable manner to the upper bound of the cognitive load of a patient. Rau: [0126] Clinicians administering and interpreting these tests use these categories, compare with anticipated or expected responses for standardized clinical tests based on a patient's illness and combine the information of significant biometric changes and thresholds provided by this system through the real-time data analytics.).
Claim(s) 14 are rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1), and in further view of Palti-Wasserman (US 20110109879 A1).
Regarding claim 14, the combination of Publicover/Rau discloses the method of claim 1. However, the combination of Publicover/Rau fails to disclose an opponent.
Palti-Wasserman teaches a system for profiling a personal aspect of a subject based on eye response to visual stimuli. Palti-Wasserman discloses wherein the task comprises a task configured to change an opponent condition in a subsequent task, wherein the predetermined expected cognitive and/or emotional response comprises anxiety ([0019] A subject's personal aspects include many things such as but not limited to: state of mind, level of stress, intensions, anxiety; [0028] A different application may be used in the electronic gaming industry. A player's profile may be prepared and used for the players benefit, or for his opponent to see. For example by calculating and displaying a player's stress level to his opponents, the game becomes more interesting and challenging. Rau: [0126] Clinicians administering and interpreting these tests use these categories, compare with anticipated or expected responses for standardized clinical tests based on a patient's illness and combine the information of significant biometric changes and thresholds provided by this system through the real-time data analytics).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to include the anxiety elicited by an opponent as disclosed in Palti-Wasserman because by calculating and displaying a player's stress level to his opponents, the game becomes more interesting and challenging (Palti-Wasserman [0028]).
Claim(s) 15 are rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1), and in further view of Aimone (US 20160077547 A1).
Regarding claim 15, the combination of Publicover/Rau discloses the method of claim 1. However, the combination of Publicover/Rau fails to disclose an attack.
Aimone teaches a training apparatus has an input device and a wearable computing device with a bio-signal sensor and a display to provide an interactive virtual reality (“VR”) environment for a user in which the user interactions and bio-signal data are scored with a user state score and a performance scored. Aimone discloses wherein the task comprises a task configured to change the level of attack on the user, wherein the predetermined expected cognitive and/or emotional response comprises stress ([0159] Dan wants to increase his performance in these games so that he can perform well under high levels of stress. The VR game presents a series of tests that put Dan under simulated stress events—i.e. a sniper attack, etc. The EEG sensors in Dan's VR headset asses his emotional/stress state during these tests and provide him with updates based on his performance. Dan's performance during these “stress tests” can unlock different achievements and new missions. Rau: [0126] Clinicians administering and interpreting these tests use these categories, compare with anticipated or expected responses for standardized clinical tests based on a patient's illness and combine the information of significant biometric changes and thresholds provided by this system through the real-time data analytics).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to include the stress elicited by an attack as disclosed in Aimone because these “stress tests” can provide a valuable metric for gauging how a subject will perform in real-life situations that are stressful (Aimone [0159]).
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1), and in further view of Sakamoto (US 8790280 B2).
Regarding claim 16, the combination of Publicover/Rau discloses the method of claim 1. However, the combination of Publicover/Rau fails to disclose details for camera placement beyond facing the user.
Sakamoto teaches a human state estimating device that uses eye movement. Sakamoto discloses wherein the camera is disposed at or between +45 degrees to -45 degrees in relation to a sagittal plane of the user and at or between +20 degrees to -45 degrees in relation to the transverse plane of the user (Col 15, lines 12-23, FIG. 12 is a configuration diagram of a measuring device used by the inventors for accurately measuring small eye movement among the eyeball movements. In FIG. 12, [a] is a lateral view and [b] is a bird's-eye view. This measuring device corresponds to another embodiment of the video obtaining unit 11 and the analyzing unit 12 of the human state estimating device 10 in the present invention. The optical axis of a high-speed camera 88 equipped with a high-magnification lens 33 is set on an equatorial plane. The angle created by the optical axis of the high-speed camera 38 and the visual axis when the eyeball looks to the front is 45 degrees as shown in FIG. 13).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to include camera position as disclosed in Sakamoto to accurately measuring small eye movement among the eyeball movements (Sakamoto, Col 15, lines 12-13).
Claim(s) 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1), and in further view of Marci (US 20100004977 A1).
Regarding claim 18, the combination of Publicover/Rau discloses the method of claim 1. However, the combination of Publicover/Rau fails to disclose a set period for the task.
Marci teaches a method and system for measuring the biometric (physically, behaviorally, biologically and self-report based) responses of an audience to a presentation or interactive experience. Marci discloses wherein the task comprises a set time period ([0168] an event window begins when a user is presented with a screen display which involves the user in an interactive presentation, task or activity and extends for a duration of five [or in some cases, up to ten] seconds).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to include set period for the task as disclosed in Marci to collect the eye tracking, behavior and biometric response data on the 20 ms intervals, providing up to 500 data points from each sensor for the event window, which is preferable for processing (Marci [0167 and 0168]).
Regarding claim 19, the combination of Publicover/Rau discloses the method of claim 1. However, the combination of Publicover/Rau fails to disclose a set period for the task.
Marci discloses wherein the task comprises a set time period of 10 seconds ([0168] an event window begins when a user is presented with a screen display which involves the user in an interactive presentation, task or activity and extends for a duration of five [or in some cases, up to ten] seconds).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to include a period of 10 seconds for the task as disclosed in Marci to collect the eye tracking, behavior and biometric response data on the 20 ms intervals, providing up to 500 data points from each sensor for the event window, which is preferable for processing (Marci [0167 and 0168]).
Claim(s) 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1), and in further view of Najarian (US 20120123232 A1).
Regarding claim 20, the combination of Publicover/Rau discloses the method of claim 1. However, the combination of Publicover/Rau fails to disclose linear regression. Najarian teaches an apparatus to analyze heart-related electronic signals, including eye movement, to identify various states of the cardiovascular system, valuable to understanding the physical and mental stress of the patient. The combination of Publicover/Rau/Najarian discloses wherein the step of identifying, by the computing device, relationships between eye movements that correlate to the outwards events (Publicover: [0186] Eye movements and patterns can also be discerned using non-head mounted cameras including those embedded within cell phones, tablets, laptop computers, and desktop computers; [0477] Gaze patterns are then further analyzed to determine if they generally correspond to activation interactables displayed on the screen at 1634. If a match is found, then actions corresponding to the selected interactable[s] are performed at 1635) comprises linear regression computing beta weights to relate (Najarian: [0143] The algorithm includes at least one context detector 1605 that produces a weight, shown as W1 through WN… a regression algorithm 1610 is provided where a continuous prediction is computed taking raw or derived channels as input. The individual regressions can be any of a variety of regression equations or methods, including, for example, multivariate linear or polynomial regression) eye movements to cognitive and/or emotional responses (Publicover: [0618] determined conditions can include a user state [i.e., generally classified over a continuous range] such as cognitive load or a degree of fatigue; Fig. 27; Rau: [0136] The patient's various autonomic physiological parameters [APPs] will respond and react to the induced stimuli. … Some of the APPs captured by the biometric devices include … eye movements… These biometric parameters can capture changes and severity of individual patient's feelings, emotions 1014 and theft innate resiliencies, coping skills, [healthy] behavior, function and responses 1012.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to use linear regression as disclosed in Najarian to compute a continuous estimate of the output of the parameter of interest in the algorithm (Najarian [0143]).
Regarding claim 21, the combination of Publicover/Rau discloses the method of claim 1. While Publicover discloses Bayesian statistics ([0571]), Publicover fails to specifically disclose a Bayesian network to identify non-linear patterns. The combination of Publicover/Rau/Najarian discloses wherein the step of identifying, by the computing device, relationships between eye movements that correlate to the outwards events (Publicover: [0186] Eye movements and patterns can also be discerned using non-head mounted cameras including those embedded within cell phones, tablets, laptop computers, and desktop computers; [0477] Gaze patterns are then further analyzed to determine if they generally correspond to activation interactables displayed on the screen at 1634. If a match is found, then actions corresponding to the selected interactable[s] are performed at 1635) comprises identifying non-linear patterns using Bayesian deep belief networks (Najarian: [0140] Further examples of the types of non-linear functions and/or machine learning method that may be used in the present invention include the following: conditionals, case statements, logical processing, probabilistic or logical inference, neural network processing, kernel based methods, memory-based lookup including kNN and SOMs, decision lists, decision-tree prediction, support vector machine prediction, clustering, boosted methods, cascade-correlation, Boltzmann classifiers, regression trees, case-based reasoning, Gaussians, Bayes nets, dynamic Bayesian networks).
Claim(s) 24 is rejected under 35 U.S.C. 103 as being unpatentable over Publicover (US 20150338915 A1) in view of Rau (US 20160022193 A1), and in further view of Proença (H. Proença, "Iris Recognition: On the Segmentation of Degraded Images Acquired in the Visible Wavelength," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 8, pp. 1502-1516, Aug. 2010, doi: 10.1109/TPAMI.2009.140.).
Regarding claim 24, the combination of Publicover/Rau discloses the method of claim 23. However, the combination of Publicover/Rau fails to disclose one that converts a noisy color image into a clear infrared image. Proença teaches an iris recognition segmentation method that can handle degraded images acquired in less constrained conditions. The combination of Publicover/Proença discloses, wherein the first time series of eye movements recorded by the first camera (Publicover: [0020] one or more cameras or at least one sensor to monitor changes in the reflection of the light; [0221] Using eye position data determined from a series of eye images, algorithmic “filters” can be constructed to identify and distinguish, in real-time, the presence of a saccadic or smooth pursuit eye movements; [0583] The use of deep learning approaches to eye signals can be classified as a “time series” data analysis) comprises a noisy color image data, and now including the step of transforming, by a neural network, the noisy color image data into a clear infrared image data (Proença: pg 1502, col 2, [2], an iris segmentation technique designed specifically for degraded iris images acquired in the VW [visible wavelength] and unconstrained scenarios; Fig. 3; pg 1507, col 1, Fig. 7. Schema for the multilayered feed-forward neural networks used in both classification stages of [the] segmentation method) for the step of comparing, by the computing device, the eye movements from the first time series and the plurality of tasks (Publicover: [0477] FIG. 16 shows an overall sequence of algorithmic steps used to detect eye signals that may lead up to the performance of an action. Images of an eye are acquired and analyzed at 1630 to determine gaze pathways … Filters at 1632 are applied to all series of eye positions to determine whether primarily saccadic or smooth pursuit [or vergence or vestibulo-ocular eye movements are present. If matches to these forms of voluntary eye movement are found, then gaze pathways are further compared with screen positions at 1633 to determine if regions corresponding to interactables or real objects are being viewed. Gaze patterns are then further analyzed to determine if they generally correspond to activation interactables displayed on the screen at 1634. If a match is found, then actions corresponding to the selected interactable[s] are performed at 1635).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Publicover/Rau to convert a noisy color image into a clear infrared image as disclosed in Proença to enable the capture of a much higher level of detail afforded by visible wavelengths despite the captured noise and artifacts (Proença, pg 1502, col 2, [2]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOLLY HALPRIN whose telephone number is (703)756-1520. The examiner can normally be reached 12PM-8PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Robert (Tse) Chen can be reached at (571) 272-3672. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.H./Examiner, Art Unit 3791
/DEVIN B HENSON/Primary Examiner, Art Unit 3791
Prosecution Insights