DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is in response to Applicant’s amendment/response filed on 11 August 2026, which has been entered and made of record.
Response to Arguments
Applicant’s arguments filed on 11 August 2026 have been fully considered but they are not persuasive.
Applicant argues that Karmarkar does not describe a link between demographic data and changing RGB values of media content: “Karmarkar does not teach or suggest changing RGB values of the displayed media based upon external demographic or trending data” (Remarks, pg. 7). The Examiner respectfully disagrees. Karmarkar discloses that current user eye-tracking data can be “matched to an averaged mental fatigue values … for a population of similar demographic attributes to the user” (para. 29). In other words, eye-tracking data indicating fatigue in one demographic of users may not indicate fatigue in another demographic of users, so Karmarkar compares current user eye-tracking data to expected eye-tracking data for their demographic in order to determine if the current user eye-tracking data indicates mental fatigue.
Karmarkar additionally discloses “modify text presentation and/or text content based on a current user mental fatigue state. For example, eye-tracking system 1002 can obtain user eye metrics … user age and other demographic information, etc. Thresholds for a particular eye metric data can be based on user profile information” (para. 54). Similar to para. 29, this recitation teaches that current user eye metrics can be evaluated in the context of their demographic data, such as by changing a threshold for an eye metric indicating mental fatigue based on demographic data. For example, a user having particular demographic data may be assigned a higher threshold that an eye metric must meet before the system concludes that the user has mental fatigue which requires text modification. Since Karmarkar teaches that text modification based on mental fatigue includes “Text can be highlighted” (para. 30), which includes a change to RGB values, one can see that Karmarkar teaches changing RGB values of text based on demographic data.
Any remaining arguments are considered moot based on the foregoing.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 9, 13, 14, 16-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Karmarkar et al. (US 2015/0213634; hereinafter "Karmarkar") in view of Krueger (US 2014/0152792), and further in view of Berme et al. (US 9,517,008; hereinafter “Berme”).
Regarding claim 1, Karmarkar discloses A method of assessing, improving or treating a condition experienced by a user ("a user mental fatigue state," para. 7), while said user is experiencing media using a computing device with a display ("augmented-reality glasses," para. 45) comprising acquiring a first value for at least one of a plurality of data using said computing device ("obtain a set of baseline eye movement attributes for a particular user," para. 28); acquiring a second value for the at least one of the plurality of data using said computing device ("a user's substantially contemporary saccadic velocity …," para. 28); using said first value and second value, determining a change in at least one of the plurality of data over time ("… can be compared with past data," para. 28), wherein the change is derived, at least in part, from measurements of differences in a gaze direction of the user's eyes ("tracks the gaze of user," para. 32; "the user's eye-tracking data (e.g. saccadic information, gaze trace, etc.)," para. 36); based upon said change in the at least one of the plurality of data over time, determining a degree of said condition, wherein said condition comprises fatigue ("determine a user's current mental fatigue state," para. 28); and based upon determining a degree of said condition, modifying visual elements of the media (“the size of text of the content can be increased, display contrast can be modified to make the text easier to view,” para. 30) by (ii) changing RGB values of the media (“Various levels of the practice exam can again be modified based on the user's substantially current mental fatigue state … Text can be highlighted,” para. 30) based upon external data, wherein the external data comprises at least one of demographic data or trending data (“Eye-tracking data that indicates a mental fatigue level may be parsed from the eye-tracking data and matched to an averaged mental fatigue values … for a population of similar demographic attributes to the user,” para. 29; “modify text presentation and/or text content based on a current user mental fatigue state. For example, eye-tracking system 1002 can obtain user eye metrics … user age and other demographic information, etc. Thresholds for a particular eye metric data can be based on user profile information,” para. 54; see also para. 28 for link between user demographic data and determining fatigue).
Karmarkar does not disclose wherein the plurality of data comprises data indicative of a degree of the user's palpebral fissure distance.
In the same art of dynamically assessing fatigue for controlling user equipment, Krueger teaches wherein the plurality of data comprises data indicative of a degree of the user's palpebral fissure distance and determining a change between first and second values of a user's palpebral fissure distance to determine a fatigue condition of the user ("a display that includes at least one display element responsive to palpebral movement information from the eye sensor," abstract; "The abnormal eye movements or eyelid responses (including … palpebral fissure height …) can be measured over time by physiologic biosensors," para. 56; "The visual alarm can be in the form of lights, which may have different colors, text information or other visual symbology," para. 60; "the eye closure information measured by the eye sensor comprises … eyelid fissure height … as a result of a physiological effect including … fatigue," published claim 10).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Krueger to Karmarkar. The motivation would have been to increase the accuracy of the fatigue determination.
The combination of Karmarkar and Krueger does not disclose modifying visual elements by changing a focal point of content displayed in the media to a more central location.
In the same art of reducing eye fatigue on electronic devices, Berme teaches modifying visual elements by changing a focal point of content displayed in the media to a more central location (“when the computing device determines that the subject has identified at least the predetermined number or percentage (i.e., 75%) of the one or more configurations of the one or more visual objects 10′ incorrectly, the computing device is specially programmed to decrease the difficulty of the training by … changing a location for displaying the one or more visual objects on the output screen of the visual display device from a peripheral location on the output screen of the visual display device to a central location on the output screen,” col. 93, lines 1-15).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Berme to the combination of Karmarkar and Krueger. The motivation would have been to “greatly minimize subject or patient fatigue” (Berme, col. 94, lines 5-10).
Regarding claim 2, the combination of Karmarkar, Krueger, and Berme renders obvious a virtual reality, augmented reality, or mixed reality view device ("augmented-reality glasses," Karmarkar, para. 45).
Regarding claim 3, the combination of Karmarkar, Krueger, and Berme renders obvious at least one of a camera configured to acquire eye movement data, a sensor configured to detect a rate and/or direction of head movement, a sensor configured to detect a heart rate, or an EEG sensor to detect brain waves ("Eye-tracking module may utilize information from at least one digital camera," Karmarkar, para. 32; "head movement," Karmarkar, para. 35; "pulse rate," Karmarkar, para. 50).
Regarding claim 4, the combination of Karmarkar, Krueger, and Berme renders obvious wherein the virtual reality, augmented reality, or mixed reality view device comprises the camera configured to acquire the eye movement data (e.g. eye-facing digital camera 712 of Karmarkar, Fig. 7) and wherein the eye movement data comprises rapid scanning ("rapid … scans," Karmarkar, para. 26), saccadic movement ("saccadic eye movement," Karmarkar, para. 26), blink rate data ("blink rate," Karmarkar, para. 28, fixation data ("fixation metrics," Karmarkar, para. 5), and pupillary diameter ("pupil sizes," Karmarkar, para. 5).
Regarding claim 5, the combination of Karmarkar, Krueger, and Berme renders obvious wherein the condition is at least one of comprehension, engagement, performance, symptoms associated with visually-induced motion sickness secondary to visual-vestibular mismatch, symptoms associated with post-traumatic stress disorder, double vision related to accommodative dysfunction, vection due to unintended peripheral field stimulation, vergence-accommodation disorders, fixation disparity, blurred vision and myopia, headaches, difficulties in focusing, disorientation, postural instability, visual discomfort, eyestrain, dry eye, eye tearing, foreign body sensation, feeling of pressure in the eyes, aching around the eyes, nausea, stomach discomfort, potential phototoxicity from overexposure to screen displays, hormonal dysregulation arising from excessive blue light exposure, heterophoria, decrease in positive emotions, or an increase in negative emotions ("Saccades may also reflect visual attention of the user and thus indicate user attributes (e.g. interest, comprehension difficulty, etc.)," Karmarkar, para. 26).
Regarding claim 6, the combination of Karmarkar, Krueger, and Berme renders obvious wherein the plurality of data further comprises at least one of rapid scanning, saccadic movement, fixation, blink rate, pupillary diameter, speed of head movement, direction of head movement, heart rate, motor reaction time, smooth pursuit, or a degree and rate of brain wave activity ("saccadic velocity," Karmarkar, para. 28).
Regarding claim 9, the combination of Karmarkar, Krueger, and Berme renders obvious wherein the change further comprises at least one of increased rapid scanning, increased saccadic movement, decreased fixation, increased blink rate, increased pupillary diameter, increased head movement, increased heart rate, decreased reaction time, decreased separation of the eyelids, changes in brain wave activity, or increased smooth pursuit ("an increase in blink rate," Karmarkar, para. 28).
Regarding claim 13, the combination of Karmarkar, Krueger, and Berme renders obvious wherein the change comprises a rate of change ("analysis of the slope," Karmarkar, para. 28) of the degree of the user's visual convergence or visual divergence ("Saccadic parameters and fixation durations also showed specific changes with increasing sleepiness … binocular vergence or divergence," Krueger, para. 140; see claim 1 for motivation to combine) and at least one of decreased fixation or increased blink rate ("an increase in blink rate," Karmarkar, para. 28).
Regarding claim 14, the combination of Karmarkar, Krueger, and Berme renders obvious wherein the determined degree of the condition indicates an increased degree of fatigue of the user ("indicate mental fatigue," Karmarkar, para. 28).
Regarding claim 16, Karmarkar discloses A method of decreasing fatigue experienced by a user ("a user mental fatigue state," para. 7), while the user is experiencing media through a virtual reality, augmented reality, or mixed reality view device ("augmented-reality glasses," para. 45), the method comprising: acquiring a first value for a plurality of data ("obtain a set of baseline eye movement attributes for a particular user," para. 28); acquiring a second value for the plurality of data ("a user's substantially contemporary saccadic velocity …," para. 28); using the first value and the second value to determine a change in the plurality of data over time ("… can be compared with past data," para. 28); based upon the change in the plurality of data over time, determining a degree of increased fatigue of the user ("comprehension difficulty," para. 26); and modifying visual elements of the media based upon determining a degree of increased fatigue (“the size of text of the content can be increased, display contrast can be modified to make the text easier to view,” para. 30) by (ii) changing RGB values of the media (“Various levels of the practice exam can again be modified based on the user's substantially current mental fatigue state … Text can be highlighted,” para. 30) based upon external data, wherein the external data comprises at least one of demographic data or trending data (“Eye-tracking data that indicates a mental fatigue level may be parsed from the eye-tracking data and matched to an averaged mental fatigue values … for a population of similar demographic attributes to the user,” para. 29; “modify text presentation and/or text content based on a current user mental fatigue state. For example, eye-tracking system 1002 can obtain user eye metrics … user age and other demographic information, etc. Thresholds for a particular eye metric data can be based on user profile information,” para. 54; see also para. 28 for link between user demographic data and determining fatigue).
Karmarkar does not disclose wherein the plurality of data comprises a degree of the user's palpebral fissure distance.
In the same art of dynamically assessing fatigue for controlling user equipment, Krueger teaches wherein the plurality of data comprises a degree of the user's palpebral fissure distance and determining a change between first and second values of a user's palpebral fissure distance to determine a fatigue condition of the user ("a display that includes at least one display element responsive to palpebral movement information from the eye sensor," abstract; "The abnormal eye movements or eyelid responses (including … palpebral fissure height …) can be measured over time by physiologic biosensors," para. 56; "The visual alarm can be in the form of lights, which may have different colors, text information or other visual symbology," para. 60; "the eye closure information measured by the eye sensor comprises … eyelid fissure height … as a result of a physiological effect including … fatigue," published claim 10).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Krueger to Karmarkar. The motivation would have been to increase the accuracy of the fatigue determination.
The combination of Karmarkar and Krueger does not disclose modifying visual elements by changing a focal point of content displayed in the media to a more central location.
In the same art of reducing eye fatigue on electronic devices, Berme teaches modifying visual elements by changing a focal point of content displayed in the media to a more central location (“when the computing device determines that the subject has identified at least the predetermined number or percentage (i.e., 75%) of the one or more configurations of the one or more visual objects 10′ incorrectly, the computing device is specially programmed to decrease the difficulty of the training by … changing a location for displaying the one or more visual objects on the output screen of the visual display device from a peripheral location on the output screen of the visual display device to a central location on the output screen,” col. 93, lines 1-15).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Berme to the combination of Karmarkar and Krueger. The motivation would have been to “greatly minimize subject or patient fatigue” (Berme, col. 94, lines 5-10).
Regarding claim 17, the combination of Karmarkar, Krueger, and Berme renders obvious acquiring at least one or more of: data from a sensor configured to detect basal body temperature, heart rate, body movement, body rotation, body direction, body velocity, or body amplitude; data from a sensor configured to measure limb movement, limb rotation, limb direction, limb velocity, or limb amplitude; a pulse oximeter; data from a sensor configured to measure auditory processing; data from a sensor configured to measure gustatory and olfactory processing; data from a sensor to measure pressure; an input device such as a traditional keyboard and mouse and or any other form of controller to collect manual user feedback; an electroencephalograph; an electrocardiograph; an electromyograph; an electrooculograph; an electroretinography; or data from a sensor configured to measure galvanic skin response ("saccadic velocity … blink rates," Karmarkar, para. 30; these are forms of body movement).
Regarding claim 18, the combination of Karmarkar, Krueger, and Berme renders obvious wherein the plurality of data further comprises at least one or more of: blink rate, pupil size, pupil position, gaze direction, gaze position, vergence, fixation position, fixation duration, fixation rate; fixation count; saccade position, saccade angle, saccade magnitude, pro-saccade, anti-saccade, inhibition of return, saccade velocity, saccade rate, screen distance, head direction, head fixation, limb tracking, weight distribution, frequency domain analysis, electroencephalography output, frequency bands, electrocardiography output, electromyography output, electrooculography output, electroretinography output, galvanic skin response, body temperature, respiration rate, oxygen saturation, heart rate, blood pressure, vocalizations, inferred efferent responses, respiration, facial expression, olfactory processing, gustatory processing, or auditory processing ("pupil dilation, blink rates," Karmarkar, para. 30).
Regarding claim 20, the combination of Karmarkar, Krueger, and Berme renders obvious wherein modifying the visual elements of the media comprises modifying the media to provide a predefined increase in comprehension ("a user's current reading level can be modified (e.g. increased, decreased) … a table can be provided that matches saccades velocity values to reading levels for a particular user," Karmarkar, para. 30).
Claims 10 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Karmarkar, Krueger, and Berme, and further in view of Gabel (US 2014/0195900).
Regarding claim 10, the combination of Karmarkar, Krueger, and Berme renders obvious determining decreased comprehension of the user ("comprehension difficulty," Karmarkar, para. 26) and wherein, based on the determined degree of fatigue of the user, said media is modified ("the size of text of the content can be increased, display contrast can be modified to make the text easier to view," Karmarkar, para. 30).
The combination of Karmarkar, Krueger, and Berme does not disclose modifying the media by changing a motion of objects in the field of view.
In the same art of compensating for eye strain/fatigue, Gabel renders obvious modifying media by changing a motion of objects in the field of view ("reduce eye strain and fatigue … by providing additional control over scrolling speed," para. 5).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Gabel to the combination of Karmarkar, Krueger, and Berme. The motivation would have been "to reduce eye strain and fatigue" (Gabel, para. 5).
Regarding claim 15, the combination of Karmarkar, Krueger, and Berme renders obvious wherein, based on said increased degree of fatigue of the user, said media is modified ("the size of text of the content can be increased, display contrast can be modified to make the text easier to view," Karmarkar, para. 30).
The combination of Karmarkar, Krueger, and Berme does not disclose modifying the media by changing a motion of objects in the field of view.
In the same art of compensating for eye strain/fatigue, Gabel renders obvious modifying media by changing a motion of objects in the field of view ("reduce eye strain and fatigue … by providing additional control over scrolling speed," para. 5).
Before the effective filing date of the claimed invention, it would have been obvious to one having ordinary skill in the art to apply the teachings of Gabel to the combination of Karmarkar, Krueger, and Berme. The motivation would have been "to reduce eye strain and fatigue" (Gabel, para. 5).
Conclusion
Applicant's amendment necessitated any new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan McCulley whose telephone number is (571)270-3754. The examiner can normally be reached on Monday through Friday, 8:00am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN MCCULLEY/Primary Examiner, Art Unit 2611
