DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The previous objections to the drawings and claims are withdrawn in light of the amended drawings and claims, each filed 02/16/2026.
Applicant’s request to hold the non-statutory double patenting rejection in abeyance is acknowledged. As necessitated by amendments, claims 1-4, 7-8, 10, 13-18, 21, and 24-25 are rejected on the ground of nonstatutory double patenting as being unpatentable over Mulliken in view of Horowitz and Ghajar; claim 5 is rejected on the ground of nonstatutory double patenting as being unpatentable over Mulliken in view of Horowitz, Ghajar, and Li; claim 9 is rejected on the ground of nonstatutory double patenting as being unpatentable over Mulliken in view of Horowitz, Ghajar, and Cao; claim 11 is rejected on the ground of nonstatutory double patenting as being unpatentable over Mulliken in view of Horowitz, Ghajar, and Devaraj; and claim 12 is rejected on the ground of nonstatutory double patenting as being unpatentable over Mulliken in view of Horowitz, Ghajar, and Boccanfuso. See rejections, as presented in detail below.
Applicant's arguments with regards to the rejection of the claims under 35 U.S.C. § 101 (Remarks, filed 02/16/2026, pp. 11-16) have been fully considered but they are not persuasive.
Applicant first argues that the steps of “assessing an attentive state during an experience in an environment using physiological signals and context of the environment, and providing tailored feedback to improve performance and well-being” cannot be practically performed in the human mind and that the claim “requires a processor coupled to computer memory to cause the device to perform each of these preceding steps (Remarks, filed 02/16/2026, p. 12). Examiner respectfully disagrees and asserts that these steps can indeed be performed by a human. For example, a human teacher could assess whether a student is paying attention based on their gaze and the context of the situation, e.g., if the student is looking at a talking teacher or if a student is looking at a presentation. Furthermore, if needed, the teacher could provide tailored feedback by calling on the student to pay attention.
Applicant next argues that the claims “do not recite any method of organizing human activity, such as fundamental economic concept or managing interactions between people” (Remarks, filed 02/16/2026, p. 12). Examiner respectfully disagrees and asserts that at least the step of providing feedback encompasses managing personal behaviors or relationships or interactions between people (including social activities, teaching, and following rules or instructions). Examiner further notes that this category includes “certain activity between a person and a computer.” See MPEP § 2106.04(a)(2)(II).
Applicant then argues that the claims integrate “any alleged abstract idea into a practical application by (i) reciting specific sensor-based data acquisition and computer vision operations; (ii) modifying XR content in real time based on a computed attentive state; and (iii) optionally closing the loop within a defined time window to update the attentive state based on the measured physiological response (Remarks, filed 02/16/2026, pp. 13-14). Examiner respectfully disagrees. The claims merely recite generic computing components used to automate the method steps. Furthermore, “sensor-based data acquisition” and “modifying XR content” appear to be directed to insignificant extra-solution activity (such as data gathering or data presentation) rather than an integration into a practical application.
Applicant next argues that the rejected claims do not merely apply a judicial exception to generic computer components and instead comprise “a non-conventional and non-generic arrangement of operations being performed in a specified manner to address the complexity of improve the functioning of an XR presentation system by adaptively modulating content attributes in response to measured physiological signals and sensor-derived scene understanding to achieve targeted cognitive-state outcomes while minimizing disruption” (Remarks, filed 02/16/2026, pp. 15-16). Examiner respectfully disagrees that Applicant’s claims are directed to any technological improvement pertaining to “the complexity of improv[ing] the functioning of an XR presentation system” and instead appears to be directed to conventional and generic computing components performing routine functions.
Applicant finally argues the claimed steps “amount to a specific improvement in XR content control and user experience through closed-loop fusion of physiological signals and scene understanding. These amendments integrate the abstract idea into a practical application and reflect an improvement in computer technology)” (Remarks, filed 02/16/2026, p. 16). Examiner respectfully disagrees and reiterates that the claims are not directed to any technological improvement of XR content control and instead utilize generic computing components to perform routine functions such as data acquisition and data presentation.
Applicant’s arguments with regards to the rejection of claims 1-4, 10, 13-16, 18, 21, and 24 under 35 U.S.C. § 102 (Remarks, filed 02/16/2026, pp. 16-20) have been fully considered and are persuasive. This rejection under 35 U.S.C. § 102 has been withdrawn. However, as necessitated by amendment, a new rejection of claims 1-4, 7-8, 10, 13-18, 21, and 24-25 under 35 U.S.C. 103 as being unpatentable over Ghajar in view of Horowitz and Karmarkar has been raised. See rejection of claims as presented in detail below.
Applicant’s arguments with respect to the rejection of the remaining claims under 35 U.S.C. § 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-4, 7-8, 10, 13-18, 21, and 24-25 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4, 7, 8-10, 15, and 19 of U.S. Patent No. 12,277,265 (hereinafter “Mulliken”) in view of Horowitz and Ghajar.
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 1: A method comprising:
at a device comprising a processor:
obtaining physiological data associated with a gaze of a user during an experience, wherein the user experience is associated with a task;
determining a context of the experience and the task based on sensor data of an environment, including generating, from image data, a scene understanding of the environment;
determining that the user has a first attentive state during the experience based on the physiological data, the context of the experience and task, and the scene understanding of the environment, the first attentive state corresponding to a lack of attention by the user in the task during the experience; and
providing, during a presentation of an extended reality (XR) environment, a feedback mechanism that modifies XR content during the experience based on determining that the user has the first attentive state during the experience, wherein modifying XR content comprises adjusting visual or auditory attributes associated with the experience and the associated task
Claim 1: A method comprising:
at a device comprising a processor:
presenting content for a plurality of time segments to a user and obtaining physiological data associated with a gaze of the user;
…
determining that the user has a first attentive state during a first time segment of the plurality of time segments based on classifying the variability of the gaze characteristic of the user, wherein the first attentive state comprises a mind wandering state that includes a different stability of the variability of the gaze characteristic compared to a focused state; and
presenting feedback content for a second time segment in response to determining that the first attentive state differs from a second attentive state intended for the content, wherein the feedback content directs the user to modify a physiological action associated with the first attentive state, and wherein the second attentive state intended for the content comprises the focused state.
Claim 9: The method of claim 1, wherein the content is an extended reality (XR) environment that is presented to the user.
Further regarding Claim 1, Mulliken is silent as to the determination of a context and scene understanding. However, Horowitz discloses determining a context of the experience and the task based on sensor data of an environment, including generating, from image data, a scene understanding of the environment (fig. 5: steps 506, 508, 510; par. 0017: “the system may determine which objects at the location are receiving attention from the users. The system then generates an attention map based on the attention information;” par. 0023: the sensor(s) obtaining data may be “client devices 110, 120, 130, and 140 may include wearable computing devices, including any hands-free devices. For example, in some implementations, one or more client devices may include devices that operate with a head-mounted camera, head-mounted eye tracking device, and/or head-mounted display;” par. 0017: “a system collects photos associated with objects in at least one location, where one or more of the photos are captured using one or more respective devices that are operable to track a gaze of a user. The system then collects attention information associated with one or more of the objects;” Examiner notes an understanding of the environment—such as objects to pay attention to—is generated from image data); and determining that the user has a first attentive state during the experience based on the physiological data, the context of the experience and task, and the scene understanding of the environment, (fig. 6: every spot on the content is given an attention value score; therefore, Examiner notes that the first attentive state can be determined when a given attention value is low or nonexistent; fig. 5: attention values are determined based on received gaze/physiological data and the associated context and scene understanding). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the determination of a context of the experience of Horowitz with the attention determination method of Mulliken in order to most effectively utilize the obtained data, such as by creating an attention map (Horowitz, abstract; par. 0006).
Mulliken is also silent as to the modification of the (XR) content specifically includes adjusting the auditory or visual attributes of the content. However, Ghajar discloses modifying content comprises adjusting visual or auditory attributes associated with the experience and the associated task (par. 0032: “Feedback module 228… generates feedback signals for presentation to the subject via display 104 or feedback devices 208. For example, display 104 may optionally present an indication of an attention state of the subject, such as an "attention meter" displayed on the display or a "traffic light" with red, yellow, and green lights, where red indicates that the subject is not attending”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the specific feedback mechanism of Ghajar with the feedback mechanism of the method of Mulliken in order to more effectively direct the user to pay attention to the content (Ghajar, par. 0032).
See also related mirror claims 21 and 24:
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 21: A device comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: (see claim 1 above)
Claim 15: A device comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: (see claim 1 above)
Claim 24: A non-transitory computer readable storage medium, storing program instructions executable on a device to perform operations comprising: (see claim 1 above)
Claim 19: A non-transitory computer readable storage medium, storing program instructions executable via one or more processors to perform operations comprising: (see claim 1 above)
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 2: The method of claim 1, further comprising: determining that the user has a second attentive state during a portion of the experience based on the physiological data, the second attentive state corresponding to attention by the user in the task during the portion of the experience
Claim 1: A method comprising:
…
determining that the user has a first attentive state during a first time segment of the plurality of time segments based on classifying the variability of the gaze characteristic of the user, wherein the first attentive state comprises a mind wandering state that includes a different stability of the variability of the gaze characteristic compared to a focused state; and…
Further regarding Claim 2, Mulliken implies the ability to determine the user is in a second attentive state (a state of focus) but does not explicitly claim as such. However, Mulliken modified by Ghajar further discloses determining that the user has a second attentive state during a portion of the experience based on the physiological data (pars. 0013-0014: “a determination of a user's level of attention… one or more of the subject's eye positions is measured. The method further includes continuously performing the operations of generating a variability metric using the one or more measured eye positions and comparing the variability metric with a predetermined baseline to determine an attention state of the subject”), the second attentive state corresponding to attention by the user in the task during the portion of the experience (par. 0019: “an attention state of the subject is categorized based on the comparison, for example, to determine that the subject's attention state is an attending state (e.g., the subject is paying attention)”).
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 3: The method of claim 2, wherein presenting the feedback mechanism is based on determining that the second attentive state differs from the first attentive state.
Claim 1: A method comprising:
… presenting feedback content for a second time segment in response to determining that the first attentive state differs from a second attentive state intended for the content…
Claim 4: The method of claim 1, wherein determining that the user has the first attentive state comprises determining a level of attentiveness.
Claim 1: A method comprising:
… determining that the user has a first attentive state during a first time segment of the plurality of time segments based on classifying the variability of the gaze characteristic of the user, wherein the first attentive state comprises a mind wandering state that includes a different stability of the variability of the gaze characteristic compared to a focused state; and…
Further regarding Claim 4, Mulliken implies the ability to determine a level of attentiveness but does not explicitly claim as such. However, Mulliken modified by Ghajar further discloses determining that the user has the first attentive state comprises determining a level of attentiveness (par. 0013: “a determination of a user's level of attention;” par. 0032: “an "attention meter" displayed on the display or a "traffic light" with red, yellow, and green lights, where red indicates that the subject is not attending”).
Regarding Claim 7, Mulliken is silent on the described context. However, Mulliken modified by Horowitz further discloses the context comprises an object upon which the user's attention should be focused during the experience (Horowitz, par. 0017: “the system may determine which objects at the location are receiving attention from the users. The system then generates an attention map based on the attention information;” fig. 5: steps 502, 504, 506).
Regarding Claim 8, Mulliken is silent on an attention map. However, Mulliken modified by Horowitz further discloses determining context comprises determining an attention map identifying portions of an image upon which attention is focused when attentive to the task (par. 0017: “the system may determine which objects at the location are receiving attention from the users. The system then generates an attention map based on the attention information;” fig. 5: steps 506, 508, 510; fig. 6).
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 10: The method of claim 1, wherein providing the feedback mechanism comprises providing a graphical indicator or sound configured to change the first attentive state to a second attentive state corresponding to attention by the user in the task during the experience.
Claim 1: A method comprising:
… presenting feedback content… wherein the feedback content directs the user to modify a physiological action associated with the first attentive state, and wherein the second attentive state intended for the content comprises the focused state.
Further regarding Claim 10, Mulliken is silent on the feedback mechanism comprising a graphical indicator or sound. However, Mulliken modified by Ghajar further discloses providing the feedback mechanism comprises providing a graphical indicator or sound configured to change the first attentive state to a second attentive state corresponding to attention by the user in the task during the experience (par. 0032: “Feedback module 228, if included, generates feedback signals for presentation to the subject via display 104 or feedback devices 208. For example, display 104 may… present an indication of an attention state of the subject, such as an "attention meter" displayed on the display or a "traffic light" with red, yellow, and green lights, where red indicates that the subject is not attending, yellow indicates that the subject is barely attending, and green indicates that the subject is attending”).
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 13: The method of claim 1, further comprising adjusting content corresponding to the experience based on the first attentive state.
Claim 1: A method comprising:
determining that the user has a first attentive state during a first time segment of the plurality of time segments based on classifying the variability of the gaze characteristic of the user, wherein the first attentive state comprises a mind wandering state that includes a different stability of the variability of the gaze characteristic compared to a focused state…
Claim 4: The method of claim 1, further comprising adjusting the content in response to determining that the first attentive state differs from a second attentive state intended for the content.
Claim 14: The method of claim 1, wherein the physiological data comprises: an image of an eye or electrooculography (EOG) data; or a gaze characteristic.
Claim 7: The method of claim 1, wherein the physiological data comprises an image of an eye or electrooculography (EOG) data.
Regarding Claim 15, Mulliken is silent on the specifics of the experience. However, Mulliken modified by Ghajar further discloses the experience is observing educational content, observing entertainment content, receiving instruction regarding a new skill, reading a document, or participating in a meditation session (par. 0021: “content is delivered… Consider an example of a pre-recorded lecture;” par. 0065: “For example, the content may include a television program;” fig. 3B: content is a document to be read).
Regarding Claim 16, Mulliken implies a display since content is presented to the user but does not explicitly claim as such. However, Mulliken modified by Ghajar further discloses the experience is presented to the user on a display (fig. 1: display 106; par. 0013: “the visual content displayed on the screen”).
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 17: The method of claim 1, wherein the experience is presented to the user on a head-mounted device.
Claim 8: The method of claim 1, wherein the device is a head-mounted device (HMD).
Claim 18: The method of claim 1, wherein the physiological data is generated based on an image of an eye captured by an eye tracking camera on a head mounted device.
Claim 10: The method of claim 1, wherein the physiological data is generated based on an image of an eye captured by an eye tracking camera on a head mounted device
Further regarding Claim 18, Mulliken discloses a head-mounted device but is silent on eye-tracking cameras mounted thereon. However, Mulliken modified by Ghajar further discloses the physiological data is generated based on an image of an eye captured by an eye tracking camera on a head mounted device (fig. 1: digital video camera(s) 104 on head equipment 108; par. 0023: “One or more digital video cameras 104 are focused on subject 102's eyes so that eye positions (and, in some embodiments, eye movements) of subject 102 are recorded… digital video cameras 104 are mounted on subject 102's head by head equipment 108”).
Regarding Claim 25, Mulliken implies but does not explicitly disclose updating the attentive state. However, Mulliken modified by Ghajar discloses obtaining, within a predetermined time following providing the feedback mechanism, physiological data associated with a physiological response of the user to the feedback mechanism (par. 0067: “the system continuously compares (522) the variability metric with a predetermined baseline to determine an attention state of the subject;” Examiner notes that the physiological data is obtained continuously and therefore would also occur instantly after providing the feedback mechanism); and updating the attentive state based on the physiological response (par. 0067: “the system continuously compares (522) the variability metric with a predetermined baseline to determine an attention state of the subject… the attention state is (524) one of an about-to-attend state, an attending state, or a not-attending state;” par. 0099: “the detected change in the attention state is used to update a profile of the subject;” Examiner notes that the attention state is constantly monitored and evaluated, and would therefore update accordingly).
Claim 5 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 8-9 of U.S. Patent No. 12,277,265 (hereinafter “Mulliken”) in view of Horowitz, Ghajar, and Li.
Pending US Application 18/283,850
US Patent 12,277,265 (Mulliken)
Claim 5: The method of claim 1, wherein determining that the user has a first attentive state comprises using a machine learning model, the machine learning model trained using ground truth data comprising self-assessments in which users labelled portions of experiences with attentive state labels.
Claim 8: The method of claim 1, wherein the variability of the gaze characteristic is classified using a statistical technique or a machine learning technique.
Further regarding Claim 5, Mulliken discloses a machine learning technique but is silent on the training of a learning model. However, Li discloses determining that the user has a first attentive state comprises using a machine learning model (abstract: “inputting the brain reaction parameter value into a machine learning model; outputting the recognition result of the emotion recognition by the machine learning model. In the embodiment of the invention, after collecting the brain reaction data (mainly eye reaction data);” p. 2: “the emotion recognition type comprises at least one of emotion type and cognition type; wherein: … the cognitive type comprises attention”), the machine learning model trained using ground truth data comprising self-assessments in which users labelled portions of experiences with attentive state labels (p. 2: “the machine learning model is trained by using the training sample with label; wherein the training sample comprises brain response parameters from a healthy individual or a patient individual; the label comprises… [an] attention state information label and pressure state information label”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the machine learning model of Li with the method of Mulliken in order to automate the determination of the user’s attentive state using a model which has been trained on verified user data (Li, abstract; p. 2).
Claim 9 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 9 of U.S. Patent No. 12,277,265 (hereinafter “Mulliken”) in view of Horowitz, Ghajar, and Cao.
Regarding Claim 9, Mulliken is silent on transition data. However, Cao discloses the attention map further comprises transition data identifying a number of transitions of the user changing focus (figs. 5, 7; par. 0086: “To understand and model user behaviors when viewing web designs, eye-tracking data of different users on a web design corpus consisting of different genres (e.g., shopping, game, social media, etc.) under different viewing conditions (i.e., task-oriented and free-viewing conditions) 312 is collected and analyzed. Based on the collected data, it is possible to build an attention transition model 310T;” examiner notes that tracking the transition data described would inherently include the number of times each transition happens) from: a first portion of the image upon which attention is focused, to a second portion of the image upon which attention is not focused, to a third portion of the image upon which attention is focused (fig. 5: user’s gaze is tracked as it transitions across various portions of the content; par. 0113: “the eye gaze path predicted from a user saliency map was compared against the actual attention transition;” Examiner notes that the collected transition data is directly compared to an attention/saliency map, and therefore the tracked transitions will include transitions from focused portions to non-focused portions and back to focused portions). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention transition data of Cao with the method of modified Mulliken (as described above for claim 8) in order to better design the content (Cao, par. 0145) or to better predict where a user is most likely to focus next at any given moment (Cao, par. 0071).
Claim 11 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 9 of U.S. Patent No. 12,277,265 (hereinafter “Mulliken”) in view of Horowitz, Ghajar, and Devaraj.
Regarding Claim 11, Mulliken discloses a feedback mechanism but is silent on any specific implementations. However, Devaraj discloses providing the feedback mechanism comprises providing a mechanism for rewinding or providing a break from content associated with the task (par. 0045: “Once the set top box has determined the level of engagement of the user 201 with the program 206, then it can be programmed to make a determination regarding what to do in the event of changes in the user's activity during the program. If the engagement has been previously measured as very high…then it can automatically pause or even rewind or store the program while the user is interrupted”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the specific features of the feedback mechanism of Devaraj with the feedback mechanism used in the method of Mulliken because the ability to rewind or pause content is a well-known and useful feedback mechanism, especially within the context of a user not paying attention to the content (Devaraj, abstract; par. 0045).
Claim 12 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 9 of U.S. Patent No. 12,277,265 (hereinafter “Mulliken”) in view of Horowitz, Ghajar, and Boccanfuso.
Regarding Claim 12, Mulliken discloses a feedback mechanism but is silent on any specific implementations. However, Boccanfuso discloses providing the feedback mechanism comprises suggesting a time for another experience based on first attentive state (pars. 0061-0062: “Once the child has been redirected 3 times during a single lesson, the robot will suggest a break… Attentional tracking data may be collected to research individual variations in attention and performance. These data will contribute to an even more personalized approach for the robot. To this end, machine learning techniques are employed to learn typical attentional behavior for each individual and determine the optimal schedule for deploying redirection and break behaviors;” Examiner notes the feedback is suggesting another experience begin after a predetermined length of time in which the user takes a break, with this suggestion being based on individualized attention data from previous sessions). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the specific features of the feedback mechanism of Boccanfuso with the feedback mechanism used in the method of Mulliken because utilizing known attention data to recommend a break until the user can pay better attention would increase the user’s intake of the content (Boccanfuso, pars. 0061-0062, 0118).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-5, 7-18, 21, and 24-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea(s) without significantly more.
Regarding Claim 1, analyzed as the representative claim:
[Step 1] Claim 1 recites “A method…” which falls within the “process” statutory category of invention under 35 U.S.C. § 101.
[Step 2A – Prong 1] Claim 1 recites “A method comprising: at a device comprising a processor: obtaining physiological data associated with a gaze of a user during an experience, wherein the user experience is associated with a task; determining a context of the experience and the task based on sensor data of an environment, including generating, from image data, a scene understanding of the environment; determining that the user has a first attentive state during the experience based on the physiological data, the context of the experience and task, and the scene understanding of the environment, the first attentive state corresponding to a lack of attention by the user in the task during the experience; and providing, during a presentation of an extended reality (XR) environment, a feedback mechanism that modifies XR content during the experience based on determining that the user has the first attentive state during the experience, wherein modifying XR content comprises adjusting visual or auditory attributes associated with the experience and the associated task.” The bolded limitations, under their broadest reasonable interpretation, encompass mental processes (including observation, evaluation, judgment, and opinion) or methods of organizing human activity (managing personal behavior or relationships or interactions between people – including social activities, teaching, and following rules or instructions). That is, other than reciting that the method is performed by “a device comprising a processor,” nothing in the claim precludes the steps from practically being performed by a human and/or in the human mind. Specifically, the claim encompasses a person, such as a teacher, observing another user, such as a student, to see whether they are gazing at presented content, determining whether they are paying attention based on that observation, and providing feedback if they are not paying attention (such as simply telling the student to pay attention or speaking louder). Accordingly, the claim recites an abstract idea(s).
[Step 2A – Prong 2] The judicial exception is not integrated into a practical application. Specifically, the claim recites the additional elements of a device comprising a processor for executing the method steps, sensors, and a presentation of extended-reality (XR) content, each of these additional elements is recited at a high level of generality and merely automate the obtaining, determining, providing, and adjusting steps. Therefore, this additional element amounts to no more than mere instructions to apply the exception using a generic computing device, which does not impose any meaningful limits on practicing the abstract idea(s). Additionally and/or alternatively, the limitations of “present[ing] an XR environment” and “modifying XR content” are drawn to insignificant extra-solution activity (data display), which do not integrate the abstract ideas into a practical application. See MPEP § 2106.05(g). Thus, the claim is directed to an abstract idea(s).
[Step 2B] The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea(s) into a practical application, the additional elements a device comprising a processor for executing the method steps, sensors, and a presentation of extended-reality (XR) content amount to no more than mere instructions to apply the exception using a generic computing device and/or insignificant extra-solution activity, which cannot provide an inventive concept. Accordingly, representative claim 1 is not patent eligible.
Claims 2-5, 7-18, and 25 are dependent on representative claim 1 and include all of the limitations of claim 1. Therefore, the dependent claims recite the same abstract idea(s) as those recited in the independent claim or contain limitations drawn to generic computer components and/or reciting extra solution activities. While the dependent claims may have a narrower scope than the representative claim, no claim contains an additional element to integrate the abstract idea(s) into a practical application or to render an inventive concept that transforms the corresponding claim into a patent eligible application of the otherwise ineligible abstract idea(s). Thereby, claims 2-5, 7-18, and 25 are also patent ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 7-8, 10, 13-18, 21, and 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over Ghajar in view of Horowitz and Karmarkar.
Regarding Claims 1, 21, and 24, Ghajar discloses (referring to Claim 1) at a device comprising a processor (par. 0017: “an electronic device with one or more processors”)…,
(referring to Claim 21) a device comprising: a non-transitory computer-readable storage medium (par. 0017: “a non-transitory computer readable storage medium storing one or more programs. The one or more programs comprise instructions, which when executed by an electronic device”); and one or more processors coupled to the non-transitory computer-readable storage medium (par. 0018: “The electronic device includes one or more processors, memory, and one or more programs”), wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations (par. 0017: “a non-transitory computer readable storage medium storing one or more programs. The one or more programs comprise instructions, which when executed by an electronic device with one or more processors and memory, cause the electronic device to perform any of the methods provided herein)…, and
(referring to Claim 24) a non-transitory computer-readable storage medium, storing program instructions executable on a device to perform operations (par. 0017: “a non-transitory computer readable storage medium storing one or more programs. The one or more programs comprise instructions, which when executed by an electronic device with one or more processors and memory, cause the electronic device to perform any of the methods provided herein”) comprising:
obtaining physiological data associated with a gaze of a user during an experience (par. 0014: “While presenting the content to the subject, one or more of the subject's eye positions is measured”), wherein the user experience is associated with a task (fig. 1; abstract; pars. 0021-0023: the associated task is to watch, observe, and/or listen in an attentive manner to the provided content);
determining that the user has a first attentive state during the experience based on the physiological data (pars. 0013-0014: “a determination of a user's level of attention… one or more of the subject's eye positions is measured. The method further includes continuously performing the operations of generating a variability metric using the one or more measured eye positions and comparing the variability metric with a predetermined baseline to determine an attention state of the subject”), the first attentive state corresponding to a lack of attention by the user in the task during the experience (par. 0019: “an attention state of the subject is categorized… a non-attending state (e.g., the subject is not paying attention)”); and
providing, during a presentation, a feedback mechanism that modifies content during the experience based on determining that the user has the first attentive state during the experience, wherein modifying content comprises adjusting visual or auditory attributes associated with the experience and the associated task (par. 0032: “Feedback module 228… generates feedback signals for presentation to the subject via display 104 or feedback devices 208. For example, display 104 may optionally present an indication of an attention state of the subject, such as an "attention meter" displayed on the display or a "traffic light" with red, yellow, and green lights, where red indicates that the subject is not attending;” Examiner notes this is an example of adjusting the visual attributes associated with the experience and task).
Ghajar does not disclose a determination of the context and scene understanding. However, Horowitz discloses determining a context of the experience and the task based on sensor data of an environment, including generating, from image data, a scene understanding of the environment (fig. 5: steps 506, 508, 510; par. 0017: “the system may determine which objects at the location are receiving attention from the users. The system then generates an attention map based on the attention information;” par. 0023: the sensor(s) obtaining data may be “client devices 110, 120, 130, and 140 may include wearable computing devices, including any hands-free devices. For example, in some implementations, one or more client devices may include devices that operate with a head-mounted camera, head-mounted eye tracking device, and/or head-mounted display;” par. 0017: “a system collects photos associated with objects in at least one location, where one or more of the photos are captured using one or more respective devices that are operable to track a gaze of a user. The system then collects attention information associated with one or more of the objects;” Examiner notes an understanding of the environment—such as objects to pay attention to—is generated from image data);
determining that the user has a first attentive state during the experience based on the physiological data, the context of the experience and task, and the scene understanding of the environment (fig. 6: every spot on the content is given an attention value score; therefore, Examiner notes that the first attentive state can be determined when a given attention value is low or nonexistent; fig. 5: attention values are determined based on received gaze/physiological data and the associated context and scene understanding).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the determination of a context of the experience of Horowitz with the attention determination method of Ghajar in order to most effectively utilize the obtained data, such as by creating an attention map (Horowitz, abstract; par. 0006).
Ghajar also does not disclose that the presentation consists of extended reality content which is what gets modified. However, Karmarkar discloses providing, during a presentation of an extended reality (XR) environment (par. 0046: “Display 710 may be included in the augmented-reality glasses 702 optical systems;” par. 0047: “wearer of augmented-reality glasses 702 may simultaneously observe from display 710 a real-world image with an overlaid displayed image”), a feedback mechanism that modifies XR content during the experience based on determining that the user has the first attentive state during the experience (par. 0047: “Augmented reality glasses 702 may also include eye-tracking system(s) that may be integrated into the display 710 of each lens. Eye-tracking system(s) may include eye-tracking module 708 to manage eye-tracking operations;” par. 0021: “system, method, and article of modifying text content presentation settings as determined by user states based on user eye metric data;” par. 0050: “provide augmented-reality message to user indicating mental fatigue level and/or request user review a particular portion of a task, etc. In one example, the system can replace a text of a particular reading level with a text about the same content but at a lower reading level”), wherein modifying XR content comprises adjusting visual or auditory attributes associated with the experience and the associated task (par. 0030: “display attributes of the reading content can be modified to compensate for mental fatigue attributes. For example, when a higher than normal level of mental fatigue is detected for a user, the size of text of the content can be increased, display contrast can be modified to make the text easier to view, audio volume can be increased, audio replay can be slowed, etc.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the XR content of Karmarkar with the attention determination method of Ghajar because presenting content in extended reality is well-known to potentially be more enjoyable for the user (Karmarkar, fig. 7) and/or to provide a single integrated device which may both provide the content and track the eye data (Karmarkar, fig. 7; par. 0047).
Regarding Claim 2, Ghajar further discloses determining that the user has a second attentive state during a portion of the experience based on the physiological data (pars. 0013-0014: “a determination of a user's level of attention… one or more of the subject's eye positions is measured. The method further includes continuously performing the operations of generating a variability metric using the one or more measured eye positions and comparing the variability metric with a predetermined baseline to determine an attention state of the subject”), the second attentive state corresponding to attention by the user in the task during the portion of the experience (par. 0019: “an attention state of the subject is categorized based on the comparison, for example, to determine that the subject's attention state is an attending state (e.g., the subject is paying attention)”).
Regarding Claim 3, Ghajar further discloses presenting the feedback mechanism is based on determining that the second attentive state differs from the first attentive state (par. 0016: “The method further includes detecting a change in the attention state of the subject. In accordance with the detected change in the attention state of the subject, the subject is presented with second content;” par. 0032: “can optionally include a tactile output device controlled by feedback control module 228 which, for example, produces a vibration on the subject's finger when the subject's attention starts to drift (e.g., when the user's attention state changes from the attending state to the not-attending state)”).
Regarding Claim 4, Ghajar further discloses determining that the user has the first attentive state comprises determining a level of attentiveness (par. 0013: “a determination of a user's level of attention;” par. 0032: “an "attention meter" displayed on the display or a "traffic light" with red, yellow, and green lights, where red indicates that the subject is not attending”).
Regarding Claim 7, Ghajar modified by Horowitz further discloses the context comprises an object upon which the user's attention should be focused during the experience (Horowitz, par. 0017: “the system may determine which objects at the location are receiving attention from the users. The system then generates an attention map based on the attention information;” fig. 5: steps 502, 504, 506).
Regarding Claim 8, Ghajar modified by Horowitz further discloses determining context comprises determining an attention map identifying portions of an image upon which attention is focused when attentive to the task (par. 0017: “the system may determine which objects at the location are receiving attention from the users. The system then generates an attention map based on the attention information;” fig. 5: steps 506, 508, 510; fig. 6).
Regarding Claim 10, Ghajar further discloses providing the feedback mechanism comprises providing a graphical indicator or sound configured to change the first attentive state to a second attentive state corresponding to attention by the user in the task during the experience (par. 0032: “Feedback module 228, if included, generates feedback signals for presentation to the subject via display 104 or feedback devices 208. For example, display 104 may… present an indication of an attention state of the subject, such as an "attention meter" displayed on the display or a "traffic light" with red, yellow, and green lights, where red indicates that the subject is not attending, yellow indicates that the subject is barely attending, and green indicates that the subject is attending”).
Regarding Claim 13, Ghajar further discloses adjusting content corresponding to the experience based on the first attentive state (par. 0016: “In accordance with the detected change in the attention state of the subject, the subject is presented with second content;” par. 0071: “Upon detection of a change in the attention state, the system modifies (532) the presentation of the content. In some implementations, presentation of the content is modified (534) in accordance with both the change in attention state (532) and the continuously generated locus of attention. In some implementations, presentation of the content is modified (536) in accordance with both the change in attention state (532) and the trajectory of the locus of attention”).
Regarding Claim 14, Ghajar further discloses the physiological data comprises: an image of an eye or electrooculography (EOG) data; or a gaze characteristic (par. 0021: “in accordance additional determinations, such as a locus of the subject's attention (e.g., a locus of the subject's gaze);” par. 0023: “eye positions (and, in some embodiments, eye movements) of subject 102 are recorded”).
Regarding Claim 15, Ghajar further discloses the experience is observing educational content, observing entertainment content, receiving instruction regarding a new skill, reading a document, or participating in a meditation session (par. 0021: “content is delivered… Considerer an example of a pre-recorded lecture;” par. 0065: “For example, the content may include a television program;” fig. 3B: content is a document to be read).
Regarding Claim 16, Ghajar further discloses the experience is presented to the user on a display (fig. 1: display 106; par. 0013: “the visual content displayed on the screen”).
Regarding Claim 17, Ghajar modified by Karmarkar further discloses the experience is presented to the user on a head-mounted device (Karmarkar, par. 0045: “this example embodiment is provided in an eyeglasses format, it will be understood that wearable systems may take other forms, such as hats, goggles, masks, headbands and helmets. Augmented-reality glasses 702 may include an optical head mounted display (OHMD)”).
Regarding Claim 18, Ghajar further discloses the physiological data is generated based on an image of an eye captured by an eye tracking camera on a head mounted device (fig. 1: digital video camera(s) 104 on head equipment 108; par. 0023: “One or more digital video cameras 104 are focused on subject 102's eyes so that eye positions (and, in some embodiments, eye movements) of subject 102 are recorded… digital video cameras 104 are mounted on subject 102's head by head equipment 108”).
Regarding Claim 25, Ghajar further discloses obtaining, within a predetermined time following providing the feedback mechanism, physiological data associated with a physiological response of the user to the feedback mechanism (par. 0067: “the system continuously compares (522) the variability metric with a predetermined baseline to determine an attention state of the subject;” Examiner notes that the physiological data is obtained continuously and therefore would also occur instantly after providing the feedback mechanism); and
updating the attentive state based on the physiological response (par. 0067: “the system continuously compares (522) the variability metric with a predetermined baseline to determine an attention state of the subject… the attention state is (524) one of an about-to-attend state, an attending state, or a not-attending state;” par. 0099: “the detected change in the attention state is used to update a profile of the subject;” Examiner notes that the attention state is constantly monitored and evaluated, and would therefore update accordingly).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Ghajar in view of Horowitz and Karmarkar as applied to claim 1 above, and further in view Li.
Regarding Claim 5, modified Ghajar does not disclose a machine learning model. However, Li discloses determining that the user has a first attentive state comprises using a machine learning model (abstract: “inputting the brain reaction parameter value into a machine learning model; outputting the recognition result of the emotion recognition by the machine learning model. In the embodiment of the invention, after collecting the brain reaction data (mainly eye reaction data);” p. 2: “the emotion recognition type comprises at least one of emotion type and cognition type; wherein: … the cognitive type comprises attention”), the machine learning model trained using ground truth data comprising self-assessments in which users labelled portions of experiences with attentive state labels (p. 2: “the machine learning model is trained by using the training sample with label; wherein the training sample comprises brain response parameters from a healthy individual or a patient individual; the label comprises… [an] attention state information label and pressure state information label”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the machine learning model of Li with the attention determination method of modified Ghajar in order to automate the determination of the user’s attentive state using a model which has been trained on verified user data (Li, abstract; p. 2).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Ghajar in view of Horowitz and Karmarkar as applied to claim 1 above, and further in view of Cao.
Regarding Claim 9, modified Ghajar does not disclose transition data. However, Cao discloses the attention map further comprises transition data identifying a number of transitions of the user changing focus (figs. 5, 7; par. 0086: “To understand and model user behaviors when viewing web designs, eye-tracking data of different users on a web design corpus consisting of different genres (e.g., shopping, game, social media, etc.) under different viewing conditions (i.e., task-oriented and free-viewing conditions) 312 is collected and analyzed. Based on the collected data, it is possible to build an attention transition model 310T;” examiner notes that tracking the transition data described would inherently include the number of times each transition happens) from: a first portion of the image upon which attention is focused, to a second portion of the image upon which attention is not focused, to a third portion of the image upon which attention is focused (fig. 5: user’s gaze is tracked as it transitions across various portions of the content; par. 0113: “the eye gaze path predicted from a user saliency map was compared against the actual attention transition;” Examiner notes that the collected transition data is directly compared to an attention/saliency map, and therefore the tracked transitions will include transitions from focused portions to non-focused portions and back to focused portions).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the attention transition data of Cao with the attention determination method of modified Ghajar in order to better design the content (Cao, par. 0145) or to better predict where a user is most likely to focus next at any given moment (Cao, par. 0071).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Ghajar in view of Horowitz and Karmarkar as applied to claim 1 above, and further in view of Devaraj.
Regarding Claim 11, modified Ghajar does not disclose rewinding or providing a break as the feedback mechanism. However, Devaraj discloses providing the feedback mechanism comprises providing a mechanism for rewinding or providing a break from content associated with the task (par. 0045: “Once the set top box has determined the level of engagement of the user 201 with the program 206, then it can be programmed to make a determination regarding what to do in the event of changes in the user's activity during the program. If the engagement has been previously measured as very high…then it can automatically pause or even rewind or store the program while the user is interrupted”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the specific features of the feedback mechanism of Devaraj with the feedback mechanism used in the method of Ghajar because the ability to rewind or pause content is a well-known and inherently useful feedback mechanism, especially within the context of a user not paying attention to the content (Devaraj, abstract; par. 0045).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Ghajar in view of Horowitz and Karmarkar as applied to claim 1 above, and further in view of Boccanfuso.
Regarding Claim 12, modified Ghajar does not disclose suggesting another time as the feedback mechanism. However, Boccanfuso discloses providing the feedback mechanism comprises suggesting a time for another experience based on the first attentive state (pars. 0061-0062: “Once the child has been redirected 3 times during a single lesson, the robot will suggest a break… Attentional tracking data may be collected to research individual variations in attention and performance. These data will contribute to an even more personalized approach for the robot. To this end, machine learning techniques are employed to learn typical attentional behavior for each individual and determine the optimal schedule for deploying redirection and break behaviors;” Examiner notes the feedback is suggesting another experience begin after a predetermined length of time in which the user takes a break, with this suggestion being based on individualized attention data from previous sessions).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIE DOSHER whose telephone number is (571) 272-4842. The examiner can normally be reached Monday - Friday, 10 a.m. - 6 p.m. ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dmitry Suhol can be reached at (571) 272-4430. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.G.D./Examiner, Art Unit 3715
/DMITRY SUHOL/Supervisory Patent Examiner, Art Unit 3715
Prosecution Insights