DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the amendment filed on 01/26/26.
Claims 1, 3, 6, 11-13, 16 have been amended and are hereby entered.
Claims 7, 14, 15 were previously canceled.
Claims 1-6, 8-13, 16-20 are currently pending and have been examined.
This action is made final.
Priority Date/Continuity
Status of this application as a 371 of PCT/US2021/029853 is acknowledged. Accordingly, a priority date of 04/29/2021 has been given to this application.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8-13, 16-20 are rejected under 35 U.S.C.101 because the claimed invention is directed to a judicial exception (an abstract idea) without significantly more.
Step 1
Claims 1-6, 7-9, 18-20 are drawn to a method, Claims 10-13 are drawn to a head-mounted display, and Claims 16-17 are drawn to a non-transitory computer-readable storage medium, each of which are within the four statutory categories. Claims 1-6, 8-13, 16-20 are further directed to an abstract idea on the grounds set out in detail below.
Step 2A Prong 1
Claim 1 recites implementing the steps of:
processing physiological signals to calculate a first parameter that is a predicted value of a current mental state characteristic of the user, and
processing the physiological signals to calculate a second parameter that is an uncertainty quantification for the predicted value.
These steps amount to managing personal behavior or relationships or interactions
between people and therefore recite certain methods of organizing human activity. Processing physiological signs to calculate a predicted value of a user’s current mental state characteristic (“first parameter”) and to calculate an uncertainty quantification of the predicted value are personal behavior that may be performed by a healthcare provider.
Claim 10 recites implementing the steps of:
calculate by processing physiological signals:
a first parameter corresponding to a predicted value of a current mental state characteristic of the user, and
a second parameter corresponding to an uncertainty quantification for the predicted value.
These steps amount to managing personal behavior or relationships or interactions
between people and therefore recite certain methods of organizing human activity. Processing physiological signals to calculate a first parameter corresponding to a predicted value of a user’s current mental state characteristic and a second parameter corresponding to an uncertainty quantification of the predicted value is a personal behavior that may be performed by a healthcare provider.
Claim 16 recites implementing the steps of:
segmenting each physiological signal over time using a sliding window to generate a plurality of signal segments;
extracting a set of features from each of the signal segments, wherein a feature includes at least one of a pupil diameter, a blink, a saccade, a fixation, a heart rate statistic, a heart rate variability, a respiration rate, or a power spectral density of a photoplethysmography (PPG) sensor
generating a learned representation for each physiological signal based on the corresponding extracted features;
fusing the learned representations into a fused multimodal representation; and
calculating: (i) a predicted value corresponding to a cognitive load experienced by the user, and (ii) an uncertainty quantification associated with the predicted value using the fused multimodal representation.
These steps amount to managing personal behavior or relationships or interactions
between people and therefore recite certain methods of organizing human activity. Segmenting signal data, extracting features, generating a learned representation, fusing the representation to generate a multimodal representation to calculate a first parameter corresponding to a predicted value of a user’s cognitive load and a second parameter corresponding to an uncertainty quantification are personal behaviors that may be performed by a healthcare provider.
Step 2A Prong 2
This judicial exception is not integrated into a practical application because the additional
elements within the claims only amount to:
A. Instructions to Implement the Judicial Exception. MPEP 2106.05(f)
The independent claims additionally recite:
a processor integrated into the wearable device as implementing the steps of the abstract idea (Claim 1 and Claim 10)
a non-transitory computer-readable storage medium storing instructions that, when executed by a processor as implementing the steps of the abstract idea (Claim 16)
a trained machine learning model as implementing the step of calculating: (i) a predicted value corresponding to a cognitive load experienced by the user, and(ii) an uncertainty quantification associated with the predicted value using the fused multimodal representation (Claim 16)
The broad recitation of general purpose computing elements at a high level of generality only amounts to mere instructions to implement the abstract idea using computing components as tools.
Regarding the processor of the wearable device, para. [0014] discloses “Processor 102 includes a central processing unit (CPU) or another suitable processor”. No further details of the processor are disclosed. Regarding the wearable display, para. [0031] discloses “In an example, wearable device 100 is a VR or AR headset or other head mounted display (HMD) device”; para. [0031] broadly discloses, “The wearable device may be a head mounted display, and the sensors may be multi-modal and may sense a plurality of different types of physiological measures of the user of the head mounted display”. Therefore, these elements are given their broadest reasonable interpretation as general purpose computing elements functioning in their ordinary capacity to implement the steps of the abstract idea.
Regarding the non-transitory CRM, para. [0014] discloses “Memory 104 includes any suitable combination of volatile and/or non-volatile memory, such as combinations of Random Access Memory (RAM), Read-Only Memory (ROM), flash memory, and/or other suitable memory. These are examples of non-transitory computer readable storage media. The memory 104 is non-transitory in the sense that it does not encompass a transitory signal but instead is made up of at least one memory component to store machine executable instructions for performing techniques described herein”. Therefore, this element is given its broadest reasonable interpretation as a general purpose computing element functioning in its ordinary capacity to implement the steps of the abstract idea.
Regarding the trained machine learning model, para. [0015] discloses “Memory 104 stores application module 106 and inference engine module 108. Processor 102 executes instructions of modules 106 and 108 to perform some techniques described herein…In an example, inference engine module 108 infers high-level insights about a user of device 100, such as cognitive load, emotion, stress, engagement, and health conditions, based on lower-level sensor data, such as that measured by physiological sensors 122. In an example, inference engine module 108 is based on a machine learning model that is trained with a training set of data to be able to predict a current cognitive load of a user along with an uncertainty quantification for that prediction. It is noted that some or all of the functionality of modules 106 and 108 may be implemented using cloud computing resources”. No particulars of the machine learning model appear to be disclosed. Therefore, the broad recitation of a trained machine learning model, in this case, to process physiological data and calculate a predicted value of cognitive load and an uncertainty quantification of the predicted value, only amounts to using the machine learning model as a tool to apply data to a model and generate a result (see MPEP 2106.05(f)(2)).
B. Insignificant Extra-Solution Activity. MPEP 2106.05(g)
Claim 1 also recites:
capturing, by a plurality of sensors integrated into a wearable device, a plurality of physiological signals indicative of a plurality of physiological characteristic of a user of the wearable device
Claim 10 also recites:
a display device integrated into the head mounted display, the display device is to display images to a user of the head mounted display;
multi-modal sensors integrated into the head mounted display, the sensors are to capture physiological signals of the user
Claim 16 also recites:
collect, from a plurality of physiological sensors integrated into the wearable device, a set of multi-modal physiological signals from a user
The steps pertaining to capturing, by a sensor integrated into a wearable device, a physiological signal indicative of a physiological characteristic of a user (Claim 1), multi-modal sensors integrated into the head mounted display, the sensors are to capture physiological signals of the user (Claim 10), and collect, from a plurality of physiological sensors integrated into the wearable device, a set of multi-modal physiological signals from a user (Claim 14) only amount to insignificant extra-solution in the form of mere data gathering.
The element of a display device integrated into the head mounted display, the display device is to display images to a user of the head mounted display only amounts to insignificant extra-solution activity. As explained above, Claim 10 is directed to an abstract idea in the form of processing physiological signals to calculate a first parameter corresponding to a predicted value of a user’s current mental state characteristic and a second parameter corresponding to an uncertainty quantification of the predicted value. As stated in MPEP 2106.05(g), "[t]he term "extra-solution activity" can be understood as activities incidental to the primary process or product that are merely a nominal or tangential addition to the claim." In the present claim, the element of a display device to display images to a user of the head mounted display is only nominally or tangentially related to the process of processing physiological signals to calculate a first parameter corresponding to a predicted value of a user’s current mental state characteristic and a second parameter corresponding to an uncertainty quantification of the predicted value, and accordingly constitutes insignificant extra-solution activity.
These elements are therefore not sufficient to integrate the abstract idea into a practical application. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually.
The above claims, as a whole, are therefore directed to an abstract idea.
Step 2B
The present claims do not include additional elements that are sufficient to amount to
more than the abstract idea because the additional elements or combination of elements amount to no more than a recitation of:
A. Instructions to Implement the Judicial Exception. MPEP 2106.05(f)
As explained above, claims 1, 10, and 16 only recite the aforementioned computing elements as tools for performing the steps of the abstract idea, and mere instructions to perform the abstract idea using a computer is not sufficient to amount to significantly more than the abstract idea. MPEP 2106.05(f).
B. Insignificant Extra-Solution Activity. MPEP 2106.05(g)
Likewise, as explained above, element of a display device to display images to a user of the head mounted display, generating a plurality of physiological measurements, and collecting multi-modal signals only amounts to insignificant extra-solution activity.
C. Well-Understood, Routine and Conventional Activities. MPEP 2106.0S(d)
In addition to amounting to insignificant extra-solution activity, the elements in Section B above constitute well-understood, routine and conventional activity. These have been re-evaluated under the “significantly more” analysis and determined to be well-understood, routine, conventional activity in the field.
As evidenced by the prior art of record, a display device integrated into the head mounted display, the display device is to display images to a user of the head mounted display is a well-understood, routine, and conventional element in the field of computerized healthcare (see Gerber et. al., US Publication 20110040547 at Para. [0074]; see Stone et. al., US Publication 20070203545 at Para. [0157]; See Border, US Publication 20160015470 A1 at Para. [0350]).
As evidenced by the prior art of record, capturing, by a sensor integrated into a wearable device, a physiological signal indicative of a physiological characteristic of a user, and collecting from a plurality of physiological sensors integrated into the wearable device, a set of multi-modal physiological signals from a user are well-understood, routine, and conventional activities in the field of computerized healthcare (see Berman et. al., US Publication 20130245396 at Paras. [0015], [0020], [0031]; see Luna et. al., US Publication 20140128754 at Para. [0002], [0018]; See Ein-Gil et. al., US Publication 20190045020 at Paras. [0018]-[0020]).
Well-understood, routine, conventional activity cannot provide an inventive concept (“significantly more”). As such the claim is not patent eligible.
Thus, taken alone, the additional elements do not amount to significantly more than the
above-identified judicial exception. Looking at the limitations as an ordered combination adds
nothing that is not already present when looking at the elements taken individually. Their
collective functions merely provide conventional computer implementation.
Depending Claims
Dependent claims recite additional subject matter which further narrows or defines the abstract idea embodied in the claims. For example, Claims 2, 4-6, 11-13, 17 recite limitations which further narrow the scope of the independent claims. Claims 3, 8-9, 18-20 further recites limitations that are certain methods of organizing human activity as described below:
Claim 3 recites limitations pertaining to generating a parametric distribution, based on sensed physiological characteristics; determining the first parameter from the parametric distribution; and determining the second parameter from the parametric distribution, which are also certain methods of organizing human activity including managing personal behaviors, as a healthcare provider could use received data to generate a parametric distribution of the data and determine first and second parameters such as a predicted value and uncertainty quantification from the distribution. Claim 3 also recites “the processor integrated into the wearable device” as implementing the steps of the abstract idea. As discussed above with respect to independent claims, this amounts to mere instructions to apply the abstract idea. MPEP 2106.05(f). This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 8 recites limitations pertaining to wherein the inference engine is based on a trained machine learning model, wherein the method further comprises training the machine learning model, and wherein the training comprises: generating a plurality of physiological measures of each of a plurality of test set users of wearable devices while the test set users perform tasks of varying difficulty; receiving, from each of the test set users for each of the tasks, a subjective rating of the mental state characteristic experienced during that task; and performing a regression analysis based on the physiological measures and the subjective ratings to maximize a likelihood that a trained probabilistic model fits in a distribution of target mental state characteristic values, which is also directed to an abstract idea in the form of certain methods of organizing human activity and/or mathematical algorithms. Examiner notes that, in light of the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, the claims recite certain methods of organizing human activity, and are not subject matter eligible. The use of a computer to train a model, utilizing the training embodiments offered in the instant specification (see at least [0021]-[0027]) amount to applying data to an algorithm and reporting the results (MPEP § 2106.05(f)(2), see case involving a commonplace business method or mathematical algorithm being applied on a general purpose computer within the “Other examples.. i.”) amounting to instruction to implement the abstract idea using a general purpose computer. Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 134 S. Ct. 2347, 1357 (2014) consistent with Example 47 claim 2. The techniques outlined are mathematical algorithms or certain methods of organizing human activity of labeling and fitting data to a particular model representation. Regarding the inference engine, para. [0015] discloses “Memory 104 stores application module 106 and inference engine module 108. Processor 102 executes instructions of modules 106 and 108 to perform some techniques described herein…In an example, inference engine module 108 infers high-level insights about a user of device 100, such as cognitive load, emotion, stress, engagement, and health conditions, based on lower-level sensor data, such as that measured by physiological sensors 122. In an example, inference engine module 108 is based on a machine learning model that is trained with a training set of data to be able to predict a current cognitive load of a user along with an uncertainty quantification for that prediction. It is noted that some or all of the functionality of modules 106 and 108 may be implemented using cloud computing resources”; para. [0020] discloses “inference engine 200 according to an example. In an example, inference engine module 108 (FIG. 1 ) is implemented with inference engine 200. Inference engine 200 includes a plurality of feature generation modules 204(1)-204(2) (collectively referred to as feature generation modules 204), a fusion model module 210, and a prediction module 214”. The broad recitation of an “inference engine”, which is understood to encompass a machine learning model per specification, in this case, to process physiological data and generate a distribution, only amounts to using the machine learning model as a tool to apply data to a model and generate a result (see MPEP 2106.05(f)(2)).
Claim 9 recites limitations pertaining to wherein the regression analysis comprises: generating a predetermined number of training probabilistic distributions for a given data input, wherein each of the training probabilistic distributions includes an associated weight; and calculating a loss function in a winner takes all manner using the training probabilistic distribution with a highest value for its associated weight, which is also directed to an abstract idea in the form of certain methods of organizing human activity including managing personal behavior, as performing data calculations/analysis are behaviors that may be performed by a healthcare provider or data analyst.
Claim 18 recites limitations pertaining to generating a plurality of signal segments by applying a sliding window over time to the physiological signal, which is also certain methods of organizing human activity as a healthcare provider could perform a data analysis by using a sliding window over time to generate a plurality of signal segments from a physiological signal. Claim 18 also recites “the processor” as implementing the steps of the abstract idea. As discussed above with respect to independent claims, this amounts to mere instructions to apply the abstract idea. MPEP 2106.05(f). This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 19 recites limitations pertaining to generating, upon extracting a set of features from the signal segments, a learned representation of the physiological signal based on the extracted features, which is also certain methods of organizing human activity as a healthcare provider could perform a data analysis by generating a learned representation of physiological signals based on extracted features. Claim 19 also recites “the processor” as implementing the steps of the abstract idea. As discussed above with respect to independent claims, this amounts to mere instructions to apply the abstract idea. MPEP 2106.05(f). This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Claim 20 recites limitations pertaining to forming a fused representation by fusing the learned representation with one or more additional learned representations corresponding to other physiological signals, and calculating, based on the fused representation, the predicted value and the uncertainty quantification, which is also certain methods of organizing human activity as a healthcare provider could perform a data analysis by fusing learned representations with additional representations corresponding to other physiological signals and calculating the predicted value and uncertainty quantification based on the fused representation. Claim 20 also recites “the processor” as implementing the steps of the abstract idea. As discussed above with respect to independent claims, this amounts to mere instructions to apply the abstract idea. MPEP 2106.05(f). This is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
The dependent claims have been given the full two-part analysis including analyzing the additional limitations both individually and in combination. The dependent claims, when analyzed individually, and in combination, are also held to be patent ineligible under 35 U.S.C. 101 as they include all of the limitations of claim 1, 10 or 16 respectively. The additional recited limitations of dependent claims 2-6, 8-9, 11-13, 17-20 fail to establish that the claims do not recite an abstract idea because the additional recited limitations of the dependent claims merely further narrow the abstract idea. Beyond the limitations which recite the abstract idea, claims 2-6, 8-9, 11-13, 17-20 recite additional elements consistent with those identified above with respect to the independent claims which encompass adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claims 2-6, 8-9, 11-13, 17-20 when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitation(s) fail(s) to establish that the claim(s) is/are not directed to an abstract idea without significantly more. These claims fail to remedy the deficiencies of their parent claims above, and are therefore rejected for at least the same rationale as applied to their parent claims above, and incorporated herein.
For the reasons stated, Claims 1-6, 8-13, 16-20 fail the Subject Matter Eligibility Test and are consequently rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Das et. al. (US Publication 20190175091A1) in view of Baker (US Publication 20090287070A1).
Regarding Claim 1, Das discloses capturing, by a plurality of sensors integrated into a wearable device, a plurality of physiological signals indicative of a plurality of physiological characteristics of a user of the wearable device ([0043] teaches on using a plurality of non-invasive physiological signals from physiological sensors embodied in at least one wearable device; signals include EEG, PPG, GSR – interpreted as physiological signals indicative of physiological characteristics of the user of the wearable device); processing, by a processor integrated into the wearable device, the physiological signals to calculate a first parameter that is a predicted value of a current mental state characteristic of the user ([0044] teaches on the system developing a classification model to differentiate between high and low stress experienced by an individual from the plurality of non-invasive signals; [0051] teaches on extracting all the feature sets related to each of the particular signal modalities from each of the physiological signals; [0052] teaches on the plurality of feature sets identified from the physiological signals correlating with the cognitive stress of the user while engaging in performance of a task; the system predicts a “stress indicator metric” comprising a quantitative estimate of the cognitive stress experienced during performance of the task – interpreted as a “predicted value” of a current mental state characteristic).
Das does not explicitly teach, but Baker, which is directed to estimation of a physiological parameter, teaches:
processing, by the processor integrated into the device, the physiological signals to calculate a second parameter that is an uncertainty quantification for the predicted value ([0007] teaches on a microprocessor capable of calculating an estimated value of oxygen saturation of a patient’s blood based on information received from a sensor (interpreted as “physiological signal”); [0057] teaches on determining an accuracy of an estimated value of oxygen saturation (a physiological value) may be determined by calculating a standard deviation (“uncertainty quantification” – interpreted as the “second parameter” as it is based on the first (estimated) value (parameter)) from which the estimated value is determined).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Das with these teachings of Baker, for the wearable device of Das to additionally use the physiological signals to calculate an uncertainty quantification (e.g., standard deviation) of the predicted cognitive load value, with the motivation of using standard deviation as a metric to represent the accuracy of the predicted (estimated) value (Baker [0057]).
Regarding Claim 2, Das/Baker teach the limitations of Claim 1. Das further discloses wherein the current mental state characteristic is a current cognitive load of the user ([0027] teaches on classifying and quantifying “cognitive stress” of a user, which is interpreted as synonymous with “cognitive load”).
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Das et. al. (US Publication 20190175091A1) in view of Baker (US Publication 20090287070A1) as applied to Claim 1 above, and further in view of Choudhury et. al. (“Distribution of data” article).
Regarding Claim 3, Das/Baker teach the limitations of Claim 1 but do not teach the following. Choudhury, which is directed to statistical analysis of data, teaches: further comprising: generating, using the processor integrated into the wearable device, a parametric distribution, based on sensed physiological characteristics page 278, Col 1, “Distribution of data” first para. teaches on quantitative data being distributed in a manner defined as normal or Gaussian distribution; Examiner submits that both “normal” and “Gaussian” distribution read on the broadest reasonable interpretation of parametric distribution; determining, by the processor integrated into the wearable device, the first parameter from the parametric distribution (Page 278 Col 2, “Normal distribution” teaches on the mean and standard deviation of normally distributed data; see as an Example, page 278 Col 2 onto page 273 Col 1 teaches on heart rate data for a population where the mean is 140 bpm and SD is 20 bpm; Normal distribution of Fig. 3 is generated showing the percentage that are 1, 2, 3 standard deviations away from the mean); and determining, by the processor integrated into the wearable device, the second parameter from the parametric distribution (Page 278 Col 2, “Normal distribution” teaches on the mean and standard deviation of normally distributed data; see as an Example, page 278 Col 2 onto page 273 Col 1 teaches on heart rate data for a population where the mean is 140 bpm and SD is 20 bpm; Normal distribution of Fig. 3 is generated showing the percentage that are 1, 2, 3 standard deviations away from the mean).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Das/Baker with these teachings of Choudhury to generate a parametric distribution using the generated first and second parameter of Das/Baker, because normal/Gaussian distributions (e.g., parametric distributions) can be used with a large number of observations (e.g., data points) and indicate how the data is dispersed around the mean – e.g., a normal (“bell shaped”) curve (Choudhury, page 278, Col 1 Introduction and Distribution of data paras).
Regarding Claim 4, Das/Baker/Choudhury teach the limitations of claim 3. Choudhury further teaches wherein the parametric distribution is a Gaussian distribution (page 278, Col 1, “Distribution of data” first para. teaches on quantitative data being distributed in a manner defined as normal or Gaussian distribution), the first parameter is a mean value for the Gaussian distribution (Page 278 Col 2, “Normal distribution” teaches on the mean and standard deviation of normally distributed data; see as an Example, page 278 Col 2 onto page 273 Col 1 teaches on heart rate data for a population where the mean is 140 bpm; Normal distribution of Fig. 3 is generated showing the percentage that are 1, 2, 3 standard deviations away from the mean; per Abstract, “normal” is interpreted as a Gaussian distribution) and the second parameter is a standard deviation for the Gaussian distribution (Page 278 Col 2, “Normal distribution” teaches on the mean and standard deviation of normally distributed data; see as an Example, page 278 Col 2 onto page 273 Col 1 teaches on heart rate data for a population where SD is 20 bpm; Normal distribution of Fig. 3 is generated showing the percentage that are 1, 2, 3 standard deviations away from the mean; per Abstract, “normal” is interpreted as a Gaussian distribution).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to further modify the combined teachings of Das/Baker/Choudhury with these teachings of Choudhury to utilize a Gaussian distribution, because Gaussian distributions can be used with a large number of observations (e.g., data points) and indicate how the data is dispersed around the mean – e.g., a normal (“bell shaped”) curve (Choudhury, page 278, Col 1 Introduction and Distribution of data paras).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Das et. al. (US Publication 20190175091A1) in view of Baker (US Publication 20090287070A1) as applied to Claim 1 above, and further in view of, and further in view of Tadi et. al. (US Publication 20160235323).
Regarding Claim 5, Das/Baker teach the limitations of Claim 1 but do not disclose the following. Tadi, which is directed to a physiological parameter measurement and feedback system, teaches: wherein the wearable device is a head mounted display (see Fig. 4a; [0101] teaches on the display unit being attached to display unit support, which supports the display unit on the user and extends proximate from the eyes and around the head of the user and is in the form of goggles), and wherein the sensor senses a physiological measures of the user of the head mounted display ([0089] teaches on the physiological parameter sensing system comprising one or more sensors to measure a physiological parameter of the user, including EEG, [0093] teaches on the sensors of Fig. 4a being configured to measure electrical potential due to eye movement; [0094], sensors may also comprise one or more of electrocorticogram (ECOG); electrocardiogram (ECG); galvanic skin response (GSR) sensor; respiration sensor; pulse-oximetry sensor; temperature sensor; single unit and multi-unit recording chips for measuring neuron response using a microelectrode system.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Das/Baker with these teachings of Tadi, to use a head-mounted display comprising sensors which sense physiological measures of the user of the head mounted display, with the motivation of using a head-mounted display to display information to the wearer ([0037]) and collecting eye-related data via the head-mounted sensors (Tadi [0108],[0118]).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Das et. al. (US Publication 20190175091) in view of Baker (US Publication 20090287070A1) as applied to Claim 1 above, and further in view of Cho et. al. (US Publication 20160021302 A1).
Regarding Claim 6, Das/Baker teach the limitations of Claim 1. Das further discloses wherein the plurality of sensors include ([0042] teaches on sensor data including PPG signals); and wherein the physiological measure characteristics comprises at least one of pupillometry information, eye movement information, and heart activity information ([0043] teaches on using PPG signals to analyze variations in heart rate and other cardiac parameters of the user; per claim construction “at least one of”, the claim requirements are fulfilled by heart activity information as taught by Das).
Das does not disclose, but Cho which is directed to a cognitive sensor, discloses:
the plurality of sensors include a first sensor to track a user's pupillometry, a second sensor to track eye movement of a user ([0088] teaches on a first sensor which may be an “eye tracking sensor” (interpreted as a first sensor which tracks eye movement of a user) and a second sensor which may be configured to detect movements of pupils (interpreted as a different sensor that tracks pupillometry)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Das/Baker with these teachings of Cho to incorporate additional sensors for eye movement and pupillometry, because eye-related data provides a metric that can identify cognitive processing of a user when the user is performing a task (Cho [0003]).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Das et. al. (US Publication 20190175091A1) in view of Baker (US Publication 20090287070A1) as applied to Claim 1 above, and further in view of Romine et. al. (“Using Machine Learning to Train a Wearable Device for Measuring Students’ Cognitive Load during Problem-Solving Activities Based on Electrodermal Activity, Body Temperature, and Heart Rate: Development of a Cognitive Load Tracker for Both Personal and Classroom Use” article).
Regarding Claim 8, Das/Baker teach the limitations of Claim 1 but do not teach the following. Romine, which is directed to using a wearable device to measure cognitive load of an individual, teaches: wherein the processor applies, upon processing the physiological signal, an inference engine based on a trained machine learning model (page 8, 4.1.1 teaches on experimental design and collection of data to obtain training data; page 9, 4.1.2 teaches on classifying each participant’s self-reported mental focus; the effect of cognitive load on a person’s TEMP, EDA, HR (physiological signals) was evaluated; a logistic regression model was used to understand the relationship between the physiological features (TEMP, EDA, HR and self-reported focus); a main effects model was fitted that included TEMP, EDA, HR as predictors for outcome of focus; page 11, last para., teaches on technological implementation, fitness trackers, smartphone, etc. which is interpreted as synonymous with a processor); wherein the method further comprises training the machine learning model (Page 6, Section 3.1.4 teaches on teaches on using the self-reported measure for mental focus training and testing of classifiers in Study 1; 95% and 90% of data were respectively used for training models as liberal/conservative; page 9, 4.1.2 teaches on classifying each participant’s self-reported mental focus; the effect of cognitive load on a person’s TEMP, EDA, HR (physiological signals) was evaluated; a logistic regression model was used to understand the relationship between the physiological features (TEMP, EDA, HR and self-reported focus); a main effects model was fitted that included TEMP, EDA, HR as predictors for outcome of focus; 90%/95% of data was used for training the models as liberal/conservative, respectively), and wherein the training comprises: generating a plurality of physiological measures of each of a plurality of test set users of the wearable devices while the test set users perform tasks of varying difficulty (page 8, section 4. Study 2 teaches on investigating efficacy of temp, EDA, HR for predicting self-reported mental focus; section 4.1.1. teaches on the participants wearing the E4 for data collection, measurement and transformation of the physiological features; each participant completed similar activities as Study 1 (varying levels of cognitive load)); receiving, from each of the test set users for each of the tasks, a subjective rating of the mental state characteristic experienced during that task (page 8, Section 4.1.4 teaches on Study 2 participants performing the cognitive tasks while wearing the E4 and being asked to rate their level of mental focus after each activity; using Qualtrics survey software, each participant was asked to complete 5 puzzles which were described as ranging in difficulty from “extremely easy” to “extremely difficult”; after completing each puzzle, each participant rated their level of mental focus while working on the puzzle as “not focused, somewhat focused, or very focused”); and performing a regression analysis based on the physiological measures and the subjective ratings to maximize a likelihood that a trained probabilistic model fits in a distribution of target mental state characteristic values (Page 6, Section 3.1.4 teaches on teaches on using the self-reported measure for mental focus training and testing of classifiers in Study 1, which incorporated logistic regression; 95% of data and 90% were used for training models as liberal/conservative; Page 9, Section 4.1.2 teaches on using logistic regression to fit a main effects model using TEMP, EDA, HR as predictors for focus outcome; page 10 section 4.2.2 teaches on the target variable of user-reported focus in which classifiers are identified which perform well or struggle with classification – selecting a model that performs best is interpreted as maximizing a likelihood that a trained probabilistic model fits in a distribution of target mental state values).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Das/Baker with these teachings of Romine incorporate a machine learning model which has been trained on physiological measures of test users while performing tasks of difficulty, with the motivation of training the model so that it provides continually stronger predictions of the individual’s learning activities and cognitive load over time (Romine page 14, last paragraph).
Claim(s) 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Das et. al. (US Publication 20190175091A1) in view of Baker (US Publication 20090287070A1) as applied to Claim 1 above, and further in view of Gjoreski et. al. (“Datasets for Cognitive Load Inference Using Wearable Sensors and Psychological Traits” article )
Regarding Claim 18, Das/Baker teach the limitations of Claim 1 but do not teach the following. Gjoreski, which is directed to cognitive load inference from multimodal sensor data, teaches: wherein processing by the processor comprises: generating a plurality of signal segments by applying a sliding window over time to the physiological signal (page 6, section 3.2, last para. teaches on using the MS band wrist device to assess participants’ physiological response during a cognitive task, including heart rate, RR intervals, GSR, temperature and ACC data; Page 5-6, section 3.1 teaches on participants solving a plurality of cognitive tasks of varying difficulty; Page 10, section 5.1 teaches on re-sampling all the data to a frequency of 1 Hz; the last 30s of each task (interpreted as sliding window) was used to extract features, thus, one segment represents one task; if participants solved a plurality of tasks, and each task has a segment extracted, it is interpreted as generating a plurality of signal segments; 5.1 further teaches on preprocessing the GSR signal using a “sliding mean filter” which is interpreted as a sliding window over the GSR physiological signal).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to further modify the combined teachings of Das/Baker with these teachings of Gjoreski, to generate a plurality of signal segments using a sliding window over time to the physiological signal, with the motivation of extracting statistical features (mean, SD, skewness, etc.) from each input signal (HR, Temp, etc.) for each segment (Gjoreski 5.1), where each segment is representing a particular task.
Regarding Claim 19, Das/Baker/Gjoreski teach the limitations of Claim 18. Das further discloses wherein processing by the processor comprises: generating, upon extracting a set of features from the signal segments, a learned representation of the physiological signal based on the extracted features ([0048] teaches on extracting a feature set individually from each non-invasive physiological signal of the plurality of non-invasive signals to obtain the plurality of feature sets; [0049]-[0050] further teach on feature extraction; [0051] teaches on applying a feature reduction method to extracted feature sets).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Das et. al. (US Publication 20190175091A1) in view of Baker (US Publication 20090287070A1), further in view of Gjoreski et. al. (“Datasets for Cognitive Load Inference Using Wearable Sensors and Psychological Traits” article) as applied to Claim 19 above, and further in view of Romine et. al. (“Using Machine Learning to Train a Wearable Device for Measuring Students’ Cognitive Load during Problem-Solving Activities Based on Electrodermal Activity, Body Temperature, and Heart Rate: Development of a Cognitive Load Tracker for Both Personal and Classroom Use” article) and further
Regarding Claim 20, Das/Baker/Gjoreski teach the limitations of Claim 19 but do not teach the following. Romine, which is directed to using a wearable device to measure cognitive load of an individual, teaches: forming a fused representation by fusing the learned representation with one or more additional learned representations corresponding to other physiological signals page 6, Table 2 shows each individual physiological signal (EDA, HR, Temp) of each individual; the mean and SD are interpreted as being the learned representation based on the extracted feature (e.g., EDA), wherein the processor calculates, based on the fused representation, the predicted value and the uncertainty quantification (page 8, section 4 Study 2, intended to utilize self-reported measures of cognitive load with sensor data to improve model classification performance; page 9 section 4.1.2 teaches on evaluating how cognitive load affects a person’s TEMP, EDA, and HR; a logistic regression model was used to understand relationship between TEMP, EDA, HR (fused multimodal representation) and self-reported focus; a main effects model was fitted using Temp, EDA, HR as predictors for focus outcome, where focus is understood to be a binary value of 1 (focused) and 0 (not focused); Fig. 1 shows the user’s automated logging data of temp, EDA, HR from wearable sensor being compiled by an app with manual labels added, using ML classification on the data and subsequently importing predictions back to the user’s smartphone for display – applying the trained ML model to the fused representation of temp, EDA, HR; per last para. on page 12, charts and tables may be used to represent “level of cognitive load” (interpreted as “value”); per tables 2 and 4 it is understood that a standard deviation (measure of uncertainty) may be calculated).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to modify Das/Baker/Gjoreski with these teachings of Romine, to form fused representation by fusing the learned representation with one or more additional learned representations corresponding to other physiological signals, wherein the processor calculates, based on the fused representation, the predicted value and the uncertainty quantification with the motivation of examining how multiple physiological signals can be used together to classify activities representing varying levels of cognitive load Romine page 4, section 3 study 1).
Subject Matter Not Rejected under 35 USC 102/103
Claim 9 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, as set forth in this Office action and to include all of the limitations of the base claim and any intervening claims. A search of publicly available prior art fails to yield a reference or combination of references that makes the combination of elements in Claim 9 obvious when considered as a whole.
Response to Applicant’s Remarks/Arguments
Please note: When referencing page numbers of Applicant’s response, references are to page numbers as printed.
Rejections under 35 USC 101
Applicant’s remarks have been fully considered but are not persuasive. Regarding remarks at top of page 8 pertaining to Step 2A Prong 1 and improvements to any other technology or technical field, the Examiner respectfully disagrees with Applicant’s position. MPEP 2106.04(d)(1) states that a practical application may be present where the claimed invention improves another technology. See also MPEP 2106.05(a)(II). Applicant’s claimed invention recites the additional element(s) of a processor integrated into a wearable device. While these additional elements implement the steps of the abstract idea, there is no indication that these additional elements operate in a manner different than they normally operate. Regarding the processor of the wearable device, para. [0014] discloses “Processor 102 includes a central processing unit (CPU) or another suitable processor”. No further details of the processor are disclosed. Regarding the wearable device, para. [0031] discloses “In an example, wearable device 100 is a VR or AR headset or other head mounted display (HMD) device”, and para. [0031] broadly discloses, “The wearable device may be a head mounted display”. Examiner submits that in the instant claims, processing physiological signals to calculate first and second parameters does not improve the wearable device itself. The wearable device is operating as it normally operates. Operating another device in the manner it normally operates is insufficient to improve that other technology. As such, these additional elements are not improved through implementation of the abstract idea and a practical application is not present. This argument is not persuasive.
Regarding remarks that underlying detection is improved by “providing an uncertainty quantification for the predicted value”, Examiner respectfully submits that providing an uncertainty qualification falls within the scope of the abstract idea itself; e.g., a human operator can calculate (provide) an uncertainty quantification for a predicted value. As such, any purported improvements from this limitation may be an improvement to the abstract idea but are not technological improvements. Please see MPEP 2106.05(a) which states, “It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements.” Applicant has not provided, nor can Examiner find evidence of, how any of the additional elements identified above in main 101 analysis section are providing an improvement over prior art systems. The additional elements identified above are understood to be computing components functioning in their normal operating capacity, which is not sufficient to integrate the judicial exception into a practical application. Similarly, Examiner submits that any purported improvements referenced with respect to para. [0011] may be improvements to the abstract idea, but are not technological improvements as they are not provided by one or more additional elements. Therefore, this argument is not persuasive.
Regarding remarks at page 9 pertaining to independent Claim 10 and the “head mounted display environment”, please see preceding remarks with respect to this element. It is understood to be a general purpose head mounted display functioning in its ordinary capacity. As shown in Step 2B analysis in 101 analysis above, a display device integrated into a head mounted display is a well-understood, routine, and conventional element in the field of computerized healthcare. This is not sufficient to render the claim subject matter eligible. Regarding remarks at page 9 pertaining to independent Claim 16 and “a specific feature”, Examiner submits that the amendments only serve to further limit the scope of the abstract idea by specifying types of features that may be extracted, e.g., pupil diameter or respiration rate. Further limiting the abstract idea is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the abstract idea. These arguments are not persuasive.
Regarding remarks at page 10 pertaining to dependent Claims 3, 4, 13, 17, Examiner submits that with respect to reciting a parametric or Gaussian distribution, these limitations only serve to further limit the scope of the abstract idea. Further limiting the abstract idea is not sufficient to integrate the judicial exception into a practical application or amount to significantly more than the abstract idea. These arguments are not persuasive.
For all of the above reasons, the rejections of Claims 1-6, 8-13, 16-20 under 35 USC 101 are maintained.
Rejections under 35 USC 103
Applicant’s arguments with respect to claim(s) 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. New grounds of rejection have been necessitated by Applicant’s amendments, specifically, amendment to include “a plurality of sensors” for capturing “a plurality of signals” indicative of “a plurality of physiological characteristics” of a user in the first limitation, and amendment to narrow the claim language of the second and third limitations to recite that the first parameter “is” a predicted value and a second parameter “is” an uncertainty quantification, rather than the first/second parameters broadly “corresponding to” a predicted value/uncertainty quantification as previously recited. Regarding the rejection of dependent Claims 2-6, 8-9, 18-20, the Applicant has not offered any specific arguments with respect to these claims. As such, the rejection of these claims is also maintained.
Applicant’s remarks regarding the rejections of independent Claims 10 and 16 and corresponding dependent claims have been considered and are persuasive. The rejections of Claims 10-13, 16-17 under 35 USC 103 are withdrawn.
Conclusion
Examiner respectfully requests that Applicant provides citations to relevant paragraphs of specification for support for amendments in future correspondence.
The following relevant prior art not cited is made of record:
WIPO Publication WO2017004362A1, teaching on systems and methods for monitoring and improving cognitive flexibility
US Publication 20200029806A1, teaching on eye-tracking for detection of cognitive load of a user
US Publication 20180310851A1, teaching on method and system for pre-processing of an EEG signal for cognitive load measurement
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANNE-MARIE K ALDERSON whose telephone number is (571)272-3370. The examiner can normally be reached on Mon-Fri 9:00am-5:00pm EST, and generally schedules interviews in the timeframe of 2:00-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fonya Long, can be reached on 571-270-5096. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNE-MARIE K ALDERSON/Primary Examiner, Art Unit 3682