DETAILED ACTION
This action is in response to the Remarks dated 25 February 2026. No claims are amended. No claims have been added or cancelled. Claims 1-3, 5-23 and 25-52 remain pending and have been considered below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Suggestion
Examiner suggests amending the independent claims to recite “correlate the plurality of localized signals to a plurality of different capabilities performed by the particular user” and “wherein a combination of the plurality of different capabilities performed by the particular user forms a perceptual experience.” Examiner believes an amendment in this manner would facilitate the advancement of prosecution
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-10, 13-19, 21-23, 25, 26, 31, 33-41, 43, 44, 48-50 and 51 are rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1).
As for independent claim 1, Yeow teaches a method comprising:
obtaining a plurality of localized signals acquired by a measurement device comprising a plurality of sensors positioned to measure brain activity from specific areas of the brain, the plurality of localized signals being acquired from a particular user being measured by the measurement device [(e.g. see Yeow paragraphs 0076, 0081, 0099, 0104) ”a portable instrument such as a wearable brainwave-sensing device 102 that can detect brainwaves via electrodes 104 placed on the scalp … The portable EEG device can have designs to allow for flexible placement of electrodes or other sensors. This preferably allows the addition of electrodes to the device to provide users with a broader range of brainwaves information at different parts of the brain … the headband design permits shifting of the electrodes 302, 304 from e.g. the frontal positions as illustrated in FIG. 3B), to the occipital positions as illustrated in FIG. 3C), depending on the type of EEG state that needs to be monitored, e.g. frontal and occipital regions correlate with attention and pain respectively … The electrode(s) can be placed anywhere along the portable EEG device in example embodiments to advantageously allow for collection of data from various parts of the scalp to allow for varied analysis. By allowing flexible placement of the electrode(s) for measurement of brain activity anywhere along the scalp, example embodiments can provide users with a variety of different brainwave information which can range from sleep tracking to anxiety monitoring to measurement of concentration levels”].
providing the plurality of localized signals … wherein at least one capability performed by the particular user forms a perceptual experience [(e.g. see Yeow paragraphs 0067, 0068, 0076, 0082) ”a brainwave-sensing device is provided which allows for the detection and display of mental states such as emotions (happiness, anger, sadness, fear, excitement), pain, anxiety, sleep, mental fatigue, comfort and pleasure … The software in example embodiments uses supervised/unsupervised algorithm(s) to detect the mental states, e.g. anxiety levels, of the user based on the brainwave data … The system 100 further comprises software 114 that, when running on an appropriate computing device, receives the transmitted brainwave signals 112 and which will subject the signals to signal processing 116 and brainwave interpretation such as emotion identification 118 … brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see plots 702-706”]. Examiner notes that, as defined by applicant’s specification at paragraphs 0035, 0037 and 0094, a capability and perceptual experience can be “emotions” experienced by the user.
generating an output corresponding to a perceptual experience, using the trained model and subsequent acquisition of the plurality of localized signals to calibrate for the particular user [(e.g. see Yeow paragraphs 0067, 0068, 0076, 0081, 0082) ”a brainwave-sensing device is provided which allows for the detection and display of mental states such as emotions (happiness, anger, sadness, fear, excitement), pain, anxiety, sleep, mental fatigue, comfort and pleasure … the headband design permits shifting of the electrodes 302, 304 from e.g. the frontal positions as illustrated in FIG. 3B), to the occipital positions as illustrated in FIG. 3C), depending on the type of EEG state that needs to be monitored, e.g. frontal and occipital regions correlate with attention and pain respectively … The software in example embodiments uses supervised/unsupervised algorithm(s) to detect the mental states, e.g. anxiety levels, of the user based on the brainwave data … The system 100 further comprises software 114 that, when running on an appropriate computing device, receives the transmitted brainwave signals 112 and which will subject the signals to signal processing 116 and brainwave interpretation such as emotion identification 118 … brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see plots 702-706 … a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see plots 702-706. This procedure allows generating user-specific calibrated brainwave scales 714 to facilitate subsequent brainwave state identification”]. Examiner notes that, as defined by applicant’s specification at paragraphs 0035, 0037 and 0094, a capability and perceptual experience can be “emotions” experienced by the user.
providing the output to an application utilizing the corresponding perceptual experience [(e.g. see Yeow paragraphs 0067, 0076, 0084, 0085 and Figs. 9A-B) ”a brainwave-sensing device is provided which allows for the detection and display of mental states such as emotions (happiness, anger, sadness, fear, excitement), pain, anxiety, sleep, mental fatigue, comfort and pleasure … The system 100 also comprises one or more of a wide-ranging scope of different applications 122 in which the identified brainwave information 124 may be used … the plot 900 can reveal the brainwave state 904 at the particular time event. The brainwave information can for example be projected on a colored scale to indicate the brainwave levels … projected 918 on a colored/numerical scale to indicate the user's brainwave levels during a certain experience or activity performed by the user”]. Examiner notes that, as defined by applicant’s specification at paragraphs 0035, 0037 and 0094, a capability and perceptual experience can be “emotions” experienced by the user. Examiner notes that, as depicted in Figs. 9A-B, multiple measured emotions of the user (e.g. happy, excited, angry) are displayed.
Yeow does not specifically teach the plurality of localized signals being in a raw and unfiltered form, without application of an artifact removal process or providing [signals] in their raw and unfiltered form, without pre-processing, to a processing system comprising at least one deep learning module, the at least one deep learning module being configured to use the raw, unfiltered, and unprocessed signals to train a model to correlate the plurality of localized signals to at least one capability performed by the particular user while being measured by the measurement device. However, in the same field of invention, Laszlo teaches:
the plurality of localized signals being in a raw and unfiltered form, without application of an artifact removal process [(e.g. see Laszlo paragraphs 0030, 0039 and Figs. 1 and 3A) ”FIG. 3A illustrates a simulated example of noisy brainwave data that may be received from one brainwave sensor … The system includes a brainwave data processing module 102 which is in communication with brainwave sensors 104”]. Examiner notes that, as shown in Figs. 1 and 3A, a noisy/raw EEG signal is provided directly from brainwave sensor (numeral 104) to a processing module (numeral 102).
providing [signals] in their raw and unfiltered form, without pre-processing, to a processing system comprising at least one deep learning module, the at least one deep learning module being configured to use the raw, unfiltered, and unprocessed signals to train a model to correlate the plurality of localized signals to at least one capability performed by the particular user while being measured by the measurement device [(e.g. see Laszlo paragraphs 0004, 0030, 0039, 0042, 0051, 0057, 0060 and Figs. 1 and 3A) ”The system includes a brainwave data processing module 102 which is in communication with brainwave sensors 104 … FIG. 3A illustrates a simulated example of noisy brainwave data that may be received from one brainwave sensor … the brainwave filtering module 110 incorporates a machine learning model to identify signal patterns associated with user physiological activities within the brainwave data … optionally includes a noise filter 114 … the machine learning model is a deep learning model that employs multiple layers of models to generate an output for a received input … Such filters may be initialized based on known brain wave patterns for muscle control (e.g. head motion) and further refined based on learned analysis of a particular user's brain wave patterns. The above processes can be performed on data from each of multiple brainwave sensors individually … the machine learning model can refine the ability to identify signal patterns associated with physiological actions of a particular user. For example, the machine learning model can continue to be trained on user specific data in order to adapt the signal pattern recognition algorithms to the those associated with a particular user … identify user brain states including, but not limited to, attentiveness, tiredness, depth of thought, physiological arousal (e.g., fear or other strong emotions), seizure or pre-seizure activity, or stage of sleep”]. The claim requires the signals from sensors measuring brain activity, in their raw and unfiltered form and without pre-processing, to be provided to a processing system that comprises at least one deep learning module. Laszlo teaches a processing system (numeral 102, brainwave data processing module) that comprises at least one deep learning module (numeral 110, brainwave filtering module contains machine learning models to perform signal pattern recognition). The noisy/raw brainwave signal (e.g. Fig. 3A) is provided directly (see path from numeral 104 to numeral 102) to the brainwave data processing module (e.g. processing system). Moreover, examiner notes that a noise pre-filter (numeral 114) within the processing module is additionally disclosed as optional; however, the noisy/raw brainwave signal has already been provided to the overarching brainwave data processing module (numeral 102) (e.g. processing system).
Therefore, considering the teachings of Yeow and Laszlo, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add the plurality of localized signals being in a raw and unfiltered form, without application of an artifact removal process or providing [signals] in their raw and unfiltered form, without pre-processing, to a processing system comprising at least one deep learning module, the at least one deep learning module being configured to use the raw, unfiltered, and unprocessed signals to train a model to correlate the plurality of localized signals to at least one capability performed by the particular user while being measured by the measurement device, as taught by Laszlo, to the teachings of Yeow because it improves the efficiency of training the model and improves the generalizability of the model (e.g. see Laszlo paragraph 0053).
As for dependent claim 2, Yeow and Laszlo teach the method as described in claim 1, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the model is trained using a machine learning algorithm executed by the deep learning module using the unprocessed signals measured during trials performed by a first user corresponding to the particular user, the trials corresponding to the at least one capability [(e.g. see Laszlo paragraphs 0004, 0057) ”Such filters may be initialized based on known brain wave patterns for muscle control (e.g. head motion) and further refined based on learned analysis of a particular user's brain wave patterns. The above processes can be performed on data from each of multiple brainwave sensors individually … the machine learning model can refine the ability to identify signal patterns associated with physiological actions of a particular user. For example, the machine learning model can continue to be trained on user specific data in order to adapt the signal pattern recognition algorithms to the those associated with a particular user”].
The motivation to combine is the same as that used for claim 1.
As for dependent claim 3, Yeow and Laszlo teach the method as described in claim 2 and Yeow further teaches:
further comprising source localization in training the model using the machine learning algorithm [(e.g. see Yeow paragraphs 0068, 0082) ”brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions … The portable device has at least one, and preferably multiple dry electrodes (and optionally other sensors or sensing devices) which can be configured in multiple ways to allow for different use cases. The software in example embodiments uses supervised/unsupervised algorithm(s) to detect the mental states, e.g. anxiety levels, of the user based on the brainwave data. A supervised approach requires the user to exhibit a specific mental state repeatedly so that the detected brainwave profile can be tagged to the desired mental state. An unsupervised approach uses a predetermined relationship between the specific mental state and the brainwave data, e.g. the brainwave profile, based on previously collected test data from a subject population. The mental state information can be stored on a centralized database for subsequent retrieval or analysis”].
As for dependent claim 5, Yeow and Laszlo teach the method as described in claim 2, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the machine learning algorithm comprises a convolution neural network (CNN) [(e.g. see Laszlo paragraph 0052) ”the neural network may apply a mathematical transformation, e.g., convolutional, to input data prior to feeding the input data to the network”].
The motivation to combine is the same as that used for claim 1.
As for dependent claim 6, Yeow and Laszlo teach the method as described in claim 5, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the CNN is trained using one of the following variants: (a) training a CNN model directly from raw signal data, (b) learning a feature representation of the signals through a plurality of different modules with a same algorithm, or (c) constructing an autoregressive dilated causal convolution neural network (ADCCNN) that directly receives the signals [(e.g. see Laszlo paragraphs 0039, 0040, 0052) ”In some implementations, the neural network may apply a mathematical transformation, e.g., convolutional, to input data prior to feeding the input data to the network … The signal in FIG. 3A represents an aggregate electrical signal that can include multiple signal patterns related to both physiological activities of the user and brainwave patterns related to mental activities of the user. Each of the signal patterns may not be easily recognizable. Furthermore, the signal patterns may interfere with each other. For example, a signal pattern related to the physiological activity of the user may be viewed as noise with respect to a signal pattern related to the mental activity of the user if the later is desired for further analysis in a given context. On the contrary, the signal pattern related to the mental activity of the user may be viewed as noise with respect to the signal pattern related to the physiological activity of the user if it is the desired signal pattern for further analysis in a different context … The brainwave sensors 104 or sensor system 105 transmit signals such as the example data signal shown in FIG. 3A to the data processing module 102”]. Both Yeow and Laszlo define using algorithms and machine learning. Given that machine learning/neural networks have a finite number of models to choose from, it would have been obvious to one of ordinary skill in the art, namely a software developer, to try any available model that would achieve the processed output, with reasonable success, as the use of particular algorithms and machine learning models is shown in Yeow and Laszlo. A person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely the product not of invention but of ordinary skill and common sense.
As for dependent claim 7, Yeow and Laszlo teach the method as described in claim 6, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein in variant (c), the ADCCNN is trained on providing an output of classes that indicated what functions were made by the user [(e.g. see Laszlo paragraphs 0051, 0052, 0054) ”the machine learning model is a shallow machine learning model, e.g., a linear regression model … The neural network may apply a mathematical transformation, e.g., convolutional, to input data prior to feeding the input data to the network … a machine learning model can be trained to recognize signal patterns associated with various different user physiological actions. For example, the machine learning model can correlate identified user physiological actions with signal patterns within the brainwave data that are related to the identified actions. For example, the machine learning model can be trained to identify noise patterns generated in brainwave sensors when a user moves their head. The machine learning model can be trained to identify interference signal patterns that occur in brainwave that are caused by non-brainwave electrical impulses (e.g., other nervous system signal) in the user's body when the user makes muscular movements (e.g., changing facial expressions, moving their eyes, head or other limbs)”]. Both Yeow and Laszlo define using algorithms and machine learning. Given that machine learning/neural networks have a finite number of models to choose from, it would have been obvious to one of ordinary skill in the art, namely a software developer, to try any available model that would achieve the processed output, with reasonable success, as the use of particular algorithms and machine learning models is shown in Yeow and Laszlo. A person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely the product not of invention but of ordinary skill and common sense.
As for dependent claim 8, Yeow and Laszlo teach the method as described in claim 2, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the machine learning algorithm comprises a generative model utilized by the deep learning module [(e.g. see Laszlo paragraph 0053) ”the machine learning model can be a supervised model. For example, for each input provided to the model during training, the machine learning model can be instructed as to what the correct output should be. The machine learning model can use batch training, e.g., training on a subset of examples before each adjustment, instead of the entire available set of examples. This may improve the efficiency of training the model and may improve the generalizability of the model. The machine learning model may use folded cross-validation. For example, some fraction (the “fold”) of the data available for training can be left out of training and used in a later testing phase to confirm how well the model generalizes”]. Both Yeow and Laszlo define using algorithms and machine learning. Given that machine learning/neural networks have a finite number of models to choose from, it would have been obvious to one of ordinary skill in the art, namely a software developer, to try any available model that would achieve the processed output, with reasonable success, as the use of particular algorithms and machine learning models is shown in Yeow and Laszlo. A person of ordinary skill has good reason to pursue the known options within his or her technical grasp. If this leads to the anticipated success, it is likely the product not of invention but of ordinary skill and common sense.
As for dependent claim 9, Yeow and Laszlo teach the method as described in claim 2 and Yeow further teaches:
further comprising: conducting a calibration for a second user of the measurement device [(e.g. see Yeow paragraphs 0073, 0082, 0083) ”a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see plots 702-706. This procedure allows generating user-specific calibrated brainwave scales 714 to facilitate subsequent brainwave state identification … it is advantageously possible to relate, for each individual subject, different localized brainwave profiles to specific levels of brainwave states … Based on the calibration, brainwave state identification can be obtained and the real-time brainwave state of users 800(1)-(N)”].
As for dependent claim 10, Yeow and Laszlo teach the method as described in claim 9 and Yeow further teaches:
having the second user conduct the same trials as the first user [(e.g. see Yeow paragraph 0082) ”a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see plots 702-706. This procedure allows generating user-specific calibrated brainwave scales 714 to facilitate subsequent brainwave state identification”].
As for dependent claim 13, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the measurement device comprises a headset used to position the plurality of sensors [(e.g. see Yeow paragraph 0081 and Fig. 3) ”a portable EEG device in the form of a wearable EEG device according to an example embodiment, as a headband 300 embedded with monitoring and reference electrodes 302, 304, together with a power board 306 and an amplifier and wireless communications board 308. In this case, the headband design permits shifting of the electrodes 302, 304 from e.g. the frontal positions as illustrated in FIG. 3B), to the occipital positions as illustrated in FIG. 3C), depending on the type of EEG state that needs to be monitored, e.g. frontal and occipital regions correlate with attention and pain respectively. Other adjustability features in example embodiments will be described below, including e.g. with reference to FIGS. 15-21. Data from the EEG device 300 can be wirelessly accessed via a computer/phone/tablet device through a real-time data display (not shown) which plots, for example, the time and frequency-domain EEG information”].
As for dependent claim 14, Yeow and Laszlo teach the method as described in claim 13 and Yeow further teaches:
wherein the signals are acquired using the headset, and the at least one of the processing system, the at least one capability, and the application is provided using a separate device [(e.g. see Yeow paragraphs 0081, 0193 and Figs. 22, 27-32) ”a portable EEG device in the form of a wearable EEG device according to an example embodiment, as a headband 300 embedded with monitoring and reference electrodes 302, 304, together with a power board 306 and an amplifier and wireless communications board 308. In this case, the headband design permits shifting of the electrodes 302, 304 from e.g. the frontal positions as illustrated in FIG. 3B), to the occipital positions as illustrated in FIG. 3C), depending on the type of EEG state that needs to be monitored, e.g. frontal and occipital regions correlate with attention and pain respectively. Other adjustability features in example embodiments will be described below, including e.g. with reference to FIGS. 15-21. Data from the EEG device 300 can be wirelessly accessed via a computer/phone/tablet device through a real-time data display (not shown) which plots, for example, the time and frequency-domain EEG information, the power of the targeted brainwave state, as well as processed data that shows the intensity level of the mental state of interest … Example embodiments can provide a mobile app platform whereby the raw and/or processed brainwave (and/or other physiological) information can be transmitted to and displayed on the mobile phone for user to utilize in a convenient and meaningful manner”].
As for dependent claim 15, Yeow and Laszlo teach the method as described in claim 14 and Yeow further teaches:
wherein the separate device comprises an edge device coupled to the headset [(e.g. see Yeow paragraph 0124 and Fig. 26) ”In addition, the EEG data can be directly fed to, for example, a forehead display 2600 on the patient 2602”].
As for dependent claim 16, Yeow and Laszlo teach the method as described in claim 15 and Yeow further teaches:
wherein the edge device communicated with a cloud device over a communication network to provide the at least one of the processing system, the at least one capability, at the application [(e.g. see Yeow paragraphs 0078, 0081, 0124) ”which receives the transmitted brainwave signals 212 and which will subject the signals to signal processing and brainwave interpretation such as emotion identification, followed by subsequent display of the brainwave status information on a Graphical User Interface (GUI) 216 for viewing. The system 200 in this embodiment also comprises a Server 218 for data storage/accessibility … The data can also be uploaded in real-time onto an online server/database, whereby the data can be remotely accessed anywhere and anytime … In addition, the EEG data can be directly fed to, for example, a forehead display 2600 on the patient 2602”].
As for dependent claim 17, Yeow and Laszlo teach the method as described in claim 14 and Yeow further teaches:
wherein the headset is configured to send at least signal data to a cloud device over a communication network [(e.g. see Yeow paragraph 0081) ”The data can also be uploaded in real-time onto an online server/database, whereby the data can be remotely accessed anywhere and anytime”].
As for dependent claim 18, Yeow and Laszlo teach the method as described in claim 1, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the at least one capability comprises body movements measured using the plurality of unprocessed signals [(e.g. see Laszlo paragraph 0034, 0047) ”For example, the brainwave filtering module 110 can correlate a particular type of user physiological action (e.g., a head movement) to known signal patterns within the brainwave data that are correlate to the particular type of action. For example, the brainwave filtering module 110 can correlate the timing of an identified head movement with changes in the brainwave signal that correlate with the timing of the head movement … identify a user's muscular activity (e.g., limb and facial movements, heartbeat, respiration, eye movements, etc.), identify signal patterns associated with an identified muscular activity”].
The motivation to combine is the same as that used for claim 1.
As for dependent claim 19, Yeow and Laszlo teach the method as described in claim 18, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the model is trained by the deep learning module by having the particular user perform a trial comprises a set of body movements [(e.g. see Laszlo paragraphs 0004, 0057) ”the machine learning model can refine the ability to identify signal patterns associated with physiological actions of a particular user. For example, the machine learning model can continue to be trained on user specific data in order to adapt the signal pattern recognition algorithms to the those associated with a particular user. For example, the machine learning model can use brainwave data from periods of time during which the user … performs only few physiological actions … Such filters may be initialized based on known brain wave patterns for muscle control (e.g. head motion) and further refined based on learned analysis of a particular user's brain wave patterns”].
The motivation to combine is the same as that used for claim 1.
As for dependent claim 21, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the at least one capability comprises measuring emotions expressed by the particular user [(e.g. see Yeow paragraphs 0068, 0076, 0082) ”The software in example embodiments uses supervised/unsupervised algorithm(s) to detect the mental states, e.g. anxiety levels, of the user based on the brainwave data … The system 100 further comprises software 114 that, when running on an appropriate computing device, receives the transmitted brainwave signals 112 and which will subject the signals to signal processing 116 and brainwave interpretation such as emotion identification 118 … brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see plots 702-706”].
As for dependent claim 22, Yeow and Laszlo teach the method as described in claim 21 and Yeow further teaches:
wherein a plurality of emotions are determined according to a predefined categorization scheme, and measuring the emotions comprises eliciting emotions and measuring the brain activity to train the model using the deep learning module to categorize emotions for the particular user [(e.g. Yeow paragraph 0082) ”brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see plots 702-706. These waveforms include alpha (8-13 Hz), delta (0.5-4 Hz), beta (14-30 Hz) and theta (4-8 Hz). In example embodiments, a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see plots 702-706. This procedure allows generating user-specific calibrated brainwave scales 714 to facilitate subsequent brainwave state identification”].
As for dependent claim 23, Yeow and Laszlo teach the method as described in claim 22, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the deep learning module is constructed and trained on detecting the emotions expressed by the particular user by using a pair of deep learning models, a recurrent neural network (RNN) as a first model that learns features from the signals and provides a feature vector as an input to a CNN as a second model that used the features vectors provided by the first model and further trains using the deep learning module through classification [(e.g. see Laszlo paragraphs 0051, 0052) ”In some cases, the neural network may be a recurrent neural network. A recurrent neural network is a neural network that receives an input sequence and generates an output sequence from the input sequence. In particular, a recurrent neural network uses some or all of the internal state of the network after processing a previous input in the input sequence to generate an output from the current input in the input sequence … In some implementations, the neural network may apply a mathematical transformation, e.g., convolutional, to input data prior to feeding the input data to the network”].
The motivation to combine is the same as that used for claim 1.
As for dependent claim 25, Yeow and Laszlo teach the method as described in claim 21 and Yeow further teaches:
wherein each of a plurality of emotions are output according to a scale [(e.g. see Yeow paragraphs 0084, 00585 and Fig. 9A) ”the brainwave state 904 at the particular time event. The brainwave information can for example be projected on a colored scale to indicate the brainwave levels … The brainwave information at the particular time event can be recorded during the video recording, and projected 918 on a colored/numerical scale to indicate the user's brainwave levels during a certain experience or activity performed by the user”].
As for dependent claim 26, Yeow and Laszlo teach the method as described in claim 25 and Yeow further teaches:
further comprising combining a plurality of the emotions output according to the scale to identify a complex emotion [(e.g. see Yeow paragraphs 0082, 0084, 0085 and Fig. 9A) ”the brainwave state 904 at the particular time event. The brainwave information can for example be projected on a colored scale to indicate the brainwave levels … The brainwave information at the particular time event can be recorded during the video recording, and projected 918 on a colored/numerical scale to indicate the user's brainwave levels during a certain experience or activity performed by the user … brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear”].
As for dependent claim 31, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the at least one capability comprises mental commands from the particular user determined from the plurality of unprocessed signals [(e.g. see Yeow paragraph 0165) ”the system can be modified for use in remote control of assistive devices, for example, a user with paraplegia dons the EEG device and is able to send EEG command, coupled with his/her head orientation, to control the direction of motion of the wheelchair”].
As for dependent claim 33, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device during a period of sleep [(e.g. see Yeow paragraph 0116) ”Sleep detection is another possible application of embodiments of the present invention … Example embodiments can also be used for sleep quality monitoring, whereby a wearable EEG device can record the brainwave states during the user's sleep, in order to determine sleep quality”].
As for dependent claim 34, Yeow and Laszlo teach the method as described in claim 33 and Yeow further teaches:
wherein the measurement device is operable to: generate the perceptual experience during the period of sleep by combining the outputs for the capabilities, and provide information indicative of the perceptual experience during the period of sleep as a recording during or following the period of sleep, through a user interface [(e.g. see Yeow paragraphs 0081, 0116) ”Sleep detection is another possible application of embodiments of the present invention … Example embodiments can also be used for sleep quality monitoring, whereby a wearable EEG device can record the brainwave states during the user's sleep … Thereafter, the user receives a post-sleep objective assessment … The data can also be uploaded in real-time onto an online server/database, whereby the data can be remotely accessed anywhere and anytime”].
As for dependent claim 35, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device to measure a consciousness of the particular user to provide as the perceptual experience to the application [(e.g. see Yeow paragraphs 0067, 0116) ”brainwave-sensing device is provided which allows for the detection and display of mental states such as … sleep … Sleep detection is another possible application of embodiments of the present invention”].
As for dependent claim 36, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device to provide the output to a medical application to utilize at least one of the capabilities [(e.g. see Yeow paragraph 0118) ”The applications of brainwave-detection and monitoring can be extended in example embodiments to healthcare and medical treatment, which is particularly useful for monitoring the mental states of people with brainwave irregularities, resulting in behavioral issues and challenges with interpersonal relationships. This brainwave information is most often important to, but is not limited to, parents, guardians, counselors, healthcare personnel or medical doctors to track the real-time brainwaves of their child, client and/or a group of people concurrently. For example, psychologists, counselors and psychiatrists could monitor their clients' disorder/condition (such as autism, bipolar disorder, ADHD etc.) in real time from their workplace, enabling tracking of the progressive condition of their client and to administer appropriate treatments to improve the patient's brainwave states”].
As for dependent claim 37, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device to provided the output to a communication application for enabling locked-in patients to communicate according to the determined perceptual experience [(e.g. see Yeow paragraph 0124) ”this application can potentially help patients who are in a vegetative state regain consciousness earlier by identifying the most optimal form of stimulation through detection of a change in their brainwave state. For example, a common technique that family members would do for the vegetative patient is to provide stimulatory experiences of their senses that mimic an old memory. Family members of patients would now be able to know if the therapy is helpful based on a change in response of the patient. Rather than doing repetitive stimulation e.g. playing of a certain musical piece that they may not respond well with, experimentation with other forms of stimulation could possibly be more useful. This way, the brainwave state of a comatose patient may be tracked, displayed and reviewed over time”].
As for dependent claim 38, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device to provided the output to the application for applying mind or gesture controlled capabilities to one or more of: emotionally adaptive gaming, an augmented reality menu or interface, or a virtual reality menu or interface [(e.g. see Yeow paragraph 0155) ”Games can be designed to incorporate the user's brainwaves in example embodiments, with the option to display, monitoring, share and also act as a feedback for controlling part-of or whole of the running of the game. This would humanize the game characters or activities, making it more interactive and realistic. Current games typically sense the physical movements to power the games, but using brainwaves with games to induce a certain function in example embodiments can help enhance the user's experience. For instance, a gamer who exhibits an excited brainwave state can trigger a special attack move for his character to defeat the opponent”].
As for dependent claim 39, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the application utilizes the output to provide live streaming vision from the particular user [(e.g. see Yeow paragraph 0081, 0245) ”embodiments of the present invention can be adapted to detect brainwaves localized at all parts of the brain, including but not limited to … vision … The data can also be uploaded in real-time onto an online server/database, whereby the data can be remotely accessed anywhere and anytime”].
As for dependent claim 40, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device during a simulation or training exercise to provide as the perceptual experience to the application [(e.g. see Yeow paragraph 0082) ”In example embodiments, a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see plots 702-706. This procedure allows generating user-specific calibrated brainwave scales 714 to facilitate subsequent brainwave state identification”].
As for dependent claim 41, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the application is used to remotely study the particular user from a distance [(e.g. see Yeow paragraph 0119) ”Parents or guardians could monitor their child more closely, who could be suffering from excessive stress levels or psychological trauma due to child abuse or bully by others; as well as elderly in the family who are struggling with managing ageing illnesses such as Alzheimer disease as well as the associated emotion-related issues. By remotely monitoring the brainwave states of these people in real-time, their mental health can be diagnosed beforehand, allowing others to empathize with their condition and provide adequate care for them”].
As for dependent claim 43, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device during consumer related activities to provide as the perceptual experience to the application for enhancing advertising [(e.g. see Yeow paragraph 0153) ”Advertisers can categorize their product/services by having a particular brainwave state tagged to it according to example embodiments, allowing the product/service to pop up according to the user's brainwave state. This could be implemented in conjunction with the online web store as mentioned above. For example, a depressed person may trigger the display of an online advertisement offering psychological hotline assistance or even shopping therapy for an upset individual may be deemed helpful. In addition, a happy person may be prompted by celebratory advertisements that sell items which of great interest to him/her”].
As for dependent claim 44, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the plurality of signals are acquired by the measurement device to provide the determined perceptual experience for the applied to conduct research [(e.g. see Yeow paragraph 0086) ”For example, a brainwave diary can be printed out 1006 into a hardcopy version of a physical diary, with brainwave information 1100, 1102 coupled to each event in the diary 1104 (Fig. 11). Depending on the user's preference, the downloaded brainwave information may also be automatically exported in e.g. an email format for easy regular sharing and updating. This can provide a digital online capability for the input of personalized documents (comments/photos/files) based on each brainwave information triggered and recorded, and can further allow for the documentation, recording, tracking of events and storage of personal brainwave data collected over a span of a specified period of time in multimedia and printable format. Such an option can be especially advantageous for both personal reference and for institutional research references (e.g. patient records and population-based healthcare monitoring of specific diseased patients)”].
As for dependent claim 48, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the application is configured to provide information back to the brain of the particular user, from a computing device hosting the information [(e.g. see Yeow paragraph 0124) ”experimentation with other forms of stimulation could possibly be more useful. This way, the brainwave state of a comatose patient may be tracked, displayed and reviewed over time by the family members to provide a form of two-way communication via brainwaves”].
As for dependent claim 49, Yeow and Laszlo teach the method as described in claim 1 and Yeow further teaches:
wherein the application comprises interactions from a plurality of users connected to each other [(e.g. see Yeow paragraphs 0133, 0134 and Fig. 27) ”Brainwave-Sharing Via Video or Voice Calls … Embodiments of the invention can be designed to improve relationships between specific individuals or groups of people by allowing their true emotions to be reflected during their virtual communications. This can be particular useful for, but is not limited to, those in long distance relationships relying on video or voice calls as their main mode of communication, as illustrated in FIG. 27. By understanding how the other party 2700, 2702 feels, it would help to facilitate communication and minimize the occurrence of unintended misunderstandings, thereby fostering stronger ties between parent-child, couples, friends, business partners etc.”].
As for independent claim 50, Yeow and Laszlo teach a non-transitory computer readable medium. Claim 50 discloses substantially the same limitations as claim 1. Therefore, it is rejected with the same rational as claim 1.
As for independent claim 51, Yeow and Laszlo teach a system. Claim 51 discloses substantially the same limitations as claim 1. Therefore, it is rejected with the same rational as claim 1.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 10 above, and further in view of Deshpande, Adit “A Beginner’s Guide to Understanding Convolutional Neural Networks Part 2”, published 29 July 2016, <URL: https://adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/>.
As for dependent claim 11, Yeow and Laszlo teach the method as described in claim 10, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the calibration for the second user comprises using a same deep learning module with weights optimized to data derived from the first user, with at least one final layer of the network removed and replaced with a new layer optimized with weights associated with signals generated by the second user [(e.g. see Laszlo paragraphs 0004, 0052, 0057) ”Such filters may be initialized based on known brain wave patterns for muscle control (e.g. head motion) and further refined based on learned analysis of a particular user's brain wave patterns … the machine learning model can refine the ability to identify signal patterns associated with physiological actions of a particular user. For example, the machine learning model can continue to be trained on user specific data in order to adapt the signal pattern recognition algorithms to the those associated with a particular user … The neural network may include an ADAM optimizer for training the network and computing updated layer weights”].
Yeow and Laszlo do not specifically teach with one or more early layers frozen or to transfer learning to the second user. However, in the same field of invention or solving similar problems, Deshpande teaches:
with one or more early layers frozen [(e.g. see Deshpande page 6 first and second paragraph) ”Rather than training the whole network through a random initialization of weights, we can use the weights of the pre-trained model (and freeze them) and focus on the more import layers for training … You will remove the last layer of the network and replace it with your own classifier. You then freeze the weights of all the other layers and train the network normally”].
to transfer learning to the second user [(e.g. see Deshpande page 6 first and second paragraph) ”Transfer learning is the process of taking a pre-trained model (the weights and parameters of a network that has been trained on a large dataset by somebody else) and ‘fine-tuning’ the model with your own dataset”].
Therefore, considering the teachings of Yeow, Laszlo and Deshpande, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add with one or more early layers frozen or to transfer learning to the second user, as taught by Deshpande, to the teachings of Yeow and Laszlo because it allows the training to be focused on the most important layers (e.g. see Deshpande page 6 second paragraph).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 10 above, and further in view of Beaubien et al. (US 10,783,801 B1).
As for dependent claim 12, Yeow and Laszlo teach the method as described in claim 10, but do not specifically teach wherein the calibration for the second user comprises: calculating a difference in weights between signals generated by the first user and signals generated by the second user for a subset of capabilities or predicting weights for remaining capabilities for the second user based on the calculated difference. However, in the same field of invention, Beaubien teaches:
wherein the calibration for the second user comprises: calculating a difference in weights between signals generated by the first user and signals generated by the second user for a subset of capabilities [(e.g. see Beaubien col 7 lines 14-22, col 11 lines 31-42, col 12 lines 6-17) ”receives data from the users 112 through sensors 120A and 120B. The sensors 120A and 120B communicate data from the users 112 real time to the cognitive load assessment subsystem 140 that determines objective CL measures for the individuals and/or teams 112. The CL measures may then be used by the simulator and trainers to quantitatively assess or ensure the effectiveness of the training for the users 112 … adjusts these weights with repeated exposure to the training data until an optimal set of weights is computed based on the difference between the predicted value of y and the expected y. The adjustment of these weights is the embodiment of the learning. A set of weights can be updated to provide personalized learning with additional data collected during training. The process is identical as described above except (1) the weights are not reinitialized to random values and (2) the new data collected from the trainee is added to the original training data. In this way, the trainees' performance is reflected in the new set of weights computed … where y is the resulting predicted values generated (at time t), w is the matrix of network weights, δ is a delta function, d is the time delay, t is time, x represents the input values of the EEG/ECG data source, i represents a number of increments and N represents the number of i inputs to be used in the TDNN. The EEG/ECG data source comprises values for the input data such as, but not limited to a heart rate, electrical activity of the brain such as voltage fluctuation or spectral content (“brain waves”) of the EEG. The time delay neural network maintains a history of model inputs via tap delay lines that store a current real-time EEG/ECG input data vector””].
predicting weights for remaining capabilities for the second user based on the calculated difference [(e.g. see Beaubien col 11 lines 31-42, col 12 lines 6-17) ”adjusts these weights with repeated exposure to the training data until an optimal set of weights is computed based on the difference between the predicted value of y and the expected y. The adjustment of these weights is the embodiment of the learning. A set of weights can be updated to provide personalized learning with additional data collected during training. The process is identical as described above except (1) the weights are not reinitialized to random values and (2) the new data collected from the trainee is added to the original training data. In this way, the trainees' performance is reflected in the new set of weights computed … where y is the resulting predicted values generated (at time t), w is the matrix of network weights, δ is a delta function, d is the time delay, t is time, x represents the input values of the EEG/ECG data source, i represents a number of increments and N represents the number of i inputs to be used in the TDNN. The EEG/ECG data source comprises values for the input data such as, but not limited to a heart rate, electrical activity of the brain such as voltage fluctuation or spectral content (“brain waves”) of the EEG. The time delay neural network maintains a history of model inputs via tap delay lines that store a current real-time EEG/ECG input data vector”].
Therefore, considering the teachings of Yeow, Laszlo and Beaubien, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add wherein the calibration for the second user comprises: calculating a difference in weights between signals generated by the first user and signals generated by the second user for a subset of capabilities or predicting weights for remaining capabilities for the second user based on the calculated difference, as taught by Beaubien, to the teachings of Yeow and Laszlo because it allows trends to be tracked across different users which assists model validation (e.g. see Beaubien col 19 lines 30-32).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 18 above, and further in view of Kang (US 2013/0123585 A1).
As for dependent claim 20, Yeow and Laszlo teach the method as described in claim 18, but Yeow does not specifically teach the following limitation. However, Laszlo teaches:
wherein the body movements are modeled for continuous free motion to provide approximations of exact body movements of the particular user [(e.g. see Laszlo paragraph 0054) ”For example, a machine learning model can be trained to recognize signal patterns associated with various different user physiological actions. For example, the machine learning model can correlate identified user physiological actions with signal patterns within the brainwave data that are related to the identified actions. For example, the machine learning model can be trained to identify noise patterns generated in brainwave sensors when a user moves their head. The machine learning model can be trained to identify interference signal patterns that occur in brainwave that are caused by non-brainwave electrical impulses (e.g., other nervous system signal) in the user's body when the user makes muscular movements (e.g., changing facial expressions, moving their eyes, head or other limbs)”].
Yeow and Laszlo do not specifically teach by generating a multi-dimensional output for the body movements that includes values for direction, degree, and speed. However, in the same field of invention, Kang teaches:
by generating a multi-dimensional output for the body movements that includes values for direction, degree, and speed [(e.g. see Kang paragraph 0014, abstract and Figs. 5 and 8) ”measures a weak brainwave signal detected from a human scalp through a noninvasive method to measure a degree of concentration through an analysis and process of the detected brainwave signal, and a control method enabled for short distance control of an electronic device or remote monitoring through the internet, by using the brainwave signal. An acceleration sensor is put on the head of a user, and a brainwave detecting means put on the head of a user to detect a brainwave and the acceleration sensor that outputs an acceleration value of three axes including XYZ axes as a signal of predetermined data are used to detect movement of the head, to thereby control the direction and speed of a brainwave-related device. In more detail, after a signal outputted from the brainwave detecting means is converted into a wireless signal and transmitted, a value of the wireless signal is inputted to a receiving unit of a display device and is expressed numerically. By setting the value inputted to the display device, a portable brainwave measuring device provides accurate device control. A control system is characterized in that a signal according to the slope direction of the head is detected and analyzed, and then numerically expressed by using 6 brainwave signals such as a delta wave (.delta.), theta wave (.theta.), alpha wave (.alpha.), SMR wave, beta wave (.beta.) and gamma wave (.sigma.), which are measured through the portable brainwave measuring device … acceleration value of a 3-axis direction formed of X, Y and Z-axes in a form of a constant data signal”].
Therefore, considering the teachings of Yeow, Laszlo and Kang, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add by generating a multi-dimensional output for the body movements that includes values for direction, degree, and speed, as taught by Kang, to the teachings of Yeow and Laszlo because it allows for efficient operation and accurate control (e.g. see Kang paragraph 0007).
Claims 27-30 are rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 1 above, and further in view of Albert (US 6,377,833 B1).
As for dependent claim 27, Yeow and Laszlo teach the method as described in claim 1, but do not specifically teach wherein the at least one capability comprises vision perceived by the particular user by decoding and reconstructing the vision based on the plurality of unprocessed signals. However, in the same field of invention, Albert teaches:
wherein the at least one capability comprises vision perceived by the particular user by decoding and reconstructing the vision based on the plurality of unprocessed signals [(e.g. see Albert col 1 lines 45-63, col 12 lines 54-64) ”Images on the retina are geometric mappings (projections) of what a person sees. These images are carried to a region in the visual cortex commonly referred to as the V1 region (or the primary visual cortex). The V1 region is retinotopically mapped, i.e., the physical locations of the activated neurons in the V1 region are a geometric mapping (homeomorphism) of the image on the retina. The image in the V1 region can be and has been read by using brain-scanning instruments such as functional magnetic resonance imaging (functional MRI) or positron emission tomography (PET). Neurons then carry the signals out of the V1 region and into deeper regions of the brain, which are not geometrically mapped. It has been recently recognized that there is feedback from those deeper regions back to the V1 region. It has also been recently recognized that this feedback includes images generated by the imagination. Accordingly, a system embodying the present invention reads these feedback signals to obtain and interpret this dynamic mental imagery as computer input … Accordingly, any image in the user's visual cortex is an imagined image. Alternatively, the image inference engine 935 may infer that the image being viewed was the last image in the user's visual cortex. For example, if the user thinks of a first image, which is transmitted to the display, the second image being considered may be an addition to the previously imagined image. Accordingly, the user 120 can build onto imagined images. If not, then the image inference engine 935 may infer that the image is the image being displayed on the output device 225”].
Therefore, considering the teachings of Yeow, Laszlo and Albert, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add wherein the at least one capability comprises vision perceived by the particular user by decoding and reconstructing the vision based on the plurality of unprocessed signals, as taught by Albert, to the teachings of Yeow and Laszlo because it allows computer input to be effected via the imagination whereby drawing ability is not as crucial (e.g. see Albert col 2 lines 54-55).
As for dependent claim 28, Yeow, Laszlo and Albert teach the method as described in claim 27, but Yeow and Laszlo do not specifically teach the following limitation. However, Albert teaches:
wherein the decoding and reconstructing vision comprises: (i) classifying vision training data using an RNN to learn features of the signal data in response to stimuli of images/videos, and (ii) generating and classifying previously unseen images/videos in different categories, as well as the same category of images, as the stimuli of images [(e.g. see Albert col 5 lines 8-12, col 10 lines 32-50) ”The information gathered includes the anatomical geometry of the user's V1 region, and images read from the V1 region based on test object patterns (see FIG. 4, element 410) being presented to the user 120 … If the user's eyes are closed, then, in a first embodiment, the inference engine 935 may infer that the user sees nothing. Alternatively, in a second embodiment, the image inference engine 935 may assume the image being viewed is the previous image imagined. The user 120 can accordingly build on previous images. For example, the user 120 can imagine a table and then a chair with the table. Despite the eyes being closed, the inferred image in the second embodiment would be the table. If the user's eyes are opened, then, in a third embodiment, the inference engine 935 may presume that the image being viewed is the image displayed on the output device 225. In yet a fourth embodiment, the inference engine 935 may obtain the image being viewed by requesting line sight information from the eye tracker 105, and resolving the image being viewed from external information received from a digital still/video camera (not shown)”]. Examiner notes that Laszlo already establishes the use of a RNN.
The motivation to combine is the same as that used for claim 27.
As for dependent claim 29, Yeow and Laszlo teach the method as described in claim 1, but do not specifically teach wherein the at least one capability comprises decoding and reconstructing the plurality of unprocessed signals to determine what the particular user is hearing. However, Albert teaches:
wherein the at least one capability comprises decoding and reconstructing the plurality of unprocessed signals to determine what the particular user is hearing [(e.g. see Albert col 1 line 64 – col 2 line 3, col 10 lines 20-23) ”It will be appreciated that other brain regions (e.g., the auditory cortex, the somatosensory cortex, etc.) may similarly provide physiological responses to other actual sensory information (e.g., sounds and voices, body movement, etc.) and may similarly receive feedback to other imagined sensory information (e.g., imagined sounds and voices, imagined body movement, etc.) … if the sensory information is voices and sound, then the sensory information format would be a chronological order of musical notes and guttural utterance”].
The motivation to combine is the same as that used for claim 27.
As for dependent claim 30, Yeow, Laszlo and Albert teach the method as described in claim 29, but Yeow and Laszlo do not specifically teach the following limitation. However, Albert teaches:
wherein the decoding and reconstructing what the particular user is hearing comprises one of the following variants for collecting and training the model for a dataset using the deep learning module: (a) collecting the dataset from a first user while the first user is listening to target words and feeding an audio derivative and text for the target word into an algorithm of neural networks or (b) collecting the dataset with the first user listening to a categorized phonology and labeling signals according to stimuli presented along with textual transcriptions of sounds [(e.g. see Albert col 7 lines 25-30, col 13 lines 10-11) ”It will be appreciated that test object patterns 410 may be sounds, voices, movements, etc. based on the brain region being examined and the sensory information being interpreted … may use the interpreted image as text data to be input to a word processor”].
The motivation to combine is the same as that used for claim 27.
Claims 32 and 45 are rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 1 above, and further in view of Pasley et al. (US 2015/0297106 A1).
As for dependent claim 32, Yeow and Laszlo teach the method as described in claim 1, but do not specifically teach wherein the at least one capability comprises correlating brain-to-text and/or brain-to-speech commands based on the plurality of unprocessed signals by listening for a trigger word and executing at least one command by interfacing with one or more internal capabilities or one or more external applications to provide a virtual assistant. However, in the same field of invention, Pasley teaches:
wherein the at least one capability comprises correlating brain-to-text and/or brain-to-speech commands based on the plurality of unprocessed signals by listening for a trigger word and executing at least one command by interfacing with one or more internal capabilities or one or more external applications to provide a virtual assistant [(e.g. see Pasley paragraphs 0051, 0088) ”the training of the decoder module may involve use of the algorithms, models, statistical methods, and/or machine learning algorithms as previously discussed. The training of the decoder module may involve fitting one or more models (e.g. linear model, non-linear modulation model) to a training set (e.g. comprising the applied stimulus, brain speech activity data and/or reconstructed speech feature data). The decoder module may reconstruct 112B speech feature data from the brain speech activity data. An output device (e.g. part of or external to the decoder 105) may output 114B reconstructed speech feature data as speech (e.g. synthesized speech and/or as readable text). In certain aspects, this method may be performed iteratively. The training 110B of the decoder module may be based on a number of speech stimuli and resulting brain speech activity data and/or original speech feature data. In certain embodiments, the output device may apply one or more statistical methods and/or machine learning algorithms (such as those described for the decoding module) to classify, or otherwise identify, the speech feature data (or sections thereof) as one or more phonemes, words, pseudowords, phrases, commands, actions, and/or sentences … The speech feature data that is output by the output device may include one or more phonemes, words, pseudowords (e.g., “heef” or “thack”), phrases, commands, actions, and/or sentences. In certain aspects, an output device may convert the speech feature data into readable text. The output device may output the readable text in a tangible (e.g., paper) or intangible form (e.g., a computer display). An output device may also, or instead, convert the speech feature data into audible sound. The audible sound may be played by the output device, such as by a speaker contained in the output device. In certain aspects, the output device may transmit the sound to a receiving device, which produces the audible sound. For instance, a smartphone may convert the speech feature data into audible sound that is transmitted to another phone, which produces the audible sound”].
Therefore, considering the teachings of Yeow, Laszlo and Pasley, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add wherein the at least one capability comprises correlating brain-to-text and/or brain-to-speech commands based on the plurality of unprocessed signals by listening for a trigger word and executing at least one command by interfacing with one or more internal capabilities or one or more external applications to provide a virtual assistant, as taught by Pasley, to the teachings of Yeow and Laszlo because it improves the ability to communicate via spoken language for certain users (e.g. see Pasley paragraph 0003).
As for dependent claim 45, Yeow and Laszlo teach the method as described in claim 1, but do not specifically teach wherein the plurality of signals are acquired by the measurement device to enable the application to generate text based on the output. However, Pasley teaches:
wherein the plurality of signals are acquired by the measurement device to enable the application to generate text based on the output [(e.g. see Pasley paragraphs 0051, 0088) ”the training of the decoder module may involve use of the algorithms, models, statistical methods, and/or machine learning algorithms as previously discussed. The training of the decoder module may involve fitting one or more models (e.g. linear model, non-linear modulation model) to a training set (e.g. comprising the applied stimulus, brain speech activity data and/or reconstructed speech feature data). The decoder module may reconstruct 112B speech feature data from the brain speech activity data. An output device (e.g. part of or external to the decoder 105) may output 114B reconstructed speech feature data as speech (e.g. synthesized speech and/or as readable text). In certain aspects, this method may be performed iteratively. The training 110B of the decoder module may be based on a number of speech stimuli and resulting brain speech activity data and/or original speech feature data. In certain embodiments, the output device may apply one or more statistical methods and/or machine learning algorithms (such as those described for the decoding module) to classify, or otherwise identify, the speech feature data (or sections thereof) as one or more phonemes, words, pseudowords, phrases, commands, actions, and/or sentences … The speech feature data that is output by the output device may include one or more phonemes, words, pseudowords (e.g., “heef” or “thack”), phrases, commands, actions, and/or sentences. In certain aspects, an output device may convert the speech feature data into readable text. The output device may output the readable text in a tangible (e.g., paper) or intangible form (e.g., a computer display). An output device may also, or instead, convert the speech feature data into audible sound. The audible sound may be played by the output device, such as by a speaker contained in the output device. In certain aspects, the output device may transmit the sound to a receiving device, which produces the audible sound. For instance, a smartphone may convert the speech feature data into audible sound that is transmitted to another phone, which produces the audible sound”].
The motivation to combine is the same as that used for claim 32.
Claim 42 is rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 41 above, and further in view of Tyler (US 2012/0289869 A1).
As for dependent claim 42, Yeow and Laszlo teach the method as described in claim 41, but do not specifically teach wherein the studying corresponds to astronauts. However, in the same field of invention, Tyler teaches:
wherein the studying corresponds to astronauts [(e.g. see Tyler paragraphs 0063, 0093) ”Information may be sent to or from an ultrasound device of the present invention and processors or microprocessors and microcontrollers, computer interfacing components, computers and software may be implemented in the transfer of such information. A device of the present invention may comprise components for measuring or detecting physiological status indicators such as heart rate, blood pressure, blood oxygenation levels such as the oxygen content of hemoglobin, hormone levels, or brain activity by detection methods, including but not limited to, EEG … may be worn by … astronauts”].
Therefore, considering the teachings of Yeow, Laszlo and Tyler, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add wherein the studying corresponds to astronauts, as taught by Tyler, to the teachings of Yeow and Laszlo because it improves performance and minimizes the risk of injury to the user and others and accidents (e.g. see Tyler paragraph 0093).
Claim 46 and 47 are rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 41 above, and further in view of Yu et al. (US 2018/0116597 A1).
As for dependent claim 46, Yeow and Laszlo teach the method as described in claim 1, but do not specifically teach wherein the plurality of signals are acquired by the measurement device from a non-human subject. However, in the same field of invention, Yu teaches:
wherein the plurality of signals are acquired by the measurement device from a non-human subject [(e.g. see Yu paragraph 0027) ”FIG. 1 shows an application scenario diagram of the physiological information acquisition system. The system may include but not limited to a physiological sign information acquisition device 101, a living body 102, and a transmission device 103. The physiological sign information acquisition device 101 may acquire, process, extract, and/or analyze physiological information from the living body 102. The living body 102 may include but not limited to a human body, and other living bodies such as animals … The physiological information may include but not limited to physical, chemical, and biological information, such as … brain wave”].
Therefore, considering the teachings of Yeow, Laszlo and Yu, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add wherein the plurality of signals are acquired by the measurement device from a non-human subject, as taught by Yu, to the teachings of Yeow and Laszlo because it improves the ability to diagnose critical care patients (e.g. see Yu paragraph 0024).
As for dependent claim 47, Yeow, Laszlo and Tyler teach the method as described in claim 46, but Yeow and Laszlo do not specifically teach wherein the non-human subject is a pet. However, Yu teaches:
wherein the non-human subject is a pet [(e.g. see Yu paragraph 0024) ”The physiological sign information acquisition system in the present disclosure may be applied to a variety of fields including but not limited to … pet care (critically sick pet care, newborn pet care, home pet care)”].
The motivation to combine is the same as that used for claim 46.
Claims 52 is rejected under 35 U.S.C. 103 as being unpatentable over Yeow et al. (US 2017/0042439 A1) in view of Laszlo et al. (US 2018/0160982 A1), as applied to claim 1 above, and further in view of Albert (US 6,377,833 B1), as applied to claim 29 above, and further in view of Yu et al. (WO 2011/025237 A2, hereinafter: Yu-237).
As for dependent claim 52, Yeow, Laszlo and Albert teach the method as described in claim 29, but do not specifically teach detecting an unknown object in a vision reconstruction; annotating the unknown object with an associated description; providing the description to an external web source; and receiving a response with at least one probability identifying the unknown object. However, in the same field of invention, Yu-237 teaches:
detecting an unknown object in a vision reconstruction; annotating the unknown object with an associated description; providing the description to an external web source; and receiving a response with at least one probability identifying the unknown object [(e.g. see Yu-237 paragraphs 0002, 0049, 0091-0093 and Figs. 9 and 14) ”the image pickup device 100 displays the category selection menu 600 on the screen. Herein, the category refers to a category of the kinds of objects included in the picked-up image. For example, as shown in FIG. 14, the category may include a plant, an insect, a mammal, a building, and others … a user can select which category the object included in the picked-up image belongs to … the image pickup device 100 may transmit the information on the selected category to the external server. The external server the recognizes the object with reference to the selected category, and thus, can recognize the object more accurately and promptly … When a person encounters an unknown object, he/she may take a picture of the object using a camera in order to find out what the object is. Then, he/she discovers information about the object by consulting an encyclopedia or searching the Internet with reference to the photograph of the object … However, if the object cannot be recognized, the control unit 140 operates to transmit the picked-up image to the external server. Herein, the external server refers to a server that recognizes the object from the image and provides information on diverse objects”].
Therefore, considering the teachings of Yeow, Laszlo, Albert and Yu-237, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to add detecting an unknown object in a vision reconstruction; annotating the unknown object with an associated description; providing the description to an external web source; and receiving a response with at least one probability identifying the unknown object, as taught by Yu-237, to the teachings of Yeow, Laszlo and Albert because it improves processing speed when the object cannot be recognized locally (e.g. see Yu-237 paragraph 0050).
Response to Arguments
Applicant's arguments, filed 25 February 2026, have been fully considered but they are not persuasive.
Applicant argues that [“[in] the claimed method … the deep learning module is not a filter and does not output a cleaned signal nor does it discard noise like in Laszlo … the brainwave filtering module (110), even if it has received data that was not filtered by the noise filtering module (104) still outputs a filtered brain activity signal by discarding noise (see paragraphs 0049, 0050) … The examiner’s rejection hinges on the assertion that it would be obvious to omit Laszlo’s optional noise filter … A person of ordinary skill would not be motived to intentionally break Laszlo’s system to pursue a completely different and unstated goal” (Pages 2-4).].
Examiner respectfully disagrees. The output of the claim is recited as a perceptual experience not a signal. Primary reference Yeow already establishes that emotions are detected from the analyzed brainwave data and displayed (see rejection of claim 1 above). Independent claim 1 additionally requires the signals from sensors measuring brain activity, in their raw and unfiltered form and without pre-processing, to be provided as input to a processing system that comprises at least one deep learning module. Laszlo explicitly defines a noise pre-filter as “optional” (see paragraph 0042). This definition in Laszlo provides a teaching of both the brainwave data processing module including a noise pre-filter and the brainwave data processing module excluding a noise pre-filter. Regardless of the optional noise pre-filter, Laszlo teaches a processing system (numeral 102, brainwave data processing module) that comprises at least one deep learning module (numeral 110, brainwave filtering module contains machine learning models to perform signal pattern recognition). The noisy/raw brainwave signal (e.g. Fig. 3A) is provided directly from brainwave sensors (see path/trace from numeral 104 to numeral 102) into the brainwave data processing module (e.g. processing system). Moreover, examiner notes that a noise pre-filter (numeral 114) within the processing module is additionally disclosed as optional; however, the noisy/raw brainwave signal has already been provided to the overarching brainwave data processing module (numeral 102) (e.g. processing system as claimed) from the brainwave sensors. Applicant points to paragraphs 0049, 0050 of Laszlo where the deep learning module begins to process the input signal to perform identification, but this is just further evidence that the signal provided as input to the deep learning module (numeral 110) is noisy/raw. Thus, the combination adequately teaches applicant’s claimed limitation. One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., Inc., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant argues that [“‘optional’ does not make the combination obvious but instead requires impermissible hindsight” (Page 5)].
Examiner respectfully disagrees. It must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant’s disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Applicant argues that [“the examiner provides no teaching, reason, or motivation for why a person of ordinary skill would have combined Yeow with Lazlo” (Page 5)].
Examiner respectfully disagrees. Obviousness can be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so. In re Kahn, 441 F.3d 977, 986, 78 USPQ2d 1329, 1335 (Fed. Cir. 2006). One of ordinary skill in the art, namely a software/hardware developer, would recognize that adding Laszlo’s training of machine learning models to refine and adapt brainwave signal pattern recognition algorithms to Yeow’s brainwave sensing device using supervised/unsupervised algorithms to detect the emotional mental states of the user improves the efficiency of training the models and improves the generalizability of the models (see Laszlo paragraph 0053). The strongest rationale for combining references is a recognition, expressly or impliedly in the prior art or drawn from a convincing line of reasoning based on established scientific principles or legal precedent, that some advantage or expected beneficial result would have been produced by their combination. In re Sernaker, 702 F.2d 989, 994-95, 217 USPQ 1, 5-6 (Fed. Cir. 1983). See also Dystar Textilfarben GmbH & Co. Deutschland KG v. C.H. Patrick, 464 F.3d 1356, 1368, 80 USPQ2d 1641, 1651 (Fed. Cir. 2006) ("Indeed, we have repeatedly held that an implicit motivation to combine exists not only when a suggestion may be gleaned from the prior art as a whole, but when the ‘improvement’ is technology-independent and the combination of references results in a product or process that is more desirable, for example because it is stronger, cheaper, cleaner, faster, lighter, smaller, more durable, or more efficient. Because the desire to enhance commercial opportunities by improving a product or process is universal—and even common-sensical—we have held that there exists in these situations a motivation to combine prior art references even absent any hint of suggestion in the references themselves.").
Applicant argues that [“the ability to generate the ‘perceptual experiences’ described – particularly the generative reconstruction of visual and auditory experiences – is an unexpected result of the counter-intuitive approach recited in claim 1.” (Page 5)].
Examiner cannot locate in independent claim 1, where the generative reconstruction of visual and auditory experiences is recited. However, those features do appear in dependent claims (see the rejections of associated dependent claims above).
Applicant argues that [“the claimed invention, which generates multiple, distinct ‘capabilities’ that form a compound ‘perceptual experience’ is not taught or suggested.” (Page 6)].
Examiner respectfully disagrees. The independent claim is drawn to only identifying “at least one capability”. The broadest reasonable interpretation of “at least one capability” is a single capability that forms a perceptual experience. Yeow teaches wherein at least one capability performed by the particular user forms a perceptual experience in paragraphs 0067, 0076, 0084, 0085 and Figs. 9A-B of Yeow’s disclosure [“a brainwave-sensing device is provided which allows for the detection and display of mental states such as emotions (happiness, anger, sadness, fear, excitement), pain, anxiety, sleep, mental fatigue, comfort and pleasure … The system 100 also comprises one or more of a wide-ranging scope of different applications 122 in which the identified brainwave information 124 may be used … the plot 900 can reveal the brainwave state 904 at the particular time event. The brainwave information can for example be projected on a colored scale to indicate the brainwave levels … projected 918 on a colored/numerical scale to indicate the user's brainwave levels during a certain experience or activity performed by the user”]. Examiner notes that, as defined by applicant’s specification at paragraphs 0035, 0037 and 0094, a capability and perceptual experience can be “emotions” experienced by the user. One of ordinary skill in the art, namely a software developer, would recognize that, as depicted in Figs. 9A-B, multiple measured emotions of the user (e.g. happy, excited, angry) are displayed. Thus, the combination adequately teaches applicant’s claimed limitation.
The Patent and Trademark Office (“PTO”) determines the scope of claims in patent applications not solely on the basis of the claim language, but upon giving claims their broadest reasonable construction “in light of the specification as it would be interpreted by one of ordinary skill in the art.” In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359, 1364[, 70 USPQ2d 1827, 1830] (Fed. Cir. 2004); however, it is impermissible to import subject matter from the specification into the claim, “[T]hough understanding the claim language may be aided by explanations contained in the written description, it is important not to import into a claim limitations that are not part of the claim. For example, a particular embodiment appearing in the written description may not be read into a claim when the claim language is broader than the embodiment.” Superguide Corp. v. DirecTV Enterprises, Inc., 358 F.3d 870, 875, 69 USPQ2d 1865, 1868 (Fed. Cir. 2004). See also Liebel-Flarsheim Co. v. Medrad Inc., 358 F.3d 898, 906, 69 USPQ2d 1801, 1807 (Fed. Cir. 2004).
Citation of Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. PGPub 2018/0189678 A1 issued to Gupta et al. on 05 July 2018. The subject matter disclosed therein is pertinent to the claims (e.g. feature extraction from raw EEG signals).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER J FIBBI whose telephone number is (571)-270-3358. The examiner can normally be reached Monday - Thursday (8am-6pm).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Bashore can be reached at (571)-272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER J FIBBI/Primary Examiner, Art Unit 2174
Prosecution Insights