DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: an information acquisition unit, a state generation unit, and a reaction generation unit in claims 1-18; a presentation unit in claim 8; a reaction learning unit in claims 9-13 and 15; a state learning unit in claims 11-13 and 16-17; an awkwardness learning unit in claim 14.
The information acquisition unit, the state generation unit, reaction generation unit, reaction learning unit, state learning unit, and awkwardness learning unit and the presentation unit will be interpreted to mean a computer comprising various components such as a display device, input unit, etc. (see paragraphs [0029]-[0030] and [0114]-[0115] of the specification).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7, 9-16 and 19-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a reaction generation apparatus, comprising: an information acquisition unit that acquires subject input information input to a subject and subject brain information of the subject when the subject input information is input; a state generation unit that generates subject state information representing a state of the subject, based on the subject brain wave information; and a reaction generation unit that generates reaction information representing a reaction of the subject, based on the subject input information and the state of the subject. Therefore, the claim is directed to an apparatus, which is one of the statutory categories of invention.
The claim recites the steps of generating subject state information representing a state of the subject, based on the subject brain wave information; and generating reaction information representing a reaction of the subject, based on the subject input information and the state of the subject, as drafted, is process that, under its broadest reasonable interpretation, covers performance of the limitations in the human mind or using a pen and paper, but for the recitation of generic computer components such as a state generation unit and a reaction generation unit to perform these steps. That is, other than reciting “a state generation unit” and “a reaction generation unit” nothing in the claim element precludes the steps from practically being performed in the human mind or using a pen and paper. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception of “Mental Processes” is not integrated into a practical application. In particular, the claim recites the additional element of an information acquisition unit that acquires subject input information input to a subject and subject brain information of the subject when the subject input information is input. This additional element is merely a data gathering step and is recited at a high level of generality, and thus is insignificant extra-solution activity. See MPEP 2106.05(g)). The claim also recites the additional elements of an information acquisition unit, a state generation unit and a reaction generation unit implemented as a computer to perform the acquiring and generating steps. The computer will be treated as generic computer components, recited at a high-level of generality (i.e., as a generic computer performing a generic computer function of the acquiring and generating steps) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements amount to no more than mere instructions to apply an exception and add insignificant extra-solution activity to the abstract idea. The additional element of using a computer to perform all the acquiring and generating steps amounts to no more than mere instructions to apply the exception using a generic computer component. These elements are simply a field of use that is an attempt to limit the abstract idea to a particular technological environment. Although these additional elements did limit the use of the abstract idea, these type of limitation merely confines the use of the abstract idea to a particular technological environment and thus fails to add an inventive concept to the claim. 838 F.3d at 1259, 120 USPQ2d at 1024. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Additionally, the additional limitations, amount to no more than limitations which amount to elements that have been recognized as well-understood, routine, and conventional activity in particular fields. To elaborate: “an information acquisition unit that acquires subject input information input to a subject and subject brain information of the subject when the subject input information is input” is equivalently, receiving or transmitting data over a network, Symantec, MPEP 2106.05(d)(II)(i). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation. The claim is not patent eligible.
Claims 2-7, 9-16 and 19-20 are rejected under 35 U.S.C. 101 based on the same analysis as claim 1 because the claimed invention is directed to an abstract idea without significantly more. For example, claim 19 recites a method for performing the steps recited in the abstract idea of claim 1, and claim 20 recites the a non-transitory computer readable medium for performing the steps recited in the abstract idea of claim 1.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3 and 19-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Aimone et al. (US 2014/0347265, hereinafter Aimone).
Regarding claim 1, Aimone teaches a reaction generation apparatus (wearable computing device) comprising:
an information acquisition unit (wearable computing device) that acquires subject input information input to a subject ([0069]: In combination with GPS (latitude and longitude), digital compass, and accelerometer (angle above plane of the ground) the wearable computing device may include an algorithm to tell what is in a person's field of view. Eye trackers can also be used to fine tune the direction of where a person is looking after an estimate using digital compass, GPS, and accelerometer) and subject brain wave information ([0051]: a wearable computing device comprising at least one bio-signal measuring sensor, the at least one bio-signal measuring sensor including at least one brainwave sensor. The method may include acquiring at least one bio-signal measurement from a user using the at least one bio-signal measuring sensor. The at least one bio-signal measurement may include at least one brainwave state measurement. The wearable computing device may process the at least one bio-signal measurement, including at least the at least one brainwave state measurement, in accordance with a profile associated with the user) of the subject when the subject input information is input ([0050]: The sensors may include one more bio-signal sensors, such as electroencephalogram (EEG) sensors, galvanometer sensors, electrocardiograph sensors, heart rate sensors, eye-tracking sensors, blood pressure sensors, pedometers, gyroscopes, and any other type of sensor. The sensors may be connected to the wearable computing device, such as a wearable headset or headband computer worn by the user. The sensors may be connected to the wearable computing device by wires or wirelessly; [0069]: Knowing what a person is looking at in combination with analysis and interpretation of their brainwaves; [0070]: The wearable computing device may be configured to associate the visual input with the at least one brainwave state measurement and update the profile with the associated visual input. The identified at least one object may comprise at least one food item. The associating the visual input may comprise updating the profile with an identification of the at least one food item associated with the at least one brainwave state measurement; [0155]: The wearable computing device may use visual data, combined with sensory input, to gauge an "attractiveness factor" between a user and another individual. In particular, brainwave and heart rate sensors may be used to attempt to assist the user in finding or making a connection with another person based on the gauged attractiveness factor; [0181]: The device detects Brain State, with available Sensors. Available Sensors include: cameras, galvanic skin response, bone vibrations, muscle twitch sensors, accelerometers, pheromone and hormone detectors, gyrometers, and basic brainwave sensors. Analysis of EEG data indicates a brain state that requires additional processing);
a state generation unit (wearable computing device) that generates subject state information representing a state of the subject (emotional state/brain state; [0085]: Brain states such as like/dislike, error perception, etc.; [0110]: The wearable computing device could be configured to determine particular brain states of the user), based on the subject brain wave information ([0069]: Knowing what a person is looking at in combination with analysis and interpretation of their brainwaves can enable useful applications such as: advertisers knowing brain state response (e.g. like/dislike/emotional etc.) response to their fixed billboard ads, knowing who the person is looking and their emotional reaction (e.g. like/dislike/emotional etc.) to that person (assuming the location of the other person is known), how often objects in a city are looked at and people's brain state response; [0117]: Both brainwaves and EMG measurements may be used by the wearable computing device to deduce the emotional or cognitive state of the user; [0132]: The wearable computing device may modify the delivery of audio or video media to the user based on measured brainwaves. The user's real-time brain state (e.g. attention, joy, synchronization) may be used to modulate or filter the audio to change the experience to enhance the user's brain state; [0157]: the user's brainwaves are analyzed to determine that the brain state of the user is one of attraction towards another human being; [0205]: In another example, "John" is a teacher. John uses data from the wearable computing device to learn about the mental and emotional states of his students in the classroom, based on changes to contextual baseline Brain State. The device regularly pings other devices of the present invention worn by students for ERNs, P300s, change in focus (beta to theta ratio) and other brainwave characteristics as well elevated heart rate; [0263]: The wearable computing device may analyze the user's brainwaves to determine the user's current emotional state); and
a reaction generation unit (wearable computing device) that generates reaction information representing a reaction of the subject (emotional eating), based on the subject input information (time, location and type of food the user has eaten) and the state of the subject (emotional state such as happy, sad, bored; [0210]: the wearable computing device may detect a user's response to a particular stimulus using brain state information; [0269]: In an application, a goal may be to make the user aware that they are about to eat based on an emotional trigger rather than hunger. Often the food choices for emotional eating are very specific to the user and can vary depending on the mood of the user, and what you reach for when eating to satisfy an emotion may depend on the emotion. People in happy moods tended to prefer foods such as pizza or steak while sad people may prefer ice cream and cookies, and bored people may prefer potato chips; [0270]: First, the wearable computing device may learn the user's patterns of emotional eating. Emotional eating patterns are usually associated with time, location, and an emotion or thought. The device may gather data about the situations and conditions that lead to emotional eating. The device may analyze the emotional state of the user on an almost continuous basis. The device may use its video camera to determine the type of food that the user has eaten; [0271]: Next, the device may monitor the brainwaves of the user to alert the user to the onset of emotional eating. After the device learns a set of rules then it can monitor the emotions of the user, their location, time of day to determine the probability that the user will engage in emotional eating; [0272]: Next, the device may attempt to help the user become more self-aware and suggests strategies. If the probability of emotional eating is high then the device may make the user aware of its prediction).
Regarding claim 2, Aimone teaches the reaction generation apparatus according to claim 1, wherein the information acquisition unit further acquires biological information of the subject when the subject input information is input (while Dan, the user, is viewing the scenery and finds someone he is attracted to, the heart rate and brainwave information is measured by the wearing computing device; [0113]: the wearable computing device may include an optical sensor and transmitter placed facing the skin to measure the user's heart rate; [0114]: measurements of the user's facial expressions or emotional response may be taken by performing ectromyogram ("EMG") measurements, and associated expressions or emotions may be determined by the wearable computing device; [0155]: The wearable computing device may use visual data, combined with sensory input, to gauge an "attractiveness factor" between a user and another individual. In particular, brainwave and heart rate sensors may be used to attempt to assist the user in finding or making a connection with another person based on the gauged attractiveness factor), and the state generation unit generates the subject state information (emotional state such as finding someone attractive) based on the subject brain wave information (heart rate) and the biological information (brainwave information; [0157]: Dan sees someone who he thinks is attractive. The attraction causes an increase in heart rate which is recognized by his heart-rate monitor. In a different example, the user's brainwaves are analyzed to determine that the brain state of the user is one of attraction towards another human being. Also, multiple sensors such as heart rate and predicted brain state may be combined to indicate that the user finds another attractive).
Regarding claim 3, Aimone teaches the reaction generation apparatus according to claim 2, wherein the information acquisition unit acquires the subject brain wave information before and after the subject input information is input (contextual baseline brain state is acquired from the brainwave information before the user sees a person he is attracted to in the scenery that he is viewing and a changed brain state is acquired from the brainwave and heart rate information after the user sees a person he is attracted to; [0191]: His therapist suggests he wear a wearable computing device of the present invention or suite of wearable computing devices paired with a wearable display device, so that the visual and environmental stimuli can be logged for discussion. The device notes mood changes and changes in sleep patterns and distraction, via brainwave characteristics. When Chris differentiates from contextual baseline Brain State, the wearable computing device takes a photograph of what he is staring at, determining the object of his gaze via EOG/eye-tracking when the emotion is over a threshold. Over time, an iterative mood profile of Brain State plus visual information develops), and the state generation unit generates the subject state (changes to brain state; [0203]: The wearable computing device of the present invention may determine Brain State changes that indicate a response to specific stimuli that is different from contextual baseline Brain State. The device may relay augmented reality information to the user through the display in those specific instances that will help them navigate situations in real time. This may be accomplished by the device establishing a contextual baseline Brain State (e.g. via an Algorithmic Pipeline) after filtering data from multiple sensor inputs) information based on a change from the subject brain wave information before the subject input information is input (baseline brainwaves) to the subject brain wave information after the subject input information is input (brain state has changed from his baseline to indicate attractiveness), and the biological information ([0170]: The wearable computing device may establish a contextual baseline Brain State based on the wearer's profile, and possibly other brainwave characteristics that are aggregated statistics from other users. It uses an Algorithmic Pipeline to determine changes to Brain State, and determines if the user is attracted to the person the user is looking at via brainwave characteristics that are associated with attraction; [0172]: a user, "Bob", is wearing the wearable computing device when he notices a person whom he finds attractive at the coffee shop. The wearable computing device detects him noticing her, because his Brain State has changed from his baseline to indicate attractiveness; [0224]: The wearable computing device may establish a contextual baseline Brain State based on prior brainwave patterns in the user, and aggregate user data from other users made publicly available in the cloud, via server, or on local devices with onboard storage. By filtering inputs through the Algorithm Pipeline, the device may notice changes to baseline Brain State via multiple sensors, such as camera eye tracking/EOG, Beta:Theta ratio changes, P300s, ERNs, galvanic skin and temperature changes, and changes to heart rate. The device turns to processing rules for each input to determine an output that is context-dependent. The device can pattern match these changes to Brain State against external inputs, stimuli, or events).
Claims 19 and 20 are similar in scope to claim 1, and therefore the examiner provides similar rationale to reject these claims. Moreover, Aimone teaches a non-transitory computer readable medium ([0007]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 4-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aimone, in view of Yeow et al. (WO 2015/122846, hereinafter Yeow), and further in view of Song (US20230218215).
Regarding claim 4, Aimone does not explicitly teach the reaction generation apparatus according to claim 3, wherein the state generation unit generates the subject state information based on a change from a proportion, in a total amplitude, of an amplitude of brain wave at a predetermined frequency band in the subject brain wave information before the subject input information is input to a proportion, in the total amplitude, of an amplitude of brain wave at the frequency band in the subject brain wave information after the subject input information is input, and a ratio of a magnitude of a first power spectrum to a magnitude of a second power spectrum in a heartbeat of the subject, the total amplitude is a sum of amplitudes of an alpha wave, a beta wave, a theta wave, a gamma wave, and a delta wave, and a frequency band of the second power spectrum is a band that is of a higher frequency than a frequency band of the first power spectrum.
Yeow teaches the state generation unit generates the subject state information ([page 11, lines 13-15] “brainwave-sensing device is provided which allows for the detection and display of mental states such as emotions (happiness, anger, sadness, fear, excitement),pain, anxiety, sleep, mental fatigue, comfort and pleasure.” Where displaying emotions comprises presenting the emotional state of a target person) based on a change from a proportion ([page 11, lines 1-10] “multiple electrode(s) can be placed on a localized area of the head to detect specific brainwaves of interest of the user. This serves to detect brainwave status information such as, but not limited to, sleep, attention, happiness, anger, sadness, pain, anxiety, fear and excitement,” and “the brainwave information can be plotted against time (plot 900), whereby selecting a time segment e.g. 902 of the plot 900 can reveal the brainwave state 904 at the particular time event. The brainwave information can for example be … plotting continuous real time brainwave information… Physiological measurements such as. heart rate and body temperature measured by incorporated sensors can also be displayed and correlated with the brainwave data for more meaningful data interpretation” where continuous physiological measurements which generate measurement of a subject’s emotion comprises a change in a brainwave and heartbeat proportion), in a total amplitude, of an amplitude of brain wave at a predetermined frequency band in the subject brain wave information before the subject input information is input to a proportion, in the total amplitude, of an amplitude of brain wave at the frequency band in the subject brain wave information after the subject input information is input ([Figure 7, and page 10 lines 14-25] “Fig. 7, brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see plots 702-706. These waveforms include alpha (8- 13Hz), delta (0.5-4Hz), beta ( 14-30Hz) and theta (4-8Hz). In example embodiments, a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see plots 702-706.” Where the waveforms of different brainwaves (comprising the first and second brainwave proportion) are measured before and after audio-visual stimuli (i.e., before and after engagement) and encompass measuring the change from one state of mind to another (comprising one proportion of amplitude to another)), and a ratio of a magnitude of a first power spectrum to a magnitude of a second power spectrum in a heartbeat of the subject (see [page 11, lines 1-10] where plotting physiological signals of heart rate against time comprises a power spectrum of a heartbeat of the target person). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Yeow’s knowledge to generate the subject state information as taught and modify the system of Aimone because such a system relates to detection, processing and display of brainwaves and utilizing the brainwave data for a variety of different applications with the aim of improving performance, quality of life and healthcare (page 5 lines 12-14).
Song teaches the total amplitude is a sum of amplitudes of an alpha wave, a beta wave, a theta wave, a gamma wave, and a delta wave ([0081] “According to the range of oscillating frequencies, EEG signals are artificially classified into and referred to as delta (δ) waves (0.2 to 3.99 Hz), theta (θ) waves (4 to 7.99 Hz), alpha (α) waves (8 to 12.99 Hz), beta (β) waves (13 to 29.99 Hz), and gamma (γ) waves (30-50 Hz).” Where classifying the signals into different waves comprise measuring the summation of each wave; see also [Fig. 22] where the proportion of frequencies are mapped out to include a changing portion of the brain wave amplitudes before and after engagement), and a frequency band of the second power spectrum is a band that is of a higher frequency than a frequency band of the first power spectrum ([0224] “In the present invention, time-series characteristics are generated from data at the time when the data changes from the normal EEG waves of joy and pleasure to the abnormal EEG waves of fear, sadness, anger, disgust, and depression by applying the CRNN to the power spectrum distribution structure separated into multiple waves in the frequency region, and brain states having different lengths are detected for each person and for each occurrence by using these characteristics, thereby generating emotional data.” Where different lengths of brain states correlates to different frequency bands of power spectrums). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Song’s knowledge of using the artificial intelligence emotional data extraction and power spectrum analysis to map the time series features of a brainwave signals before and after events and modify the system of Aimone and Yeow because such a system would have provided a more in depth display of information for healthcare providers and thus better communicating an emotional state of a user.
Regarding claim 5, the combination of Aimone, Yeow and Song teaches the reaction generation apparatus according to claim 4, wherein the state generation unit generates the subject state information based on a magnitude relationship of a ratio of a magnitude of the first power spectrum to a magnitude of the second power spectrum and a predetermined threshold of a ratio of the magnitude of the first power spectrum to the magnitude of the second power spectrum (Yeow - [page 11, lines 1-10] “multiple electrode(s) can be placed on a localized area of the head to detect specific brainwaves of interest of the user. This serves to detect brainwave status information such as, but not limited to, sleep, attention, happiness, anger, sadness, pain, anxiety, fear and excitement,” and “the brainwave information can be plotted against time (plot 900), whereby selecting a time segment e.g. 902 of the plot 900 can reveal the brainwave state 904 at the particular time event. The brainwave information can for example be … plotting continuous real time brainwave information… Physiological measurements such as. heart rate and body temperature measured by incorporated sensors can also be displayed and correlated with the brainwave data for more meaningful data interpretation” where continuous physiological measurements which generate measurement of a subject’s emotion comprises a change in a brainwave and heartbeat proportion; see also Yeow- [page 27, lines 39-44] “addition of electrodes to the device can provide users with a broader range of brainwave information at different parts of the brain; or the addition of sensors such as heart rate or temperature sensors for measurement of other physiological signals. This expands the scope of use of the device in its ability to read brainwave data whilst correlating it with other vital signs.” Where the proportions of heart rates increases to a threshold that indicates a different emotion), and the change from the proportion, in the total amplitude, of an amplitude of the brain wave of the frequency band before the subject input information is input to the proportion, in the total amplitude, of an amplitude of the brain wave of the frequency band after the subject input information is input (Yeow - [Figure 7, and page 10 lines 14-25] “Fig. 7, brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see Yeow - plots 702-706. These waveforms include alpha (8- 13Hz), delta (0.5-4Hz), beta ( 14-30Hz) and theta (4-8Hz). In example embodiments, a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see Yeow - plots 702-706.” Where the waveforms of different brainwaves (comprising the first and second brainwave proportion) are measured before and after audio-visual stimuli (i.e., before and after engagement) and encompass measuring the change from one state of mind to another (comprising one proportion of amplitude to another)).
Regarding claim 6, the combination of Aimone, Yeow and Song teaches the reaction generation apparatus according to claim 4, wherein the subject state information includes information according to a plurality of states of the subject (Aimone – [0085]: Brain states such as like/dislike, error perception, etc.; Aimone – [0210]: Daniel cycles through positive and negative mental states as he listens to the debating points; Aimone - [0213]: The following are example brain states that could be inferred: like, dislike, emotional valence (i.e. positive or negative emotions) and arousal (low energy to high energy)), and the state generation unit generates the subject state information according to one state among the plurality of states ([page 11, lines 13-15] “brainwave-sensing device is provided which allows for the detection and display of mental states such as emotions (happiness, anger, sadness, fear, excitement),pain, anxiety, sleep, mental fatigue, comfort and pleasure.” Where displaying emotions comprises presenting the emotional state of a target person) based on a ratio of a magnitude of the first power spectrum to a magnitude of the second power spectrum and the change from the proportion (Yeow - [page 11, lines 1-10] “multiple electrode(s) can be placed on a localized area of the head to detect specific brainwaves of interest of the user. This serves to detect brainwave status information such as, but not limited to, sleep, attention, happiness, anger, sadness, pain, anxiety, fear and excitement,” and “the brainwave information can be plotted against time (plot 900), whereby selecting a time segment e.g. 902 of the plot 900 can reveal the brainwave state 904 at the particular time event. The brainwave information can for example be … plotting continuous real time brainwave information… Physiological measurements such as. heart rate and body temperature measured by incorporated sensors can also be displayed and correlated with the brainwave data for more meaningful data interpretation” where continuous physiological measurements which generate measurement of a subject’s emotion comprises a change in a brainwave and heartbeat proportion; see also Yeow- [page 27, lines 39-44] “addition of electrodes to the device can provide users with a broader range of brainwave information at different parts of the brain; or the addition of sensors such as heart rate or temperature sensors for measurement of other physiological signals. This expands the scope of use of the device in its ability to read brainwave data whilst correlating it with other vital signs.” Where the proportions of heart rates increases to a threshold that indicates a different emotion), in the total amplitude, of an amplitude of the brain wave the frequency band before the subject input information is input to the proportion, in the total amplitude, of an amplitude of the brain wave of the frequency band after the subject input information is input (Yeow - [Figure 7, and page 10 lines 14-25] “Fig. 7, brainwaves of interest that can be captured in example embodiments include, but are not limited to, brainwave states of happiness, excitement, attention, motivation, anger, sadness/depression, pain, sleep, anxiety and fear. Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see Yeow - plots 702-706. These waveforms include alpha (8- 13Hz), delta (0.5-4Hz), beta ( 14-30Hz) and theta (4-8Hz). In example embodiments, a calibration procedure/system of the basal level of the user's brainwave states can be performed. The user 707 will for example first be exposed to different audio-visual brainwave stimuli for a particular brainwave state 708-712, and the waveforms that arise from the triggered brainwaves will be detected, see Yeow - plots 702-706.” Where the waveforms of different brainwaves (comprising the first and second brainwave proportion) are measured before and after audio-visual stimuli (i.e., before and after engagement) and encompass measuring the change from one state of mind to another (comprising one proportion of amplitude to another)).
Regarding claim 7, the combination of Aimone, Yeow and Song teaches the reaction generation apparatus according to claim 6, wherein the brain wave of the frequency band is at least one of a delta wave, a theta wave, a low alpha wave, or a medium alpha wave (Yeow - [page 10, lines 18-19] “These waveforms include alpha (8-13Hz), delta (0.5-4Hz), beta (14-30Hz) and theta (4-8Hz).” Where depicting the various wave forms to label emotions comprise categorizing predetermined frequencies), or at least one of a high alpha wave, a low beta wave, a high beta wave, or a gamma wave (Yeow - [page 10, lines 18-19] “Different brainwave states tend to show activation in different brain regions and exhibit different waveforms of varying frequency, see plots 702-706. These waveforms include alpha (8- 13Hz), delta (0.5-4Hz), beta (14-30Hz) and theta (4-8Hz).” Where the range of alpha waves comprise both a low and high beta wave and depicting the various wave forms to label emotions comprise categorizing predetermined frequencies).
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aimone, and further in view of Kwalwasser et al. (US 2023/0218221, Kwalwasser).
Regarding claim 16, the combination of Aimone, Yeow and Song does not explicitly teach the reaction generation apparatus according to claim 2, further comprising a state learning unit that generates a state inference model for inferring a state of the subject based on the subject input information through machine learning of a relationship between the subject input information and the subject state information.
Kwalwasser teaches a state learning unit that generates a state inference model for inferring a state of the subject based on the subject input information through machine learning of a relationship between the subject input information and the subject state information (Abstract, [0021]-[0022], [0025], [0028], [0030]-[0033], [0046]-[0049], and FIG. 1-3: The brain state model generator 160 trains one or more brain state models based on the extracted features from the brain activity signal associated with an audio stimulus and the user survey responses. In one embodiment, the brain state models are random forest regression models. In other embodiments, the brain state models utilize different machine learning techniques, e.g., neural networks, multinomial regressors, other decision trees, etc. The trained brain state models are configured to predict a value for the brain state (or the brain state value over time) based on an input brain activity signal that was captured when performing tasks with an audio stimulus. The brain state models may be stored in the data store 180. For a given user, the server 150 may select a brain state model that best fits the user's survey responses and brain activity data associated with the audio stimulus. The best fit model provides the closest prediction of the user's brain state value based on the brain activity data associated with the audio stimulus. The selected brain state model may be stored in a user profile for that user, such that the server 150 may provide tailored content to each user). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply Kwalwasser’s knowledge of using a state inference model as taught and modify the system of Aimone because such a system enhances the user experience by providing optimal stimulations to the user based on his/her condition.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hazur et al. (US 2018/0018540) describes a method for affect based learning is provided comprising: presenting one or more instructional component(s) to a learner; capturing emotional state data of the learner in response to the one or more instructional component; inferring emotional states, using one or more processors, based on the emotional state data which was captured and analysis of the facial data; capturing cognitive data for the learner based on the learner's response to the one or more instructional component; assigning a positive or negative adaptivity score based on the individual's emotional state and cognitive data; populating a learner profile with the adaptivity score; and providing alternative learning relating to the one or more instructional components in the event of a negative adaptivity score, and presenting a higher level of learning to the learner in the event of a positive adaptivity score.
Chandrasekaran et al. (US 2019/0266999) describes how, given a conversational state 20 and inferred emotional state 24, response selector 26 may vary the text response sent to the TTS subsystem 28 whereby responses may be varied based on rules or lookups against pre-selected responses or encoded as responses generated by a learned model such as a deep neural network to generate conversationally and emotionally appropriate responses having the appropriately contextualized text and tone.
Allowable Subject Matter
Claim 18 is allowed.
The following is an examiner’s statement of reasons for allowance:
Regarding claim 18, Aimone et al. (US 2016/0077547) describes the reaction of the subject is presented by a presentation unit that presents a virtual person model ([0167], [0170], and FIG. 10-11: The EEG readings from other members of the group affects the appearance of their avatars. e.g. if the person is relaxed, their avatars can glow blue; if they are tense, their avatars would glow red. The individual participants can look at their own avatars to determine how they are proceeding with the meditation session; their own avatars will change colour like those in the rest of the group. Emotions can also be displayed on the avatars; i.e. the words “ANGRY” or “ANXIOUS” can appear on their faces; [0113], [0169], and FIG. 10-11: The device measures her brainwaves and changes the characteristics of the virtual pet accordingly to provide visual feedback. For example, the pet changes colour from green to red when Danielle is upset; it changes back from red to green when she enters a relaxed state. Alternatively, the pet can change its own behaviour: irritated, relaxed, angry, etc. The pet can be used as a mindfulness/meditation aide - Danielle tries to get her pet to change colour to a certain state to match whatever mindfulness goals she is aiming towards; [0112], [0167], [0170], and FIG. 10-11: The usage of the facial sensors (an example of bio-signal sensors of wearable device 1002, 1004) may allow for the mapping of a user's expression to the face of their avatar 1010, 1012 in a VR environment (e.g. to represent detected facial states with associated smiles, squints, winks, furrows, frowns, etc.). This can be augmented with brain signals from wearable device 1002, 1004 to do emotion estimation by device 1008, 1006. This estimate can further augment a characters appearance in the VR environment as an example of feedback). Yeow describes the information acquisition unit further acquires user brain wave information of a user who came into contact with the reaction presented by the user (page 20 lines 16-20: embodiments of the invention can be extended to group chats where a customized cluster of people, such as family or close friends are given the option to share and track one another's brainwaves via a platform 2800 that could allow simultaneous exchanges of short text messages through web or mobile communication systems, as illustrated in Fig. 28); the state generation unit generates user state information indicating a state of the user based on the user brain wave information (fig. 27 shows the video chat displays the emotions of the other user to the respective users). Further, Moll (US 2023/0410441) describes awkward state of a user ([0017]: although users may see that a virtual object is being touched, without haptic feedback being provided in response to a user's interaction, the user's interaction with the virtual object may be awkward or result in frustration for the user).
However, none of the cited prior art references of record, teach, either individually or in combination, “the information acquisition unit acquires, from the user, a feedback for the reaction presented by the virtual person model, when the user state information is a predetermined awkward state, and the reaction generation unit corrects at least one of the subject state information or the reaction information based on the feedback”.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Claims 8 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 8, none of the cited prior art references of record, teach, either individually or in combination, “the information acquisition unit acquires, from the user, a feedback for the reaction presented by the virtual person model, when the user state information is a predetermined awkward state, and the reaction generation unit corrects at least one of the subject state information or the reaction information based on the feedback”.
Regarding claim 17, none of the cited prior art references of record, teach, either individually or in combination, “the information acquisition unit acquires, from the user, a feedback for the reaction presented by the virtual person model, when the user state information is a predetermined awkward state, and the state learning unit corrects the state inference model based on the feedback”.
Further, the following subject matter is also considered to be allowable over cited prior art references of record:
the reaction generation apparatus further comprising a reaction learning unit that generates a reaction inference model for inferring a reaction of the subject based on the subject state information through machine learning of a relationship between the subject state information and the reaction of the subject acquired by the information acquisition unit.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JWALANT B AMIN whose telephone number is (571)272-2455. The examiner can normally be reached Monday-Friday 10am - 630pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JWALANT AMIN/Primary Examiner, Art Unit 2612
Prosecution Insights