DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5th June, 2026 has been entered.
Response to Amendment
This action is in response to the amendment filed on 6th May, 2026. Claims 1, 11, and 20 have been amended. Claim 21 has been added. Claims 1-21 remain rejected in the application.
Response to Arguments
Applicant's arguments with respect to Claims 1, 11, and 20 filed on 6th May, 2026, with respect to the rejection under 35 U.S.C. § 103, regarding that the prior art does not teach the limitation(s): "based on a determination that an intended emotion of the user is different from the predicted emotion of the user, modifying a virtual environment theme of the computer generated virtual environment to elicit the intended emotion from the user, wherein each location within the computer generated virtual environment is associated with respective metadata specifying an intended emotion for the user for that location, wherein the intended emotion of the user is determined based at least in part on metadata associated with a location corresponding to the user within the computer generated virtual environment, and wherein for another user at the location within the computer generated virtual environment, the metadata associated with the location specifies another intended emotion that is different from the intended emotion of the user" has been fully considered, but are moot because of new grounds for rejection. It has now been taught by the combination of Baughman, Sumant, and Sterchi.
Regarding arguments to Claims 2-10, 12-19, and 21, they directly/indirectly depend on independent Claims 1, 11, and 20 respectively. Applicant does not argue anything other than independent Claims 1, 11, and 20. The limitations in those claims, in conjunction with combination, was previously established as explained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-13, and 16-21 are rejected under 35 U.S.C. 103 as being unpatentable over Baughman et al. (US 20180217808 A1, previously cited), hereinafter referenced as Baughman, in view of Sumant et al. (US 20200206631 A1, previously cited), hereinafter referenced as Sumant, and further in view of Sterchi et al. (US 20020171647 A1), hereinafter referenced as Sterchi.
Regarding Claim 1, Baughman discloses a method, in a data processing system, for personalizing a computer generated virtual environment (Baughman, [0021]: teaches "a method for modifying a user response to visual and/or audio experience by augmenting the visual and/or audio experience of a user with alternative sounds determined to modify the user response by applying machine learning techniques, based on user feedback <read on personalizing a computer generated virtual environment>"), the method comprising:
collecting, from one or more sensors associated with a user, emotion data representing physiological conditions of the user in response to stimuli (Baughman, [0049]: teaches a machine learning program 300 comparing collective biometric data <read on physiological data of user> of the user's response to a pre-existing repository of biometric data associated with emotional and state-of-mind responses; [0048]: teaches the biometric feedback data coming from a biometric sensor 140; [0046]: teaches "in response to a given stimulus, the menu of populated responses can include a range of emotions <read on emotion data>, or states or mind <read on physiological conditions>, including: fear, anger, sadness, joy, disgust, trust, anticipation, surprise, shame, pity, indignation, envy, and love");
collecting, from one or more data source computing systems, stimuli context data and correlating the stimuli context data with the emotion data (Baughman, [0033]: teaches a repository 130 being a database, where it "includes stored data associating <read on correlating stimuli context data> the effect of one stimulus/response association based on machine learning");
[[training, via a machine learning training process, one or more machine learning computer models based on the emotion data and correlated stimuli context data to thereby generate one or more trained machine learning computer models that are trained to predict an emotion of the user from patterns of input data;]]
receiving runtime emotion data from the one or more sensors associated with the user (Baughman, [0042]: teaches camera 240 being used "to determine a user's response to a given stimulus by associating facial expressions to the relevant emotion <read on runtime emotion data>");
receiving runtime stimuli context data from a virtual environment provider computing system for the computer generated virtual environment (Baughman, [0033]: teaches a repository 130 being a database that can be updated in real-time, where it "includes stored data associating the effect of one stimulus/response association <read on runtime stimuli context data> based on machine learning"; [0040]: teaches an augmented display 235 of an AR headset 115 <read on environment provider computing system> displaying a "realistic representation of a user's surrounding environment <read on computer generated virtual environment>, replacing the real-world view"); and
[[generating, by the one or more trained machine learning computer models, a predicted emotion of the user based on the runtime emotion data and runtime stimuli context data which are input to the one or more trained machine learning computer models; and]]
[[based on a determination that an intended emotion of the user is different from the predicted emotion of the user, modifying a virtual environment theme of the computer generated virtual environment to elicit the intended emotion from the user, wherein]]
[[each location within the computer generated virtual environment is associated with respective metadata specifying an intended emotion for the user for that location, wherein]]
[[the intended emotion of the user is determined based at least in part on metadata associated with a location corresponding to the user within the computer generated virtual environment, and wherein]]
[[for another user at the location within the computer generated virtual environment, the metadata associated with the location specifies another intended emotion that is different from the intended emotion of the user.]]
However, Baughman does not expressly disclose
training, via a machine learning training process, one or more machine learning computer models based on the emotion data and correlated stimuli context data to thereby generate one or more trained machine learning computer models that are trained to predict an emotion of the user from patterns of input data;
generating, by the one or more trained machine learning computer models, a predicted emotion of the user based on the runtime emotion data and runtime stimuli context data which are input to the one or more trained machine learning computer models; and
based on a determination that an intended emotion of the user is different from the predicted emotion of the user, modifying a virtual environment theme of the computer generated virtual environment to elicit the intended emotion from the user, wherein
each location within the computer generated virtual environment is associated with respective metadata specifying an intended emotion for the user for that location, wherein
the intended emotion of the user is determined based at least in part on metadata associated with a location corresponding to the user within the computer generated virtual environment, and wherein
for another user at the location within the computer generated virtual environment, the metadata associated with the location specifies another intended emotion that is different from the intended emotion of the user.
Sumant discloses
training, via a machine learning training process, one or more machine learning computer models based on the emotion data and correlated stimuli context data to thereby generate one or more trained machine learning computer models that are trained to predict an emotion of the user from patterns of input data (Sumant, [0065]: teaches a model generation system 146 that includes a model generation rule set 170 for generating prediction model 160 <read on generate trained machine learning computer model> to predict the emotional state of the user, where "the prediction model 160 and/or the respective parameters 162 of the prediction models 160 may be derived during a training process based on particular input data, such as the historical data 152 <read on patterns of input data>, feedback data 154, and control data 156, and defined output criteria, which may be included with the control data 156, used for training purposes"; [0070]: teaches a retention analysis system 140 receiving input data 172 and applying said input data 172 to prediction model(s) 160, where the input data 172 includes "one or more pieces of sensory data associated with a user who is playing the video game 112"; [0077]: teaches sensory data including sensor data associated with user emotion/emotional state <read on emotion data>, which includes associated audio data, visual/image data, physiological data, etc. <read on correlated stimuli context data> that would trigger the emotional state);
generating, by the one or more trained machine learning computer models, a predicted emotion of the user based on the runtime emotion data and runtime stimuli context data which are input to the one or more trained machine learning computer models (Sumant, [0098]: teaches an emotion analysis system 126 of a retention analysis system 140 using a prediction model <read on machine learning model> to "determine a predicted emotional state <read on generating predicted emotion> for the user based at least in part on the set of sensory <read on runtime stimuli context data> and biometric data <read on runtime emotion data> received or otherwise obtained at the block 402"; [0065]: teaches "the prediction model 160 and/or the respective parameters 162 of the prediction models 160 may be derived during a training process based on particular input data, such as the historical data 152, feedback data 154, and control data 156, and defined output criteria, which may be included with the control data 156, used for training purposes"); and
based on a determination that an intended emotion of the user is different from the predicted emotion of the user, modifying a virtual environment theme of the computer generated virtual environment to elicit the intended emotion from the user (Sumant, [0102]: teaches decision block 408, where if it determines that "the desired emotional state <read on intended emotion> does not match <read on different> the predicted emotional state, the process 400 proceeds to the block 410," which selects a video game configuration that is associated with the desired emotional state for the user, such as reducing the lighting in the game <read on modifying virtual theme> as shown in FIG. 4), wherein
PNG
media_image1.png
674
392
media_image1.png
Greyscale
[[each location within the computer generated virtual environment is associated with respective metadata specifying an intended emotion for the user for that location, wherein]]
the intended emotion of the user is determined based at least in part on metadata associated with a location corresponding to the user within the computer generated virtual environment (Sumant, [0086]: teaches sensors 116 being included as part of a VR/AR system <read on user within computer generated virtual environment>; [0092]: teaches changing state variables <read on metadata> and/or game settings that changes certain aspects of the video game, such as "difficulty, the amount of blood or gore, the music, the sound effects, the volume level of music or sound effects, the lighting, textures or shaders used, particular game events <read on location>, levels made available to the user, and order or sequence of stages or levels, numbers or types of enemies or opponents, numbers or types of non-playable characters, in-game items dropped or made available, game options or expansions presented to the user, or any other factor related to the video game 112 that can have an impact on the user's emotional state when playing or interacting with the video game 112"; Note: the game events are being interpreted as location-based in-game events that occur within a virtual environment, such as cutscene or a triggerable effect (i.e., a jump scare); furthermore, it would be obvious for one skilled in the art to use metadata to affect parts of the level as this is a core aspect of level design; [0096]: teaches process 400, which describes modifying a video game state/configuration based at least in part on a desired emotional state <read on intended emotion> of a user playing video game 112), and wherein
for another user at the location within the computer generated virtual environment, the metadata associated with the location specifies another intended emotion that is different from the intended emotion of the user (Sumant, [0057]: teaches some users expressing a disgust emotion when encountering gore in a video game adventure, and other users <read on another user> not being affected by the presence of gore, where those users may illicit "a more positive reaction <read on another intended emotion> to the realism the gore may introduce to the video game," which would be different from the desired/intended emotion).
Sumant is analogous art with respect to Baughman because they are from the same field of endeavor, namely utilizing neural networks to elicit emotion responses of users via stimuli/triggers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a state modification process based on the elicited emotion of the user, an emotion analysis system, and a retention analysis system as taught by Sumant into the teaching of Baughman. The suggestion for doing so would allow for real-time dynamic scene modification that ensures the user stays engaged with the virtual content, thereby improving the overall user experience. Therefore, it would have been obvious to combine Sumant with Baughman.
However, the combination of Baughman and Sumant does not expressly disclose
each location within the computer generated virtual environment is associated with respective metadata specifying an intended emotion for the user for that location.
Sterchi discloses
each location within the computer generated virtual environment is associated with respective metadata specifying an intended emotion for the user for that location (Sterchi, [0049]: teaches a character moving within a game environment, where the character can read tags <read on respective metadata> that are within proximity, and where "when a tag is detected (step 1308), the tag is read to determine the type of reaction <read on specified intended emotion> that the tag is supposed to elicit from the character and the exact location of the tag in the 3D world (step 1312)").
Sterchi is analogous art with respect to Baughman, in view of Sumant because they are from the same field of endeavor, namely processing emotions in virtual environments. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement tags for objects within a virtual game environment as taught by Sterchi into the teaching of Baughman, in view of Sumant. The suggestion for doing so would allow the system to determine what the intended emotional reaction is, which can then be compared against the actual emotion of the user, thereby yielding similar results. Therefore, it would have been obvious to combine Sterchi with Baughman, in view of Sumant.
Regarding Claim 11, it recites the limitations that are similar in scope to Claim 1, but in a non-transitory computer-readable medium. As shown in the rejection, the combination of Baughman, Sumant, and Sterchi discloses the limitations of Claim 1. Additionally, Baughman discloses a non-transitory computer-readable medium storing a set of instructions for wireless communication (Baughman, [0067]: teaches memory 506 <read on non-transitory computer-readable medium> and persistent storage 508 being computer-readable storage media, where memory 506 "can include any suitable volatile or non-volatile computer-readable storage media"; [0068]: teaches memory 506 being computer-readable storage media that is capable of storing program instructions or digital information),
the set of instructions (Baughman, [0068]: teaches a machine learning program and sound association program 400 being stored "in persistent storage 508 for execution by one or more of the respective computer processors 504 via one or more memories of memory 506") comprising:…
Thus, Claim 11 is met by Baughman according to the mapping presented in the rejection of Claim 1, given the method corresponds to a non-transitory computer-readable medium.
Regarding Claim 20, it recites the limitations that are similar in scope to Claim 1, but in an apparatus. As shown in the rejection, the combination of Baughman, Sumant, and Sterchi discloses the limitations of Claim 1. Additionally, Baughman discloses an apparatus (Baughman, [0079]: teaches an apparatus) comprising:
one or more processors (Baughman, [0079]: teaches a processor of a general purpose computer); and
one or more memory devices coupled to the one or more processors (Baughman, [0068]: teaches a machine learning program and sound association program 400 being stored "in persistent storage 508 <read on memory devices> for execution by one or more of the respective computer processors 504 via one or more memories of memory 506"), wherein
the one or more processors are configured to (Baughman, [0079]: teaches the processor of a general purpose computer):…
Thus, Claim 20 is met by Baughman according to the mapping presented in the rejection of Claim 1, given the method corresponds to an apparatus.
Regarding Claims 2 and 12, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 1 and 11 respectively. The combination of Baughman and Sterchi does not expressly disclose the limitations of Claims 2 and 12; however, Sumant discloses
modifying the virtual environment theme of the computer generated virtual environment based on the predicted emotion of the user (Sumant, [0102]: teaches decision block 408, where if it determines that "the desired emotional state does not match the predicted emotional state, the process 400 proceeds to the block 410," which selects a video game configuration that is associated with the desired emotional state for the user, such as reducing the lighting in the game <read on modifying virtual theme>, to match the predicted emotional state with the desired emotional state).
Sumant is analogous art with respect to Baughman, in view of Sterchi because they are from the same field of endeavor, namely utilizing neural networks to elicit emotion responses of users via stimuli/triggers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a state modification process based on the elicited emotion of the user, an emotion analysis system, and a retention analysis system as taught by Sumant into the teaching of Baughman, in view of Sterchi. The suggestion for doing so would allow for real-time dynamic scene modification that ensures the user stays engaged with the virtual content, thereby improving the overall user experience. Therefore, it would have been obvious to combine Sumant with Baughman, in view of Sterchi.
Regarding Claims 3 and 13, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 2 and 12 respectively. The combination of Baughman and Sterchi does not expressly disclose the limitations of Claims 3 and 13; however, Sumant discloses wherein modifying the virtual environment theme of the computer generated virtual environment based on the predicted emotion of the user comprises
modifying the virtual environment theme to elicit the predicted emotion of the user from the user (Sumant, [0102]: teaches decision block 408, where if it determines that "the desired emotional state does not match the predicted emotional state, the process 400 proceeds to the block 410," which selects a video game configuration that is associated with the desired emotional state for the user, such as reducing the lighting in the game <read on modifying virtual theme>, to match the predicted emotional state <read on elicit predicted emotion> with the desired emotional state).
Sumant is analogous art with respect to Baughman, in view of Sterchi because they are from the same field of endeavor, namely utilizing neural networks to elicit emotion responses of users via stimuli/triggers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a state modification process based on the elicited emotion of the user, an emotion analysis system, and a retention analysis system as taught by Sumant into the teaching of Baughman, in view of Sterchi. The suggestion for doing so would allow for real-time dynamic scene modification that ensures the user stays engaged with the virtual content, thereby improving the overall user experience. Therefore, it would have been obvious to combine Sumant with Baughman, in view of Sterchi.
Regarding Claims 6 and 16, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 1 and 11 respectively. Additionally, Baughman further discloses wherein
the emotion data is collected from the one or more sensors associated with the user in response to stimuli present in a physical environment (Baughman, [0035]: teaches the biometric sensor 140 being a collection of sensors that "are arranged in an area in which a user is presented a stimulus, and the user's response in terms of the biometric measurements are received and associated with the particular stimulus"; [0042]: teaches camera 240 being used "to determine a user's response to a given stimulus by associating facial expressions to the relevant emotion <read on emotion data>"), and wherein
the stimuli context data represents the stimuli present in the physical environment, such that the one or more machine learning computer models learn associations of input patterns of emotional responses of the user to stimuli in the physical environment (Baughman, [0034]: teaches a machine learning program learning a user's response to a particular sound within a distributed computer processing environment 100; [0033]: teaches the machine learning program 300 learning the user responses to various sounds and objects, where it associates one stimulus/response to another stimulus/response to determine the user's behavioral response based on the two stimuli <read on input patterns of emotional responses>), and
applies the learning to virtual stimuli in the computer generated virtual environment (Baughman, [0062]: teaches the sound association program modifying a sound <read on apply learning to virtual stimuli in computer generated virtual environment> transmitted to a user's headphone to obtain the user's response).
Regarding Claims 7 and 17, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 1 and 11 respectively. Additionally, Baughman further discloses wherein
the emotion data is collected from the one or more sensors associated with the user in response to virtual stimuli present in a virtual world environment (Baughman, [0045]: teaches the machine learning program 300 presenting the user a stimulus <read on presented virtual stimuli> and receiving the user's response <read on emotion data> as shown in FIG. 3A), and wherein
PNG
media_image2.png
282
401
media_image2.png
Greyscale
the stimuli context data represents the virtual stimuli present in the virtual world environment, such that the one or more machine learning computer models learn associations of input patterns of emotional responses of the user to virtual stimuli in the virtual world environment (Baughman, [0049]: teaches a machine learning program 300 comparing collective biometric data of the user's response to a pre-existing repository of biometric data associated with emotional and state-of-mind responses, where it "selects the response that most closely matches the biometric data of the user's response, and associates the matched response <read on associations of input patterns of emotional responses> from the pre-existing repository, with the stimulus as the type of response"), and
applies the learning to virtual stimuli in the computer generated virtual environment (Baughman, [0062]: teaches the sound association program modifying a sound <read on apply learning to virtual stimuli in computer generated virtual environment> transmitted to a user's headphone to obtain the user's response).
Regarding Claims 8 and 18, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 1 and 11 respectively. Additionally, Baughman further discloses wherein the emotion data comprises at least one of
brain wave pattern data, heart rate pattern data, perspiration level data, eye dilation data, breathing rate data, facial expression data, body temperature data, or blood pressure data (Baughman, [0035]: teaches "biometric information <read on emotion data> being measured by biometric sensor 140", which includes: "heartrate, breathing rate, voice tonality, skin conductivity levels, temperature, eye dilation, facial expression, iris and retinal recognition, audio response of the user, gait, and vein recognition").
Regarding Claims 9 and 19, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 1 and 11 respectively. Additionally, Baughman further discloses wherein the one or more sensors comprise
at least one of a user wearable sensor or a sensor physically positioned in a physical environment occupied by the user, to monitor the user while the user occupies the physical environment (Baughman, [0031]: teaches the AR Headset 115 <read on user wearable sensor> and the mobile device 120 communicating with each other to provide an AR experience to the user, such as "if a user is in a living room environment and sees a chair while wearing AR Headset 115 <read on monitoring the user in the physical environment>, mobile device 120 will prompt the user to select at least one option relevant to the relationship between the user and the chair").
Regarding Claim 10, the combination of Baughman, Sumant, and Sterchi discloses the method of Claim 1. Additionally, Baughman further discloses wherein the stimuli context data comprises at least one of:
physical environment stimuli context data, collected from the one or more sensors, specifying objects, entities, or physical conditions of a physical environment in which the user occupies at a substantially same time as the emotion data is collected;social networking stimuli context data, from one or more social networking computing systems, specifying relationships between the user and other entities present in the physical environment in which the user occupies at a substantially same time as the emotion data is collected;event stimuli context data, from one or more event data source computing systems, specifying events occurring in the physical environment in which the user occupies at a substantially same time as the emotion data is collected; orlocation stimuli context data, from one or more location services computing systems, specifying characteristics of a physical location of the user corresponding to the physical environment in which the user occupies at a substantially same time as the emotion data is collected (Baughman, [0013]: teaches a stimulus/given stimulus being referred to "a natural or non-natural input sound that is associated with a given physical object <read on physical environment stimuli context data specifying objects in physical environment>, or a digital or non-digital representation of a physical object"; [0014]: teaches a response/user response being referred to "the emotional or psychological state of mind that a user is experiencing upon being exposed to a stimulus"; [0035]: teaches the biometric sensor 140 being a collection of sensors that "are arranged in an area <read on physical location of user> in which a user is presented a stimulus, and the user's response in terms of the biometric measurements are received <read on simultaneous emotion data collection> and associated with the particular stimulus").
Regarding Claim 21, the combination of Baughman, Sumant, and Sterchi discloses the method of Claim 1. The combination of Baughman and Sterchi does not expressly disclose the limitations of Claim 21; however, Sumant discloses wherein the intended emotion from the user is
an emotional response of being afraid or apprehensive (Sumant, [0089]: teaches a negative predicted emotional state <read on emotional response> of a user being feelings of fear <read on afraid> or disgust).
Sumant is analogous art with respect to Baughman, in view of Sterchi because they are from the same field of endeavor, namely utilizing neural networks to elicit emotion responses of users via stimuli/triggers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a state modification process based on the elicited emotion of the user, an emotion analysis system, and a retention analysis system as taught by Sumant into the teaching of Baughman, in view of Sterchi. The suggestion for doing so would allow for real-time dynamic scene modification that ensures the user stays engaged with the virtual content, thereby improving the overall user experience. Therefore, it would have been obvious to combine Sumant with Baughman, in view of Sterchi.
Claims 4-5 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Baughman et al. (US 20180217808 A1, previously cited), hereinafter referenced as Baughman, in view of Sumant et al. (US 20200206631 A1, previously cited), hereinafter referenced as Sumant, and further in view of Sterchi et al. (US 20020171647 A1), hereinafter referenced as Sterchi as applied to Claims 1 and 11 above respectively, and further in view of Cowen et al. (US 20230096485 A1, previously cited), hereinafter referenced as Cowen.
Regarding Claims 4 and 14, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 1 and 11 respectively. Additionally, Baughman further discloses wherein the correlated stimuli context data comprises at least one of
visual stimuli, auditory stimuli, environmental stimuli, social stimuli, object stimuli, or contextual stimuli as specified in a stimuli ontology data structure (Baughman, [0021]: teaches enabling "augmentation of sound detected by a user in response to exposure of the user to a stimulus of detected sounds <read on auditory stimuli>, objects, or both"; [0023]: teaches machine learning algorithms defining characteristics <read on stimuli ontology data structure> of the types of sounds associated with an object), and wherein
[[the stimuli ontology data structure is input to the one or more machine learning computer models to train the one or more machine learning computer models to predict the emotion of the user from patterns of input data.]]
However, the combination of Baughman, Sumant, and Sterchi does not expressly disclose
the stimuli ontology data structure is input to the one or more machine learning computer models to train the one or more machine learning computer models to predict the emotion of the user from patterns of input data.
Cowen discloses
the stimuli ontology data structure is input to the one or more machine learning computer models to train the one or more machine learning computer models to predict the emotion of the user from patterns of input data (Cowen, [0098]: teaches "a model can be trained to predict emotion(s) based on a recording of a user and the system can create/modify virtual characters, or aspects of an AR/VR environment <read on modifying a virtual environment theme>, based on the predicted emotion(s)"; [0099]: teaches "the training data can comprise a plurality of images", where "each training image can be an image that is presented to a participant (e.g., the stimuli <read on stimuli ontology data structure>) and is labeled with the participant's ratings of emotion").
Cowen is analogous art with respect to the combination of Baughman, Sumant, and Sterchi because they are from the same field of endeavor, namely associating stimuli with user emotions. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to train a machine learning algorithm to predict an emotional rating of a user based on stimulus input and image/video training data as taught by Cowen into the combined teaching of Baughman, Sumant, and Sterchi. The suggestion for doing so would allow the machine learning algorithm to assess how specific people, such as therapy patients, would react to certain stimuli, thereby offering ways to aid people in achieving their goals, such as calming down from anxiety. Therefore, it would have been obvious to combine Cowen with the combination of Baughman, Sumant, and Sterchi.
Regarding Claims 5 and 15, the combination of Baughman, Sumant, and Sterchi discloses the method and the non-transitory computer-readable medium of Claims 1 and 11 respectively. The combination of Baughman, Sumant, and Sterchi does not expressly disclose the limitations of Claims 5 and 15; however, Cowen discloses wherein the emotion data comprises
an emotional state ontology data structure specifying a plurality of predefined emotional states of the user (Cowen, [0081]: teaches predefined emotion tags <read on predefined emotional states of a user> being associated with media content <read on emotional state ontology data structure> and/or recording as ratings), and wherein
portions of the emotion data are correlated with corresponding ones of predefined emotional states in the emotional state ontology data structure (Cowen, [0090]: teaches the system being trained to associate certain stimuli with emotions based on the percentage overlap in emotion tags <read on corresponding ones of predefined emotional states> in the input media content <read on emotional state ontology data structure>), and wherein
the emotional state ontology data structure is input to the one or more machine learning computer models to train the one or more machine learning computer models to predict the emotion of the user from patterns of input data (Cowen, [0098]: teaches "a model can be trained to predict emotion(s) based on a recording of a user and the system can create/modify virtual characters, or aspects of an AR/VR environment <read on modifying a virtual environment theme>, based on the predicted emotion(s)"; [0099]: teaches "the training data can comprise a plurality of images", where "each training image can be an image that is presented to a participant (e.g., the stimuli) and is labeled with the participant's ratings of emotion <read on emotional state ontology data structure>").
Cowen is analogous art with respect to the combination of Baughman, Sumant, and Sterchi because they are from the same field of endeavor, namely associating stimuli with user emotions. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement emotion tags to the training data for the machine learning algorithm to predict emotions as taught by Cowen into the combined teaching of Baughman, Sumant, and Sterchi. The suggestion for doing so would allow the machine learning program to infer behavioral patterns and responses from certain environmental settings and stimuli, thereby creating a machine learning model that can accurately anticipate a user's future emotional response. Therefore, it would have been obvious to combine Cowen with the combination of Baughman, Sumant, and Sterchi.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Czerwinski et al. (US 20160063874 A1) discloses a digital personal assistant that determines a mental or emotional state of a user based on signals and providing feedback as a result; and
Primus et al. (US 20180314321 A1) discloses an emotion-based experience feedback system that collects a user's emotional response in response to an event.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KARL TRUONG whose telephone number is (703)756-5915. The examiner can normally be reached 10:30 AM - 7:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.D.T./Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights