DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Claim 20 does not fall within at least one of the four categories of patent eligible subject matter because it is directed to a computer readable storage medium per se, which may encompass transitory propagating signals. Moreover, claims 1-20 are directed to an abstract idea without significantly more. The claims recite a method and system for estimating the cognitive state of a user and generating content on the basis of that cognitive state. Observing the behavior of a human and adapting entertainment to that human is an abstract idea in the form of mental activities and/or certain methods of organizing human activity. These judicial exceptions are not integrated into a practical application because they do not improve the functioning of any particular computer or item. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the claims use a variety of computerized elements in generic and conventional ways that are known in the art of video gaming (see prior art cited herein).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 14-16, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 2020/0342648 to Shimizu et al. (hereinafter Shimizu).
Regarding claims 1, 15, and 20, Shimizu teaches a computer-implemented system and method of providing dynamic content in a virtual environment hosted on a cloud platform (e.g., a cloud server that analyzes a user to estimate user emotion in ¶ 139), comprising:
providing a virtual environment to a user (e.g., a game environment in ¶ 129);
estimating a cognitive state of the user (e.g., emotions such as bored, joy, anger, etc. in ¶¶ 188-190); and
generating content in the virtual environment according to the cognitive state of the user (e.g., generate an image that presents a quest for reinforcement of target behavior in ¶ 157).
Regarding claims 2 and 16, Shimizu teaches wherein estimating the cognitive state of the user comprises at least one of: estimating the cognitive state of the user in real time; and estimating a change in cognitive state of the user over a predetermined period (e.g., real-time biometric information and emotions in ¶ 201).
Regarding claim 14, Shimizu teaches wherein the cognitive state of the user indicates the user being at least one of: tired, stressed, bored, or engaged (e.g., emotions such as bored, joy, anger, etc. in ¶¶ 188-190).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 5-13, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Shimizu in view of US 2023/0047787 to Chappell, III et al. (hereinafter Chappell).
Regarding claims 3 and 17, Shimizu teaches further comprising estimating cognitive state by: receiving sensor data from user peripherals, the sensor data indicative of behavioral and/or physiological signals (e.g., electroencephalograph 31 and the biometric information supplied from the biometric sensor 32 in the server 43, and displays the emotion of “excited” in ¶ 202). However, Shimizu lacks in explicitly teaching generating, in a machine learning cloud-deployed model, a state vector from the sensor data, the state vector representing the estimated cognitive state of the user.
In a related disclosure, Chappell teaches a system for controlling progress of audio-video content based on sensor data of multiple users, among other methods, using sensor data positioned on users (abstract). More particularly, Chappell teaches that a correlation module 206 may include instructions that when executed by the processor 202 and/or 214 cause the server to correlate biometric sensor data to one or more neuro-physiological (e.g., emotional) states of the user, using machine learning (ML) or other processes (see e.g., ¶ 52). Furthermore, Chappell teaches a multi-dimensional vector with values representing intensity of psychological qualities such as cognitive load, arousal, and valence (see e.g., ¶ 73). Finally, Chappell teaches that the server may be a cloud-based data server 122 (see e.g., ¶ 46). It would have been obvious to one of ordinary skill in the art before the effective date to modify the system of Shimizu to include the features of Chappell discussed above in order to allow centralized collection of player emotion data in a cloud server to be analyzed with a machine learning vector system that can then be applied to other users of the system.
Regarding claim 5, Shimizu teaches wherein sensor data indicative of physiological signals includes at least one of: hand poses estimated from an accelerometer and/or gyroscope in a user peripheral; eye or gaze tracking in VR-based games; headphone audio pickup; head poses estimated from an accelerometer and/or gyroscope in a head mounted display, HMD, or headphones (e.g., the electroencephalogram can be detected using the time tr as a timing based on an operation such as the attitude or gesture of the information processing apparatus 41 including the imaging unit 94 measured by the accelerometer in ¶ 243); or galvanic skin response or electrochemical activity detected in a user peripheral. See also Chappell at ¶ 55 discussing several sensor types claimed.
Regarding claims 6 and 18, Chappell teaches or suggests wherein the machine learning cloud-deployed model is a generalized user agnostic model that is trained from an offline dataset and deployed on the cloud platform (e.g., engagement measures are generalized to a model space and an algorithm can use the neuro-physiological model for operation of the application dynamically based on the psychological state or predisposition of the user in ¶¶ 82-83).
Regarding claim 7, Chappell teaches or suggests wherein the training data set is a labelled collection of cognitive states according to different physiological and behavioral signals from a plurality of users (e.g., data may be associated with a specific user or cohort or may be generic, such that if most users exhibit similar biometric tells when engaged with similar social interactions (e.g., friendly, happy, angry, scary, seductive, etc.), each similar interaction can be classified with like interactions that provoke similar biometric data from users in ¶ 77).
Regarding claim 8, Chappell teaches or suggests wherein the machine learning cloud-deployed model is trained based on a specific player through federated learning and deployed on the cloud platform, thereby training personalized models (e.g., correlation module 206 may include instructions that when executed by the processor 202 and/or 214 cause the server to correlate biometric sensor data to one or more neuro-physiological (e.g., emotional) states of the user, using machine learning (ML) or other processes in ¶ 52; see also ¶ 77 discussing training of a machine learning model based on a specific user or plurality of users).
Regarding claim 9, Chappell teaches or suggests wherein training through federated learning includes performing user calibration during a first setup phase for said specific player (e.g., Generic input data can be used to calibrate a baseline for neuro-physiological response to a scene, to classify a baseline neuro-physiological response to stimuli that simulates social interaction in ¶ 77; see also ¶ 74 discussing a personal calibration session).
Regarding claim 10, Chappell teaches or suggests wherein said user calibration includes, for a predetermined application, collection of data from interaction with a predetermined calibration application (e.g., to obtain baseline correlations between sentic modulations and neuro-physiological states, player actors may be shown a known visual stimulus (e.g., from focus group testing or a personal calibration session) to elicit a certain type of emotion in ¶ 74).
Regarding claims 11-12, Chappell teaches or suggests wherein user calibration data is stored locally or wherein further updates corresponding to changes in user interactions are transmitted to the cloud-based model (e.g., a component or module may be localized on one computer and/or distributed between two or more computers in ¶ 203).
Regarding claims 13 and 19, Chappell teaches or suggests wherein once the machine learning cloud-deployed model is trained and deployed on the cloud platform, the method further comprises providing a query-based interface thereby rendering estimates of the cognitive state vector from the model accessible from an engine providing the virtual environment (e.g., predicting, by a processor in conjunction with an artificial intelligence (AI) engine, one or more recommendations for one or more action items for the first user based on the sensor data of the first user and the one or more second users, and the at least one of the CNS value and the CEP value in ¶ 12).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Shimizu and Chappell in view of US 2016/0042566 to Mao (hereinafter Mao).
Regarding claim 4, the combination of Shimizu and Chappell teaches or suggests the invention above, but lacks in explicitly teaching wherein the sensor data indicative of behavioral signals includes at least one of reaction times, frequency of button presses, and speed on left/right thumb-sticks. In a similar disclosure, Mao teaches sensory stimulus management in head mounted displays during game play (abstract). More particularly, Mao teaches that the game calculates a current user intensity value based on one or more parameters, including tracking physical manifestation of the user behavior (e.g., equilibrium, reaction time, erratic movements, erratic motion of the eyes, elevated heart rate, user falling down, etc.) in ¶ 132. It would have been obvious to one of ordinary skill in the art before the effective date to modify the system of Shimizu and Chappell to include sensing of reaction time or erratic movements as indications of behavior, as taught by Mao, in order to more accurately track the emotional behavior of the player.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and is listed on the attached Notice of References Cited.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM H MCCULLOCH whose telephone number is (571)272-2818. The examiner can normally be reached M-F 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Lewis can be reached at 571-272-7673. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM H MCCULLOCH JR/Primary Examiner, Art Unit 3715