Prosecution Insights
Last updated: November 15, 2026
Application No. 18/966,590

Video Diary and Facial Predictive Analysis

Non-Final OA §101§103§112
Filed
Dec 03, 2024
Examiner
ROBINSON, KYLE G
Art Unit
3685
Tech Center
3600 — Transportation & Electronic Commerce
Assignee
Gaia AG
OA Round
2 (Non-Final)
12%
Grant Probability
At Risk
2-3
OA Rounds
28%
With Interview

Examiner Intelligence

Grants only 12% of cases
12%
Career Allowance Rate
25 granted / 215 resolved
-40.4% vs TC avg
Strong +16% interview lift
Without
With
+16.3%
Interview Lift
resolved cases with interview
Typical timeline
Avg Prosecution
25 currently pending
Career history
256
Total Applications
across all art units

Office Action

§101 §103 §112
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Response to Amendment This action is in response to the amendment filed on 05/20/2026. Claims 1-9 have been canceled, and new claims 10-18 have been added. Claims 10-18 are examined below. Claim Rejections - 35 USC § 112 The following is a quotation of the first paragraph of 35 U.S.C. 112(a): (a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention. The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112: The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention. Claims 15-18 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Regarding claim 15, the limitations “software executing on the computer that assigns an assessment value to the emotional portion based on fluctuations and adaptations of emotional valence captured within the video of said patient message”, and “software executing on the computer that assigns an assessment value to the text portion based on emotional valence adaptations captured within the text portion of said patient message” lack a proper written description. The specification does not appear to disclose what the assessment value represents, how it is calculated or assigned, or how fluctuations and adaptations of emotional valence are translated into the value. Therefore, the application does not reasonably convey possession of the presently claimed relationship between changes in emotional valence and an assessment value. Claims 16-18 are rejected as well since they inherit the limitations of claim 15. The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. Claims 10-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding claim 10, the limitation “software executing on the computer for separating the text portion from the emotional portion of said patient message and aligns a first timestamped data set capturing emotional expressions from the video with a second timestamped data set capturing nuances from the text portion to identify an emotional incongruity between spoken words and facial expressions within the patient message” is indefinite. It is unclear as to how the timestamped data sets are aligned; particularly due to the nature of the text portion of the patient message. The Examiner points to paragraph [0060] which states, in part, “When the partner 10 completes the video recording, the mobile computing device 160 may transmit the video recording, as well as any audio and/or textual messages to the computer 170. The entirety of the transmitted message, which may include audio and/or textual messages in addition to the video recording, is described as a message 112.” Based on this, it appears that the text portion may just be a textual message (ex., “I had a good day”). Consequently, it is unclear how a textual message, even if timestamped, is “aligned” with a video since a textual message would only have one timestamp, while a video would consist of multiple timestamps. Further, it is unclear as to the manner in which the textual messages would be timestamped. For example, would the timestamp be based upon its time of creation, or the time at which it is received by the computer? Moreover, it is unclear as to how a comparison of textual data and video data is used to identify emotional incongruity between spoken words and facial expressions as there is no link between the textual data and the spoken words. Claims 11-14 are rejected as well since they inherit the limitations of independent claim 10. Regarding claim 11, the limitation “…wherein identifying the emotional incongruity includes identifying a mismatch in audio expressions” is indefinite. It is unclear as to what the mismatch is in relation to. For example, it is unclear if the mismatch is in regards to differences between just the audio expressions, between the audio expressions and the text portion, or between the audio expressions and facial expressions. Regarding claim 15, the limitations “software executing on the computer that assigns an assessment value to the emotional portion based on fluctuations and adaptations of emotional valence captured within the video of said patient message”, and “software executing on the computer that assigns an assessment value to the text portion based on emotional valence adaptations captured within the text portion of said patient message” are indefinite. The claim requires assigning an assessment value based on “fluctuations and adaptations of emotional valence”, but the specification characterizes valence merely as a “positive or negative charge” (See Para. [0007]) without establishing an objective measure, scale, threshold, or other criteria for determining the magnitude or change in valence. Therefore it is unclear as to what constitutes a “fluctuation” or “adaptation” of emotional valence, particularly where valence is characterized as binary (i.e., positive or negative). Claims 16-18 are rejected as well since they inherit the limitations of independent claim . Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 10-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Independent claim 1 recites (additional elements crossed out): A s The above limitations, as drafted, are processes that, under their broadest reasonable interpretation, is a process that, under its broadest reasonable interpretation covers managing personal behavior or relationships or interactions between people, as well as performance of limitations by the human mind or with pen and paper. That is, other than reciting the claims as being performed by a “computer in communication with a telecom network”, a “mobile computing device” and “software executing on the computer”, nothing in the claims precludes the steps as being described as managing personal behavior or relationships or interactions between people, or performance of limitations by the human mind or with pen and paper. The claims, as written describe analyzing a video/text message recorded/written by a patient by aligning first timestamped data capturing emotional expressions from the video with second timestamped data from the text message to identify emotional incongruity between spoken words and facial expressions (i.e., observe that the text message says the patient is feeling happy, while the video is of them crying), and formulating a response based on the patient’s text message and identified incongruity. If a claim limitation, under its broadest reasonable interpretation, describes managing personal behavior or relationships or interactions between people, then it falls within the “Certain Methods of Organizing Human Activities” grouping of abstract ideas. Further, if a claim limitation, under its broadest reasonable interpretation, describes steps that may be performed mentally or with pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. The judicial exception is not integrated into a practical application. In particular, the claims recite the additional elements of a “computer in communication with a telecom network”, a “mobile computing device” and “software executing on the computer” to perform the steps. These additional elements are recited at a high level of generality (see at least para. [0015]) such that they amount to no more than mere instructions to apply the exception using generic computing components. The claims also recite the additional limitation of “the mobile computing device configured to capture a patient message recorded by the patient, said patient message having a text portion and an emotional portion comprising a video of at least a portion of the patient's face, and to transmit said patient message to the computer over the telecom network”. However, this amounts to insignificant extra-solution activity. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are therefore still directed to an abstract idea. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of a “computer in communication with a telecom network”, a “mobile computing device” and “software executing on the computer” to perform the claimed steps amounts to no more mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Further, the limitation “the mobile computing device configured to capture a patient message recorded by the patient, said patient message having a text portion and an emotional portion comprising a video of at least a portion of the patient's face, and to transmit said patient message to the computer over the telecom network” does not amount to significantly more than the judicial exception because it is a well-understood, routine and conventional function as evidenced by at least Symantec which featured the conventional activities of receiving or transmitting data over a network, as well as Official Notice as mobile devices have been capable of capturing video for at least two decades. The performance of conventional functions by generic computers does not provide an inventive concept. Therefore, the claims are not found to be patent eligible. Independent claim 15 recites (additional limitations crossed out): A s The above limitations, as drafted, are processes that, under their broadest reasonable interpretation, is a process that, under its broadest reasonable interpretation covers managing personal behavior or relationships or interactions between people, as well as performance of limitations by the human mind or with pen and paper. That is, other than reciting the claims as being performed by a “computer in communication with a telecom network”, a “mobile computing device” and “software executing on the computer”, nothing in the claims precludes the steps as being described as managing personal behavior or relationships or interactions between people, or performance of limitations by the human mind or with pen and paper. The claims, as written describe analyzing a message recorded by a patient by assigning a value to a text portion of the message (i.e., message reads as being happy), assigning a value to an emotional portion of the message (i.e., patient’s face looks sad), and formulating a response based on the assigned values (i.e., identified incongruity). If a claim limitation, under its broadest reasonable interpretation, describes managing personal behavior or relationships or interactions between people, then it falls within the “Certain Methods of Organizing Human Activities” grouping of abstract ideas. Further, if a claim limitation, under its broadest reasonable interpretation, describes steps that may be performed mentally or with pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. The judicial exception is not integrated into a practical application. In particular, the claims recite the additional elements of a “computer in communication with a telecom network”, a “mobile computing device” and “software executing on the computer” to perform the steps. These additional elements are recited at a high level of generality (see at least paras. [0041] such that they amount to no more than mere instructions to apply the exception using generic computing components. The claims also recite the additional limitations of “the mobile computing device configured to capture a patient message recorded by the patient, said patient message having a text portion and an emotional portion comprising a video of at least a portion of the patient's face, and to transmit said patient message to the computer over the telecom network”. However, this amounts to insignificant extra-solution activity. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are therefore still directed to an abstract idea. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of a “computer in communication with a telecom network”, a “mobile computing device” and “software executing on the computer” to perform the claimed steps amounts to no more mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Further, the limitations of “the mobile computing device configured to capture a patient message recorded by the patient, said patient message having a text portion and an emotional portion comprising a video of at least a portion of the patient's face, and to transmit said patient message to the computer over the telecom network” does not amount to significantly more than the judicial exception because it is a well-understood, routine and conventional function as evidenced by at least Symantec which featured the conventional activities of receiving or transmitting data over a network, as well as Official Notice as mobile devices have been capable of capturing video for at least two decades.. The performance of conventional functions by generic computers does not provide an inventive concept. Therefore, the claims are not found to be patent eligible. Claims 11-14 are dependent on claim 10, and include all the limitations of claim 10. Claims 16-18 are dependent on claim 15, and include all the limitations of claim 15. Therefore, they are also found to be directed to an abstract idea. Claim 12 features the additional limitation of “The computerized cognitive behavioral therapy system of claim 10, wherein the mobile computing device displays a periodic alert reminding a patient to submit a patient message.”, however the display of data is merely insignificant extra-solution activity. Claim 14 features the additional limitation of “The computerized cognitive behavioral therapy system of claim 13, wherein the computer is in communication with a database, and wherein said database stores the plurality of patient messages.”, however this amounts to no more than mere instructions to apply the exception (i.e., storing data) using generic computing components. Claim 16 features limitations similar to those of claim 14, and is therefore also found to be no more than mere instructions to apply the exception (i.e., storing data) using generic computing components. The remaining dependent claims have not been found to integrate the judicial exception into a practical application, or provide significantly more than the abstract idea since they merely further narrow the abstract idea. Therefore, the dependent claims are found to be directed to an abstract idea without significantly more Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim(s) 10, 11, 13, 15, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Valstar (US 12,135,943) in view of Bozorgtabar (US 2019/0328300). Regarding claim 10, Valstar discloses A computerized cognitive behavioral therapy system comprising: a mobile computing device for use by a patient, the mobile computing device configured to capture a patient message recorded by the patient, said patient message having a text portion and an emotional portion comprising a video of at least a portion of the patient's face; (See at least Col. 3, Lines 53-55 – “Video and audio of the users may be captured using standard camera and microphone on a computing device while they are interacting with the system.”, and Col. 4, Lines 1-3 – “The system has inputs related to text from a keyboard input 105, audio input 108, visual input 109, and biometrics input 112, all from a User 102.”, Col. 6, Lines 63-66 – “Expressed emotion from the Face module 124 estimates VAD values by analyzing the user's facial expressive cues and their temporal dynamics in the input video in near real-time on consumer hardware such as a mobile phone.”) Valstar also discloses software executing on the computer for separating the text portion from the emotional portion of said patient message (See at least Col. 4, Lines 1-12 – “The system has inputs related to text from a keyboard input 105, audio input 108, visual input 109, and biometrics input 112, all from a User 102. The text from the keyboard input 105 is fed into a Text Input module 115, along with a Speech to Text conversion module 110 of the audio input 108. The output of the Text Input module is fed into the VAD from Language module 120. The audio input 108 is fed into the VAD from Voice module 122, and into the Mental State Estimator module 128.” Valstar also discloses software executing on the computer that formulates a cognitive behavioral therapy response based on both the text portion of the patient message text portion and the identified emotional incongruity (see at least Claim 1 – “2) A first recognizing mood module using valence, arousal, and dominance to review the first input prompt module output, and create a first recognizing mood module output; 3) A first customizing prompt module to customize the first recognizing mood module output, and create a first customizing prompt module output;” While implied, Valstar does not explicitly disclose: A computer in communication with a telecom network; Transmit said patient message to the computer over the telecom network (See Bozorgtabar, Para. [0020] – “ FIG. 1 is a schematic illustrating a system for real-time annotation of facial symptoms in video conferencing in accordance with exemplary embodiments of the present invention. A patient subject 10 may utilize a camera and microphone 11 and the sounds and appearance of the patient subject 10 may be acquired therefrom. Although element 11 is illustrated as a camera device, this depiction is merely an example, and the actual device may be instantiated as teleconferencing equipment, as a personal computer, or even as a mobile electronic device such as a smartphone or tablet computer including a camera/microphone.”, and Para. [0021] – “The camera/microphone 11 may send the audio/video signals to the teleconferencing server 14 both as the high-quality acquired signal and as the scaled down/compressed signals, which may be referred to herein as the low quality signals.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Valstar to utilize the teachings of Bozorgtabar since they are both within the same field of endeavor (i.e. determining mood/emotion of user), and all of the claimed elements were known in the prior art and one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions and the combination would have yielded predictable results to one of ordinary skill in the art at the time of the invention.) Valstar also does not fully disclose aligns a first timestamped data set capturing emotional expressions from the video with a second timestamped data set capturing nuances from the text portion to identify an emotional incongruity between spoken words and facial expressions within the patient message (While Valstar discloses aligned and timestamped data sets (See at least Col. 5, Lines 41-42 – “The Speech to Text conversion module 110 transcribes an input voice signal into text in near-real time.”, Col. 5, Lines 53-55 – “This direct input text or transcribed text from speech is first modulated by the user's mood and mental state and then fed as an input prompt into LLM.”, Col. 5, Lines 60-62 – “A prompt may include all non-verbal cues (facial expressions, tone of voice, gaze actions, etc.) in addition to the verbal cues.”, , Col. 6, Lines 22-28 – “Unimodal Mood Inference Module may output a temporal aggregate of expressed emotions to the user's apparent and/or felt mood state predictions from each modality, i.e., face, voice, text, and physiological signals, separately. These modality-wise mood predictions may be fused into a single apparent mood state in the Multimodal Mood Fusion module.”, Col. 6, Lines 32-43 – “Mood may be computed through temporal aggregation of expressed emotion information embedded in the input prompt. The prompt may include non-verbal data from face, voice, and physiological signals. To estimate mood from each modality, first modality-wise expressive features (e.g., Facial Action Unit intensities, voice Low Level Descriptors, etc.) may be extracted, and then the scores of VAD may be predicted on a given time unit basis (per frame, word, sentence, turn, utterance). These per-time-unit VAD predictions may then be temporally aggregated into modality-wise mood scores for the whole prompt.”, Col.7, Lines 12-18 – “ The expressed emotion prediction model may take per-frame outputs of the expression recognition module to compute the expressed emotion scores either on a fixed time basis measured in ms (using a sequence to sequence map- 15 ping) or a variable length basis ( e.g., per word, sentence, or utterance), depending on the temporal prediction architecture.”), Valstar does not explicitly disclose the identification of an emotional incongruity. In light of the 112 rejection above, Bozorgtabar teaches this. See Bozorgtabar, Para. [0031] – “From the speech-to-text transcribed text (305), natural language processing may be performed (310. Natural language processing may be used to determine a contextual understanding of what the patient subject is saying and may be used to determine both the sentiment of what is being said (312), as well as the content of what is being said, as determined through language structure extraction (313).”, Para. [0032] – “The extracted action units (306), the semantic feature extraction (314), the body posture (308), the head movement (309), the detected tone (304), the sentiment analysis (312), and the language structure extraction (313) may all be sent to multimodal recurrent neural networks (315). The multimodal recurrent neural networks may use this data to determine an extent of expression of emotional intensity and facial movement (316) as well as an expression of correlation of features to language (317). The expression of emotional intensity and facial movement may represent a level of emotion displayed by the patient subject while the correlation of features to language may represent an extent to which a patient subject's non-verbal communication aligns with the content of what is being said. For example, discrepancy between facial/body movement and language/speech may be considered. These factors may be used to determine a probability of symptom display, as excessive emotional display may represent symptoms of health disorder and so might represent a deviation between features and language.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Valstar to utilize the teachings of Bozorgtabar since it would serve to allow Valstar to more accurately identify the emotional status of a user.) Regarding claim 11, Valstar discloses the computerized cognitive behavioral therapy system of claim 10, wherein the emotional portion of the patient message further includes an audio recording captured by the mobile computing device. (See at least Col. 4, Lines 1-12 – “The system has inputs related to text from a keyboard input 105, audio input 108, visual input 109, and biometrics input 112, all from a User 102. The text from the keyboard input 105 is fed into a Text Input module 115, along with a Speech to Text conversion module 110 of the audio input 108. The output of the Text Input module is fed into the VAD from Language module 120. The audio input 108 is fed into the VAD from Voice module 122, and into the Mental State Estimator module 128.”) Valstar does not disclose wherein identifying the emotional incongruity includes identifying a mismatch in audio expressions (In light of the 112 rejection above, Bozorgtabar teaches this. See Bozorgtabar, Para. [0027] – “Meanwhile, from the audio signal, vocal tone may be detected (304) and language may be transcribed using speech-to-text processing (305).”, Para. [0031] – “From the speech-to-text transcribed text (305), natural language processing may be performed (310. Natural language processing may be used to determine a contextual understanding of what the patient subject is saying and may be used to determine both the sentiment of what is being said (312), as well as the content of what is being said, as determined through language structure extraction (313).”, Para. [0032] – “The extracted action units (306), the semantic feature extraction (314), the body posture (308), the head movement (309), the detected tone (304), the sentiment analysis (312), and the language structure extraction (313) may all be sent to multimodal recurrent neural networks (315). The multimodal recurrent neural networks may use this data to determine an extent of expression of emotional intensity and facial movement (316) as well as an expression of correlation of features to language (317). The expression of emotional intensity and facial movement may represent a level of emotion displayed by the patient subject while the correlation of features to language may represent an extent to which a patient subject's non-verbal communication aligns with the content of what is being said. For example, discrepancy between facial/body movement and language/speech may be considered. These factors may be used to determine a probability of symptom display, as excessive emotional display may represent symptoms of health disorder and so might represent a deviation between features and language.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Valstar to utilize the teachings of Bozorgtabar since it would serve to allow Valstar to more accurately identify the emotional status of a user. Regarding claim 13, Valstar discloses The computerized cognitive behavioral therapy system of claim 10, wherein the system is configured to process a plurality of patient messages over a period of time. (See at least Col. 6, Lines 63-66 – “Expressed emotion from the Face module 124 estimates VAD values by analyzing the user's facial expressive cues and their temporal dynamics in the input video in near real-time on consumer hardware such as a mobile phone.” However, while Valstar discloses a user’s video message, Valstar does not explicitly disclose a plurality of video messages. The Examiner asserts that this represents a mere duplication of parts. See MPEP §2144 VI B, and In re Harza, 274 F.2d 669, 671; 124 USPQ 378, 380 (CCPA 1960), where it is stated that "the mere duplication of parts has no patentable significance unless a new and unexpected result is produced…". As no unexpected result is produced by the requirement of a plurality of messages being provided, there is no patentable significance in doing so.) Regarding claim 15, in light of the 112 rejection above, Bozorgtabar teaches: software executing on the computer that assigns an assessment value to the emotional portion based on fluctuations and adaptations of emotional valence captured within the video of said patient message; software executing on the computer that assigns an assessment value to the text portion based on emotional valence adaptations captured within the text portion of said patient message; (See Para. [0032] – “The extracted action units (306), the semantic feature extraction (314), the body posture (308), the head movement (309), the detected tone (304), the sentiment analysis (312), and the language structure extraction (313) may all be sent to multimodal recurrent neural networks (315). The multimodal recurrent neural networks may use this data to determine an extent of expression of emotional intensity and facial movement (316) as well as an expression of correlation of features to language (317). The expression of emotional intensity and facial movement may represent a level of emotion displayed by the patient subject while the correlation of features to language may represent an extent to which a patient subject's non-verbal communication aligns with the content of what is being said. For example, discrepancy between facial/body movement and language/speech may be considered.” The Examiner asserts that the language regarding correlation between non-verbal communication (i.e., emotional portion) and what is being said (i.e., text portion) is indicative that “values” are assigned to them. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Valstar to utilize the teachings of Bozorgtabar since it would serve to allow Valstar to more accurately identify the emotional status of a user.) The remaining limitations of claim 6 are similar to those of claim 1, and are therefore rejected using the same rationale. Regarding claim 17, Valstar does not explicitly disclose The computerized cognitive behavioral therapy system of claim 15, further comprising software executing on the computer that identifies an emotional incongruity between the assessment value assigned to the text portion and the assessment value assigned to the emotional portion. (See Bozorgtabar, Para. [0032] – “The extracted action units (306), the semantic feature extraction (314), the body posture (308), the head movement (309), the detected tone (304), the sentiment analysis (312), and the language structure extraction (313) may all be sent to multimodal recurrent neural networks (315). The multimodal recurrent neural networks may use this data to determine an extent of expression of emotional intensity and facial movement (316) as well as an expression of correlation of features to language (317). The expression of emotional intensity and facial movement may represent a level of emotion displayed by the patient subject while the correlation of features to language may represent an extent to which a patient subject's non-verbal communication aligns with the content of what is being said. For example, discrepancy between facial/body movement and language/speech may be considered.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Valstar to utilize the teachings of Bozorgtabar since it would serve to allow Valstar to more accurately identify the emotional status of a user.) Claim(s) 12 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Valstar (US 12,135,943) in view of Bozorgtabar (US 2019/0328300), and in further view of (Shankar (US 2024/0136051) Regarding claim 12, Valstar and Bozorgtabar do not explicitly disclose The computerized cognitive behavioral therapy system of claim 1, wherein the mobile computing device displays a periodic alert reminding a patient to submit a patient message. (See Shankar, Para. [0032] – “In particular, the digital therapy content 106 may be designed for developing skills to help manage symptoms of anxiety and depression, promoting social support and relationship quality, and encouraging help-seeking behavior in users. In one embodiment, the digital therapy content 106 provided via the digital therapy app 122 may encourage users to upload audio and/or video content, which may in turn be received by the digital therapeutic system 100. For example, the digital therapy content 106 may request that users upload video (e.g., video journal entries) and/or audio of themselves. The digital therapy content 106 may request that users upload the video and/or audio on a periodic (e.g., daily), a nonperiodic basis, or in response to various user inputs or other parameters.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Valstar and Bozorgtabar to utilize the teachings of Shankar since it would encourage consistent monitoring of the patient’s status.) Regarding claim 14, Valstar and Bozorgtabar do not explicitly disclose The computerized cognitive behavioral therapy system of claim 5, wherein the computer is in communication with a database, and wherein said database stores the plurality of patient messages. (See Shankar, Para. [0032] – “The uploaded user video and/or audio may be stored in , e.g., a database 112 associated with the digital therapeutic system 100” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the system of Valstar and Bozorgtabar to utilize the teachings of Shankar since they are all within the same field of endeavor (i.e. determining a mental state of a user), and all of the claimed elements were known in the prior art and one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions and the combination would have yielded predictable results to one of ordinary skill in the art at the time of the invention.) Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Valstar (US 12,135,943) in view of Bozorgtabar (US 2019/0328300), and in further view of Taylert (US 11,960,514). Regarding claim 16, Valstar and Bozorgtabar do not explicitly disclose The computerized cognitive behavioral therapy system of claim 15, further comprising a database in communication with the computer and storing the patient message and its formulated cognitive behavioral therapy response. (See Taylert, Col. 16, Lines 64-67 – “A conversational history cache stores in a database the text of previous user inputs and replies from either the generative model or the live-agent.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the system of Valstar and Bozorgtabar to utilize the teachings of Taylert since the use of the history may potentially provide more accurate replies (Col. 16, Lines 60-61).) Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Valstar (US 12,135,943) in view of Bozorgtabar (US 2019/0328300), and in further view of Khaleghi (US 10,559,307) Regarding claim 18, Valstar and Bozorgtabar do not explicitly disclose The computerized cognitive behavioral therapy system of claim 17, wherein the software compares the assessment value assigned to the text portion and the assessment value assigned to the emotional portion with a plurality of historical values derived from previous patient messages stored in a database. (See Col. 16, Lines 21-28 – “An optional speech delta detection module 412 accesses historical speech data samples of the subject from a reference speech database 412, and compares the historical speech data samples with the output of the natural language processing engine 406 and audible expression characteristics analysis module 408 to determine if there has been a material change (which may indicate a change in the subject's impairment or health status).”, and Col. 17, Lines 13-31 – “An optional feature delta detection module 418 accesses historical feature data samples of the subject from a reference features database 420, and compares the historical feature data samples with the outputs of the feature extraction module 414 and the eye and eyelid detection module 416 to determine if there has been a change (which may indicate a change in the subject's impairment or health status). By way of illustration, changes in the subject's smile, frown, lips (where one side of the user's mouth now droops), eyebrow position, facial droop, eye movement, eyelid characteristics, weight ( e.g., by determining if the subject's face and/or body is thinner or fatter) etc., may be detected. The historical feature data samples may optionally have been recorded as a baseline by the subject when setting up an account and/or afterwards, as similarly discussed elsewhere herein. Optionally, the historical feature data samples may include still and/or video images of the subject reading a script and/or engaging in free form speech.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Valstar and Bozorgtabar to utilize the teachings of Khaleghi since it may allow the system to determine a material change in a user’s health status (see at least Col. 16, Lines 21-28).) Response to Arguments Applicant's arguments regarding claims rejected under 35 U.S.C. 112(b) have been fully considered but they are not persuasive. Applicant’s amendments do not clarify the issues brought forth in the previous Office Action. See body of rejection above. Applicant's arguments regarding claims rejected under 35 U.S.C. 101 have been fully considered but they are not persuasive. Applicant argues with substance: Applicant argues that a human mind cannot perform “aligns a first timestamped data set capturing emotional expressions from the video with a second timestamped data set capturing nuances from the text portion to identify an emotional incongruity between spoken words and facial expressions within the patient message”. The Examiner respectfully disagrees. A simple example of this is a human observing a video with closed captions (i.e., aligned video and text), and making note that their facial expressions do not match the spoken works (e.g., frowning while stating “I’m happy”). This is not a data-engineering solution as the Applicant contends, and there is no mention of the alleged problem of “synchronizing separate text tokenization streams and video pixel manipulation frames” within the specification whatsoever.. Regarding claim 15, the Applicant argues that “”Emotional valence” is a precise digital signal processing metric representing the positive or negative charge coordinates within a semantic or computer-vision vector space”. The Examiner respectfully disagrees. Emotional valence is merely the indication that something feels “good” or “bad”. Tracking fluctuations and adaptations of emotional valance is merely an observation, and is therefore an abstract concept. Applicant argues that the claims solves a technical problem by forcing the computing machine to detect hidden data mismatches. The Examiner respectfully disagrees. The claim does not “force” the computer to operate differently at the hardware or software-architecture level. It merely instructs software to perform an analysis. Claims 10 and 15 merely amount to receiving data, processing data, comparing data, and generating an output. These are conventional computing operations. If anything, the claims merely apply conventional computer processing to a particular type of information, rather than a modification of the computer itself. Further, neither the specification or the original claims make mention of modifying the machine into a “specialized assessment loop”. Applicant’s arguments concerning Berkheimer are not persuasive. Applicant argues that the Examiner offered “zero evidence showing that the sub-second synchronization of multi-modal video/text timestamp arrays to automate cognitive behavioral responses is well-understood, routine, or conventional in the data science sector”. The Examiner notes that none of these particular features are not included in the claim language. Further, the Examiner was pointing out that the additional element of “a mobile computing device for use by a patient, said mobile computing device receiving the patient message from the patient and transmitting said patient message to the computer over the telecom network” and “said cognitive behavioral therapy response transmitted to the mobile computing device” were conventional, as they merely featured the transmitting or receiving of data. The other limitations were found to be part of the abstract idea itself. Based on at least the above, the 101 rejection is maintained. Applicant's arguments regarding claims rejected under 35 U.S.C. 103 have been fully considered but they are not persuasive. Applicant argues with substance: Applicant appears to argue that Valstar and Bozorgtabar do not disclose the newly amended language of “aligns a first timestamped data set capturing emotional expressions from the video with a second timestamped data set capturing nuances from the text portion to identify an emotional incongruity between spoken words and facial expressions within the patient message”. The Examiner respectfully disagrees and points to the body of the rejection above. Applicant’s arguments concerning claim 15 are not persuasive. In light of the latest 112 rejections, the cited prior art was found to disclose the claim language. Based on at least the above, the 103 rejections are maintained. Conclusion Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE G ROBINSON whose telephone number is (571)272-9261. The examiner can normally be reached Monday - Thursday, 7:00 - 4:30 EST; Friday 7:00-11:00 EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kambiz Abdi can be reached at 571-272-6702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /KYLE G ROBINSON/Examiner, Art Unit 3685 /KAMBIZ ABDI/Supervisory Patent Examiner, Art Unit 3685
Read full office action

Prosecution Timeline

Dec 03, 2024
Application Filed
Oct 10, 2025
Non-Final Rejection mailed — §101, §103, §112
Jan 07, 2026
Response after Non-Final Action
Jan 07, 2026
Response Filed
May 20, 2026
Response Filed
Aug 21, 2026
Final Rejection mailed — §101, §103, §112
Oct 06, 2026
Response after Non-Final Action

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12307475
CONSUMER-SPECIFIC ADVERTISEMENT PRESENTATION AND OFFER LIBRARY
5y 3m to grant Granted May 20, 2025
Patent 12093977
ATTENTION APPLICATION USER CLASSIFICATION PRIVACY
1y 8m to grant Granted Sep 17, 2024
Patent 12039574
PROGRAMMATIC ADVERTISING SERVER
11m to grant Granted Jul 16, 2024
Patent 12026746
INSTRUMENT SYSTEM INTERACTION TRACKING
1y 6m to grant Granted Jul 02, 2024
Patent 12020283
Content Feedback and Customization
1y 11m to grant Granted Jun 25, 2024
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

2-3
Expected OA Rounds
12%
Grant Probability
28%
With Interview (+16.3%)
Moderate
PTA Risk
Based on 215 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month