DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Amendment
This is a response to Applicant’s amendment filed on 05 March 2026, wherein:
Claims 1-18 are amended.
Claims 19 and 20 are canceled.
Claims 21 and 22 are new.
Claims 1-18, 21, and 22 are pending.
Information Disclosure Statement
The information disclosure statement filed 05 April 2024 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered. In particular, foreign patent document cite no. 2 (JP 6550460 B2) is missing a translation to the English language and/or an explanation of relevance.
The listing of references in the specification is not a proper information disclosure statement. 37 CFR 1.98(b) requires a list of all patents, publications, or other information submitted for consideration by the Office, and MPEP § 609.04(a) states, "the list may not be incorporated into the specification but must be submitted in a separate paper." Therefore, unless the references have been cited by the examiner on form PTO-892, they have not been considered.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 112(a) as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed application, Provisional Application No. 63/548,341, fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for one or more claims of this application.
In particular, the disclosure of the prior-filed application is silent regarding the claimed invention. In particular, the specification of US 63/548,341 is a copy of a journal article on Charisma virtual social training that discusses low immersion virtual reality for social skills training but is silent regarding speech to text analysis, any aspect of generating a social competency score, nor the use of artificial intelligence and training an artificial intelligence model, including all of the claimed “systems”. Similarly, the abstract and the single claim in the Provisional Application only recite “a system and method for virtual technology tools to identify, quantify, and improve social skills.”
Thus, claims 1-18, 21, and 22 do not gain benefit of priority to US Application 63/548,341. Therefore, claims 1-18, 21, and 22 have an effective filing date of 17 January 2024.
Drawings
Figures 9-24 should be designated by a legend such as --Prior Art-- because only that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
Para. 26 copies para. 25 and incorrectly describes Fig. 21.
The specification inconsistently addresses trade names/marks. Some are entirely capitalized. Some are succeeded with an improperly marked “TM”. Proper marking and uniformity is recommended.
Appropriate correction is required.
The amendment filed 05 May 2026 is objected to under 35 U.S.C. 132(a) because it introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment shall introduce new matter into the disclosure of the invention. The added material which is not supported by the original disclosure is as follows: All language involving “social competency construct” and “social competency measure” throughout the amendments. “In some embodiments, the second ‘speaker’ referred to above can include a clinician, practitioner, or other interaction participant providing input, observation, or evaluation, and need not actively generate spoken audio within the interaction” in para. 3. “As used herein, references to a ‘clinician,’ ‘practitioner,’ or ‘second speaker’ can refer to a user, including a research clinician or other interaction participant, providing input, feedback, observation, evaluation, or coaching within an interaction, such as a role-play session conducted via an immersive social communication training system” in para. 34. “IMMERSIVE XR USE CASE/STUDY” preceding para. 113. (Underlined for emphasis). “various XR embodiments are completed within its scope” in para. 120. “In particular, the interactive role-play sessions, coaching interactions, and user-driven behaviors described above generate interaction data, including speech, responses, pauses, and contextual interaction events, which can be captured, processed, and analyzed using the dialog management, scoring, and competency evaluation systems previously described. In this manner, the immersive training environments and associated protocols serve as one example of a data generation and interaction framework that enables the transformation of subjective social interactions into structured interaction data and objective social competency metrics, as performed by the systems and methods of the present disclosure” in para. 160.
Applicant is required to cancel the new matter in the reply to this Office Action.
The incorporation of essential material in the specification by reference to an unpublished U.S. application, foreign application or patent, or to a publication is improper. Applicant is required to amend the disclosure to include the material incorporated by reference, if the material is relied upon to overcome any objection, rejection, or other requirement imposed by the Office. The amendment must be accompanied by a statement executed by the applicant, or a practitioner representing the applicant, stating that the material being inserted is the material previously incorporated by reference and that the amendment contains no new matter. 37 CFR 1.57(g). The amended specification lists multiple non-patent literature publications which the specification also attempts to incorporate by reference.
Claim Objections
Claim 2 is objected to because of the following informalities:
Claim 2 includes marked and unmarked amendments. This fails 37 CFR 1.121 which requires all amendments to be appropriately marked. Continued improper amendments will result in amendments not being entered.
Appropriate correction is required.
Applicant is advised that should claim 8 be found allowable, claim 9 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 112
The text of those sections of Title 35, U.S. Code 112(b) not included in this action can be found in a prior Office action.
Regarding claim 21, it is unclear what constitutes “one or more interactive virtual environment elements and at least one avatar configured to engage in social role-play interactions with the participant”. The only mention of a virtual environment element is found in para. 67 of the amended specification which recites “one or more virtual environment elements, such as where a practitioner is controlling a number of avatars, where an avatar controlled by a third party, a script or an artificial intelligence engine is used, where a virtual environment activity is generated”. Thus, the limitation, in effect, recites “one or more avatars and at least one avatar” which is redundant and unclear causing one of ordinary skill in the art to not be apprised of the metes and bounds of the patent protection sought. For the purposes of compact prosecution, the limitation is construed as “one or more avatars configured to engage in social role-play interactions with the participant”. Dependent claim 22 inherits the deficiencies of its respective parent claim, and is thus rejected under the same rationale.
Claim 21 recites the limitation "the updated weights" in line 19 of the claim. There is insufficient antecedent basis for this limitation in the claim. In particular, the preceding limitation recites “update one or more weights” which is distinctly different from “update weights”. Dependent claim 22 inherits the deficiencies of its respective parent claim, and is thus rejected under the same rationale.
Claim 21 recites the limitation "identified social communication deficits of the participant" in lines 19-20 of the claim. There is insufficient antecedent basis for this limitation in the claim. In particular, the claim is silent regarding identifying social communication deficits of the participant. The only preceding mention of social communication deficits of the participant is found in the preamble which only generically recites “improving social communication deficits of a participant”. Dependent claim 22 inherits the deficiencies of its respective parent claim, and is thus rejected under the same rationale.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 1-18, 21, and 22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 1, 11, and 21, the originally filed disclosure is silent regarding “generate a social competency score based on the evaluation of the structured interaction data relative to the predefined social communication competency constructs for the participant for at least a portion of the interactive social role-play session” in claim 1, “evaluate the structured interaction data relative to predefined social communication competency constructs corresponding to on-target and off-target social behaviors” in claim 11, and “a competency evaluation engine configured to evaluate the structured interaction data relative to predefined social communication competency constructs corresponding to on-target and off-target social behaviors and to generate a social competency score comprising construct-specific competency values” in claim 21. There are two statutory provisions that prohibit the introduction of new matter. The first provision is 35 USC 132, which provides that no amendment shall introduce new matter into the disclosure of the invention. If new matter is added to the claims, the examiner should reject the claims under 35 USC 112(a) – written description requirement. See MPEP 2163.06. In particular, the originally filed disclosure is silent regarding “social communication competency constructs”. Regarding claim 22, the disclosure is also silent regarding “a competency evaluation engine” and any function such an engine is claimed to perform. Therefore, this is new matter. Dependent claims 2-10, 12-18 and 22 inherit the deficiencies of their respective parent claims, and are thus rejected under the same rationale.
Claim Rejections - 35 USC § 101
The text of those sections of Title 35, U.S. Code 101 not included in this action can be found in a prior Office action.
Claims 1-18, 21, and 22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without including additional elements that are sufficient to amount to significantly more than the judicial exception itself.
Step 1
The claims are directed to a method and a product which falls under the four statutory categories (STEP 1: YES).
Step 2A, Prong 1
Independent claim 1 recites:
A system for generating objective assessment criteria of social skills during interactive social role-play sessions within a virtual immersive environment, comprising:
a speech to text system configured to receive audio data and to convert the audio data into text data associated with a participant engaged in an interactive social role-play session occurring within a virtual immersive environment, the participant having perceived social communication deficits;
a dialog management system configured to:
generate structured interaction data based at least in part on the text data for the interactive social role-play session, the structured interaction data including identified questions, responses, pauses, and associated interaction metadata;
evaluate the structured interaction data relative to predefined social communication competency constructs corresponding to on-target and off-target social behaviors; and
generate a social competency score based on the evaluation of the structured interaction data relative to the predefined social communication competency constructs for the participant for at least a portion of the interactive social role-play session;
a clinician verification interface configured to:
present the structured interaction data and the social competency score to a clinician having specialized training relating to the perceived social communication deficits; and
receive clinician feedback modifying or confirming the social competency score, wherein the structured interaction data, the social competency score, and the clinician feedback are used to improve subsequent social competency determinations for the participant during subsequent interactive social role-play sessions within the virtual immersive environment.
Independent claim 11 recites:
A method for generating objective assessment criteria of social skills during interactive role-play sessions within a virtual immersive environment, comprising:
receiving audio data generated during an interactive social role-play within a virtual immersive environment;
converting the audio data into text data associated with a participant engaged in the interactive social role-play session, the participant having perceived social communication deficits;
processing text data using a dialog management system to:
generate structured interaction data based at least in part on the text data, the structured interaction data including identified questions, responses, pauses, and associated interaction metadata;
evaluate the structured interaction data relative to predefined social communication competency constructs corresponding to on-target and off-target social behaviors; and
generate a social competency score for the participant for at least a portion of the interactive social role-play session;
presenting the structured interaction data and the social competency score to a clinician having specialized training relating to the perceived social communication deficits;
receiving clinician feedback data modifying or confirming the social competency score; and
using the structured interaction data, the social competency score, and the clinician feedback to improve subsequent social competency determinations for the participant during subsequent interactive social role-play sessions within the virtual immersive environment.
Independent claim 21 recites:
A computer-implemented immersive social communication training system for improving social communication deficits of a participant, comprising:
a virtual immersive environment comprising one or more interactive virtual environment elements and at least one avatar configured to engage in social role-play interactions with the participant;
an interaction analysis system configured to process data generated during the interactive social role-play interactions to generate structured interaction data including identified questions, responses, pauses, and associated metadata;
a competency evaluation engine configured to evaluate the structured interaction data relative to predefined social communication competency constructs corresponding to on-target and off-target social behaviors and to generate a social competency score comprising construct-specific competency values;
a clinician verification interface configured to present the structured interaction data and the social competency score to a clinician and to receive clinician feedback modifying or confirming at least a portion of the social competency score;
a training module configured to update one or more weights of the competency evaluation engine based on the clinician feedback to improve subsequent competency determinations; and
an interaction control module configured to modify at least one subsequent social role-play session within the virtual immersive environment based on the updated weights to target identified social communication deficits of the participant.
All of the foregoing underlined elements amount to the abstract idea grouping of a certain method of organizing human activity because it is managing personal behavior or interactions between people (including social activities, teaching, and following rules or instructions) as it is merely following rules or instructions by collecting information, analyzing the information, and outputting the results of the collection and analysis while performing an immersive social communication training between two people. They all also amount to the abstract idea grouping of mental processes as the claims, under their broadest reasonable interpretation, cover performance of the limitations in the mind with the aid of pen and paper but for the recitation of generic computer components. See MPEP 2106.04(a)(2)(III)(C) - A Claim That Requires a Computer May Still Recite a Mental Process. Even if humans would use a physical aid to help them complete the recited steps, the use of such physical aid does not negate the mental nature of these limitations. Lastly, the steps in the independent and dependent claims associated with “generating a social competency score” amount to the abstract idea grouping of mathematical concepts because they recite mathematical calculations as defined in MPEP 2106.05(a)(2)(I) which recites that a “claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the ‘mathematical concepts’ grouping” because a “mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation. There is no particular word or set of words that indicates a claim recites a mathematical calculation. That is, a claim does not have to recite the word ‘calculating’ in order to be considered a mathematical calculation. For example, a step of ‘determining’ a variable or number using mathematical methods or ‘performing’ a mathematical operation may also be considered mathematical calculations when the broadest reasonable interpretation of the claim in light of the specification encompasses a mathematical calculation."
The dependent claims amount to merely further defining the judicial exception.
Therefore, the claim recites a judicial exception. (STEP 2A, PRONG 1: YES).
Step 2A, Prong 2
This judicial exception is not integrated into a practical application because the claim does not include additional elements that are sufficient to integrate the exception into a practical application under the considerations set forth in MPEP 2106.04(d). The elements of the claims above that are not underlined constitute additional elements.
The following additional elements, both individually and as a whole, merely generally link the judicial exception to a particular technological environment or field of use: a system (claim 1), a speech to text system (claims 1 and 11), a dialog management system (claims 1 and 11), a clinician verification interface (claims 1, 11, and 21), reciting the immersive environment is “virtual” (claims 1, 11, and 21), a weight analysis system (claims 2 and 12), a strategic attention weight analysis system (claims 3 and 13), a discourse weight analysis system (claims 4 and 14), a theory of mind weight analysis system (claims 5 and 15), an expressive reasoning weight analysis system (claims 6 and 16), a transform weight analysis system (claims 7 and 17), a plurality of weight analysis systems (claims 8, 9, and 18), a computer-implemented immersive social communication training system (claim 21), reciting environment elements are “virtual” (claim 21), a competency evaluation engine (claim 21), a training module (claim 21), and an interaction control module (claim 21). This is evidenced by the manner in which these elements are disclosed. See, for example, at least Fig. 1-3 which illustrate the components as non-descript black boxes in a conventional arrangement while Fig. 4-10, 12, 13, 22, and 23 illustrate the claimed invention as purely software, and the specification which identifies that the “systems” are, at best, software modules operating on a processor. The claims do not recite any limitations that improve the functionality of the computer system because the claimed steps are merely performing the steps of processing data but are not tied to improving any functionality of a computer system. In particular, para. 32 of the specification incorrectly asserts that an “essential part of artificial intelligence is training to improve the function of the computer systems”. “Training” in the context of artificial intelligence is for “teaching” the model (otherwise referred to as setting the weights in an automated manner) to optimize performance on a dataset of sample tasks resembling its intended use. Similarly, the “clinician verification interface” and its function is merely an identification that the artificial intelligence model is supervised. Thus, the claims do not recite any specific rules with specific characteristics that improve the functionality of the computer system. The system is merely recited to be used, not improved. For instance, the speech to text system, as claimed and organized, merely adds insignificant extra-solution activity to the judicial exception (e.g., mere data gathering and processing in conjunction with a law of nature or abstract idea) while the remaining additional elements identified above merely indicate a field of use (i.e., use of a computer to implement the judicial exception). Thus, the components, identified above, are merely an attempt to link the abstract idea to a particular technological environment, but do not result in an improvement to the technology or computer functions employed. Additionally, the claims do not apply or use a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition nor do they apply or use a judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment. For instance, para. 32 of the specification recites that the “present disclosure provides systems for assisting humans with artificial intelligence processing of psychological counseling data that allow humans to independently assess the accuracy of outputs and to provide training data to the artificial intelligence systems to improve their ability to process data for the purpose of providing psychological counseling.” Again, this is merely just an identification that the artificial intelligence model is supervised. Accordingly, based on all of the considered factors, these additional elements do not integrate the abstract idea into a practical application. Therefore, the claims are directed to the judicial exception. (STEP 2A, PRONG 2: YES).
Step 2B
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception under the considerations set forth in MPEP 2106.05. As addressed in Step 2A, Prong 2, above, the system and the process it performs does not require the use of a particular machine, nor does it result in the transformation of an article. The claims do not involve an improvement in a computer or other technology. Although the claims recite components (identified in Step 2A, Prong 2) for performing at least some of the recited functions, these elements are recited at a high level of generality and are not tied to performing any of the steps of the claimed method. This is evidenced by the lack of significant structure in the figures (i.e., Fig. 1-3 which illustrate the components as non-descript black boxes in a conventional arrangement while Fig. 4-10, 12, 13, 22, and 23 illustrate the claimed invention as purely software) and the generic nature in which any structural items are described in the claims and the specification (all of the “systems” are merely recited to be loaded into a working memory of a processor to cause the processor to perform the algorithm indicating they are software modules, at best). Thus, the judicial exception is not implemented with, or used in, a particular machine or manufacture. Furthermore, this also evidences that the components are an attempt to link the abstract idea to a particular technological environment, but does not result in an improvement to the technology or computer functions employed. This also applies specifically to the speech to text system which merely adds insignificant extra-solution data-gathering and processing activity to the judicial exception (e.g., mere data gathering in conjunction with a law of nature or abstract idea) also found to not add significantly more, while the remaining additional elements identified above merely indicate a field of use (i.e., use of a computer to implement the judicial exception). For instance, the mere training of an artificial intelligence system, and use artificial intelligence as a whole, does not improve computer functionality as it merely invokes the use of a computer or other machinery in its ordinary capacity to process information. This is at least evidenced by the manner in which this is disclosed that indicates that Applicant believes the additional elements are sufficiently well-known that the specification does not need to describe the particulars of such additional elements to satisfy 35 USC 112(a). The lack of improvement to the computer or other technology is evidenced by the lack of incorporation of specific rules which enable the automation of a computer-implemented task that previously could only be performed subjectively by humans. In contrast, the focus of the claimed invention is on the analysis of the collected data, which is itself at best merely an improvement within the abstract idea. See pg. 2-3 in SAP America Inc. v. lnvestpic, LLC (890 F.3d 1016, 126 USPQ2d 1638 (Fed. Cir. 2018) which proffered “[w]e may assume that the techniques claimed are groundbreaking, innovative, or even brilliant, but that is not enough for eligibility. Nor is it enough for subject-matter eligibility that claimed techniques be novel and nonobvious in light of prior art, passing muster under 35 U.S.C. §§ 102 and 103. The claims here are ineligible because their innovation is an innovation in ineligible subject matter. Their subject is nothing but a series of mathematical calculations based on selected information and the presentation of the results of those calculations.” None of the hardware offer a meaningful limitation beyond, at best, generally linking the performance of the steps to a particular technological environment, that is, implementation via computers. Viewed as a whole, the additional claim elements do not provide a meaningful limitation to transform the abstract idea into a patent eligible application of the abstract idea such that the claim amounts to significantly more than the abstract idea of itself (STEP 2B: NO).
Therefore, the claims are rejected under 35 USC 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code 103 not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 11, 21, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Reece et al. (US 2022/0343911, hereinafter referred to as Reece) in view of Ravindran et al. (US 10,311,645 B1, hereinafter referred to as Ravindran).
Regarding claims 1 and 11, Reece teaches a system (claim 1) and a method (claim 11) for generating objective assessment criteria of social skills during interactive social role-play sessions, comprising:
a speech to text system configured to receive audio data and to convert the audio data into text data associated with a participant engaged in an interactive social role-play session (Reece, para. 39, “the conversation analytics system is configured to automatically generate a transcript based on the acoustic/video data.”), the participant having perceived social communication deficits;
a dialog management system and configured to:
generate structured interaction data based at least in part on the text data for the interactive social role-play session, the structured interaction data including identified questions, responses, pauses, and associated interaction metadata (Reece, para. 39, “The conversation analytics system is configured to apply natural language processing algorithms to the text data. In one implementation, conversation keywords, interruptions, topic changes, questions, use of passive voice, and so on, may be identified.” At least para. 156-161 describe the structured interaction data, calling them conversation features.);
evaluate the structured interaction data relative to predefined social communication competency constructs corresponding to on-target and off-target social behaviors (Reece, para. 101, “conversation analysis indicators 612 include engagement scores, enthusiasm scores, ownership score, goal score, interruptions score, and ‘time spent listening’ score, and/or attention scores. For example, engagement scores may be influenced by the number of questions asked and the length of statements made. Enthusiasm scores may be influenced by changes in pitch, changes in voice volume, and excited facial expressions. Attention scores may be defined by gaze detection (e.g., on-screen gaze, eye contact gaze) and body pose detection from the video data.”); and
generate a social competency score based on the evaluation of the structured interaction data relative to the predefined social communication competency constructs for the participant for at least a portion of the interactive social role-play session (Reece, para. 35, “the communication skills of the mentee can be evaluated by performing computational methods (e.g., neural network based analysis) on the acoustic/video data.” Para. 69, “conversation analysis indicators can be one or more scores for the entire conversation or parts of the conversation, such as an overall effectiveness or quality rating for the conversation or parts of the conversation.”); and
an annotator verification interface configured to:
present the structured interaction data and the social competency score to an annotator having specialized training relating to the perceived social communication deficits (Reece, para. 134, “Application 1108, executing at annotator 1102, displays utterance 1104 to a human annotator”; para. 135, “Additionally or alternatively, server computing device may transmit machine generated label data 1110 to annotator 1102. In other words, annotator 1102 may be configured to review and modify label data generated by a machine learning system.” Para. 145, “the evaluation system includes an annotator configured for manual evaluation by a trained human annotator.”); and
receive annotator feedback modifying or confirming the social competency score, wherein the structured interaction data, the social competency score, and the annotator feedback are used to improve subsequent social competency determinations for the participant during subsequent interactive social role-play sessions (Reece, para. 169, “In another implementation, ML training system 1848 communicates with an annotator for supervised learning, as shown in FIG. 11. For example, ML training system 1848 may transmit utterance features to an annotator, receive synthesized conversation features identified by the annotator, and subsequently train conversation synthesis ML system 1844 to automatically identify those conversation features.”).
Reece does not explicitly teach that the social role-play sessions are within a virtual immersive environment and that the human annotator is a clinician.
However, in a related art, Ravindran teaches the social role-play sessions are within a virtual immersive environment (Ravindran, Col. 5, lines 13-20, “The virtual world, and the one or more stimulations (e.g., visual, auditory, haptic) presented in the virtual world, can be designed to treat the given mental developmental disorder and/or improve the given social skill. For example, the virtual world can be tailored for a given mental or developmental disorder. Alternatively or in addition, the virtual world can be tailored for a given social skill.” Line 47 of Col. 9 – line 6 of Col. 10 provide multiple examples of social role-play sessions performed within a virtual immersive environment.) and that the annotator is a clinician (Ravindran, Col. 2, line 3, “human expert (e.g., therapist)”. It is noted that the term “therapist” in this context is synonymous with the term “clinician”.).
It would have been obvious to a person having ordinary skill in the art before the effective filing of the claimed invention for the computer-based social role-play sessions in Reece to be held within a virtual immersive environment and for the annotator to be a clinician because “[u]sing the virtual or augmented reality systems may advantageously provide various therapeutic values to a subject (or user) receiving treatment… that need a more gentle and controlled introduction to certain aspects of the real world” or “any and all individuals in the general population that may benefit from practicing social skills or routines in a controlled environment.” Ravindran at Col. 7, lines 10-45.
Claims 2-10 and 12-18 are rejected under 35 U.S.C. 103 as being unpatentable over Reece in view of Ravindran as applied to claims 1 and 11 above, in view of Griffin (US 2023/0178217).
Regarding claims 2 and 12, Reece in view of Ravindran teaches the system of claim 1 and the method of claim 11 wherein the dialog management system is configured to generate the social competency score using the structured interaction data associated with the interactive social role-play session (Reece, para. 71, “generates multiple conversation analysis indicators (e.g., conversation scores, conversation analysis indicators, openness scores, engagement scores, ownership scores, goal scores, interruptions scores, ‘time spent listening’ scores”; para. 100, “a conversation including multiple interactions between two speakers may indicate a conversation quality score”).
Reece in view of Ravindran does not explicitly teach using a weight analysis system configured to apply one or more weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include using a weight analysis system configured to apply one or more weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claims 3 and 13, Reece in view of Ravindran teaches the system of claim 1 and the method of claim 11 wherein the dialog management system is configured to generate the social competency score using the structured interaction data to evaluate strategic attention (Reece, para. 101, “conversation analysis indicators 612 include engagement scores, enthusiasm scores, ownership score, goal score, interruptions score, and ‘time spent listening’ score, and/or attention scores. For example, engagement scores may be influenced by the number of questions asked and the length of statements made. Enthusiasm scores may be influenced by changes in pitch, changes in voice volume, and excited facial expressions. Attention scores may be defined by gaze detection (e.g., on-screen gaze, eye contact gaze) and body pose detection from the video data.” This is includes strategic attention as described in at least para. 39-41 of the instant amended specification.).
Reece in view of Ravindran does not explicitly teach using a strategic attention weight analysis system configured to apply one or more strategic attention weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include using a strategic attention weight analysis system configured to apply one or more strategic attention weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claims 4 and 14, Reece in view of Ravindran teaches the system of claim 1 and the method of claim 11 wherein the dialog management system is configured to generate the social competency score using the structured interaction data to evaluate discourse (Reece, para. 101, “conversation analysis indicators 612 include engagement scores, enthusiasm scores, ownership score, goal score, interruptions score, and ‘time spent listening’ score, and/or attention scores. For example, engagement scores may be influenced by the number of questions asked and the length of statements made. Enthusiasm scores may be influenced by changes in pitch, changes in voice volume, and excited facial expressions. Attention scores may be defined by gaze detection (e.g., on-screen gaze, eye contact gaze) and body pose detection from the video data.” This is includes discourse as described in at least para. 42-44 of the instant amended specification.).
Reece in view of Ravindran does not explicitly teach using a discourse weight analysis system configured to apply one or more discourse weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include using a discourse weight analysis system configured to apply one or more discourse weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claims 5 and 15, Reece in view of Ravindran teaches the system of claim 1 and the method of claim 11 wherein the dialog management system is configured to generate the social competency score using the structured interaction data to evaluate theory of mind (Reece, para. 101, “conversation analysis indicators 612 include engagement scores, enthusiasm scores, ownership score, goal score, interruptions score, and ‘time spent listening’ score, and/or attention scores. For example, engagement scores may be influenced by the number of questions asked and the length of statements made. Enthusiasm scores may be influenced by changes in pitch, changes in voice volume, and excited facial expressions. Attention scores may be defined by gaze detection (e.g., on-screen gaze, eye contact gaze) and body pose detection from the video data.” This is includes theory of mind as described in at least para. 45-47 of the instant amended specification.).
Reece in view of Ravindran does not explicitly teach using a theory of mind weight analysis system configured to apply one or more theory of mind weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include using a theory of mind weight analysis system configured to apply one or more theory of mind weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claims 6 and 16, Reece in view of Ravindran teaches the system of claim 1 and the method of claim 11 wherein the dialog management system is configured to generate the social competency score using the structured interaction data to evaluate expressive reasoning (Reece, para. 140, “conversation labels 1210 include conversation effectiveness ratings. Conversation effectiveness ratings may be a subjective score or rating defining the effectiveness of the conversation towards a particular goal or activity. For example, conversation labels 1210 may include coaching effectiveness scores.” This is includes expressive reasoning as described in at least para. 48-50 of the instant amended specification.).
Reece in view of Ravindran does not explicitly teach using an expressive reasoning weight analysis system configured to apply one or more expressive reasoning weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include using an expressive reasoning weight analysis system configured to apply one or more expressive reasoning weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claims 7 and 17, Reece in view of Ravindran teaches the system of claim 1 and the method of claim 11 wherein the dialog management system is configured to generate the social competency score using the structured interaction data to evaluate transform (Reece, para. 71, “openness scores,… ownership scores”; This is includes transform as described in at least para. 51-52 of the instant amended specification.).
Reece in view of Ravindran does not explicitly teach using a transform weight analysis system configured to apply one or more transform weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include using a transform weight analysis system configured to apply one or more transform weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claims 8, 9, and 18, Reece in view of Ravindran teaches the system of claim 1 and the method of claim 11 wherein the dialog management system is configured to generate the social competency score using the structured interaction data (Reece, para. 71, “conversation analytics system 400 generates multiple conversation analysis indicators (e.g., conversation scores, conversation analysis indicators, openness scores, engagement scores, ownership scores, goal scores, interruptions scores, ‘time spent listening’ scores, and emotional labels).”).
Reece in view of Ravindran does not explicitly teach using a plurality of weight analysis systems that are each configured to apply one or more weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include using a plurality of weight analysis systems that are each configured to apply one or more weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claim 10, Reece in view of Ravindran, further in view of Griffin teaches the system of claim 9 wherein the scores for the plurality of weight analysis systems are used to generate proposed interactions for subsequent interactive social role-play sessions within the virtual immersive environment (Reece, para. 77, “the conversation analysis indicators can be mapped to coaching suggestion inferences or actions providing the coach suggestions on topics, coaching materials, engagement techniques, etc., to better engage with the mentee or improve coaching.” Para. 207, “rules are applied to conversation analysis indicators to determine actions (e.g., notifications, triggering new coaching pairing, suggesting training materials, selecting coaching techniques, etc.).”).
Claims 21 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Reece et al. (US 2022/0343911, hereinafter referred to as Reece) in view of Ravindran et al. (US 10,311,645 B1, hereinafter referred to as Ravindran) and Griffin (US 2023/0178217).
Regarding claim 21, Reece teaches a computer-implemented immersive social communication training system for improving social communication deficits of a participant, comprising:
an interaction analysis system configured to process data generated during the interactive social role-play interactions to generate structured interaction data including identified questions, responses, pauses, and associated metadata (Reece, para. 39, “The conversation analytics system is configured to apply natural language processing algorithms to the text data. In one implementation, conversation keywords, interruptions, topic changes, questions, use of passive voice, and so on, may be identified.” At least para. 156-161 describe the structured interaction data, calling them conversation features.);
a competency evaluation engine configured to evaluate the structured interaction data relative to predefined social communication competency constructs corresponding to on-target and off-target social behaviors (Reece, para. 101, “conversation analysis indicators 612 include engagement scores, enthusiasm scores, ownership score, goal score, interruptions score, and ‘time spent listening’ score, and/or attention scores. For example, engagement scores may be influenced by the number of questions asked and the length of statements made. Enthusiasm scores may be influenced by changes in pitch, changes in voice volume, and excited facial expressions. Attention scores may be defined by gaze detection (e.g., on-screen gaze, eye contact gaze) and body pose detection from the video data.”) and to generate a social competency score comprising construct-specific competency values (Reece, para. 35, “the communication skills of the mentee can be evaluated by performing computational methods (e.g., neural network based analysis) on the acoustic/video data.” Para. 69, “conversation analysis indicators can be one or more scores for the entire conversation or parts of the conversation, such as an overall effectiveness or quality rating for the conversation or parts of the conversation.”);
an annotator verification interface configured to present the structured interaction data and the social competency score to a annotator and to receive annotator feedback modifying or confirming at least a portion of the social competency score (Reece, para. 169, “In another implementation, ML training system 1848 communicates with an annotator for supervised learning, as shown in FIG. 11. For example, ML training system 1848 may transmit utterance features to an annotator, receive synthesized conversation features identified by the annotator, and subsequently train conversation synthesis ML system 1844 to automatically identify those conversation features.”);
a training module configured to update the competency evaluation engine based on the annotator feedback to improve subsequent competency determinations (Reece, para. 169, “ML training system 1848 trains conversation synthesis ML system 1844 to identify synthesized conversation features in response to features from the data modalities. In one implementation.”); and
an interaction control module configured to modify at least one subsequent social role-play session to target identified social communication deficits of the participant (Reece, para. 35, “The rich acoustic/video data generated by the videoconference can be analyzed to quantify the underlying coaching relationship, and guide further developments in the relationship. For example, the effectiveness of the coach may be identified, and further effectiveness of scores may be determined to identify areas for improvement… the communication skills of the mentee can be evaluated by performing computational methods (e.g., neural network based analysis) on the acoustic/video data. Tracking these conversation analysis indicators throughout an individual conversation, and across multiple conversations, may encourage skill development in the coach and the mentee.” Para. 77, “the conversation analysis indicators can be mapped to coaching suggestion inferences or actions providing the coach suggestions on topics, coaching materials, engagement techniques, etc., to better engage with the mentee or improve coaching.”).
Reece does not explicitly teach a virtual immersive environment comprising one or more interactive virtual environment elements and at least one avatar configured to engage in social role-play interactions with the participant;
However, in a related art, Ravindran teaches the social role-play sessions are within a virtual immersive environment (Ravindran, Col. 5, lines 13-20, “The virtual world, and the one or more stimulations (e.g., visual, auditory, haptic) presented in the virtual world, can be designed to treat the given mental developmental disorder and/or improve the given social skill. For example, the virtual world can be tailored for a given mental or developmental disorder. Alternatively or in addition, the virtual world can be tailored for a given social skill.” Line 47 of Col. 9 – line 6 of Col. 10 provide multiple examples of social role-play sessions performed within a virtual immersive environment.) and that the annotator is a clinician (Ravindran, Col. 2, line 3, “human expert (e.g., therapist)”. It is noted that the term “therapist” in this context is synonymous with the term “clinician”.).
It would have been obvious to a person having ordinary skill in the art before the effective filing of the claimed invention for the computer-based social role-play sessions in Reece to be held within a virtual immersive environment and for the annotator to be a clinician because “[u]sing the virtual or augmented reality systems may advantageously provide various therapeutic values to a subject (or user) receiving treatment… that need a more gentle and controlled introduction to certain aspects of the real world” or “any and all individuals in the general population that may benefit from practicing social skills or routines in a controlled environment.” Ravindran at Col. 7, lines 10-45.
Reece in view of Ravindran does not explicitly teach update one or more weights.
However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention for Reece in view of Ravindran to include a training module updating one or more weights and then the system using the one or more weights because “’training’ a computer-implemented machine learning model refers to any process by which parameter, hyper parameters, weights, and/or any other value related model accuracy are adjusted to improve the fit of the computer-implemented machine learning model to the training data.” See Griffin at para. 157.
Regarding claim 22, Reece in view of Ravindran and Griffin teaches the computer-implemented immersive social communication training system of claim 21, wherein the predefined social communication competency constructs comprise one or more of:
a strategic social attention competency construct (Reece, para. 71, “conversation analytics system 400 generates multiple conversation analysis indicators (e.g., conversation scores, conversation analysis indicators, openness scores, engagement scores, ownership scores, goal scores, interruptions scores, ‘time spent listening’ scores, and emotional labels).” This is includes a strategic social attention competency as described in at least para. 39-41 of the instant amended specification.),
a discourse competency construct (Reece, para. 71, “conversation analytics system 400 generates multiple conversation analysis indicators (e.g., conversation scores, conversation analysis indicators, openness scores, engagement scores, ownership scores, goal scores, interruptions scores, ‘time spent listening’ scores, and emotional labels).” This is includes a discourse competency as described in at least para. 42-44 of the instant amended specification.),
a theory of mind competency construct (Reece, para. 71, “conversation analytics system 400 generates multiple conversation analysis indicators (e.g., conversation scores, conversation analysis indicators, openness scores, engagement scores, ownership scores, goal scores, interruptions scores, ‘time spent listening’ scores, and emotional labels).” This is includes a theory of mind competency as described in at least para. 45-47 of the instant amended specification.),
an expressive reasoning competency construct (Reece, para. 71, “conversation analytics system 400 generates multiple conversation analysis indicators (e.g., conversation scores, conversation analysis indicators, openness scores, engagement scores, ownership scores, goal scores, interruptions scores, ‘time spent listening’ scores, and emotional labels).” This is includes an expressive reasoning competency as described in at least para. 48-50 of the instant amended specification.), and
a transformational social competency construct (Reece, para. 71, “conversation analytics system 400 generates multiple conversation analysis indicators (e.g., conversation scores, conversation analysis indicators, openness scores, engagement scores, ownership scores, goal scores, interruptions scores, ‘time spent listening’ scores, and emotional labels).” This is includes a transformational social competency as described in at least para. 51-52 of the instant amended specification.).
Response to Arguments
Applicant's arguments with respect to specification and drawing amendments have been fully considered but they are not persuasive. Applicant is directed to the objections to both the specification and drawings above which address the amendments.
Applicant's arguments with respect to specification and drawing amendments have been fully considered but they are not persuasive. Applicant is directed to the objections to both the specification and drawings above which address the amendments, including the introduction of new matter.
Applicant's arguments with respect to amendments to the claims have been fully considered but they are not persuasive. Applicant is directed to objections and rejections of the claims which address the amendments, including the introduction of new matter.
Applicant's arguments with respect to the denial of benefit of priority have been fully considered but they are not persuasive. Applicant asserts that any determination of priority is not material to the patentability of the presently amended claims. Applicant also does not concede the denial of benefit of priority and asserts a reservation of right to address issues reality to priority and entitlement to the benefit of US Provisional Application 63/548,341 if and when such issues become relevant. This is construed as repeating the determination of priority is not material.
Examiner is not persuaded. Contrary to such assertions, the determination that the pending claims do not gain benefit of priority to US Provisional Application 63/548,341 further evidences the introduction of new matter into the amended disclosure and claims, which directly affects patentability determinations.
Applicant's arguments with respect to the rejections of claims 8-10, 18, and 19 under 35 USC 112(b) have been fully considered but they are not persuasive. Applicant asserts that the amendments fully address the rejections.
Examiner is not persuaded. Applicant is directed to the rejections of the claims which have been updated to address the amendments to the claims. It is noted that the canceling of claims 19 and 20 render the associated rejections of these claims moot, and those rejections have thus been withdrawn.
Applicant's arguments with respect to the rejections of the claims under 35 USC 101 have been fully considered but they are not persuasive. Applicant asserts that the amended claims recite a specific technological implementation for generating objective assessment criteria of social skills, including (i) generation of structured interaction data from speech-to-text processing of interactive role-play sessions, (ii) evaluation of such data relative to predefined social communication competency constructs corresponding to on-target and off-target behaviors, (iii) generation of social competency scores based on such evaluation, and (iv) incorporation of clinician feedback to modify and improve subsequent system operation. Applicant then asserts that this integrates any judicial exception into a practical application.
Examiner is not persuaded. Steps (i)-(iv), described in the disclosure, merely amount to the conventional use of natural language processing and supervised machine learning. Such arguments further evidence that the focus of the claimed invention is on the analysis of the collected data, which is itself at best merely an improvement within the abstract idea. See pg. 2-3 in SAP America Inc. v. lnvestpic, LLC (890 F.3d 1016, 126 USPQ2d 1638 (Fed. Cir. 2018) which proffered “[w]e may assume that the techniques claimed are groundbreaking, innovative, or even brilliant, but that is not enough for eligibility. Nor is it enough for subject-matter eligibility that claimed techniques be novel and nonobvious in light of prior art, passing muster under 35 U.S.C. §§ 102 and 103. The claims here are ineligible because their innovation is an innovation in ineligible subject matter. Their subject is nothing but a series of mathematical calculations based on selected information and the presentation of the results of those calculations.”
Applicant's arguments with respect to the rejections of the claims under 35 USC 102/103 have been fully considered but they are not persuasive. Applicant asserts that the cited prior art do not teach the amended claims.
Examiner is not persuaded. Applicant is directed to the rejections above which have been updated to address the amendments to the claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL LANE whose telephone number is (303)297-4311. The examiner can normally be reached Monday - Friday 8:00 - 4:30 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xuan Thai can be reached at (571) 272-7147. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL LANE/Examiner, Art Unit 3715
Prosecution Insights