DETAILED CORRESPONDANCE
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims
This final office action on merits is in response to the communication received on 04/22/2026. Claim 16 is cancelled. Amendments to claims 1, 6-12, 14-15, and 17 are acknowledged and have been carefully considered. Claims 1-15, and 17 are pending and considered below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15, and 17 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
Under step 1, the analysis is based on MPEP 2106.03, and claims 1-13 are drawn to a method and claims 14-15, and 17 are drawn to a system. Thus, each claim, on its face, is directed to one of the statutory categories (i.e., useful process, machine, manufacture, or composition of matter) of 35 U.S.C. §101.
Step 2A Prong One
Claim 1 recites the limitations of determining a content module structured as a plurality of sequential sections to be played on an XR user device, wherein the determining comprises analyzing, at least a patient profile stored in the profile module of the XR educational system, clinical data stored in the records module of the XR educational system, or historical interaction data stored in the memory device of the XR educational system, to generate a prioritized selection of the content module tailored to an intended user; transcribing audio recording of the microphone into machine- readable text; analyzing user context including user behavior, user preferences, and learning patterns, via live data captured by said microphone, said camera, or said motion sensor, and said stored data within the XR educational system; and dynamically selecting a pace of a subsequent rendering of the content module, of the immersive three-dimensional XR environment according to the detected user context and said stored data and tracking a user's progress on viewing the content module on the XR user device, wherein the tracking comprises collecting, biometric feedback data and user interaction data generated during execution of the content module. These limitations, as drafted, are processes that, under their broadest reasonable interpretations, cover performance of the limitations in the mind or by using a pen and paper. Even when considering the “by the Al module” or “by the XR user device” language, the claim encompasses a user or healthcare professional reviewing patient information, evaluating user behavior, learning patterns, and prior interactions, selecting and prioritizing education content, determining an appropriate pace for presenting educational content based on those evaluations, and monitoring user’s progress by observing the user’s responses or interactions in their mind or by using a pen and paper. The mere nominal recitation of “by the Al module” or “by the XR user device” does not take the claim limitations out of the mental processes grouping. Thus, the claim recites a mental process which is an abstract idea.
Claim 1 also recites as a whole a method of organizing human activity (i.e., managing personal behavior or relationships or interactions between people, (including social activities, teaching, and following rules or instructions) because the claim recites a method that allows users to execute the content module on the XR user device, wherein the executing comprises rendering, an immersive three- dimensional XR environment including interactive three-dimensional content elements comprising at least one of: animated anatomical representations and spatial audio narration. This is a method of managing personal behavior through teaching by providing educational content and instructional information to a user in order to improve the user’s understanding of a healthcare topic. The mere nominal recitation of a generic XR user device does not take the claim out of the methods of certain methods of organizing human activity. Thus, the claim recites an abstract idea.
The types of identified abstract ideas are considered together as a single abstract idea for analysis purposes.
Claim 6 recites the limitations of extracting, from the elements library, one or more content elements based on the subject of the content module, wherein the extracting comprises applying, a subject classification algorithm to the subject-matter metadata of the plurality of content elements to identify and rank content elements corresponding to the identified subject; generating, the content module from the one or more extracted content elements, wherein the generating comprises assemble and sequence the extracted content elements into a structured content module. These limitations, as drafted, are processes that, under their broadest reasonable interpretations, cover performance of the limitations in the mind or by using a pen and paper. Even when considering the “by the Al module” language, the claim encompasses a user evaluating a requested subject, reviewing available educational content, identifying and ranking content relevant to the subject, and organizing the selected content into a structured educational model in their mind or by using a pen and paper. The mere nominal recitation of by the Al module does not take the claim limitations out of the mental processes grouping. Thus, the claim recites a mental process which is an abstract idea.
Under Step 2A Prong Two
The claimed limitations, as per claim 1, include:
a processing device, a memory device, an artificial intelligence (Al) module, a records module, and a profile module, the method comprising:
determining by the Al module of the XR educational system, a content module structured as a plurality of sequential sections to be played on an XR user device, wherein the determining comprises analyzing, by the Al module, at least a patient profile stored in the profile module of the XR educational system, clinical data stored in the records module of the XR educational system, or historical interaction data stored in the memory device of the XR educational system, to generate a prioritized selection of the content module tailored to an intended user;
downloading the content module to the XR user device, wherein the XR user device comprises:
a camera; a microphone: a motion sensor; and an operational interface having a touch-sensitive surface;
executing the content module on the XR user device, wherein the executing comprises rendering, by the XR user device, an immersive three- dimensional XR environment including interactive three-dimensional content elements comprising at least one of: animated anatomical representations and spatial audio narration;
wherein said Al module is programmatically configured to:
transcribe audio recording of the microphone into machine- readable text;
analyze user context including user behavior, user preferences, and learning patterns, via live data captured by said microphone, said camera, or said motion sensor, and said stored data within the XR educational system; and
dynamically selecting a pace of a subsequent rendering of the content module, by the XR user device, of the immersive three-dimensional XR environment according to the detected user context and said stored data and
tracking a user's progress on viewing the content module on the XR user device, wherein the tracking comprises collecting, by sensors of the user device, biometric feedback data and user interaction data generated during execution of the content module, and storing the collected biometric feedback data and user interaction data in the memory device of the XR educational system to update the historical interaction data.
The claimed limitations, as per method claim 6, include:
storing in the elements library of the XR educational system, content elements tagged with subject-matter metadata;
receiving a request for a content module, the request comprising an identification of a subject of the content module;
extracting, by the Al module, from the elements library, one or more content elements based on the subject of the content module, wherein the extracting comprises applying, by the Al module, a subject classification algorithm to the subject-matter metadata of the plurality of content elements to identify and rank content elements corresponding to the identified subject;
generating by the content builder module, the content module from the one or more extracted content elements, wherein the generating comprises applying one or more machine learning algorithms trained to assemble and sequence the extracted content elements into a structured content module formatted for rendering as an immersive three-dimensional XR environment on a head-mounted XR user device; and
storing the content module in a computer-readable storage medium.
Examiner Note: underlined elements indicate additional elements of the claimed invention identified as performing the steps of the claimed invention.
The judicial exception expressed in claim 1 is not integrated into a practical application. The claim as a whole merely describes how to generally “apply” the concept of evaluating patient information to determine and present personalized educational content, adjusting the presentation based on user context, and tracking user progress in a computer environment. The claimed computer components (i.e., a processing device, a memory device, an artificial intelligence (Al) module, a records module, and a profile module, by the Al module of the XR educational system, by the Al module, a camera; a microphone: a motion sensor; and an operational interface having a touch-sensitive surface, by the XR user device, wherein said Al module is programmatically configured to, by the XR user device, by sensors of the user device) are recited at a high level of generality and are merely invoked as tools to perform an existing process of evaluating information, selecting and presenting educational content, adapting the presentation based on observations, and monitoring s user’s educational progress. Simply implementing the abstract idea on a generic computer is not a practical application of the abstract idea. Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application.
The judicial exception expressed in claim 1 is not integrated into a practical application. The claim recites the additional elements of downloading the content module to the XR user device, wherein the XR user device comprises; and storing the collected biometric feedback data and user interaction data in the memory device of the XR educational system to update the historical interaction data. These limitations are recited at a high level of generality (i.e., as a general means of collecting data and storing data for subsequent use), and amount to merely data gathering and insignificant application, which are forms of insignificant extra-solution activities. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application. The claim is directed to an abstract idea.
The judicial exception expressed in claim 6 is not integrated into a practical application. The claim as a whole merely describes how to generally “apply” the concept of selecting, organizing, and generating educational content based on a requested subject in a computer environment. The claimed computer components (i.e., storing in the elements library, content elements tagged with subject-matter metadata; by the Al module; by the content builder module; and applying one or more machine learning algorithms trained to) are recited at a high level of generality and are merely invoked as tools to perform an existing process of reviewing available educational content, identifying and ranking content relevant to a requested subject, and assembling and sequencing the selected content into a structured educational module. Simply implementing the abstract idea on a generic computer is not a practical application of the abstract idea. Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application.
The judicial exception expressed in claim 6 is not integrated into a practical application. The abstract idea is merely carried out in a technical environment or field (i.e., an extended reality (XR) educational system for generating healthcare educational content modules), however fails to contain meaningful limitations beyond generally linking the use of an abstract idea to a particular technological environment (see MPEP 2106.05(h)). The additional elements that are carried out in a technical environment includes of the XR educational system and formatted for rendering as an immersive three-dimensional XR environment on a head-mounted XR user device. Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application.
The judicial exception expressed in claim 6 is not integrated into a practical application. The claim recites the additional elements of receiving a request for a content module, the request comprising an identification of a subject of the content module and storing the content module in a computer-readable storage medium. These limitations are recited at a high level of generality (i.e., as a general means of gathering information and storing the results), and amounts to merely data gathering and insignificant application, which are forms of insignificant extra-solution activities. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application. The claim is directed to an abstract idea.
Therefore, under step 2A, the claims are directed to the abstract idea, and require further analysis under Step 2B.
Under step 2B
Claims 1 and 6 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed with respect to Step 2A, the claims as a whole merely describes how to generally “apply” the concepts of evaluating patient information to determine and present personalized educational content, adjusting the presentation based on user context, and tracking user progress (claim 1) and selecting, organizing, and generating educational content based on a requested subject (claim 6) in a computer environment. Thus, even when viewed as a whole, nothing in the claims add significantly more (i.e., an inventive concept) to the abstract idea.
For claim 1, under step 2B, the additional elements of downloading the content module to the XR user device, wherein the XR user device comprises; and storing the collected biometric feedback data and user interaction data in the memory device of the XR educational system to update the historical interaction datal have been evaluated. The method comprising a processing device performs a general function of receiving patient data for transmitting, receiving, and storing electronic data for subsequent processing, which represents a well-understood, routine, and conventional activity in the field of computer systems and electronic data processing The specification discloses that the processor is used in its ordinary capacity as a data input device and does not describe any improvement to the computer itself or to the functioning of the overall computer system (see [0057]). Also noted in Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016), merely collecting information for analysis without a technological improvement does not add significantly more to an abstract idea. The use of the method is no more than collecting information before evaluating patient information, selecting educational content, adjusting the presentation, and tracking user progress and does not integrate the abstract idea into a practical application. Additionally, as noted in In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016), merely storing the collected biometric feedback data and user interaction data in the memory device of the XR educational system to update the historical interaction data represents an insignificant application of the underlying mental process, as the storing simply records the results of the data collection for later use and does not impose any meaningful limitation or add any technological improvement. Therefore, the claim does not recite an inventive concept and is not patent eligible.
Claim 6 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed with respect to Step 2A, the abstract idea is merely carried out in a technical environment or field, however fails to contain meaningful limitations beyond generally linking the use of an abstract idea to a particular technological environment. Thus, even when viewed as a whole, nothing in the claim adds significantly more (i.e., an inventive concept) to the abstract idea.
For claim 6, under step 2B, the additional elements of receiving a request for a content module, the request comprising an identification of a subject of the content module and storing the content module in a computer-readable storage medium have been evaluated. As noted in Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016), merely collecting information for analysis without a technological improvement does not add significantly more to an abstract idea. The use of the method is no more than collecting information before selecting, identifying, ranking, and assembling educational content into a structured content module and does not integrate the abstract idea into a practical application. Additionally, as noted in In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016), merely storing the content module in a computer-readable storage medium represents an insignificant application of the underlying mental process, as the storage preserves the generated result for later use and does not alter the manner in which the abstract idea is performed and does not impose any meaningful limitation or add any technological improvement. Therefore, the claim does not recite an inventive concept and is not patent eligible.
Claims 2-5, 7-13, 15 and 17 recite the additional element of the content module (2-5, 7-13, and 15 and 17), and a healthcare content module (claim 9). However, these additional elements amount to implementing an abstract idea on a generic computing device. As such, these additional elements, when considered individually or in combination with the prior devices, do not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea.
Thus, as the dependent claims remain directed to a judicial exception, and as the additional elements of the claims do not amount to significantly more, the dependent claims are not patent eligible.
Therefore, the claims here fail to contain any additional element(s) or combination of additional elements that can be considered as significantly more and the claims are rejected under 35 U.S.C. 101 for lacking eligible subject matter.
Claim Rejections - 35 USC § 102
Applicant’s amendments overcome the rejection under 35 U.S.C. 102 because the cited reference (Morgan et al.) no longer teaches or suggests al of the claimed limitations. Accordingly, the rejection is withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-15, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Morgan et al. (U.S. Patent 11961197 B1), referred to hereinafter as Morgan, in view of Kurani (U.S. Patent Publication 2020/0258420 A1), referred to hereinafter as Kurani.
Regarding claim 1, Morgan teaches a method of delivering interactive content through an extended reality (XR) educational system comprising a processing device, a memory device, an artificial intelligence (Al) module, a records module, and a profile module, the method comprising (Morgan, Col. 9, “In one exemplary embodiment, the present XR Health Platform utilizes a combined extended reality head-mounted display and computing device (referred to herein as “HMD and Computing Device”). The exemplary HMD and Computing Device comprises a standalone (meaning wireless and untethered) extended reality system using an operating system such as Google's Android, Apple's iOS, and others. The exemplary HMD and Computing Device may include a virtual reality head-mounted display, an augmented reality head-mounted display, or a mixed reality head-mounted display. The HMD and Computing Device may also include a web browser, high-speed data access via Wi-Fi and mobile broadband, Bluetooth capability, an expandable memory slot for micro SD cards.”
Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”
Morgan, Col. 3, ““ML/AI model” (also ML/AI) includes any models, functions, algorithms, code, and/or programming that involve machine learning, artificial intelligence, mathematical, statistical, logic-based processes, and/or control functionalities.”, and
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”):
determining by the Al module of the XR educational system, a content module structured as a plurality of sequential sections to be played on an XR user device, wherein the determining comprises analyzing, by the Al module, at least a patient profile stored in the profile module of the XR educational system, clinical data stored in the records module of the XR educational system, or historical interaction data stored in the memory device of the XR educational system (Morgan, Col. 12, “Any of the exemplary platform features may be applied as any portion of applications relating to the health, wellness, and/or health education of individuals. For example, features within the Anatomy Module discussed below may be used for having an individual identify the location of a symptom (as part of a diagnostic), may be used to identify an area in need of strengthening (as part of a therapeutic), may be used by clinicians in documenting changes in symptoms over time (as part of care delivery), as an anatomy simulator, and/or to educate patients regarding anatomy-related subject matter. Additionally, any of the platform features may be combined with any other set of platform features to form applications of the exemplary XR Health Platform. Additionally, any set of items (any item within any module, feature, or sub-feature) may be combined with any other set of items for any implementation(s) of the XR Health Platform.”
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”, and
Morgan, Col. 110, “According to this exemplary feature, at the conclusion of any use, the patient and/or the patient's legal guardian may be given a unique and/or one-time use code that may be linked to electronic medical records. This allows the patient to utilize data in subsequent procedures to further optimize and/or personalize his digital anesthetic. This use code may also identify any patient preferences and/or characterize the patient's dynamic titration profile from previous usages of the system. The unique identifier may be delivered via a scratch off card with the code or by email. Alternatively, the above functions may be enabled without the “unique and/or one-time use code”, but through integration with electronic medical records and/or one or more similar programs handing patient data.”);
downloading the content module to the XR user device, wherein the XR user device comprises: a camera; a microphone: a motion sensor; and an operational interface having a touch-sensitive surface (Morgan, Col. 11, “In other embodiments, the present XR Health Platform may utilize other computer networks; for example, a wide area network (WAN), local area network (LAN), or intranet. The host server may comprise a processor and a computer readable medium, such as random access memory (RAM). The processor is operable to execute certain programs for performing the present XR Health Platform and other computer program instructions stored in memory. Such processor may comprise a microprocessor (or any other processor) and may also include, for example, a display device, internal and external data storage devices, cursor control devices, and/or any combination of these components, or any number of different components, peripherals, input and output devices, and other devices. Such processors may also communicate with other computer-readable media that store computer program instructions, such that when the stored instructions are executed by the processor, the processor performs the acts described further herein. Those skilled in the art will also recognize that the exemplary environments described herein are not intended to limit application of the present XR Health Platform, and that alternative environments may be used without departing from the scope of the invention. Various problem-solving programs incorporated into the present XR Health Platform and discussed further herein, may utilize as inputs, data from a data storage device or location. In one embodiment, the data storage device comprises an electronic database. In other embodiments, the data storage device may comprise an electronic file, disk, or other data storage medium. The data storage device may store features of the disclosure applicable for performing the present XR Health Platform. The data storage device may also include other items useful to carry out the functions of the present XR Health Platform. In one example, the exemplary computer programs may further comprise algorithms designed and configured to perform the present XR Health Platform.”
Morgan, Col. 2, ““XR device” refers to any device that can be used for simulating, viewing, engaging, experiencing, controlling and/or interacting with XR. This includes headsets, head-mounted displays (HMD), augmented reality glasses, 2D displays viewing XR content, 2D displays, 3D displays, computers, controllers, projectors, other interaction devices, mobile phones, speakers, microphones, cameras, headphones, haptic devices, and the like.”
Morgan, Col. 116, “In another exemplary embodiment, the present device may comprise sensors, transceivers, computers, and/or Bluetooth and/or Wi-Fi connectivity. This wireless technology may be used to send and/or receive points of platform data to and/or from other platform features and/or modules.”
Morgan, Col. 4, ““Patient input methods” (synonymous with “patient input(s)”) refers to patient interactions with platform features which may be accomplished using one or more of the following: controller inputs (which includes controller buttons, joysticks, and/or touch sensors, and/or any other controller functionalities), keyboard inputs, computer mouse inputs, touchscreen inputs, physical movements, spoken voice/verbal inputs, gaze inputs, text inputs, question response inputs, communication inputs, and/or by any other visual, tactile, movement-based, and/or auditory means, and/or by any other means described herein.”);
executing the content module on the XR user device, wherein the executing comprises rendering, by the XR user device, an immersive three- dimensional XR environment including interactive three-dimensional content elements comprising at least one of: animated anatomical representations and spatial audio narration (Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”
Morgan, Col. 57, “The exemplary movement integration feature may also comprise variations where three-dimensional objects, two-dimensional overlays, and/or camera filters are instantiated one or more times in XR as a feedback mechanism for the patient. This may occur if the patient fails to traverse waypoints; and/or if the patient fails to traverse waypoints with the appropriate anatomical structure(s) and/or appropriate XR hardware; and/or upon hyperextension of an anatomical structure and/or skeletal joint; and/or upon hyperflexion of an anatomical structure and/or skeletal joint; and/or to encourage the patient to complete a greater percentage of movements; and/or to complete movements and/or series of movements.”, and
Morgan, Col. 57 “The exemplary movement integration feature may also comprise variations combined with the telecommunication module for real-time voice, and/or video, and/or text interactions between the patient in XR and clinicians using a companion application, and/or web portal, and/or in XR.”);
wherein said Al module is programmatically configured to: transcribe audio recording of the microphone into machine- readable text (Morgan, Col. 39, “Using the present feature, a clinician and/or ML/AI models select snippets to deliver to a patient using the content object and snippet feature as described herein. Any selected text snippets are converted to audio files containing spoken language using text-to-speech models (TTS). Audio voice over(s) of text snippets may be configured to be recited out loud to patients in XR. Patient voice response(s) may be entered using patient input methods. STT models are applied to audio to convert the audio into text. Patient audio and/or text transcripts of patient voice may be saved in database(s) and/or application(s) for later analysis. Natural language processing (NLP) models may be applied to either the text and/or original audio with feature extraction and/or analysis results of text and/or audio (vocal biomarkers, semantics, sentiment, clinical meaning, and the like) being saved. The most appropriate snippet(s) are then selected to deliver to the patient next using one or more platform features.”)
analyze user context including user behavior, user preferences, and learning patterns, via live data captured by said microphone, said camera, or said motion sensor, and said stored data within the XR educational system (Morgan, Col. 39, “Using the present feature, a clinician and/or ML/AI models select snippets to deliver to a patient using the content object and snippet feature as described herein. Any selected text snippets are converted to audio files containing spoken language using text-to-speech models (TTS). Audio voice over(s) of text snippets may be configured to be recited out loud to patients in XR. Patient voice response(s) may be entered using patient input methods. STT models are applied to audio to convert the audio into text. Patient audio and/or text transcripts of patient voice may be saved in database(s) and/or application(s) for later analysis. Natural language processing (NLP) models may be applied to either the text and/or original audio with feature extraction and/or analysis results of text and/or audio (vocal biomarkers, semantics, sentiment, clinical meaning, and the like) being saved. The most appropriate snippet(s) are then selected to deliver to the patient next using one or more platform features.”
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”
Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”
Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”, and
Morgan, Col. 110, “According to this exemplary feature, at the conclusion of any use, the patient and/or the patient's legal guardian may be given a unique and/or one-time use code that may be linked to electronic medical records. This allows the patient to utilize data in subsequent procedures to further optimize and/or personalize his digital anesthetic. This use code may also identify any patient preferences and/or characterize the patient's dynamic titration profile from previous usages of the system. The unique identifier may be delivered via a scratch off card with the code or by email. Alternatively, the above functions may be enabled without the “unique and/or one-time use code”, but through integration with electronic medical records and/or one or more similar programs handing patient data.”); and
by the XR user device, of the immersive three-dimensional XR environment according to the detected user context and said stored data (Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”
Morgan, Col. 3, ““ML/AI model” (also ML/AI) includes any models, functions, algorithms, code, and/or programming that involve machine learning, artificial intelligence, mathematical, statistical, logic-based processes, and/or control functionalities.”
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”
Morgan, Col. 110, “According to this exemplary feature, at the conclusion of any use, the patient and/or the patient's legal guardian may be given a unique and/or one-time use code that may be linked to electronic medical records. This allows the patient to utilize data in subsequent procedures to further optimize and/or personalize his digital anesthetic. This use code may also identify any patient preferences and/or characterize the patient's dynamic titration profile from previous usages of the system. The unique identifier may be delivered via a scratch off card with the code or by email. Alternatively, the above functions may be enabled without the “unique and/or one-time use code”, but through integration with electronic medical records and/or one or more similar programs handing patient data.”
Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”, and
Morgan, Col. 39, “Using the present feature, a clinician and/or ML/AI models select snippets to deliver to a patient using the content object and snippet feature as described herein. Any selected text snippets are converted to audio files containing spoken language using text-to-speech models (TTS). Audio voice over(s) of text snippets may be configured to be recited out loud to patients in XR. Patient voice response(s) may be entered using patient input methods. STT models are applied to audio to convert the audio into text. Patient audio and/or text transcripts of patient voice may be saved in database(s) and/or application(s) for later analysis. Natural language processing (NLP) models may be applied to either the text and/or original audio with feature extraction and/or analysis results of text and/or audio (vocal biomarkers, semantics, sentiment, clinical meaning, and the like) being saved. The most appropriate snippet(s) are then selected to deliver to the patient next using one or more platform features.”); and
tracking a user's progress on viewing the content module on the XR user device, wherein the tracking comprises collecting, by sensors of the user device, biometric feedback data and user interaction data generated during execution of the content module, and storing the collected biometric feedback data and user interaction data in the memory device of the XR educational system to update the historical interaction data (Morgan, Col. 22, “In the next session in the simulation and/or for the patient (and/or in one or more later sessions as configured per items within this feature), specific data fields on each applicable question template are automatically populated using, for example: patient's and/or virtual object's proximity and/or position relative to one or more 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and virtual human avatars; data captured through simulated interactions between a platform feature and 2D and/or 3D virtual objects and/or other platform features; data captured through the use of scene and/or session metadata; data captured through the use of biometric data; data captured through the use of any other platform data; and data captured through the use of data from other interactions and/or simulations.”
Morgan, Col. 97, “Other exemplary scenes and/or features may include XR scene(s) and/or features within XR scenes where past accomplishment(s), achieved goal(s), and/or positive progress are highlighted. The past accomplishment(s), achieved goal(s), and/or positive progress may be highlighted using sentences/statements, snippets, content objects, audio items, video items, 2D or 3D effects, animations, and/or replays of previously experienced scenes or sessions showing an avatar representing the patient to him/herself in 3rd person. Other exemplary scenes and/or features may include application of Socratic questioning/dialogue to identify negative thought patterns and/or cognitive distortions, and/or to attenuate negative or counterproductive thoughts or feelings, and/or to enhance positive or productive thoughts or feelings as described above.”
Morgan, Col. 40, “In yet another exemplary embodiment, the present feature includes a sub-feature that allows for the personalized, timely, and actionable delivery of feedback through points of other platform data. Exemplary points of data include biometric data, data from neuro-behavioral and/or cognitive assessments, data from compliance monitoring systems, imaging studies, data produced by one or more computer vision algorithms, vocal biomarker data, and/or data from diagnostic studies. For clarity, this feature also may utilize data points produced by the use of scenes, sessions, and/or regimens.”
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”
Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”
Morgan, Col. 116, “In another exemplary embodiment, the present device may comprise sensors, transceivers, computers, and/or Bluetooth and/or Wi-Fi connectivity. This wireless technology may be used to send and/or receive points of platform data to and/or from other platform features and/or modules.”
Morgan, Col. 2, ““XR device” refers to any device that can be used for simulating, viewing, engaging, experiencing, controlling and/or interacting with XR. This includes headsets, head-mounted displays (HMD), augmented reality glasses, 2D displays viewing XR content, 2D displays, 3D displays, computers, controllers, projectors, other interaction devices, mobile phones, speakers, microphones, cameras, headphones, haptic devices, and the like.”, and
Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”).
Morgan fails to explicitly teach to generate a prioritized selection of the content module tailored to an intended user; dynamically selecting a pace of a subsequent rendering of the content module.
Kurani teaches to generate a prioritized selection of the content module tailored to an intended user (Kurani [0059] “FIG. 2 depicts an exemplary personalized and adaptive math learning end to end application workflow to offer customized lessons to learners, according to some embodiments. The administrator 204 creates all the course activities and the personalized and adaptive machine learning method rearranges lesson plan to create fit for the student. Then the user 202 can learn from these lessons and quizzes. Every time the learner submits a quiz, the algorithm can change based on their performance on the quiz. The adaptive machine learning method using student's attribute data can keep changing dynamically like this until the learner is to the final quiz of the lesson. It consists of machine learning algorithms and statistical models to perform the task of learning math. It uses unsupervised learning algorithms since it only consists of set of data that contains only student inputs, and find structure in the data, like grouping or clustering of data points. In case, if all the student data is not available it uses the predictive model to best guess the data as part of the input set.”, and
Kurani [0060] “FIG. 3 illustrates personalized and adaptive machine learning process 300 used to create customized lessons for a learner, according to some embodiments. Process 300 includes personalization step 302, which is implemented after the learner attributes are entered. In personalization step 302, the attribute data is analyzed, classified and clustered.”); and
dynamically selecting a pace of a subsequent rendering of the content module (Kurani [0062] “FIG. 4 is a diagram of a personalized and adaptive machine learning process 400 to analyze, classify and cluster various attributes and associated parameters, according to some embodiments. Process 400 includes of students' personal profile attributes (PPA) 402. Example students' personal profile attributes 402 can include, inter alia: age, gender, weight, height and so on. Personal interest attributes (PIA) include sports activities like soccer, baseball, table tennis, swimming, running, jogging and arts activities like drawing, painting. Personal instructional format attribute (IFA) includes teaching format like audio, video, step-by-step, slide, animations, class room and so on. Performance attributes (PMA) include their grade, competency in the subject matter, level of the student's current understanding of the domain content. Performance attributes can be based on previous learning scores, tests, quizzes, homework etc. Cognitive attributes (CGA) include working memory capacity, associative learning skill, inductive reasoning ability, information processing speed.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Morgan's XR healthcare education platform with the adaptive machine learning techniques taught by Kurani. Morgan teaches delivering personalized educational content within an XR environment based on patient profiles, historical interactions, biometric information, and adaptive platform data, while iteratively modifying subsequent sessions based on previous patient results. Kurani further teaches analyzing learner attributes, performance, instructional preferences, and cognitive characteristics using machine learning to dynamically personalize and rearrange instructional content according to the learner's current understanding and progress. A person of ordinary skill in the art would have recognized that incorporating Kurani's adaptive instructional techniques into Morgan's personalized XR educational platform would have predictably improved the personalization and effectiveness of patient education by dynamically tailoring the presentation of educational content to the individual patient's characteristics, preferences, and performance.
Furthermore, it would have been obvious to utilize Kurani's adaptive learning techniques to dynamically adjust the presentation, sequencing, and pacing of Morgan's XR educational sessions based on detected user context and stored patient data. Both references are directed to improving individualized educational experiences through the analysis of user specific information and prior interactions. Applying Kurani's known adaptive machine learning techniques to Morgan's personalized XR healthcare platform would have involved the predictable use of prior art elements according to their established functions to provide a more responsive and individualized educational experience, yielding no more than the expected benefit of improved user engagement and instructional effectiveness.
Regarding claim 2, Morgan and Kurani teach the invention in claim 1, as discussed above, and further teach further comprising: compiling feedback collected based on user interaction with the content module executing the XR user device (Morgan, Col. 55, “A real-time movement biofeedback feature of the exemplary Movement Module may be used for real-time biofeedback where the scoring in one or more gamified XR experiences is at least partially determined by establishing and/or maintaining physical movements and/or positions. The movements and/or positions are determined as described herein, and whereby visual and/or auditory stimuli provide real-time feedback in terms of the correctness or incorrectness of the movements and/or physical positions. This correctness or incorrectness results in a higher or lower score, respectively. In one embodiment of this feature, scores are proportional to the angle and height of the controllers relative to the HMD as well as the length of time that this position is maintained. For example, if the patient holds his arms straight out in front of him at eye level, the score goes up proportionally to the time that this position is maintained.”, and
Morgan, Col. 22, “In the next session in the simulation and/or for the patient (and/or in one or more later sessions as configured per items within this feature), specific data fields on each applicable question template are automatically populated using, for example: patient's and/or virtual object's proximity and/or position relative to one or more 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and virtual human avatars; data captured through simulated interactions between a platform feature and 2D and/or 3D virtual objects and/or other platform features; data captured through the use of scene and/or session metadata; data captured through the use of biometric data; data captured through the use of any other platform data; and data captured through the use of data from other interactions and/or simulations.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to compile feedback collected from a user's interaction with the XR content module, as taught by Morgan. Morgan teaches collecting real time user interaction information, including movement performance, biometric feedback, and user responses during XR experiences, and providing corresponding visual and auditory feedback based on the user's performance. Compiling this collected feedback would have been a predictable use of the disclosed interaction and performance data to evaluate user progress, improve subsequent instructional sessions, and provide personalized guidance. Doing this represents the application of known feedback aggregation techniques to Morgan's XR educational platform to achieve the expected benefit of monitoring user performance and enhancing the effectiveness of personalized educational content.
Regarding claim 3, Morgan and Kurani teach the invention in claim 1, as discussed above, and further teach further comprising: customizing the content module based on a medical history of an intended user of the content module (Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”
Morgan, Col. 110, “According to this exemplary feature, at the conclusion of any use, the patient and/or the patient's legal guardian may be given a unique and/or one-time use code that may be linked to electronic medical records. This allows the patient to utilize data in subsequent procedures to further optimize and/or personalize his digital anesthetic. This use code may also identify any patient preferences and/or characterize the patient's dynamic titration profile from previous usages of the system. The unique identifier may be delivered via a scratch off card with the code or by email. Alternatively, the above functions may be enabled without the “unique and/or one-time use code”, but through integration with electronic medical records and/or one or more similar programs handing patient data.”, and
Morgan, Col. 12, “Any of the exemplary platform features may be applied as any portion of applications relating to the health, wellness, and/or health education of individuals. For example, features within the Anatomy Module discussed below may be used for having an individual identify the location of a symptom (as part of a diagnostic), may be used to identify an area in need of strengthening (as part of a therapeutic), may be used by clinicians in documenting changes in symptoms over time (as part of care delivery), as an anatomy simulator, and/or to educate patients regarding anatomy-related subject matter. Additionally, any of the platform features may be combined with any other set of platform features to form applications of the exemplary XR Health Platform. Additionally, any set of items (any item within any module, feature, or sub-feature) may be combined with any other set of items for any implementation(s) of the XR Health Platform.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to customize the content module based on the medical history of the intended user as taught and suggested by Morgan. Morgan teaches that the XR Health Platform provides healthcare educational applications by combining various platform features to educate patients regarding anatomy and other health subject matter, while maintaining a patient level profile containing patient characteristics and other relevant platform data to enable personalized, dynamic, and adaptive applications for individual patients. Morgan further teaches integrating electronic medical records, previous patient data, patient preferences, and prior system usage to further optimize and personalize subsequent patient interactions. A person of ordinary skill in the art would have recognized that utilizing a patient's medical history maintained within the patient profile and electronic medical records to customize the educational content presented to that patient would have been a predictable use of Morgan's disclosed personalization framework, yielding the expected benefit of providing educational content that is more individualized and clinically appropriate for the patient's medical condition.
Regarding claim 4, Morgan and Kurani teach the invention in claim 1, as discussed above, and further teach wherein determining the content module comprises: generating a visual representation of an intended user of the content module based on one or more images of the intended user; and integrating the visual representation of the intended user into the content module (Morgan, Col. 3., ““Virtual human avatar” refers to a humanoid virtual avatar which may be animated, simulated, programmatically controlled (using ML/AI models, for example), and/or represented through other types of rendered content and/or other media, and is designed to interact with, educate, instruct, demonstrate, advise, assist, guide, escort, diagnose, screen, test, treat, and/or manage disease(s) and/or health-related issues for patients in XR. Virtual human avatars may interact with patients and/or clinicians through spoken dialogue, text, rendered content, through visual means, and/or through any other method of communication. Virtual human avatars may possess characteristics that are virtual approximations and/or facsimiles of characteristics of real-world clinicians and/or patients. When used in this context, the term “virtual human avatar(s)” is synonymous with “digital twin(s)”.”
Morgan, Col. 57, “The exemplary movement integration feature may also comprise variations where avatar silhouettes are used for producing visual biofeedback for the patient, and where the color, size, texture, and/or shader on the avatar silhouette may be used to indicate the level of correctness or incorrectness of physical movements being performed by the patient.”
Morgan, Col. 86-87, “Any the logged datasets from assessments are compared to repeat assessments and/or population normal values (customized for the age and gender of the patient) by clinicians and/or ML/AI models. According to one exemplary embodiment, this comparison is implemented to identify if patient has had significant changes and/or remarkable results indicative of Alzheimer's and/or dementia in cognitive domains, movement assessments, and/or other neurological characteristics and/or features as described herein. The changes and/or clinically remarkable results may include (a) change and/or clinically remarkable results in voice features detectable through vocal analysis, NLP, STT, and/or similar ML/AI models; (b) change and/or clinically remarkable results in facial features detectable through computer vision algorithms, and/or other ML/AI models; (c) change and/or clinically remarkable results in performance on one or more cognitive domain tests with or without using ML/AI models; (d) change and/or clinically remarkable results in performance on one or more assessments of ability to carry out ADLs and/or IADLs with or without using ML/AI models; (e) change and/or clinically remarkable results in performance on one or more assessments of movement, and in particular gait and/or ability to ambulate using ML/AI models; (f) change and/or clinically remarkable results in patterns of patient inputs using data obtained from patient input methods during sessions using ML/AI models; (g) clinically remarkable results relating to other platform data points; and (h) clinically remarkable results relating to other platform features.”, and
Morgan, Col. 104, “In another exemplary embodiment, the present module feature may utilize ML/AI models to automatically characterize anatomic locations of pain, duration of pain, severity pain, date and time of onset of pain, and/or other aspects of pain. This may be completed using analyses of points of biometric data, for example, vocal biomarkers identifying a painful scream upon the onset or exacerbation of pain; aspects of pain identified through analysis of spoken and/or written dialogue obtained from patients while in XR; computer vision models identifying aspects of images that are consistent with or related to the body language, posturing, and/or facial expressions of someone that is experiencing pain; and/or the inputs and/or outputs of other ML/AI models.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to generate a visual representation of the intended user based on one or more images of the intended user and integrate that visual representation into the XR content module in view of Morgan's teachings. Morgan teaches the use of patient-specific virtual human avatars, or digital twins, that are configured to educate, instruct, and interact with patients within an XR environment, as well as incorporating avatar representations into XR sessions to provide visual feedback to the patient. Morgan further teaches utilizing computer vision algorithms and ML/AI models to analyze patient facial features, facial expressions, body language, and other image characteristics for patient specific analysis and personalization. A person of ordinary skill in the art would have recognized that utilizing the patient images analyzed by Morgan's disclosed computer vision techniques to generate or customize the patient virtual avatar integrated into the XR educational content would have been a predictable implementation of Morgan's personalization framework, yielding the expected benefit of providing a more realistic, individualized, and engaging educational experience tailored to the intended user.
Regarding claim 5, Morgan and Kurani teach the invention in claim 4, as discussed above, and further teach wherein determining the content module comprises: modifying the visual representation of the intended user based on a progress of one or more health conditions of the intended user (Morgan, Col. 27, “The exemplary Clinical Platform Module may further comprise a ML/AI to influence patient behavior feature which uses ML/AI models combined with points of platform data to influence a patient's and/or clinician's behavior and/or influence the patient to carry out desirable actions in XR by creating, deriving, configuring, triggering, modifying, deploying and/or controlling platform content and/or by utilizing other platform features. Points of platform data for a given patient are used as inputs for ML/AI models and/or one or more other platform features. Code and/or configuration instructions may be used to programmatically or otherwise modify, configure, instantiate, and/or control “non-player characters”, virtual human avatars, content and/or features, objects, and/or other features within scenes, sessions and/or regimens. The exemplary feature may use specific measurable and desirable platform actions and/or series of desirable and measurable platform actions over time (“platform behaviors”). For the purpose of this feature, “desirable actions” above also includes mitigating, decreasing, and/or eliminating undesirable actions (for example, decreasing the amount or frequency of cigarette smoking). Inputs or outputs for other ML/AI models, and/or inputs or outputs for one or more iterations of the same ML/AI model(s) may also be utilized within this feature.”
Morgan, Col. 86, “An Alzheimer's and dementia feature of the exemplary Neurological Module may be used for detecting and/or screening for Alzheimer's disease and/or dementia using ML/AI models, voice and/or vocal biomarker analysis features, facial tracking, facial computer vision analyses, pupil and/or eye tracking, positional tracking, the Q&A feature, Movement Module features, Mental Health Module features, Clinical Platform Module features, Neurological Module features, other platform features, and/or using other platform data points. In one exemplary embodiment, this exemplary feature comprises neurocognitive assessments wherein features of the Neurological Module and/or as described in the Clinical Platform Module may be performed (either intentionally and/or passively), and patient inputs and/or actions are logged and stored in a database. In another exemplary embodiment, the present feature comprises movement assessments wherein features of the Neurological Module and/or Movement Module and/or as described in the Clinical Platform Module may be performed (either intentionally and/or passively). Patient inputs and/or actions are logged and stored in a database.”, and
Morgan, Col. 86-87, “Any the logged datasets from assessments are compared to repeat assessments and/or population normal values (customized for the age and gender of the patient) by clinicians and/or ML/AI models. According to one exemplary embodiment, this comparison is implemented to identify if patient has had significant changes and/or remarkable results indicative of Alzheimer's and/or dementia in cognitive domains, movement assessments, and/or other neurological characteristics and/or features as described herein. The changes and/or clinically remarkable results may include (a) change and/or clinically remarkable results in voice features detectable through vocal analysis, NLP, STT, and/or similar ML/AI models; (b) change and/or clinically remarkable results in facial features detectable through computer vision algorithms, and/or other ML/AI models; (c) change and/or clinically remarkable results in performance on one or more cognitive domain tests with or without using ML/AI models; (d) change and/or clinically remarkable results in performance on one or more assessments of ability to carry out ADLs and/or IADLs with or without using ML/AI models; (e) change and/or clinically remarkable results in performance on one or more assessments of movement, and in particular gait and/or ability to ambulate using ML/AI models; (f) change and/or clinically remarkable results in patterns of patient inputs using data obtained from patient input methods during sessions using ML/AI models; (g) clinically remarkable results relating to other platform data points; and (h) clinically remarkable results relating to other platform features.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the visual representation of the intended user based on the progress of one or more health conditions as taught and suggested by Morgan. Morgan teaches utilizing ML/AI models and patient specific platform data to programmatically modify virtual human avatars, platform content, and other features within XR scenes and sessions to influence patient behavior and personalize the patient's experience. Morgan further teaches performing neurological assessments in which patient inputs and actions are logged and stored, with the resulting assessment data being compared to prior assessments to identify significant changes in the patient's health condition, including changes indicative of Alzheimer's disease or dementia. A person of ordinary skill in the art would have recognized that using the detected progression or changes in the patient's health condition obtained from Morgan's repeated assessments to modify the patient's visual representation within the XR environment would have been a predictable implementation of Morgan's disclosed personalization techniques, thereby providing a more accurate, individualized, and clinically relevant representation of the patient's current condition while enhancing the educational and therapeutic effectiveness of the XR platform.
Regarding claim 6, Morgan teaches a method of generating a healthcare educational content module using an extended reality (XR) educational system comprising an Al module, a content builder module, and an elements library, the method comprising (Morgan, Col. 3, ““ML/AI model” (also ML/AI) includes any models, functions, algorithms, code, and/or programming that involve machine learning, artificial intelligence, mathematical, statistical, logic-based processes, and/or control functionalities.”
Morgan, Col. 9, “In one exemplary embodiment, the present XR Health Platform utilizes a combined extended reality head-mounted display and computing device (referred to herein as “HMD and Computing Device”). The exemplary HMD and Computing Device comprises a standalone (meaning wireless and untethered) extended reality system using an operating system such as Google's Android, Apple's iOS, and others. The exemplary HMD and Computing Device may include a virtual reality head-mounted display, an augmented reality head-mounted display, or a mixed reality head-mounted display. The HMD and Computing Device may also include a web browser, high-speed data access via Wi-Fi and mobile broadband, Bluetooth capability, an expandable memory slot for micro SD cards.”
Morgan, Col. 12, “Any of the exemplary platform features may be applied as any portion of applications relating to the health, wellness, and/or health education of individuals. For example, features within the Anatomy Module discussed below may be used for having an individual identify the location of a symptom (as part of a diagnostic), may be used to identify an area in need of strengthening (as part of a therapeutic), may be used by clinicians in documenting changes in symptoms over time (as part of care delivery), as an anatomy simulator, and/or to educate patients regarding anatomy-related subject matter. Additionally, any of the platform features may be combined with any other set of platform features to form applications of the exemplary XR Health Platform. Additionally, any set of items (any item within any module, feature, or sub-feature) may be combined with any other set of items for any implementation(s) of the XR Health Platform.”
Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”, and
Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”):
storing in the elements library of the XR educational system, content elements tagged with subject-matter metadata (Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, configurations can be made via a web portal, a companion application, and/or in XR. Clinicians and/or admins can specify the scenes comprising a session. On the server side, a many-to-many relationship will be established between the patient's ID and the ID of the scenes, as well as ordinality and any other necessary metadata. Clinician and/or admin can specify objects, features, and configurations that are within a scene. Options for a scene may include configurations specific to that scene, objects/features that are compatible with that scene, and scene agnostic objects/features. On the server side, a one-to-one relationship will be established between a patient's scene and a scene configuration object. A one-to-one relationship will be established between a patient's scene and scene objects configuration.”, and
Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, the XR Health Platform uses a customized Scene Manager for switching scenes as well as other scene management activities. The Scene Manager works with Scene Configurations, which are collections of metadata to fully configure any scene type available. Scene Configurations are specific to a single scene, and while they can be duplicated and reused, their purpose is for them to be customized to the specific scenario to maximize the efficacy of the XR session. Scenes can be configured statically, i.e. default scenes. Scenes can also be configured dynamically by the user either via controller input, voice input, question responses, etc. Scenes can also be configured dynamically by the clinician through the companion application. Scenes can also be configured dynamically by the system through AI/ML means to detect sub-optimal experiences and compensate. This uses biometric, positional, and other sensor inputs to determine the configuration changes needed.”);
receiving a request for a content module, the request comprising an identification of a subject of the content module (Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”
Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, configurations can be made via a web portal, a companion application, and/or in XR. Clinicians and/or admins can specify the scenes comprising a session. On the server side, a many-to-many relationship will be established between the patient's ID and the ID of the scenes, as well as ordinality and any other necessary metadata. Clinician and/or admin can specify objects, features, and configurations that are within a scene. Options for a scene may include configurations specific to that scene, objects/features that are compatible with that scene, and scene agnostic objects/features. On the server side, a one-to-one relationship will be established between a patient's scene and a scene configuration object. A one-to-one relationship will be established between a patient's scene and scene objects configuration.”, and
Morgan, Col. 12, “Any of the exemplary platform features may be applied as any portion of applications relating to the health, wellness, and/or health education of individuals. For example, features within the Anatomy Module discussed below may be used for having an individual identify the location of a symptom (as part of a diagnostic), may be used to identify an area in need of strengthening (as part of a therapeutic), may be used by clinicians in documenting changes in symptoms over time (as part of care delivery), as an anatomy simulator, and/or to educate patients regarding anatomy-related subject matter. Additionally, any of the platform features may be combined with any other set of platform features to form applications of the exemplary XR Health Platform. Additionally, any set of items (any item within any module, feature, or sub-feature) may be combined with any other set of items for any implementation(s) of the XR Health Platform.”);
extracting, by the Al module, from the elements library, one or more content elements based on the subject of the content module, wherein the extracting comprises applying, by the Al module; and to the subject-matter metadata of the plurality of content elements (Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”
Morgan, Col. 39, “Using the present feature, a clinician and/or ML/AI models select snippets to deliver to a patient using the content object and snippet feature as described herein. Any selected text snippets are converted to audio files containing spoken language using text-to-speech models (TTS). Audio voice over(s) of text snippets may be configured to be recited out loud to patients in XR. Patient voice response(s) may be entered using patient input methods. STT models are applied to audio to convert the audio into text. Patient audio and/or text transcripts of patient voice may be saved in database(s) and/or application(s) for later analysis. Natural language processing (NLP) models may be applied to either the text and/or original audio with feature extraction and/or analysis results of text and/or audio (vocal biomarkers, semantics, sentiment, clinical meaning, and the like) being saved. The most appropriate snippet(s) are then selected to deliver to the patient next using one or more platform features.”
Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, configurations can be made via a web portal, a companion application, and/or in XR. Clinicians and/or admins can specify the scenes comprising a session. On the server side, a many-to-many relationship will be established between the patient's ID and the ID of the scenes, as well as ordinality and any other necessary metadata. Clinician and/or admin can specify objects, features, and configurations that are within a scene. Options for a scene may include configurations specific to that scene, objects/features that are compatible with that scene, and scene agnostic objects/features. On the server side, a one-to-one relationship will be established between a patient's scene and a scene configuration object. A one-to-one relationship will be established between a patient's scene and scene objects configuration.”, and
Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, the XR Health Platform uses a customized Scene Manager for switching scenes as well as other scene management activities. The Scene Manager works with Scene Configurations, which are collections of metadata to fully configure any scene type available. Scene Configurations are specific to a single scene, and while they can be duplicated and reused, their purpose is for them to be customized to the specific scenario to maximize the efficacy of the XR session. Scenes can be configured statically, i.e. default scenes. Scenes can also be configured dynamically by the user either via controller input, voice input, question responses, etc. Scenes can also be configured dynamically by the clinician through the companion application. Scenes can also be configured dynamically by the system through AI/ML means to detect sub-optimal experiences and compensate. This uses biometric, positional, and other sensor inputs to determine the configuration changes needed.”);
generating by the content builder module, the content module from the one or more extracted content elements, wherein the generating comprises; and the extracted elements into a structured content module formatted for rendering as an immersive three-dimensional XR environment on a head-mounted XR user device (Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”
Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”
Morgan, Col. 12, “Any of the exemplary platform features may be applied as any portion of applications relating to the health, wellness, and/or health education of individuals. For example, features within the Anatomy Module discussed below may be used for having an individual identify the location of a symptom (as part of a diagnostic), may be used to identify an area in need of strengthening (as part of a therapeutic), may be used by clinicians in documenting changes in symptoms over time (as part of care delivery), as an anatomy simulator, and/or to educate patients regarding anatomy-related subject matter. Additionally, any of the platform features may be combined with any other set of platform features to form applications of the exemplary XR Health Platform. Additionally, any set of items (any item within any module, feature, or sub-feature) may be combined with any other set of items for any implementation(s) of the XR Health Platform.”
Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, the XR Health Platform uses a customized Scene Manager for switching scenes as well as other scene management activities. The Scene Manager works with Scene Configurations, which are collections of metadata to fully configure any scene type available. Scene Configurations are specific to a single scene, and while they can be duplicated and reused, their purpose is for them to be customized to the specific scenario to maximize the efficacy of the XR session. Scenes can be configured statically, i.e. default scenes. Scenes can also be configured dynamically by the user either via controller input, voice input, question responses, etc. Scenes can also be configured dynamically by the clinician through the companion application. Scenes can also be configured dynamically by the system through AI/ML means to detect sub-optimal experiences and compensate. This uses biometric, positional, and other sensor inputs to determine the configuration changes needed.”
Morgan, Col. 3, ““ML/AI model” (also ML/AI) includes any models, functions, algorithms, code, and/or programming that involve machine learning, artificial intelligence, mathematical, statistical, logic-based processes, and/or control functionalities.”, and
Morgan, Col. 9, “In one exemplary embodiment, the present XR Health Platform utilizes a combined extended reality head-mounted display and computing device (referred to herein as “HMD and Computing Device”). The exemplary HMD and Computing Device comprises a standalone (meaning wireless and untethered) extended reality system using an operating system such as Google's Android, Apple's iOS, and others. The exemplary HMD and Computing Device may include a virtual reality head-mounted display, an augmented reality head-mounted display, or a mixed reality head-mounted display. The HMD and Computing Device may also include a web browser, high-speed data access via Wi-Fi and mobile broadband, Bluetooth capability, an expandable memory slot for micro SD cards.”); and
storing the content module in a computer-readable storage medium (Morgan, Col. 11, “In other embodiments, the present XR Health Platform may utilize other computer networks; for example, a wide area network (WAN), local area network (LAN), or intranet. The host server may comprise a processor and a computer readable medium, such as random access memory (RAM). The processor is operable to execute certain programs for performing the present XR Health Platform and other computer program instructions stored in memory. Such processor may comprise a microprocessor (or any other processor) and may also include, for example, a display device, internal and external data storage devices, cursor control devices, and/or any combination of these components, or any number of different components, peripherals, input and output devices, and other devices. Such processors may also communicate with other computer-readable media that store computer program instructions, such that when the stored instructions are executed by the processor, the processor performs the acts described further herein. Those skilled in the art will also recognize that the exemplary environments described herein are not intended to limit application of the present XR Health Platform, and that alternative environments may be used without departing from the scope of the invention. Various problem-solving programs incorporated into the present XR Health Platform and discussed further herein, may utilize as inputs, data from a data storage device or location. In one embodiment, the data storage device comprises an electronic database. In other embodiments, the data storage device may comprise an electronic file, disk, or other data storage medium. The data storage device may store features of the disclosure applicable for performing the present XR Health Platform. The data storage device may also include other items useful to carry out the functions of the present XR Health Platform. In one example, the exemplary computer programs may further comprise algorithms designed and configured to perform the present XR Health Platform.”).
Morgan fails to explicitly teach a subject classification algorithm and to identify and rank content elements corresponding to the identified subject; applying one or more machine learning algorithms trained to assemble and sequence the content; and applying one or more machine learning algorithms trained to assemble and sequence the content.
Kurani teaches a subject classification algorithm and to identify and rank content elements corresponding to the identified subject (Kurani [0059] “FIG. 2 depicts an exemplary personalized and adaptive math learning end to end application workflow to offer customized lessons to learners, according to some embodiments. The administrator 204 creates all the course activities and the personalized and adaptive machine learning method rearranges lesson plan to create fit for the student. Then the user 202 can learn from these lessons and quizzes. Every time the learner submits a quiz, the algorithm can change based on their performance on the quiz. The adaptive machine learning method using student's attribute data can keep changing dynamically like this until the learner is to the final quiz of the lesson. It consists of machine learning algorithms and statistical models to perform the task of learning math. It uses unsupervised learning algorithms since it only consists of set of data that contains only student inputs, and find structure in the data, like grouping or clustering of data points. In case, if all the student data is not available it uses the predictive model to best guess the data as part of the input set.”, and
Kurani [0060] “FIG. 3 illustrates personalized and adaptive machine learning process 300 used to create customized lessons for a learner, according to some embodiments. Process 300 includes personalization step 302, which is implemented after the learner attributes are entered. In personalization step 302, the attribute data is analyzed, classified and clustered.”);
applying one or more machine learning algorithms trained to assemble and sequence the content (Kurani [0059] “FIG. 2 depicts an exemplary personalized and adaptive math learning end to end application workflow to offer customized lessons to learners, according to some embodiments. The administrator 204 creates all the course activities and the personalized and adaptive machine learning method rearranges lesson plan to create fit for the student. Then the user 202 can learn from these lessons and quizzes. Every time the learner submits a quiz, the algorithm can change based on their performance on the quiz. The adaptive machine learning method using student's attribute data can keep changing dynamically like this until the learner is to the final quiz of the lesson. It consists of machine learning algorithms and statistical models to perform the task of learning math. It uses unsupervised learning algorithms since it only consists of set of data that contains only student inputs, and find structure in the data, like grouping or clustering of data points. In case, if all the student data is not available it uses the predictive model to best guess the data as part of the input set.”, and
Kurani [0060] “FIG. 3 illustrates personalized and adaptive machine learning process 300 used to create customized lessons for a learner, according to some embodiments. Process 300 includes personalization step 302, which is implemented after the learner attributes are entered. In personalization step 302, the attribute data is analyzed, classified and clustered.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Morgan's XR healthcare education platform with the adaptive machine learning techniques taught by Kurani. Morgan teaches an XR healthcare platform capable of creating, configuring, storing, and dynamically managing educational scenes, scene configurations, and educational content for individual patients using AI/ML models, scene metadata, and patient platform data. Morgan further teaches selecting and presenting educational snippets, configuring scenes using metadata, and dynamically adapting educational content based on patient information and prior interactions. Kurani teaches analyzing, classifying, and clustering learner attribute data using machine learning algorithms to generate customized educational lessons that are rearranged and adapted according to learner characteristics and performance. A person of ordinary skill in the art would have recognized that incorporating Kurani's machine learning techniques into Morgan's XR educational platform would have predictably improve the identification, classification, and organization of educational content elements for individual patients.
Furthermore, it would have been obvious to apply Kurani's machine learning classification and clustering techniques when selecting and assembling Morgan's stored educational content elements into structured XR educational modules. Morgan teaches maintaining educational scenes, scene configurations, metadata, and reusable educational components that can be combined and dynamically configured for individual patients, while Kurani teaches organizing instructional content according to analyzed learner attributes to create personalized lesson plans. Combining these teachings represents the predictable use of known adaptive learning techniques to automatically assemble and sequence healthcare educational content for presentation within Morgan's XR environment, yielding the expected benefit of providing more personalized, efficient, and clinically relevant educational experiences tailored to the needs of individual users.
Regarding claim 7, Morgan and Kurani teach the invention in claim 6, as discussed above, and further teach further comprising: providing the content module to an XR playback device or other user device operated by a user (Morgan, Col. 2, ““XR device” refers to any device that can be used for simulating, viewing, engaging, experiencing, controlling and/or interacting with XR. This includes headsets, head-mounted displays (HMD), augmented reality glasses, 2D displays viewing XR content, 2D displays, 3D displays, computers, controllers, projectors, other interaction devices, mobile phones, speakers, microphones, cameras, headphones, haptic devices, and the like.”
Morgan, Col. 11, “In other embodiments, the present XR Health Platform may utilize other computer networks; for example, a wide area network (WAN), local area network (LAN), or intranet. The host server may comprise a processor and a computer readable medium, such as random access memory (RAM). The processor is operable to execute certain programs for performing the present XR Health Platform and other computer program instructions stored in memory. Such processor may comprise a microprocessor (or any other processor) and may also include, for example, a display device, internal and external data storage devices, cursor control devices, and/or any combination of these components, or any number of different components, peripherals, input and output devices, and other devices. Such processors may also communicate with other computer-readable media that store computer program instructions, such that when the stored instructions are executed by the processor, the processor performs the acts described further herein. Those skilled in the art will also recognize that the exemplary environments described herein are not intended to limit application of the present XR Health Platform, and that alternative environments may be used without departing from the scope of the invention. Various problem-solving programs incorporated into the present XR Health Platform and discussed further herein, may utilize as inputs, data from a data storage device or location. In one embodiment, the data storage device comprises an electronic database. In other embodiments, the data storage device may comprise an electronic file, disk, or other data storage medium. The data storage device may store features of the disclosure applicable for performing the present XR Health Platform. The data storage device may also include other items useful to carry out the functions of the present XR Health Platform. In one example, the exemplary computer programs may further comprise algorithms designed and configured to perform the present XR Health Platform.”, and
Morgan, Col. 31, “A hardware agnostic feature of the exemplary XR Platform Module allows systems within the XR Health Platform to work in a hardware agnostic manner and/or to be distributed at scale (see FIG. 13) and consists of one or more of the following items described below. The XR Health Platform may be paid for, downloaded, and/or updated remotely using XR and/or other web-based interface. Tooltips may be provided for showing patients how to use patient input methods with the tooltips automatically adjusting to point to the correct locations on the virtual representations of one or more real-world input devices. Deep links and/or other methods may be utilized to recognize a user's hardware device(s) and/or facilitate the remote delivery of compatible platform/software package(s). A floor recognition feature may be utilized whereby the location and/or shape of a real-world floor is determined and integrated into XR using ML/AI models and/or sensors and/or one or more other platform features. A spatial understanding feature may be utilized whereby the real-world space and/or objects within the real-world space surrounding an individual are identified, recognized, and/or contextualized using ML/AI models, other platform features, and/or other points of platform data. The exemplary module may also comprise health-related spatial understanding feature whereby health-related objects, people, places, and/or activities within the real-world space surrounding an individual are identified, recognized, and/or contextualized using ML/AI models for the purposes of improving aspects relating to the health of individuals. The exemplary platform features conditional compilation in necessary areas to account for differences in platform specific libraries. These areas may include threading, data persistence and input/output, and Bluetooth and communications library usage”); and
receiving, from the XR playback device or other user device, feedback collected based on user interaction with the content module executing on the XR playback device or other user device (Morgan, Col. 22, “In the next session in the simulation and/or for the patient (and/or in one or more later sessions as configured per items within this feature), specific data fields on each applicable question template are automatically populated using, for example: patient's and/or virtual object's proximity and/or position relative to one or more 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and virtual human avatars; data captured through simulated interactions between a platform feature and 2D and/or 3D virtual objects and/or other platform features; data captured through the use of scene and/or session metadata; data captured through the use of biometric data; data captured through the use of any other platform data; and data captured through the use of data from other interactions and/or simulations.”
Morgan, Col. 97, “Other exemplary scenes and/or features may include XR scene(s) and/or features within XR scenes where past accomplishment(s), achieved goal(s), and/or positive progress are highlighted. The past accomplishment(s), achieved goal(s), and/or positive progress may be highlighted using sentences/statements, snippets, content objects, audio items, video items, 2D or 3D effects, animations, and/or replays of previously experienced scenes or sessions showing an avatar representing the patient to him/herself in 3rd person. Other exemplary scenes and/or features may include application of Socratic questioning/dialogue to identify negative thought patterns and/or cognitive distortions, and/or to attenuate negative or counterproductive thoughts or feelings, and/or to enhance positive or productive thoughts or feelings as described above.”
Morgan, Col. 40, “In yet another exemplary embodiment, the present feature includes a sub-feature that allows for the personalized, timely, and actionable delivery of feedback through points of other platform data. Exemplary points of data include biometric data, data from neuro-behavioral and/or cognitive assessments, data from compliance monitoring systems, imaging studies, data produced by one or more computer vision algorithms, vocal biomarker data, and/or data from diagnostic studies. For clarity, this feature also may utilize data points produced by the use of scenes, sessions, and/or regimens.”
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”, and
Morgan, Col. 55, “A real-time movement biofeedback feature of the exemplary Movement Module may be used for real-time biofeedback where the scoring in one or more gamified XR experiences is at least partially determined by establishing and/or maintaining physical movements and/or positions. The movements and/or positions are determined as described herein, and whereby visual and/or auditory stimuli provide real-time feedback in terms of the correctness or incorrectness of the movements and/or physical positions. This correctness or incorrectness results in a higher or lower score, respectively. In one embodiment of this feature, scores are proportional to the angle and height of the controllers relative to the HMD as well as the length of time that this position is maintained. For example, if the patient holds his arms straight out in front of him at eye level, the score goes up proportionally to the time that this position is maintained.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to provide the content module to an XR playback device or other user device and receive feedback collected from user interaction with the content module as taught by Morgan. Morgan teaches an XR Health Platform configured to remotely deliver, download, and update compatible software packages to a variety of XR devices and user devices while recognizing device compatibility. Morgan further teaches collecting user interaction data, biometric data, movement data, and other platform data generated during XR sessions to provide real time biofeedback, evaluate user performance, and personalize subsequent interactions. A person of ordinary skill in the art would have recognized that receiving such feedback from users interacting with the delivered XR content would have been a predictable use of Morgan's disclosed XR platform to monitor user progress, improve subsequent educational content, and enhance the effectiveness of personalized XR educational experiences.
Regarding claim 8, Morgan and Kurani teach the invention in claim 7, as discussed above, and further teach wherein the feedback comprises at least one of biometric feedback collected by the playback device or other user device, user input during the user interaction with the content module executing on the XR playback device or other user device, and feedback based on the user interaction with the content module executing on the XR playback device or other user device (Morgan, Col. 40, “In yet another exemplary embodiment, the present feature includes a sub-feature that allows for the personalized, timely, and actionable delivery of feedback through points of other platform data. Exemplary points of data include biometric data, data from neuro-behavioral and/or cognitive assessments, data from compliance monitoring systems, imaging studies, data produced by one or more computer vision algorithms, vocal biomarker data, and/or data from diagnostic studies. For clarity, this feature also may utilize data points produced by the use of scenes, sessions, and/or regimens.”
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”
Morgan, Col. 22, “In the next session in the simulation and/or for the patient (and/or in one or more later sessions as configured per items within this feature), specific data fields on each applicable question template are automatically populated using, for example: patient's and/or virtual object's proximity and/or position relative to one or more 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and virtual human avatars; data captured through simulated interactions between a platform feature and 2D and/or 3D virtual objects and/or other platform features; data captured through the use of scene and/or session metadata; data captured through the use of biometric data; data captured through the use of any other platform data; and data captured through the use of data from other interactions and/or simulations.”
Morgan, Col. 39, “Using the present feature, a clinician and/or ML/AI models select snippets to deliver to a patient using the content object and snippet feature as described herein. Any selected text snippets are converted to audio files containing spoken language using text-to-speech models (TTS). Audio voice over(s) of text snippets may be configured to be recited out loud to patients in XR. Patient voice response(s) may be entered using patient input methods. STT models are applied to audio to convert the audio into text. Patient audio and/or text transcripts of patient voice may be saved in database(s) and/or application(s) for later analysis. Natural language processing (NLP) models may be applied to either the text and/or original audio with feature extraction and/or analysis results of text and/or audio (vocal biomarkers, semantics, sentiment, clinical meaning, and the like) being saved. The most appropriate snippet(s) are then selected to deliver to the patient next using one or more platform features.”, and
Morgan, Col. 55, “A real-time movement biofeedback feature of the exemplary Movement Module may be used for real-time biofeedback where the scoring in one or more gamified XR experiences is at least partially determined by establishing and/or maintaining physical movements and/or positions. The movements and/or positions are determined as described herein, and whereby visual and/or auditory stimuli provide real-time feedback in terms of the correctness or incorrectness of the movements and/or physical positions. This correctness or incorrectness results in a higher or lower score, respectively. In one embodiment of this feature, scores are proportional to the angle and height of the controllers relative to the HMD as well as the length of time that this position is maintained. For example, if the patient holds his arms straight out in front of him at eye level, the score goes up proportionally to the time that this position is maintained.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention for the feedback received from the XR playback device or other user device to comprise biometric feedback, user input, and feedback based on user interaction as taught by Morgan. Morgan teaches collecting biometric data, including vocal biomarkers, computer vision data, neurobehavioral assessments, and other diagnostic information generated during XR sessions, receiving user inputs such as patient voice responses that are processed using speech-to-text and natural language processing models, and collecting interaction data generated through patient interactions with virtual objects, virtual human avatars, and other XR platform features. Morgan further teaches generating real-time movement biofeedback based on a user's interactions within the XR environment to evaluate user performance. A person of ordinary skill in the art would have recognized that incorporating these known forms of user feedback into Morgan's XR educational platform would have been a predictable use of the disclosed interaction and biometric data to monitor user performance, personalize subsequent educational content, and improve the effectiveness of the XR educational experience.
Regarding claim 9, Morgan and Kurani teach the invention in claim 8, as discussed above, and further teach wherein the content module is a healthcare content module and the intended user is a patient (Morgan, Col. 12, “Any of the exemplary platform features may be applied as any portion of applications relating to the health, wellness, and/or health education of individuals. For example, features within the Anatomy Module discussed below may be used for having an individual identify the location of a symptom (as part of a diagnostic), may be used to identify an area in need of strengthening (as part of a therapeutic), may be used by clinicians in documenting changes in symptoms over time (as part of care delivery), as an anatomy simulator, and/or to educate patients regarding anatomy-related subject matter. Additionally, any of the platform features may be combined with any other set of platform features to form applications of the exemplary XR Health Platform. Additionally, any set of items (any item within any module, feature, or sub-feature) may be combined with any other set of items for any implementation(s) of the XR Health Platform.”
Morgan, Col. 15, “In exemplary embodiments, the present Clinical Platform Module further includes a patient-level profile feature which maintains an up-to-date record of goals, outcome-focused patient characteristics, and/or other points of relevant platform data between scenes, sessions, and/or regimens to enable personalized, dynamic, and/or adaptive applications of platform features for individual patients.”, and
Morgan, Col. 86, “An Alzheimer's and dementia feature of the exemplary Neurological Module may be used for detecting and/or screening for Alzheimer's disease and/or dementia using ML/AI models, voice and/or vocal biomarker analysis features, facial tracking, facial computer vision analyses, pupil and/or eye tracking, positional tracking, the Q&A feature, Movement Module features, Mental Health Module features, Clinical Platform Module features, Neurological Module features, other platform features, and/or using other platform data points. In one exemplary embodiment, this exemplary feature comprises neurocognitive assessments wherein features of the Neurological Module and/or as described in the Clinical Platform Module may be performed (either intentionally and/or passively), and patient inputs and/or actions are logged and stored in a database. In another exemplary embodiment, the present feature comprises movement assessments wherein features of the Neurological Module and/or Movement Module and/or as described in the Clinical Platform Module may be performed (either intentionally and/or passively). Patient inputs and/or actions are logged and stored in a database.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention for the content module to be a healthcare content module and for the intended user to be a patient as taught by Morgan. Morgan teaches that the XR Health Platform provides healthcare educational applications that educate patients regarding anatomy and other health subject matter, while maintaining patient-level profiles and patient platform data to provide personalized and adaptive applications for individual patients. Morgan further teaches healthcare-specific features directed to patients, including neurological and movement assessments in which patient inputs and actions are collected and analyzed to detect or monitor health conditions. A person of ordinary skill in the art would have recognized that implementing Morgan's educational content as healthcare content for presentation to patients would have been a predictable use of the disclosed XR healthcare platform, yielding the expected benefit of providing individualized health education and clinically relevant information tailored to the patient's medical needs.
Regarding claim 10, Morgan and Kurani teach the invention in claim 7, as discussed above, and further teach further comprising: modifying, by way of the Al module, the content module based on the feedback collected based on user interaction with the content module executing on the XR playback device or other user device (Morgan, Col. 27, “The exemplary Clinical Platform Module may further comprise a ML/AI to influence patient behavior feature which uses ML/AI models combined with points of platform data to influence a patient's and/or clinician's behavior and/or influence the patient to carry out desirable actions in XR by creating, deriving, configuring, triggering, modifying, deploying and/or controlling platform content and/or by utilizing other platform features. Points of platform data for a given patient are used as inputs for ML/AI models and/or one or more other platform features. Code and/or configuration instructions may be used to programmatically or otherwise modify, configure, instantiate, and/or control “non-player characters”, virtual human avatars, content and/or features, objects, and/or other features within scenes, sessions and/or regimens. The exemplary feature may use specific measurable and desirable platform actions and/or series of desirable and measurable platform actions over time (“platform behaviors”). For the purpose of this feature, “desirable actions” above also includes mitigating, decreasing, and/or eliminating undesirable actions (for example, decreasing the amount or frequency of cigarette smoking). Inputs or outputs for other ML/AI models, and/or inputs or outputs for one or more iterations of the same ML/AI model(s) may also be utilized within this feature.”
Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”
Morgan, Col. 39, “Using the present feature, a clinician and/or ML/AI models select snippets to deliver to a patient using the content object and snippet feature as described herein. Any selected text snippets are converted to audio files containing spoken language using text-to-speech models (TTS). Audio voice over(s) of text snippets may be configured to be recited out loud to patients in XR. Patient voice response(s) may be entered using patient input methods. STT models are applied to audio to convert the audio into text. Patient audio and/or text transcripts of patient voice may be saved in database(s) and/or application(s) for later analysis. Natural language processing (NLP) models may be applied to either the text and/or original audio with feature extraction and/or analysis results of text and/or audio (vocal biomarkers, semantics, sentiment, clinical meaning, and the like) being saved. The most appropriate snippet(s) are then selected to deliver to the patient next using one or more platform features.”, and
Morgan, Col. 22, “In the next session in the simulation and/or for the patient (and/or in one or more later sessions as configured per items within this feature), specific data fields on each applicable question template are automatically populated using, for example: patient's and/or virtual object's proximity and/or position relative to one or more 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and 2D and/or 3D virtual objects and/or features; data captured through interactions between a patient and virtual human avatars; data captured through simulated interactions between a platform feature and 2D and/or 3D virtual objects and/or other platform features; data captured through the use of scene and/or session metadata; data captured through the use of biometric data; data captured through the use of any other platform data; and data captured through the use of data from other interactions and/or simulations.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the content module using an AI module based on feedback collected from a user's interaction with the content module as taught by Morgan. Morgan teaches utilizing ML/AI models to modify platform content, virtual human avatars, scenes, and other XR platform features based on patient-specific platform data and user behaviors. Morgan further teaches collecting user interaction data, biometric data, and other platform data during XR sessions, processing patient responses using speech-to-text, natural language processing, and other ML/AI models to determine the most appropriate educational content to deliver next, and iteratively adapting subsequent XR sessions and scenes based on previous patient results. A person of ordinary skill in the art would have recognized that using the collected user interaction feedback to modify the content module through Morgan's disclosed AI-driven personalization techniques would have been a predictable implementation of the disclosed XR healthcare platform, yielding the expected benefit of providing increasingly personalized, responsive, and effective educational content tailored to the user's performance and interactions.
Regarding claim 11, Morgan and Kurani teach the invention in claim 10, as discussed above, and further teach wherein the modifying of the content module based on the feedback comprises modifying the content module in real-time while the content module executes on the XR playback device or other user device (Morgan, Col. 27, “The exemplary Clinical Platform Module may further comprise a ML/AI to influence patient behavior feature which uses ML/AI models combined with points of platform data to influence a patient's and/or clinician's behavior and/or influence the patient to carry out desirable actions in XR by creating, deriving, configuring, triggering, modifying, deploying and/or controlling platform content and/or by utilizing other platform features. Points of platform data for a given patient are used as inputs for ML/AI models and/or one or more other platform features. Code and/or configuration instructions may be used to programmatically or otherwise modify, configure, instantiate, and/or control “non-player characters”, virtual human avatars, content and/or features, objects, and/or other features within scenes, sessions and/or regimens. The exemplary feature may use specific measurable and desirable platform actions and/or series of desirable and measurable platform actions over time (“platform behaviors”). For the purpose of this feature, “desirable actions” above also includes mitigating, decreasing, and/or eliminating undesirable actions (for example, decreasing the amount or frequency of cigarette smoking). Inputs or outputs for other ML/AI models, and/or inputs or outputs for one or more iterations of the same ML/AI model(s) may also be utilized within this feature.”
Morgan, Col. 55, “A real-time movement biofeedback feature of the exemplary Movement Module may be used for real-time biofeedback where the scoring in one or more gamified XR experiences is at least partially determined by establishing and/or maintaining physical movements and/or positions. The movements and/or positions are determined as described herein, and whereby visual and/or auditory stimuli provide real-time feedback in terms of the correctness or incorrectness of the movements and/or physical positions. This correctness or incorrectness results in a higher or lower score, respectively. In one embodiment of this feature, scores are proportional to the angle and height of the controllers relative to the HMD as well as the length of time that this position is maintained. For example, if the patient holds his arms straight out in front of him at eye level, the score goes up proportionally to the time that this position is maintained.”, and
Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the content module in real time while the content module executes on the XR playback device or other user device as taught and suggested by Morgan. Morgan teaches utilizing ML/AI models to programmatically modify platform content, virtual human avatars, scenes, and other XR platform features based on patient-specific platform data and user behaviors. Morgan further teaches providing real-time movement biofeedback during execution of XR experiences, where user interactions are continuously evaluated and visual and auditory feedback are immediately generated based on the user's performance, and executing XR content at runtime by loading animations, virtual objects, and instructional content while tracking patient interactions throughout the XR session. A person of ordinary skill in the art would have recognized that applying Morgan's AI-driven content modification techniques during runtime based on the real time feedback collected from the user's interactions would have been a predictable implementation of the disclosed XR platform, yielding the expected benefit of providing an immediately adaptive, personalized, and more effective educational experience.
Regarding claim 12, Morgan and Kurani teach the invention in claim 10, as discussed above, and further teach wherein the modifying of the content module is performed by one or more machine learning algorithms of the Al module, wherein the Al module is trained to modify the content module based on the feedback (Morgan, Col. 27, “The exemplary Clinical Platform Module may further comprise a ML/AI to influence patient behavior feature which uses ML/AI models combined with points of platform data to influence a patient's and/or clinician's behavior and/or influence the patient to carry out desirable actions in XR by creating, deriving, configuring, triggering, modifying, deploying and/or controlling platform content and/or by utilizing other platform features. Points of platform data for a given patient are used as inputs for ML/AI models and/or one or more other platform features. Code and/or configuration instructions may be used to programmatically or otherwise modify, configure, instantiate, and/or control “non-player characters”, virtual human avatars, content and/or features, objects, and/or other features within scenes, sessions and/or regimens. The exemplary feature may use specific measurable and desirable platform actions and/or series of desirable and measurable platform actions over time (“platform behaviors”). For the purpose of this feature, “desirable actions” above also includes mitigating, decreasing, and/or eliminating undesirable actions (for example, decreasing the amount or frequency of cigarette smoking). Inputs or outputs for other ML/AI models, and/or inputs or outputs for one or more iterations of the same ML/AI model(s) may also be utilized within this feature.”
Morgan, Col. 3, ““ML/AI model” (also ML/AI) includes any models, functions, algorithms, code, and/or programming that involve machine learning, artificial intelligence, mathematical, statistical, logic-based processes, and/or control functionalities.”
Morgan, Col. 37, “This overall approach of utilizing a session comprised of personalized and interactive scenes, delivering personalized feedback and/or educational information, and then iteratively adapting subsequent sessions and/or scenes based on previous results can be appreciated when reviewing FIGS. 19-27.”, and
Morgan, Col. 39, “Using the present feature, a clinician and/or ML/AI models select snippets to deliver to a patient using the content object and snippet feature as described herein. Any selected text snippets are converted to audio files containing spoken language using text-to-speech models (TTS). Audio voice over(s) of text snippets may be configured to be recited out loud to patients in XR. Patient voice response(s) may be entered using patient input methods. STT models are applied to audio to convert the audio into text. Patient audio and/or text transcripts of patient voice may be saved in database(s) and/or application(s) for later analysis. Natural language processing (NLP) models may be applied to either the text and/or original audio with feature extraction and/or analysis results of text and/or audio (vocal biomarkers, semantics, sentiment, clinical meaning, and the like) being saved. The most appropriate snippet(s) are then selected to deliver to the patient next using one or more platform features.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention for the modification of the content module to be performed by one or more machine learning algorithms of an AI module trained to modify the content module based on user feedback as taught and suggested by Morgan. Morgan teaches utilizing ML/AI models to modify platform content, virtual human avatars, scenes, and other XR platform features using patient-specific platform data and user behaviors as inputs, while further employing machine learning models, including speech-to-text, natural language processing, and other ML/AI models, to analyze patient responses and determine the most appropriate educational content to deliver next. Morgan additionally teaches iteratively adapting subsequent XR sessions based on previous patient results and defines ML/AI models as including machine learning algorithms and related computational processes for controlling platform functionality. A person of ordinary skill in the art would have recognized that employing trained machine learning algorithms to modify educational content based on collected user feedback would have been a predictable implementation of Morgan's disclosed AI driven personalization framework, yielding the expected benefit of continuously adapting the educational experience to improve personalization, user engagement, and instructional effectiveness.
Regarding claim 13, Morgan and Kurani teach the invention in claim 6, as discussed above, and further teach wherein the content module is generated by one or more machine learning algorithms trained to generate the content module based on one or more inputs comprising the subject of the content module (Morgan, Col. 3, ““ML/AI model” (also ML/AI) includes any models, functions, algorithms, code, and/or programming that involve machine learning, artificial intelligence, mathematical, statistical, logic-based processes, and/or control functionalities.”,
Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, configurations can be made via a web portal, a companion application, and/or in XR. Clinicians and/or admins can specify the scenes comprising a session. On the server side, a many-to-many relationship will be established between the patient's ID and the ID of the scenes, as well as ordinality and any other necessary metadata. Clinician and/or admin can specify objects, features, and configurations that are within a scene. Options for a scene may include configurations specific to that scene, objects/features that are compatible with that scene, and scene agnostic objects/features. On the server side, a one-to-one relationship will be established between a patient's scene and a scene configuration object. A one-to-one relationship will be established between a patient's scene and scene objects configuration.”
Morgan, Col. 44, “In an exemplary embodiment of the scene creation and configuration feature, the XR Health Platform uses a customized Scene Manager for switching scenes as well as other scene management activities. The Scene Manager works with Scene Configurations, which are collections of metadata to fully configure any scene type available. Scene Configurations are specific to a single scene, and while they can be duplicated and reused, their purpose is for them to be customized to the specific scenario to maximize the efficacy of the XR session. Scenes can be configured statically, i.e. default scenes. Scenes can also be configured dynamically by the user either via controller input, voice input, question responses, etc. Scenes can also be configured dynamically by the clinician through the companion application. Scenes can also be configured dynamically by the system through AI/ML means to detect sub-optimal experiences and compensate. This uses biometric, positional, and other sensor inputs to determine the configuration changes needed.”, and
Morgan, Col. 56-57, “A movement integration feature of the exemplary Movement Module may utilize and/or integrate features within the Movement Module in diagnostic, therapeutic, and/or other clinically related uses. This includes capabilities for creating, deriving, configuring, teaching, modifying, deploying and/or controlling physical movements and/or physical activities for diagnostic and/or therapeutic purposes. Using a graphical user interface, clinicians and/or ML/AI models may configure diagnostic and/or therapeutic tasks to be completed by a patient. Based on the clinically-related tasks selected, pre-configured sets of animations and instructional audio clips appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations, along with their corresponding text-to-speech instructional audio clips. Based on diagnostic and/or therapeutic tasks selected, appropriate virtual object data may be automatically imported Animations pre-configured as appropriate for the tasks may be automatically loaded from a stored library of previously captured and/or computer-generated 2D and/or 3D animations. At runtime, prior to each movement carried out by the patient, the corresponding pre-configured animation with any clinician modifications may be loaded onto an instructional virtual human avatar and played to demonstrate to the patient how to correctly complete the movement. Each instructional avatar movement may be preceded by instructional and/or educational audio clips which are played in XR and appear to the patient to be recited by the instructional avatar. At runtime, imported virtual object data may be used to instantiate virtual objects as a function of time based upon the preconfigured set of animations determined to be appropriate for each diagnostic and/or therapeutic task. At runtime, each virtual object may be instantiated as a function of time at a position and/or rotation relative to the patient using the system, such that a time-dependent series of one or more virtual waypoints appears in the appropriate proximity to the patient. At runtime, three-dimensional positional tracking of the patient may be used to guide the patient through movements and/or to traverse virtual waypoints with either a particular anatomical structure and/or XR hardware. The tracking, biometric, and/or other platform data may also be used to evaluate, measure and/or detect a range of motion of a particular anatomical structure and/or set of anatomical structures and/or other features, and a maximum distance that a particular anatomical structure has traversed relative to its starting position. The maximum distance may be determined during a series of movements or during an XR session.”, and
Kurani [0059] “FIG. 2 depicts an exemplary personalized and adaptive math learning end to end application workflow to offer customized lessons to learners, according to some embodiments. The administrator 204 creates all the course activities and the personalized and adaptive machine learning method rearranges lesson plan to create fit for the student. Then the user 202 can learn from these lessons and quizzes. Every time the learner submits a quiz, the algorithm can change based on their performance on the quiz. The adaptive machine learning method using student's attribute data can keep changing dynamically like this until the learner is to the final quiz of the lesson. It consists of machine learning algorithms and statistical models to perform the task of learning math. It uses unsupervised learning algorithms since it only consists of set of data that contains only student inputs, and find structure in the data, like grouping or clustering of data points. In case, if all the student data is not available it uses the predictive model to best guess the data as part of the input set.”
Kurani [0060] “FIG. 3 illustrates personalized and adaptive machine learning process 300 used to create customized lessons for a learner, according to some embodiments. Process 300 includes personalization step 302, which is implemented after the learner attributes are entered. In personalization step 302, the attribute data is analyzed, classified and clustered.”).
It would have been obvious to one of ordinary skill in the art at the time of the invention to generate the content module using one or more machine learning algorithms trained to generate the content module based on one or more inputs comprising the subject of the content module by combining the teachings of Morgan and Kurani. Morgan teaches an XR healthcare platform that utilizes ML/AI models to create, configure, manage, and present educational scenes and instructional content for healthcare applications, while Kurani teaches adaptive machine learning algorithms that analyze input information and learner attributes to automatically classify, organize, and generate customized educational lessons based on the subject matter and learner needs. A person of ordinary skill in the art would have recognized that incorporating Kurani's adaptive machine learning techniques into Morgan's XR healthcare platform would have been a predictable use of known machine learning algorithms to automatically generate structured healthcare content modules based on input subject information, thereby improving the personalization, organization, and educational relevance of the XR content for the intended user.
Claim 14 is analogous to claim 1, thus claim 14 is similarly analyzed and rejected in a manner consistent with the rejection of claim 1.
Claim 15 is analogous to claim 7, thus claim 15 is similarly analyzed and rejected in a manner consistent with the rejection of claim 7.
Claim 17 is analogous to claim 12, thus claim 17 is similarly analyzed and rejected in a manner consistent with the rejection of claim 12.
Response to Arguments
Applicant’s arguments and amendments, see Remarks/Amendments submitted on 04/22/2026 with respect to the rejection of the claims have been carefully considered and is addressed below.
Claim Rejections - 35 USC § 101
Applicant's arguments have been fully considered but are not persuasive. Although Applicant states that the amended claims cannot be performed mentally because they recite an AI module, machine learning algorithms, an immersive XR environment, head-mounted XR devices, and biometric sensors, the claims remain directed to the abstract idea identified in the rejection. Specifically, claim 1 is directed to evaluating patient information, selecting and presenting personalized educational content, adjusting the presentation based on user context, and monitoring user progress, and claim 6 is directed to reviewing available educational content, identifying and ranking content relevant to a requested subject, and assembling and sequencing the selected content into a structured educational module. These concepts are observations, evaluations, and judgments that, under their broadest reasonable interpretation, can be performed mentally or with the aid of pen and paper. The recited AI module, subject classification algorithm, machine learning algorithms, and XR user device automate or implement these otherwise abstract processes using generic computer technology and do not alter the claims as being directed to mental processes and certain methods of organizing human activity.
Applicant's reliance on Enfish, McRO, and USPTO Eligibility Example 47 is also unpersuasive. Unlike Enfish, the present claims do not improve the functioning of a computer, database, AI system, XR rendering engine, or any other computer technology. Likewise, unlike McRO, the claims do not recite specific rules or a particular technological technique that improves computer animation or XR rendering; instead, they recite AI and machine learning algorithms only at a high level of generality to perform the abstract processes of selecting, organizing, generating, and presenting educational content. Furthermore, USPTO Eligibility Example 47 is distinguishable because the claimed artificial neural network there was directed to solving a technological problem in computer network security through a specific machine learning implementation. Here, the recited AI and machine learning are used as tools to organize, generate, and deliver educational content and do not provide any improvement to the operation of the AI itself, the XR hardware, or other computer technology.
Applicant further states that the claims integrate any alleged abstract idea into a practical application through the use of XR hardware, sensors, biometric feedback, and a closed-loop feedback system. However, these additional elements apply the abstract idea in the technological environment of an XR educational system. The claimed processing device, memory device, AI module, records module, profile module, content builder module, XR user device, and machine learning algorithms perform their ordinary and conventional functions of receiving, processing, storing, analyzing, and presenting data. The claims do not recite any technological improvement to these components or to XR technology itself, but instead use them as generic tools to implement the abstract idea. Accordingly, the additional elements, individually and in combination, do not integrate the judicial exception into a practical application or provide an inventive concept sufficient to amount to significantly more than the abstract idea. Therefore, the rejection under 35 U.S.C. 101 is maintained.
Claim Rejections - 35 USC § 102
Applicant’s amendments overcome the rejection under 35 U.S.C. 102 because the cited reference (Morgan et al.) no longer teaches or suggests all of the claimed limitations. Accordingly, the rejection is withdrawn. However, prior art rejections have been provided under 35 U.S.C. 103 necessitated by the amendments to the claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure.
Chidambaran et al. (U.S. Patent Publication 2012/0251993) teaches a method and system for promoting health education using an integrated platform that delivers personalized educational content, customized reminders, and incentive based rewards to support patient engagement.
Westhoff et al. (U.S. Patent Publication 2023/0052960 A1) teaches a system and method that enables users to create, modify, and operate customizable extended reality educational experiences, such as patient cases, through interactive graphical rapid case creation tool integrating case data and logic modules.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYRA R LAGOY whose telephone number is (703)756-1773. The examiner can normally be reached Monday - Friday, 8:00 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kambiz Abdi can be reached at (571)272-6702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.L./Examiner, Art Unit 3685
/KAMBIZ ABDI/Supervisory Patent Examiner, Art Unit 3685
Prosecution Insights