DETAILED CORRESPONDANCE
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims
This final office action on merits is in response to the communication received on 03/02/2026. Claims 1-20, 22-23, 25-27, 34, 36, 38-39 are cancelled. Amendments to claims 21, 24, 28-29, 33, 37, 40-51 are acknowledged and have been carefully considered. Claims 21, 24, 28-33, 35, 37, 40-51 are pending and considered below.
Information Disclosure Statement
The information disclosure statements (IDSs) filed on 01/12/2026 and 04/03/2026 have been acknowledged. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “data analytics process modules” , “a first data analytics process module”, “a second data analytics process module”, “the first data analytics process module”, and “the second data analytics process module” in claim 21; and “the first data analytics process module and/or the second data analytics process module” in claims 45 and 48.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 44 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 44 recites that “wherein a second-level classifier is trained to receive diagnostic outputs as inputs used to generate the unified final diagnosis” The disclosure does not teach or suggests a trained secondary classifier combining module outputs. No training procedure, data set, architecture, or inference process is disclosed. The term “classifier” or any equivalent concept does not appear in the specification.
Because the originally filed specification contains no mention of these features, either individually or in combination, one of ordinary skill in the art would not have reasonably concluded that the inventor(s) had possession of these elements at the time of filing. These recited elements were not present in the original claims and are not supported by the originally filed specification. The absence of any written description for these features and their associated functions means the application fails to meet the written description requirement of 35 U.S.C. 112(a).
Additionally, the rejection under 35 U.S.C. § 112(a) is withdrawn with respect to claims 21, 24, 28-33, 35, 37, 40-43, and 45-51. With respect to claims 21, 24, 28-33, 35, 37, 40-43, and 45-51, as amended, the cited portions of the specification reasonably convey possession of the amended subject matter, including the voting and agreement criteria, weighted diagnostic processing, condition-specific integrators, triggered follow-up actions, and machine learning models trained on different databases.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
In view of the amendments clarifying the limitations, the rejections under 35 U.S.C. § 112(b) of claims 21, 24, 28-33, 35, 37, and 40-51 are withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 21, 24, 28-33, 35, 37, 40-51 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
Under step 1, the analysis is based on MPEP 2106.03, and claims 21, 24, 28-33, 35, 37, 40-51 are drawn to a medical device. Thus, each claim, on its face, is directed to one of the statutory categories (i.e., useful process, machine, manufacture, or composition of matter) of 35 U.S.C. §101.
Step 2A Prong One
Claim 21 recites the limitations of determining a first diagnostic output by inputting at least one the first data, the second data, and/or the third data; determining a second diagnostic output by inputting at least one the first data, the second data, and/or the third data; and determine a unified final diagnosis by integrating the first diagnostic output and the second diagnostic output. These limitations, as drafted, are processes that, under their broadest reasonable interpretations, cover performance of the limitations in the mind or by using a pen and paper. Even when considering the “into the first data or second data analytics process module” language, the claim encompasses a user evaluating the collected data, applying different diagnostic analyses to the data, comparing the resulting diagnostic conclusions, and integrating the conclusions to reach a final diagnosis in their mind or by using a pen and paper. The nominal recitation of into the first data or second data analytics process module does not take the claim limitations out of the mental processes grouping. Thus, the claim recites a mental process which is an abstract idea.
Under Step 2A Prong Two
The claimed limitations, as per method claim 21, include:
a headset configured to be worn by a subject, the headset comprises one or more internal camera facing toward the subject's face and eyes, one or more external camera facing toward the subject's body, one or more speakers, one or more microphone oriented at the subject's mouth, at least one sensory stimulator configured to deliver vibratory and at least one of thermal or tactile stimulation to at least one unique location on the subject's body, and at least semi-transparent augmented reality visor, and
a medical diagnostic system coupled to the headset, the medical diagnostic system comprises:
a non-transitory memory storing instructions and a plurality of data analytics process modules comprising at least a first data analytics process module and a second data analytics process module, wherein the first data analytics process module and the second data analytics process module are different; and
a processor configured to access the memory and execute the instructions stored in the memory to:
cause the one or more internal camera facing toward the subject's face and eyes to capture images and/or video of the subject's face and store the images and/or video of the subject's face as first data;
cause the one or more external camera facing toward the subject's body to capture images and/or video of the subject's body and store the images and/or video of the subject's body as second data;
cause the one or more microphone oriented at the subject's mouth to capture an audio signal from the subject and store the audio signal as third data;
determine a first diagnostic output by inputting at least one the first data, the second data, and/or the third data into the first data analytics process module;
determine a second diagnostic output by inputting at least one the first data, the second data, and/or the third data into the second data analytics process module;
determine a unified final diagnosis by integrating the first diagnostic output and the second diagnostic output; and
communicate the unified final diagnosis to the subject via the speaker and/or a display screen on at least a portion of the at least semi-transparent augmented reality visor,
wherein at least one sensory stimulation is delivered to the at least one unique location on the subject's body via the at least one sensory stimulator based on the unified final diagnosis.
Examiner Note: underlined elements indicate additional elements of the claimed invention identified as performing the steps of the claimed invention.
The judicial exception expressed in claim 21 is not integrated into a practical application. The claim as a whole merely describes how to generally “apply” the concept of evaluating patient data to generate diagnostic conclusions in a computer environment. The claimed computer components (i.e., a medical diagnostic system coupled to the headset, the medical diagnostic system comprises: a non-transitory memory storing instructions and a plurality of data analytics process modules comprising at least a first data analytics process module and a second data analytics process module, wherein the first data analytics process module and the second data analytics process module are different; and a processor configured to access the memory and execute the instructions stored in the memory to) are recited at a high level of generality and are merely invoked as tools to perform an existing process of observing patient information, analyzing the information, and determining a diagnostic. Simply implementing the abstract idea on a generic computer is not a practical application of the abstract idea. Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application.
The judicial exception expressed in claim 21 is not integrated into a practical application. The claim recites the additional elements of 1) a headset configured to be worn by a subject, the headset comprises one or more internal camera facing toward the subject's face and eyes, one or more external camera facing toward the subject's body, one or more speakers, one or more microphone oriented at the subject's mouth, at least one sensory stimulator configured to deliver vibratory and at least one of thermal or tactile stimulation to at least one unique location on the subject's body, and at least semi-transparent augmented reality visor; 2) cause the one or more internal camera facing toward the subject's face and eyes to capture images and/or video of the subject's face and store the images and/or video of the subject's face as first data; cause the one or more external camera facing toward the subject's body to capture images and/or video of the subject's body and store the images and/or video of the subject's body as second data; cause the one or more microphone oriented at the subject's mouth to capture an audio signal from the subject and store the audio signal as third data; 3) communicate the unified final diagnosis to the subject via the speaker and/or a display screen on at least a portion of the at least semi-transparent augmented reality visor; and 4) wherein at least one sensory stimulation is delivered to the at least one unique location on the subject's body via the at least one sensory stimulator based on the unified final diagnosis. These limitations are recited at a high level of generality (i.e., as a general means of interfacing with a user, collecting information, communicating diagnostic information, and providing sensory feedback), and amounts to merely providing a generic environment for the abstract idea, data gathering, outputting or displaying the results, and insignificant application of the abstract idea which are forms of insignificant extra-solution activities. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application. The claim is directed to an abstract idea.
Therefore, under step 2A, the claims are directed to the abstract idea, and require further analysis under Step 2B.
Under step 2B
Claim 21 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed with respect to Step 2A, the claim as a whole merely describes how to generally “apply” the concept of evaluating patient data to generate diagnostic conclusions in a computer environment. Thus, even when viewed as a whole, nothing in the claim adds significantly more (i.e., an inventive concept) to the abstract idea.
For claim 21, under step 2B, the additional elements of 1) a headset configured to be worn by a subject, the headset comprises one or more internal camera facing toward the subject's face and eyes, one or more external camera facing toward the subject's body, one or more speakers, one or more microphone oriented at the subject's mouth, at least one sensory stimulator configured to deliver vibratory and at least one of thermal or tactile stimulation to at least one unique location on the subject's body, and at least semi-transparent augmented reality visor; 2) cause the one or more internal camera facing toward the subject's face and eyes to capture images and/or video of the subject's face and store the images and/or video of the subject's face as first data; cause the one or more external camera facing toward the subject's body to capture images and/or video of the subject's body and store the images and/or video of the subject's body as second data; cause the one or more microphone oriented at the subject's mouth to capture an audio signal from the subject and store the audio signal as third data; 3) communicate the unified final diagnosis to the subject via the speaker and/or a display screen on at least a portion of the at least semi-transparent augmented reality visor; and 4) wherein at least one sensory stimulation is delivered to the at least one unique location on the subject's body via the at least one sensory stimulator based on the unified final diagnosis have been evaluated. The medical device comprising a processor performs a general function of receiving subject data for subsequent processing, which represents a well-understood, routine, and conventional activity in the field of computer implemented medical data analysis and diagnosis. The specification discloses that the processor is used in its ordinary capacity as a data input device and does not describe any improvement to the computer itself or to the functioning of the overall computer system (see [0040] and [0050]). Also noted in Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016), merely collecting information for analysis without a technological improvement does not add significantly more to an abstract idea. The use of the medical device is no more than collecting information before performing diagnosis analysis and displaying the result and does not integrate the abstract idea into a practical application. Additionally, as noted in In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016), merely delivering sensory stimulation based on diagnosis represents an insignificant application of the underlying mental process, as the sensory stimulation is merely triggered in response to the diagnosis conclusion and does not impose any meaningful limitation or add any technological improvement. Lastly, the additionally recited headset components, including cameras, microphones, speakers, visor, and sensory stimulator and recited as generic wearable input and output components performing their routine and conventional functions of data collection, presentation, and feedback (see White (2019) and Ungureanu et al. (2020, Introduction)). Therefore, the claim does not recite an inventive concept and is not patent eligible.
Claims 28-32, 37, 42, 44, 46, 49-51 recite no further additional elements, and only further narrow the abstract idea. The previously identified additional elements, individually and as a combination, do not integrate the narrowed abstract idea into a practical application for reasons similar to those explained above, and do not amount to significantly more than the narrowed abstract idea for reasons similar to those explained above.
Claims 24, 33, 35, 40-41, 43, 45, 47-48 recite the additional element of the headset and/or the medical diagnostic system (claim 24), the processor (claims 33 and 35), cause the at least semi-transparent augmented reality visor to display graphical and/or other visual information to the subject (claim 33), wherein the drug delivery device delivers one of injection of medication to the subject based on the unified final diagnosis (claim 40), a rule-based decision engine (claim 41), confidence-weighted combiner (claim 43), the first data analytics process module and/or the second data analytics process module (claims 45 and 48), and wherein the unified final diagnosis triggers, prompting targeted follow-up questions, initiating therapy delivery, transmitting an emergency alert, or routing the subject for physician evaluation (claim 47). However, this additional element amounts to implementing an abstract idea on a generic computing device or outputting a result or insignificant application of the abstract idea (i.e., an insignificant extra-solution activity)). As such, these additional elements, when considered individually or in combination with the prior devices, do not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea.
Thus, as the dependent claims remain directed to a judicial exception, and as the additional elements of the claims do not amount to significantly more, the dependent claims are not patent eligible.
Therefore, the claims here fail to contain any additional element(s) or combination of additional elements that can be considered as significantly more and the claims are rejected under 35 U.S.C. 101 for lacking eligible subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 21, 24, 28, 33, 35, 37, and 40-51 are rejected under 35 U.S.C. 103 as being unpatentable over Wall et al. (U.S. Patent Publication 2021/0133509 A1), referred to hereinafter as Wall, in view of Ungureanu et al. (Ungureanu et al., HoloLens 2 Research Mode as a Tool for Computer Vision Research, 2020, arXiv, pages 1-7 (Year: 2020)), referred to hereinafter as Ungureanu, Harris et al. (U.S. Patent Publication 2019/0392952 A1), referred to hereinafter as Harris, and Leuthardt et al. (U.S. Patent Publication 2021/0361948A1), referred to hereinafter as Leuthardt.
Regarding claim 21, Wall teaches a medical device comprising (Wall [0022] “In one aspect, the digital personalized medicine system can comprise digital devices with processors and associated software that can be configured to:”):
one or more internal camera facing toward the subject's face and eyes, one or more external camera facing toward the subject's body, one or more speakers, one or more microphone oriented at the subject's mouth (Wall [0307] “The digital therapeutics utilized by the methods and devices described herein can be implemented using various digital applications, including augmented reality, virtual reality, real-time cognitive assistance, or other behavioral therapies augmented using technology. Digital therapeutics can be implemented using any device configured to produce a virtual or augmented reality environment. Such devices can be configured to include one or more sensor inputs such as video and/or audio captured using a camera and/or microphone. Non-limiting examples of devices suitable for providing digital therapy as described herein include wearable devices, smartphones, tablet computing devices, laptops, projectors and any other device suitable for producing virtual or augmented reality experiences.”), Wall [0448] “The system can track expressive events in faces using the outward-facing camera on the smartphone and read the facial expressions or emotions by passing video and/or image or photographic data to a smartphone app for real-time machine learning-based classification of commonly used emotions (e.g., standardized Ekman “basic” emotions).” Wall [0449] “Specific improvements can be tracked, for example, by monitoring and assessing performance and other metrics of the subject during the various digital therapeutic activities. As an example, an inward facing camera and/or microphone can be used to monitor facial engagement, emotional expression, gaze, verbal interactions (e.g., whether child verbally responds to a caregiver's question), and other behavior by the subject.”, and Wall [0320] “The device also has an inward-facing digital display having a peripheral monitor and a speaker.”);
a medical diagnostic system (Wall [0022] “In one aspect, the digital personalized medicine system can comprise digital devices with processors and associated software that can be configured to: use data to assess and diagnose a patient; capture interaction and feedback data that identify relative levels of efficacy, compliance and response resulting from the therapeutic interventions; and perform data analysis. Such data analysis can include artificial intelligence, including for example machine learning, and/or statistical models to assess user data and user profiles to further personalize, improve or assess efficacy of the therapeutic interventions.”);
a non-transitory memory storing instructions and a plurality of data analytics process modules comprising at least a first data analytics process module, wherein the first data analytics process module and the second data analytics process module are different (Wall [0163] “In one aspect, a method of providing an evaluation of at least one cognitive function attribute of a subject may comprise: on a computer system having a processor and a memory storing a computer program for execution by the processor, the computer program comprising instructions for:”, Wall [0022] “In one aspect, the digital personalized medicine system can comprise digital devices with processors and associated software that can be configured to: use data to assess and diagnose a patient; capture interaction and feedback data that identify relative levels of efficacy, compliance and response resulting from the therapeutic interventions; and perform data analysis. Such data analysis can include artificial intelligence, including for example machine learning, and/or statistical models to assess user data and user profiles to further personalize, improve or assess efficacy of the therapeutic interventions.”, Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, Wall [0021] “In addition, the digital personalized medicine systems and methods described herein can provide digital diagnostics and digital therapeutics to patients.”);
a processor configured to access the memory and execute the instructions stored in the memory to (Wall [0163] “In one aspect, a method of providing an evaluation of at least one cognitive function attribute of a subject may comprise: on a computer system having a processor and a memory storing a computer program for execution by the processor, the computer program comprising instructions for:”):
cause the one or more internal camera facing toward the subject's face and eyes to capture images and/or video of the subject's face and store the images and/or video of the subject's face as first data (Wall [0321] “In some cases, the device also comprises an inward-facing camera (e.g., a “selfie” camera) and tracks and classifies the emotions of the digital therapy recipient. The tracking and classification of the emotions of the social partner and the emotions of the digital therapy recipient can be performed in real time simultaneously or in close temporal proximity (e.g., within 1, 2, 3, 4, or 5 seconds of each other, or some other appropriate time frame). Alternatively, images may be captured of the social partner and/or digital therapy recipient and then evaluated to track and classify their respective emotions at a later time (i.e., not in real time).”);
cause the one or more external camera to capture images and/or video of the subject's body as second data (Wall [0448] “The system can track expressive events in faces using the outward-facing camera on the smartphone and read the facial expressions or emotions by passing video and/or image or photographic data to a smartphone app for real-time machine learning-based classification of commonly used emotions (e.g., standardized Ekman “basic” emotions). Examples of such emotions include anger, disgust, fear, happiness, sadness, surprise, contempt, and neutral. The system can then provide real-time social cues about the facial expressions (e.g., “happy,” “angry,” etc.) to the subject or user through the smartphone. The cues can be visual, shown on the app, and/or auditory, through a speaker on the smartphone, or any combination thereof.”);
cause the one or more microphone oriented at the subject's mouth to capture an audio signal from the subject and store the audio signal as third data (Wall [0222] “In some embodiments, said computing device comprises a microphone configured to capture audio from said augmented reality experience.”, and Wall [0223] “In some embodiments, said processor is configured with further instructions to categorize a sound from said microphone as associated with an emotion.” and Wall [0163] “In one aspect, a method of providing an evaluation of at least one cognitive function attribute of a subject may comprise: on a computer system having a processor and a memory storing a computer program for execution by the processor, the computer program comprising instructions for:”);
determine a first diagnostic output into the first data analytics process module; determine a second diagnostic output into the second data analytics process module (Wall [0633] “The scalar outputs from each of the three classification algorithms are passed as inputs into a second stage combinatorial classification model, which is trained independently on 350 historical data submissions collected in clinical studies. This combinatorial model is probabilistic in nature and is trained to take into account the covariance matrix between all three individual module classifiers. It outputs a single scalar value that represents a combined output of all three modules, and its output is then compared to preset thresholds in order to produce a categorical outcome that can be considered a determination of whether the child is Positive for ASD or Negative for ASD.”, Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”);
determine a unified final diagnosis by integrating the first diagnostic output and the second diagnostic output (Wall [0633] “The scalar outputs from each of the three classification algorithms are passed as inputs into a second stage combinatorial classification model, which is trained independently on 350 historical data submissions collected in clinical studies. This combinatorial model is probabilistic in nature and is trained to take into account the covariance matrix between all three individual module classifiers. It outputs a single scalar value that represents a combined output of all three modules, and its output is then compared to preset thresholds in order to produce a categorical outcome that can be considered a determination of whether the child is Positive for ASD or Negative for ASD.”, Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, and Wall [0167] “The method may further comprise employing rule-based logic or combinatorial techniques for combining the numerical scores for each of the preliminary subset of assessment models and for combining the numerical scores for each of the additional assessment model”); and
communicate the unified final diagnosis to the subject via the speaker and/or a display screen (Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, and Wall [0167] “The method may further comprise employing rule-based logic or combinatorial techniques for combining the numerical scores for each of the preliminary subset of assessment models and for combining the numerical scores for each of the additional assessment model”Wall [0380] “FIG. 7 illustrates a method 700 of administering an assessment procedure as described herein. The method 700 may be performed with a user interface provided on a computing device, the computing device comprising a display and a user interface for receiving user input in response to the instructions provided on the display.”, Wall [0320] “The device also has an inward-facing digital display having a peripheral monitor and a speaker.”); and
based on the unified final diagnosis (Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, and Wall [0167] “The method may further comprise employing rule-based logic or combinatorial techniques for combining the numerical scores for each of the preliminary subset of assessment models and for combining the numerical scores for each of the additional assessment model”).
Wall fails to explicitly teach a headset configured to be worn by a subject, and at least one sensory stimulator configured to deliver vibratory and at least one of thermal or tactile stimulation to at least one unique location on the subject's body, and at least semi-transparent augmented reality visor; coupled to the headset; a second data analytics process module; facing toward the subject's body and store the images and/or video of the subject's body; by inputting at least one the first data, the second data, and/or the third data; on at least a portion of the at least semi-transparent augmented reality visor; and wherein at least one sensory stimulation is delivered to the at least one unique location on the subject's body via the at least one sensory stimulator.
Ungureanu teaches at least one sensory stimulator configured to deliver vibratory and at least one of thermal or tactile stimulation to at least one unique location on the subject's body, and at least semi-transparent augmented reality visor (Ungureanu, page 1, “The first-generation HoloLens Research Mode, released in 2018, enabled computer vision research on device by providing access to all raw image sensor streams – including depth and IR. Released together with a public repository collecting auxiliary tools and sample applications [4], ResearchMode promoted the use of HoloLens as a powerful tool for doing research in computer vision and robotics [1]. HoloLens 2 (Fig. 1), announced in 2019, brings a number of improvements with respect to the first-generation device – like a dedicated DNN core, articulated hand tracking and eye gaze tracking [2]. However, being a novel platform built on new hardware, HoloLens 2 is not compatible with the previous version of Research Mode. In this technical report, we introduce the second generation HoloLens Research Mode, made available in 2020. Research Mode provides a set of C++ APIs and tools to access the HoloLens 2 sensor streams. We discuss the main novelties with respect to the previous version and present a set of applications built on top of it. For additional material and API documentation, we refer the reader to the GitHub repository [3].”);
coupled to the headset (Ungureanu, page 6, “We presented Research Mode for HoloLens 2, an API and a set of functionalities enabling access to the raw sensor streams on device. Together with Research Mode, we release a public repository collecting sample apps and examples showcasing how to leverage Research Mode to build computer vision applications based on HoloLens. With these tools, we hope to facilitate further research in the fields of computer vision and robotics, and encourage contributions from the research community.”):
facing toward the subject's body and store the images and/or video of the subject's body (Ungureanu, page 5-6 “Stream Recorder: The StreamRecorder app allows one to capture simultaneously Research Mode streams (depth and VLC), the RGB stream from the HoloLens PV (PhotoVideo) camera, head, hand and eye tracking. The application provides a simple user interface to start and stop capture (Fig. 8, top). Streams are captured and stored on device, and can be downloaded and postprocessed by using a set of python scripts. Namely, we provide scripts to correlate Research Mode, PV, head, hand and eye tracking streams in time and put them in a common coordinate frame.”);
on at least a portion of the at least semi-transparent augmented reality visor (Ungureanu, page 1, “The first-generation HoloLens Research Mode, released in 2018, enabled computer vision research on device by providing access to all raw image sensor streams – including depth and IR. Released together with a public repository collecting auxiliary tools and sample applications [4], ResearchMode promoted the use of HoloLens as a powerful tool for doing research in computer vision and robotics [1]. HoloLens 2 (Fig. 1), announced in 2019, brings a number of improvements with respect to the first-generation device – like a dedicated DNN core, articulated hand tracking and eye gaze tracking [2]. However, being a novel platform built on new hardware, HoloLens 2 is not compatible with the previous version of Research Mode. In this technical report, we introduce the second generation HoloLens Research Mode, made available in 2020. Research Mode provides a set of C++ APIs and tools to access the HoloLens 2 sensor streams. We discuss the main novelties with respect to the previous version and present a set of applications built on top of it. For additional material and API documentation, we refer the reader to the GitHub repository [3].”);
Harris teaches a second data analytics process module (Harris [0004] “One aspect of the invention provides a computer-implemented method for diagnosing a condition on a computer including a processor, memory, at least one of a user interface and a communication interface, a user profile module, a question module, a vitals module, and a diagnostic module. The computer-implemented method includes: (a) controlling the processor and one or more of the user interface, the communication interface, the user profile module, the question module, and the vitals module on the computer to receive one or more inputs regarding a subject's symptoms” and Harris [0009] “Another aspect of the invention provides a system for diagnosing a condition. The system includes: a user profile module; a diagnostic module; and a question module. The user profile module includes computer-readable program code including steps for: obtaining one or more user profile inputs regarding a subject's medical status and history; and recording the one or more user profile inputs.”);
by inputting at least one the first data, the second data, and/or the third data (Harris [0053] “Diagnostic module results can be based on the three data structure inputs being observed and computed through each of the module's active diagnoses. The resulting tabulations can then be compared to assign a diagnosis ranking, along with distance vectors between diagnoses being used to further separate the active diagnosis selection(s) from the inactive ones.”, Harris [0054] “The diagnostic module data structure can be a mutable array of key/value dictionaries that define each diagnosis based on the three data inputs. The diagnosesVitalMatrix and the diagnosesQuestionMatrix dictionaries can hold the description of a given condition, based on point values being assigned to a user's medical history, her answers to questions, and what the device hardware is reading for their current vitals.” Harris [0056] “As the three data input modules collect data, the diagnosis module can constantly update the running score for each diagnosis. These values can be used to determine the next line of questions and assess if the user has met enough of the criteria to be diagnosed for any particular disease. For example, a particular datum (e.g., blood glucose level, blood type) can affect the probability of n diagnoses to varying degrees and can be stored as a vector of probability influence weights [1.0, −0.5, 0, . . . , pn] that can be added to update a plurality of n diagnoses. The actual weights can vary between implementations. The probability influence weights can be stored in one or more databases and/or data structures, e.g., in a rule/action or IF/THEN format. For example, base diagnosis data store 118 can store a plurality of rules instructing the diagnostic module 102 to modify one or more probabilities based on a detected condition. The running probabilities for a particular patient can be stored in the session diagnosis data store 120.”, Harris [0004] “One aspect of the invention provides a computer-implemented method for diagnosing a condition on a computer including a processor, memory, at least one of a user interface and a communication interface, a user profile module, a question module, a vitals module, and a diagnostic module. The computer-implemented method includes: (a) controlling the processor and one or more of the user interface, the communication interface, the user profile module, the question module, and the vitals module on the computer to receive one or more inputs regarding a subject's symptoms” and Harris [0009] “Another aspect of the invention provides a system for diagnosing a condition. The system includes: a user profile module; a diagnostic module; and a question module. The user profile module includes computer-readable program code including steps for: obtaining one or more user profile inputs regarding a subject's medical status and history; and recording the one or more user profile inputs.”);
Leuthardt teaches wherein at least one sensory stimulation is delivered to the at least one unique location on the subject's body via the at least one sensory stimulator (Leuthardt [0080] “The systems and methods disclosed in the present application disclose a multiple closed loop cortical neuromodulation system that delivers brain electrical stimulation therapy based on sensed patient's cortical signals and on one or more relevant patient inputs in the form of ecological momentary assessments and/or other patient's physiological biomarkers. The “Patient and Sensor Informed Closed-loop Cortical” (PASICC) neuromodulation system does not require a priori identification of cortical signal or a physiological biomarker, but rather learns the biomarker with ongoing utilization by the patient. The system may include an intra-calvarial implant that is capable of stimulating and recording from a focal region in the cortex, a mobile communication device (such as, for example, a mobile phone, a smartphone a laptop, a tablet, a notebook, a phablet, an augmented reality (AR) headset having communication capabilities) that can engage with the patient to either actively or passively provide patient's mood assessments such, for example, ecological momentary mood assessment (EMA) to the system. The system also includes software for correlating the sensed cortical electrical activity with mood assessments to enable the detection of a mood state requiring treatment and deliver a selected stimulation regime. The system may adapt to each individual patient by using suitable training and/or test periods and may provide patient specific cortical biomarkers that may be used for optimized cortical stimulation to address the mood related symptoms of a patient with depression.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wall’s digital personalized medicine system with the AR headset and sensor-stream implementation taught by Ungureanu. Wall teaches that a digital diagnostic and therapeutic system may be implemented using wearable devices and devices configured to produce augmented reality environments, and that such devices may include video and audio sensor inputs, cameras, microphones, speakers, and displays. Ungureanu further demonstrates that HoloLens 2 was a known AR headset platform with sensor streams, eye gaze tracking, hand tracking, raw image streams, and on-device computer vision capabilities. A person of ordinary skill would have been motivated to use a known AR headset such as the HoloLens 2 as Wall’s wearable augmented reality implementation in order to provide hands free patient interaction, real time multimodal data acquisition, and visual and audio presentation of diagnostic or therapeutic information through an integrated wearable interface.
It would also have been obvious to incorporate Harris’s symptom diagnostic logic into Wall’s machine-learning and combinatorial diagnostic framework. Wall teaches combining outputs from multiple assessment models or classification algorithms to generate a combined diagnostic determination, while Harris teaches rule based diagnostic scoring based on patient symptoms, questions, medical history, and vitals. A person of ordinary skill would have been motivated to combine these teachings because using both machine learning assessment and symptom and rule-based diagnostic analysis would predictably improve diagnostic reliability by providing independent analytic outputs that may be compared or integrated into a unified diagnostic result.
Further, it would have been obvious to include the stimulation functionality taught by Leuthardt in the combined system. Wall teaches digital therapeutics and feedback based on diagnostic or assessment information, and Leuthardt teaches patient responsive stimulation therapy based on sensed patient information and patient inputs, including use of a mobile communication device or AR headset to engage the patient. A person of ordinary skill would have been motivated to provide sensory or therapeutic stimulation based on the unified diagnostic result in order to deliver immediate patient feedback or treatment-responsive stimulation after the diagnostic determination. The combination applies known AR headset sensing and display technology, known multi module diagnostic analytics, and known patient responsive stimulation in a predictable manner to Wall’s digital diagnostic and therapeutic system.
Regarding claim 24, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the headset and/or the medical diagnostic system is coupled to one or more of a thermal camera, thermometer, electrocardiography sensor, photoplethysmography sensor, electromagnetic pulse monitor, accelerometer, and a gyroscope (Wall [0022] “In one aspect, the digital personalized medicine system can comprise digital devices with processors and associated software that can be configured to: use data to assess and diagnose a patient; capture interaction and feedback data that identify relative levels of efficacy, compliance and response resulting from the therapeutic interventions; and perform data analysis. Such data analysis can include artificial intelligence, including for example machine learning, and/or statistical models to assess user data and user profiles to further personalize, improve or assess efficacy of the therapeutic interventions.” and Wall [0327] “Disclosed herein are platforms, systems, devices, methods, and media that provide a machine learning framework for detecting emotional or social cues. Input data can include image and/or video data and optionally additional sensor data (e.g., accelerometer data, audio data, etc.). The input data is provided into an emotion detection system that detects or identifies emotional or social cues, which can be output to the user such as in real-time via a user interface on a computing device.” and
Ungureanu, page 6, “We presented Research Mode for HoloLens 2, an API and a set of functionalities enabling access to the raw sensor streams on device. Together with Research Mode, we release a public repository collecting sample apps and examples showcasing how to leverage Research Mode to build computer vision applications based on HoloLens. With these tools, we hope to facilitate further research in the fields of computer vision and robotics, and encourage contributions from the research community.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to configure the wearable headset and/or medical diagnostic system of Wall in view of Ungureanu to be coupled to additional physiological and motion sensors, including one or more of a thermal camera, thermometer, electrocardiography (ECG) sensor, photoplethysmography (PPG) sensor, electromagnetic pulse monitor, accelerometer, and gyroscope, because Wall teaches using multiple sensor inputs and machine learning analysis for patient assessment and diagnosis, including accelerometer and audio/video sensor data, while Ungureanu teaches that HoloLens 2 provides access to raw sensor streams and supports integration of multiple sensing modalities for computer vision and wearable applications. A person of ordinary skill in the art would have been motivated to incorporate known physiological monitoring sensors into the wearable diagnostic platform in order to improve physiological data collection, patient monitoring, and diagnostic accuracy, with the combination yielding predictable results through the use of known sensor technologies for their established functions in wearable medical and monitoring systems.
Regarding claim 28, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the at least one sensory stimulator comprises an electrical stimulator configured to deliver stimulation delivered is one or more of thermal stimulation, vibratory stimulation, tactile stimulation, and electrical stimulation (Leuthardt [0080] “The systems and methods disclosed in the present application disclose a multiple closed loop cortical neuromodulation system that delivers brain electrical stimulation therapy based on sensed patient's cortical signals and on one or more relevant patient inputs in the form of ecological momentary assessments and/or other patient's physiological biomarkers. The “Patient and Sensor Informed Closed-loop Cortical” (PASICC) neuromodulation system does not require a priori identification of cortical signal or a physiological biomarker, but rather learns the biomarker with ongoing utilization by the patient. The system may include an intra-calvarial implant that is capable of stimulating and recording from a focal region in the cortex, a mobile communication device (such as, for example, a mobile phone, a smartphone a laptop, a tablet, a notebook, a phablet, an augmented reality (AR) headset having communication capabilities) that can engage with the patient to either actively or passively provide patient's mood assessments such, for example, ecological momentary mood assessment (EMA) to the system. The system also includes software for correlating the sensed cortical electrical activity with mood assessments to enable the detection of a mood state requiring treatment and deliver a selected stimulation regime. The system may adapt to each individual patient by using suitable training and/or test periods and may provide patient specific cortical biomarkers that may be used for optimized cortical stimulation to address the mood related symptoms of a patient with depression.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the wearable medical diagnostic system of Wall in view of Leuthardt to include at least one sensory stimulator comprising an electrical stimulator configured to deliver electrical stimulation, because Wall teaches wearable digital diagnostic and therapeutic systems that collect patient data and provide therapeutic interventions based on diagnostic analysis, while Leuthardt teaches delivering electrical stimulation therapy based on sensed patient biomarkers and patient assessments in conjunction with an augmented reality headset environment. A person of ordinary skill in the art would have been motivated to incorporate Leuthardt’s electrical stimulation functionality into Wall’s wearable diagnostic platform in order to provide responsive therapeutic treatment or feedback based on the determined patient condition, yielding predictable results through the use of known electrical stimulation techniques for their established therapeutic purpose.
Regarding claim 33, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the processor is further configured to, responsive to a spoken verbal response from the subject to a displayed or played prompt, cause the at least semi-transparent augmented reality visor to display graphical and/or other visual information to the subject (Wall [0460] “Each of diagnosis modules 2332 and 2334 communicate with the user digital device 2310 during a course of treatment. The diagnosis module provides diagnostic tests to and receives diagnostic feedback from the digital device 2310, and uses the feedback to determine a diagnosis of a patient. An initial diagnosis may be based on a comprehensive set of tests and questions, for example, while incremental updates may be made to a diagnosis using smaller data samples. For example, the diagnostic module may diagnose autism-related speech delay based on questions asked to the caregiver and tests administered to the patient such as vocabulary or verbal communication tests. The diagnosis may indicate a number of months or years delay in speech abilities. Later tests may be administered and questions asked to update this diagnosis, for example showing a smaller or larger degree of delay.” and
Ungureanu, page 1, “The first-generation HoloLens Research Mode, released in 2018, enabled computer vision research on device by providing access to all raw image sensor streams – including depth and IR. Released together with a public repository collecting auxiliary tools and sample applications [4], ResearchMode promoted the use of HoloLens as a powerful tool for doing research in computer vision and robotics [1]. HoloLens 2 (Fig. 1), announced in 2019, brings a number of improvements with respect to the first-generation device – like a dedicated DNN core, articulated hand tracking and eye gaze tracking [2]. However, being a novel platform built on new hardware, HoloLens 2 is not compatible with the previous version of Research Mode. In this technical report, we introduce the second generation HoloLens Research Mode, made available in 2020. Research Mode provides a set of C++ APIs and tools to access the HoloLens 2 sensor streams. We discuss the main novelties with respect to the previous version and present a set of applications built on top of it. For additional material and API documentation, we refer the reader to the GitHub repository [3].”;).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the interactive diagnostic system of Wall with the augmented reality headset platform of Ungureanu in order to provide responsive graphical and visual information to a patient through an augmented reality visor based on spoken verbal responses to diagnostic prompts. Wall teaches presenting diagnostic questions and tests to a patient and receiving diagnostic feedback and verbal responses for updating diagnostic evaluations, while Ungureanu teaches the HoloLens 2 augmented reality platform capable of real time sensor processing and visual display interaction. A person of ordinary skill in the art would have been motivated to combine these teachings to improve user interaction, hands free diagnostic guidance, and real time visual feedback in wearable medical diagnostic environments, yielding predictable results through the use of known AR interface technologies and interactive diagnostic systems.
Regarding claim 35, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the processor is further configured to convert patient speech to text and cause speakers to auditorily respond to patient with spoken text (Wall [0470] “The analysis of the data to determine progress and current diagnosis can include automated analysis such as question scoring and voice-recognition for vocabulary and speech analysis. The analysis can also include human scoring by analysis reviewing video, audio, and text data.” and Wall [0311] “In a non-limiting example of an augmented reality digital therapy experience, an individual may react with a real person, and in this example, a video, image, and/or sound recording of the person is taken by the computing device that is delivering the digital therapy. Then, the video, image, and/or sound recording is analyzed using analysis classifier that determines an emotion associated with a facial expression (or other social cue) of the person interacted with by the individual in the augmented reality environment. An analysis of the facial expression (or other social cue) may comprise an assessment of an emotion or a mood associated with the facial expression and/or other social cue. The result of the analysis is then provided to the individual receiving the digital therapy. In some embodiments, the result of the analysis is displayed within the augmented reality environment. In some embodiments, the result of the analysis is displayed on a screen of a computing device. In some embodiments, the result of the analysis is provided via an audible sound or message.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to configure the medical diagnostic system of Wall to convert patient speech to text and provide audible spoken responses to the patient because Wall expressly teaches voice recognition vocabulary and speech analysis using audio and text data, as well as providing analysis results through audible sounds or messages within an augmented reality therapeutic environment. A person of ordinary skill in the art would have been motivated to employ speech to text and spoken response functionality in Wall’s interactive diagnostic platform in order to facilitate hands free patient interaction, improve communication during diagnostic evaluations, and enhance accessibility and user feedback in wearable medical diagnostic systems, yielding predictable results through the use of known voice recognition and audio output technologies.
Regarding claim 37, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein a likelihood of proper diagnosis of a given disease or condition of the subject is determined based on a combined association of the first data, the second data, and/or the third data and the incidence of a given disease or condition of the unified final diagnosis, where the first data, the second data, and/or the third data includes one or more of presence of sudden numbness or weakness in body of the subject, a National Institutes of Health Stroke Scale (NIHSS) score, indication of tobacco, an age, a race, a sex, indication of dyslipidemia, indication of atrial fibrillation, indication of high blood pressure, current systolic blood pressure, current diastolic blood pressure, current glucose level, medications the subject is currently taking, indication of subject family history of stroke, indication of coronary artery disease, and current heart rate (Wall [0346] “The training module 110 can utilize a machine learning algorithm or other algorithm to construct and train an assessment model to be used in the assessment procedure, for example. An assessment model can be constructed to capture, based on the training data, the statistical relationship, if any, between a given feature value and a specific developmental disorder to be screened by the assessment procedure. The assessment model may, for example, comprise the statistical correlations between a plurality of clinical characteristics and clinical diagnoses of one or more developmental disorders. A given feature value may have a different predictive utility for classifying each of the plurality of developmental disorders to be evaluated in the assessment procedure. For example, in the aforementioned example of a feature comprising the ability of the subject to engage in imaginative or pretend play, the feature value of “3” or “no variety of pretend play” may have a high predictive utility for classifying autism, while the same feature value may have low predictive utility for classifying ADHD. Accordingly, for each feature value, a probability distribution may be extracted that describes the probability of the specific feature value for predicting each of the plurality of developmental disorders to be screened by the assessment procedure. The machine learning algorithm can be used to extract these statistical relationships from the training data and build an assessment model that can yield an accurate prediction of a developmental disorder when a dataset comprising one or more feature values is fitted to the model.”, Wall [0302] “The diagnostic process may enable a caregiver or family member to perform an assessment or provide data for an assessment of the subject. The subject may have a risk selected from the group consisting of a behavioral disorder, neurological disorder, and mental health disorder. The risk may be selected from the group consisting of autism, autistic spectrum, attention deficit disorder, depression, obsessive compulsive disorder, schizophrenia, Alzheimer's disease, dementia, attention deficit hyperactive disorder, and speech and learning disability.).” Wall [0633] “The scalar outputs from each of the three classification algorithms are passed as inputs into a second stage combinatorial classification model, which is trained independently on 350 historical data submissions collected in clinical studies. This combinatorial model is probabilistic in nature and is trained to take into account the covariance matrix between all three individual module classifiers. It outputs a single scalar value that represents a combined output of all three modules, and its output is then compared to preset thresholds in order to produce a categorical outcome that can be considered a determination of whether the child is Positive for ASD or Negative for ASD.”, Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, and Wall [0167] “The method may further comprise employing rule-based logic or combinatorial techniques for combining the numerical scores for each of the preliminary subset of assessment models and for combining the numerical scores for each of the additional assessment model”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to determine a likelihood of a disease or condition based on combined patient data and statistical relationships between clinical features and diagnostic outcomes as taught by Wall, because Wall teaches machine learning assessment models that evaluate multiple feature values, clinical characteristics, and combined classifier outputs to determine probabilistic diagnostic outcomes for medical conditions. Wall further teaches extracting statistical correlations between patient features and diagnoses and combining outputs from multiple assessment models to generate unified diagnostic determinations. A person of ordinary skill in the art would have been motivated to apply this probabilistic and combinatorial diagnostic techniques to multimodal patient data in order to improve diagnostic accuracy and predictive reliability, yielding predictable results through the use of known machine learning and statistical diagnostic modeling techniques.
Regarding claim 40, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the headset further communicates with a drug delivery device, wherein the drug delivery device delivers one of injection of medication to the subject based on the unified final diagnosis (Ungureanu, page 1, “The first-generation HoloLens Research Mode, released in 2018, enabled computer vision research on device by providing access to all raw image sensor streams – including depth and IR. Released together with a public repository collecting auxiliary tools and sample applications [4], ResearchMode promoted the use of HoloLens as a powerful tool for doing research in computer vision and robotics [1]. HoloLens 2 (Fig. 1), announced in 2019, brings a number of improvements with respect to the first-generation device – like a dedicated DNN core, articulated hand tracking and eye gaze tracking [2]. However, being a novel platform built on new hardware, HoloLens 2 is not compatible with the previous version of Research Mode. In this technical report, we introduce the second generation HoloLens Research Mode, made available in 2020. Research Mode provides a set of C++ APIs and tools to access the HoloLens 2 sensor streams. We discuss the main novelties with respect to the previous version and present a set of applications built on top of it. For additional material and API documentation, we refer the reader to the GitHub repository [3].”; and
Leuthardt [0080] “The systems and methods disclosed in the present application disclose a multiple closed loop cortical neuromodulation system that delivers brain electrical stimulation therapy based on sensed patient's cortical signals and on one or more relevant patient inputs in the form of ecological momentary assessments and/or other patient's physiological biomarkers.”; and
Wall [0633] “The scalar outputs from each of the three classification algorithms are passed as inputs into a second stage combinatorial classification model, which is trained independently on 350 historical data submissions collected in clinical studies. This combinatorial model is probabilistic in nature and is trained to take into account the covariance matrix between all three individual module classifiers. It outputs a single scalar value that represents a combined output of all three modules, and its output is then compared to preset thresholds in order to produce a categorical outcome that can be considered a determination of whether the child is Positive for ASD or Negative for ASD.”, Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, and Wall [0167] “The method may further comprise employing rule-based logic or combinatorial techniques for combining the numerical scores for each of the preliminary subset of assessment models and for combining the numerical scores for each of the additional assessment model”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to further configure the diagnostic headset and medical diagnostic system of Wall and Ungureanu to communicate with a therapeutic delivery device that delivers treatment based on a unified diagnostic determination as taught by Leuthardt, because Leuthardt teaches a closed loop therapeutic system in which physiological and diagnostic information is analyzed to automatically trigger patient specific therapy delivery. Wall further teaches generating unified diagnostic outputs using combined classification models, while Ungureanu teaches an augmented reality headset platform capable of interactive sensor processing and communication. A person of ordinary skill in the art would have been motivated to combine these teachings in order to provide responsive and automated treatment delivery based on diagnostic determinations, thereby improving speed of care, treatment responsiveness, and integration of diagnostic and therapeutic systems, yielding predictable results through the use of known closed loop medical treatment architectures.
Regarding claim 41, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein a rule-based decision engine is configured to apply predefined agreement criteria to the diagnostic outputs, including a unanimity rule requiring both outputs to agree or a disjunction rule permitting either output to determine the unified final diagnosis (Wall [0167] “The method may further comprise employing rule-based logic or combinatorial techniques for combining the numerical scores for each of the preliminary subset of assessment models and for combining the numerical scores for each of the additional assessment models. The ratio of inconclusive to categorical determinations may be adjusted by specifying an inclusion rate. The categorical determination as to the presence or absence of the developmental condition in the subject may be assessed by providing a sensitivity and specificity metric. The inclusion rate may be no less than 70% and the categorical determination may result in a sensitivity of at least 70 with a corresponding specificity of at least 70. The inclusion rate may be no less than 70% and the categorical determination may result in a sensitivity of at least 80 with a corresponding specificity of at least 80. The inclusion rate may be no less than 70% and the categorical determination may result in a sensitivity of at least 90 with a corresponding specificity of at least 90.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to employ predefined agreement rules for combining multiple diagnostic outputs in the medical diagnostic system of Wall because Wall teaches rule-based logic and combinatorial techniques for combining outputs from multiple assessment models and adjusting categorical diagnostic determinations based on predefined criteria and thresholds. A person of ordinary skill in the art would have been motivated to implement known logical combination approaches, including agreement and disjunctive decision rules, in order to improve diagnostic reliability, manage inconclusive determinations, and optimize diagnostic sensitivity and specificity, yielding predictable results through the use of conventional multi-model decision integration techniques.
Regarding claim 42, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein a voting rule is applied when three diagnostic outputs are available, requiring at least two of the outputs to concur on a health condition before providing the unified final diagnosis (Wall [0349] “FIG. 3 is a schematic diagram illustrating a portion of an assessment model 160 based on a Random Forest classifier. The assessment module may comprise a plurality of individual decision trees 165, such as decision trees 165 a and 165 b, each of which can be generated independently using a random subset of features in the training data. Each decision tree may comprise one or more decision nodes such as decision nodes 166 and 167 shown in FIG. 3, wherein each decision node specifies a predicate condition. For example, decision node 16 predicates the condition that, for a given dataset of an individual, the answer to question #86 (age when abnormality is first evident) is 4 or less. Decision node 167 predicates the condition that, for the given dataset, the answer to question #52 (showing and direction attention) is 8 or less. At each decision node, a decision tree can be split based on whether the predicate condition attached to the decision node holds true, leading to prediction nodes (e.g., 166 a, 166 b, 167 a, 167 b). Each prediction node can comprise output values (‘value’ in FIG. 3) that represent “votes” for one or more of the classifications or conditions being evaluated by the assessment model. For example, in the prediction nodes shown in FIG. 3, the output values comprise votes for the individual being classified as having autism or being non-spectrum. A prediction node can lead to one or more additional decision nodes downstream (not shown in FIG. 3), each decision node leading to an additional split in the decision tree associated with corresponding prediction nodes having corresponding output values. The Gini impurity can be used as a criterion to find informative features based on which the splits in each decision tree may be constructed. An assessment model can be configured to detect or evaluate a subject for the presence of a disorder or condition. In some cases, a separate assessment model is configured to determine whether a subject having the disorder or condition will be improved by a digital therapy, for example, a digital therapy configured to promote social reciprocity.”.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to apply a voting rule requiring agreement among multiple diagnostic outputs before determining a unified final diagnosis because Wall teaches assessment models employing multiple decision trees and prediction nodes that generate classification “votes” for evaluated conditions. A person of ordinary skill in the art would have been motivated to utilize majority voting or concurrence decision logic, such as requiring at least two outputs to agree, in order to improve diagnostic reliability, reduce false positives, and increase confidence in final diagnostic determinations, yielding predictable results through the use of known ensemble classifier and voting diagnostic techniques.
Regarding claim 43, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein a confidence-weighted combiner assigns weights to diagnostic outputs based on operational performance and subject-specific measures, and to compute a confidence-weighted consensus as the unified final diagnosis (Wall [0354] “In at least some instances, available training data may be skewed towards individuals diagnosed with a specific developmental disorder. In such instances, the training data may produce an assessment model reflecting that sample bias, such that the model assumes that subjects are at risk for the specific developmental disorder unless there is a strong case to be made otherwise. An assessment model incorporating such a particular sample bias can have less than ideal performance in generating predictions of new or unclassified data, since the new data may be drawn from a subject population which may not comprise a sample bias similar to that present in the training data. To reduce sample bias in constructing an assessment model using skewed training data, sample weighting may be applied in training the assessment model. Sample weighting can comprise lending a relatively greater degree of significance to a specific set of samples during the model training process. For example, during model training, if the training data is skewed towards individuals diagnosed with autism, higher significance can be attributed to the data from individuals not diagnosed with autism (e.g., up to 50 times more significance than data from individuals diagnosed with autism). Such a sample weighting technique can substantially balance the sample bias present in the training data, thereby producing an assessment model with reduced bias and improved accuracy in classifying data in the real world. To further reduce the contribution of training data sample bias to the generation of an assessment model, a boosting technique may be implemented during the training process. Boosting comprises an iterative process, wherein after one iteration of training, the weighting of each sample data point is updated. For example, samples that are misclassified after the iteration can be updated with higher significances. The training process may then be repeated with the updated weightings for the training data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to employ a confidence weighted combiner that assigns weights to diagnostic outputs based on operational performance and subject characteristics because Wall teaches weighting techniques, boosting techniques, and assigning differing significance levels to data and classification outputs in order to reduce bias and improve diagnostic accuracy. Wall further teaches updating weighting values based on model performance and classification behavior during iterative training processes. A person of ordinary skill in the art would have been motivated to apply such weighted consensus techniques to combined diagnostic outputs in order to improve reliability, confidence, and predictive accuracy of unified diagnostic determinations, yielding predictable results through the use of known weighted ensemble machine learning techniques.
Regarding claim 44, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein a second-level classifier is trained to receive diagnostic outputs as inputs used to generate the unified final diagnosis (Wall [0347] “One or more machine learning algorithms may be used to construct the assessment model, such as support vector machines that deploy stepwise backwards feature selection and/or graphical models, both of which can have advantages of inferring interactions between features. For example, machine learning algorithms or other statistical algorithms may be used, such as alternating decision trees (ADTree), Decision Stumps, functional trees (FT), logistic model trees (LMT), logistic regression, Random Forests, linear classifiers, or any machine learning algorithm or statistical algorithm known in the art. One or more algorithms may be used together to generate an ensemble method, wherein the ensemble method may be optimized using a machine learning ensemble meta-algorithm such as a boosting (e.g., AdaBoost, LPBoost, TotalBoost, BrownBoost, MadaBoost, LogitBoost, etc.) to reduce bias and/or variance. Once an assessment model is derived from the training data, the model may be used as a prediction tool to assess the risk of a subject for having one or more developmental disorders. Machine learning analyses may be performed using one or more of many programming languages and platforms known in the art, such as R, Weka, Python, and/or Matlab, for example.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to employ a second-level classifier trained to receive diagnostic outputs as inputs for generating a unified final diagnosis because Wall teaches the use of ensemble machine learning methods and ensemble meta algorithms, including boosting techniques, for combining outputs from multiple machine learning algorithms to improve predictive performance. A person of ordinary skill in the art would have recognized that such ensemble and meta classification techniques inherently involve higher level classifiers receiving outputs from underlying classifiers as inputs to generate a final prediction. A PHOSITA would have been motivated to utilize such second level classification architectures in order to improve diagnostic accuracy, reduce classification bias and variance, and increase robustness of unified diagnostic determinations, yielding predictable results through the application of known ensemble machine learning techniques.
Regarding claim 45, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the first data analytics process module and/or the second data analytics process module comprises a hierarchy of condition-specific integrators, each configured to confirm or exclude diagnoses for a respective medical condition (Wall [0633] “The scalar outputs from each of the three classification algorithms are passed as inputs into a second stage combinatorial classification model, which is trained independently on 350 historical data submissions collected in clinical studies. This combinatorial model is probabilistic in nature and is trained to take into account the covariance matrix between all three individual module classifiers. It outputs a single scalar value that represents a combined output of all three modules, and its output is then compared to preset thresholds in order to produce a categorical outcome that can be considered a determination of whether the child is Positive for ASD or Negative for ASD.”, Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, and Wall [0420] “FIG. 14 is a non-limiting operational flow of the model output combining step depicted in FIG. 13. As shown in FIG. 14, a combiner module 1400 can collect the outputs from multiple assessment models 1410, 1420, 1430, and 1440, which are received by a model combinatory or combinatorial model 1450. The combinatorial model can employ simple rule-based logic to combine the outputs, which can be numerical scores. Alternatively, the combinatorial model can use more sophisticated combinatorial techniques such as logistic regression, probabilistic modeling, discriminative modeling, or any other combinatorial technique as known in the art. The combinatorial model can also rely on context to determine the best way to combine the model outputs. For example, it can be configured to trust the questionnaire-based model output only in a certain range, or to defer to the video-based model otherwise. In another case, it can use the questionnaire-based model output more significantly for younger subjects than older ones. In another case, it can exclude the output of the video-based model for female subjects, but include the video-based model for male subjects.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to implement condition specific integrators for confirming or excluding diagnostic determinations because Wall teaches combining outputs from multiple assessment models using staged combinatorial models, rule based logic, probabilistic modeling, and context dependent selection or exclusion of classifier outputs. Wall further teaches selectively relying upon or excluding certain model outputs depending on subject characteristics and diagnostic context in order to improve classification accuracy. A person of ordinary skill in the art would have been motivated to organize such combinatorial diagnostic processing into hierarchical condition integration logic in order to improve diagnostic confirmation, reduce erroneous classifications, and tailor diagnostic determinations to particular medical conditions, yielding predictable results through the use of known multi model ensemble and classifier integration techniques.
Regarding claim 46, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach configured to initiate additional actions are initiated including physician review, acquisition of additional data, or further diagnostic testing (Wall [0298] “The diagnostic data may comprise an evaluation for the subject, the evaluation selected from the group consisting of an inconclusive determination and a categorical determination in response to data received from the subject. The therapeutic module may comprise one or more models built using machine learning or statistical modeling based on at least a portion the subject population to determine and output the personal therapeutic treatment plan of the subject. The diagnostic module may be configured to receive updated subject data from the subject in response to therapy of the subject and generate updated diagnostic data from the subject.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to configure the medical diagnostic system of Wall to initiate acquisition of additional data following a diagnostic determination because Wall teaches receiving updated subject data and generating updated diagnostic data in response to continued subject evaluation and therapy. A person of ordinary skill in the art would have been motivated to obtain additional patient information after an initial diagnostic assessment in order to improve diagnostic accuracy, refine subsequent diagnostic determinations, and enhance treatment planning, yielding predictable results through the use of known iterative medical diagnostic and patient monitoring techniques.
Regarding claim 47, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the unified final diagnosis triggers, prompting targeted follow-up questions, initiating therapy delivery, transmitting an emergency alert, or routing the subject for physician evaluation (Wall [0490] “If the digital personalized medicine device predicts that the user is likely to have a diagnosable condition (e.g. Autism Spectrum Disorder), then a therapy module can provide a behavioral treatment (2630) which can comprise behavioral interventions; prescribed activities or trainings; interventions with medical devices or other therapeutics for specific durations or, at specific times or instances. As the subject undergoes the therapy, data (e.g. passive data and diagnostic question data) can continue to be collected to perform follow-up assessments, to determine for example, whether the therapy is working.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to configure the medical diagnostic system of Wall such that a unified final diagnosis initiates therapy delivery because Wall teaches that when the system predicts a diagnosable condition, a therapy module provides behavioral treatment and continues collecting follow up assessment data. A person of ordinary skill in the art would have been motivated to automatically initiate therapy responsive to diagnostic determinations in order to improve treatment responsiveness, patient monitoring, and continuity of care, yielding predictable results through the use of known automated diagnostic and therapeutic systems.
Regarding claim 48, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein the first data analytics process module and/or the second data analytics process module comprises a plurality of machine learning models trained on different subsets of databases (Wall [0633] “The scalar outputs from each of the three classification algorithms are passed as inputs into a second stage combinatorial classification model, which is trained independently on 350 historical data submissions collected in clinical studies. This combinatorial model is probabilistic in nature and is trained to take into account the covariance matrix between all three individual module classifiers. It outputs a single scalar value that represents a combined output of all three modules, and its output is then compared to preset thresholds in order to produce a categorical outcome that can be considered a determination of whether the child is Positive for ASD or Negative for ASD.”, Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, Wall [0404] “FIG. 12 is an operational flow of a model training module as described herein. As shown in FIG. 12, a method 1200 is provided for using machine learning to train an assessment model and tune its configuration parameters optimally. Multiple machine learning predictive models can be trained and tuned using the method 1200, each using datasets prepared offline and comprising a representative sample of a standardized clinical instrument such as ADI-R, ADOS, or SRS. Models can also be trained using datasets comprising data other than clinical instruments, such as demographic data. The model training module pre-processes diagnostic data from a plurality of subjects using machine learning techniques at 1210. Datasets can be pre-processed using well-established machine learning techniques such as data cleaning, filtering, aggregation, imputation, normalization, and other machine learning techniques as known in the art.” and Wall [0431] “Additional datasets may be obtained from large archival data repositories as described herein, such as the Autism Genetic Resource Exchange (AGRE), Boston Autism Consortium (AC), Simons Foundation, National Database for Autism Research, and the like. Alternatively or in combination, additional datasets may comprise mathematically simulated data, generated based on archival data using various simulation algorithms. Alternatively or in combination, additional datasets may be obtained via crowd-sourcing, wherein subjects self-administer the assessment procedure as described herein and contribute data from their assessment. In addition to data from the self-administered assessment, subjects may also provide a clinical diagnosis obtained from a qualified clinician, so as to provide a standard of comparison for the assessment procedure.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to employ a plurality of machine learning models trained on different subsets of databases because Wall teaches training multiple predictive and assessment models using different datasets, including clinical instrument datasets, demographic datasets, archival repositories, and crowd sourced datasets. Wall further teaches combining outputs from independently trained classifiers to generate unified diagnostic determinations. A person of ordinary skill in the art would have been motivated to utilize multiple models trained on differing subsets of data in order to improve diagnostic robustness, increase predictive accuracy, and reduce bias associated with individual datasets, yielding predictable results through the use of known ensemble machine learning training techniques.
Regarding claim 49, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein a syndrome is provisionally diagnosed when a predetermined portion of definitional elements of the syndrome are satisfied by the first data, the second data, and/or the third data (Harris [0056] “As the three data input modules collect data, the diagnosis module can constantly update the running score for each diagnosis. These values can be used to determine the next line of questions and assess if the user has met enough of the criteria to be diagnosed for any particular disease. For example, a particular datum (e.g., blood glucose level, blood type) can affect the probability of n diagnoses to varying degrees and can be stored as a vector of probability influence weights [1.0, −0.5, 0, . . . , pn] that can be added to update a plurality of n diagnoses. The actual weights can vary between implementations. The probability influence weights can be stored in one or more databases and/or data structures, e.g., in a rule/action or IF/THEN format. For example, base diagnosis data store 118 can store a plurality of rules instructing the diagnostic module 102 to modify one or more probabilities based on a detected condition. The running probabilities for a particular patient can be stored in the session diagnosis data store 120.”, Harris [0004] “One aspect of the invention provides a computer-implemented method for diagnosing a condition on a computer including a processor, memory, at least one of a user interface and a communication interface, a user profile module, a question module, a vitals module, and a diagnostic module. The computer-implemented method includes: (a) controlling the processor and one or more of the user interface, the communication interface, the user profile module, the question module, and the vitals module on the computer to receive one or more inputs regarding a subject's symptoms” and Harris [0009] “Another aspect of the invention provides a system for diagnosing a condition. The system includes: a user profile module; a diagnostic module; and a question module. The user profile module includes computer-readable program code including steps for: obtaining one or more user profile inputs regarding a subject's medical status and history; and recording the one or more user profile inputs.”, and Harris [0053] “Diagnostic module results can be based on the three data structure inputs being observed and computed through each of the module's active diagnoses. The resulting tabulations can then be compared to assign a diagnosis ranking, along with distance vectors between diagnoses being used to further separate the active diagnosis selection(s) from the inactive ones.”, Harris [0054] “The diagnostic module data structure can be a mutable array of key/value dictionaries that define each diagnosis based on the three data inputs. The diagnosesVitalMatrix and the diagnosesQuestionMatrix dictionaries can hold the description of a given condition, based on point values being assigned to a user's medical history, her answers to questions, and what the device hardware is reading for their current vitals.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to provisionally diagnose a syndrome when a predetermined portion of definitional elements are satisfied by collected patient data because Harris expressly teaches collecting multiple categories of patient information, comparing the information against diagnostic criteria for particular diseases, assigning weighted diagnostic probabilities, and determining whether enough criteria have been satisfied to diagnose a condition. Harris further teaches rule and probability diagnostic evaluation using symptom, medical history, and physiological input data. A person of ordinary skill in the art would have been motivated to utilize threshold satisfaction of diagnostic criteria in order to improve automated syndrome identification and diagnostic consistency, yielding predictable results through the use of known computer medical diagnostic techniques.
Regarding claim 50, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein a measure of certainty is assigned to a linkage between a keyword or phrase and a potential diagnosis, the measure of certainty comprising at least one of: a numerical value, a proportional value, a frequency-based value, or a specificity-based value (Wall [0358] “At step 410, the prediction module can load a previously saved assessment model, constructed by the training module, from a local memory and/or a remote server configured to store the model. At step 415, the new data is fitted to the assessment model to generate a predicted classification of the subject. At step 420, the module can check whether the fitting of the data can generate a prediction of one or more specific disorders (e.g., autism, ADHD, etc.) within a confidence interval exceeding a threshold value, for example within a 90% or higher confidence interval, for example 95% or more. If so, as shown in step 425, the prediction module can output the one or more developmental disorders as diagnoses of the subject or as disorders for which the subject is at risk. The prediction module may output a plurality of developmental disorders for which the subject is determined to at risk beyond the set threshold, optionally presenting the plurality of disorders in order of risk. The prediction module may output one developmental disorder for which the subject is determined to be at greatest risk. The prediction module may output two or more development disorders for which the subject is determined to risk with comorbidity. The prediction module may output determined risk for each of the one or more developmental disorders in the assessment model. If the prediction module cannot fit the data to any specific developmental disorder within a confidence interval at or exceeding the designated threshold value, the prediction module may determine, in step 430, whether there are any additional features that can be queried. If the new data comprises a previously-collected, complete dataset, and the subject cannot be queried for any additional feature values, “no diagnosis” may be output as the predicted classification, as shown in step 440.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to assign a measure of certainty to associations between analyzed patient information and potential diagnoses because Wall teaches generating diagnostic predictions associated with confidence intervals, threshold values, and quantified risk determinations for multiple disorders. Wall further teaches ranking potential diagnoses according to determined risk levels and outputting diagnostic confidence measures based on fitted assessment data. A person of ordinary skill in the art would have been motivated to utilize numerical or proportional certainty measures in connection with diagnostic associations in order to improve diagnostic reliability, communicate confidence levels, and support automated medical assessment decisions, yielding predictable results through the use of known probabilistic machine learning diagnostic techniques.
Regarding claim 51, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach wherein multiple matching records are ranked or weighted based on degree of similarity or dissimilarity to the first data, the second data, and/or the third data to generate a confidence measure for the unified final diagnosis (Wall [0463] “FIG. 23B illustrates a detailed diagram of diagnosis module 2332. The diagnosis module 2332 comprises a test administration module 2342 that generates tests and corresponding instructions for administration to a subject. The diagnosis module 2332 also comprises a subject data receiving module 2344 in which subject data are received, such as test results; caregiver feedback; meta-data from patient and caregiver interactions with the device; and video, audio, and gaming interactions with the device, for example. A subject assessment module 2346 generates a diagnosis of the subject based on the data from subject data receiving module 2344, as well as past diagnoses of the subject and of similar subjects. A machine learning module 2348 assesses the relative sensitivity of each input to the diagnosis to determine which types of measurement provide the most information regarding a patient's diagnosis. These results can be used by test administration module 2342 to provide tests which most efficiently inform diagnoses and by subject assessment module 2346 to apply weights to diagnosis data in order to improve diagnostic accuracy and consistency. Diagnostic data relating to each treated patient are stored, for example in a database, to form a library of diagnostic data for pattern matching and machine learning. A large number of subject profiles can be simultaneously stored in such a database, for example 10,000 or more.”, and Wall [0166] “The method may further comprise: combining the numerical scores for each of the preliminary subset of assessment models to generate a combined preliminary output score; and mapping the combined preliminary output score to a categorical determination or to an inconclusive determination as to the presence or absence of the cognitive function attribute in the subject, wherein the ratio of inconclusive to categorical determinations can be adjusted.”, and Wall [0167] “The method may further comprise employing rule-based logic or combinatorial techniques for combining the numerical scores for each of the preliminary subset of assessment models and for combining the numerical scores for each of the additional assessment model”, and
Harris [0053] “Diagnostic module results can be based on the three data structure inputs being observed and computed through each of the module's active diagnoses. The resulting tabulations can then be compared to assign a diagnosis ranking, along with distance vectors between diagnoses being used to further separate the active diagnosis selection(s) from the inactive ones.”, Harris [0054] “The diagnostic module data structure can be a mutable array of key/value dictionaries that define each diagnosis based on the three data inputs. The diagnosesVitalMatrix and the diagnosesQuestionMatrix dictionaries can hold the description of a given condition, based on point values being assigned to a user's medical history, her answers to questions, and what the device hardware is reading for their current vitals.” Harris [0056] “As the three data input modules collect data, the diagnosis module can constantly update the running score for each diagnosis. These values can be used to determine the next line of questions and assess if the user has met enough of the criteria to be diagnosed for any particular disease. For example, a particular datum (e.g., blood glucose level, blood type) can affect the probability of n diagnoses to varying degrees and can be stored as a vector of probability influence weights [1.0, −0.5, 0, . . . , pn] that can be added to update a plurality of n diagnoses. The actual weights can vary between implementations. The probability influence weights can be stored in one or more databases and/or data structures, e.g., in a rule/action or IF/THEN format. For example, base diagnosis data store 118 can store a plurality of rules instructing the diagnostic module 102 to modify one or more probabilities based on a detected condition. The running probabilities for a particular patient can be stored in the session diagnosis data store 120.”,).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to rank or weight multiple matching records based on similarity or dissimilarity to collected patient data in order to generate a confidence measure for a unified final diagnosis because Wall teaches comparing subject data against databases of similar subject profiles, applying weights to diagnostic data, and using pattern matching and machine learning to improve diagnostic accuracy and consistency. Harris further teaches diagnosis ranking, distance vectors between diagnoses, and weighted probability scoring based on patient inputs and diagnostic criteria. A person of ordinary skill in the art would have been motivated to combine such ranking, weighting, and similarity diagnostic techniques in order to improve confidence estimation and reliability of automated diagnostic determinations, yielding predictable results through the use of known machine learning and probabilistic diagnostic analysis methods.
Claims 29-32 are rejected under 35 U.S.C. 103 as being unpatentable over Wall et al. (U.S. Patent Publication 2021/0133509 A1), referred to hereinafter as Wall, in view of Ungureanu et al. (Ungureanu et al., HoloLens 2 Research Mode as a Tool for Computer Vision Research, 2020, arXiv, pages 1-7 (Year: 2020)), referred to hereinafter as Ungureanu, Harris et al. (U.S. Patent Publication 2019/0392952 A1), referred to hereinafter as Harris, and Leuthardt et al. (U.S. Patent Publication 2021/0361948A1), referred to hereinafter as Leuthardt, and further in view of Pierce (U.S. Patent Publication 2007/0294105 A1), referred to hereinafter as Pierce.
Regarding claim 29, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above, and further teach further comprising one or more peripherals, the peripherals including one or more sensors and/or one or more stimulators (Wall [0445] “The diagnostic and therapeutic modules as disclosed herein can receive data from a plurality of sources, such as data acquired from the group consisting of genetic data, floral data, a sleep sensor, a wearable anklet sleep monitor, a booty to monitor sleep, and eye tracking of the subject.”, and Wall [0309] “In some embodiments of the methods and devices described herein, a method or device for delivering a virtual or augmented reality based digital therapy receives inputs and in some of these embodiments an input affects how a virtual or augmented reality is presented to an individual receiving therapy. In some embodiments, an input is received from a camera and/or microphone of a computing device used to deliver the digital therapy. In some instances, an inputs is received from a sensor such as, for example, a motion sensor or a vital sign sensor.”).
Wall, Ungureanu, Harris, and Leuthardt fail to explicitly teach positioned on one or both ankles and/or one or both wrists of the subject, the peripherals including adhesive-and/or having a circular shape to remain frictionally attached to the subject when wrapped around a limb of the patient.
Pierce teaches positioned on one or both ankles and/or one or both wrists of the subject, the peripherals including adhesive-and/or having a circular shape to remain frictionally attached to the subject when wrapped around a limb of the patient (Pierce [0039] “Wristband WB includes a remote actuator, which could be a transmitter and/or transceiver, generally 90, attached to a band 92, which is adjustable to fit the size of the wrist of a patient. Band 92 includes adjustment openings 94 which are received by a fastener, generally 96, for attachment of band 92 about the patient's wrist.”, Pierce [0043] “While a wristband WB has been shown in the drawings as including devices for operation of camera C, it is to be understood that such devices could be in other forms other than wristband WB, and could be, for example, identification tags, key fobs, handheld controllers, ankle bracelets, necklaces, keys, etc., if desired.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate limb mounted wearable peripherals, such as wristbands or ankle mounted devices including sensors and/or stimulators, into the medical diagnostic system of Wall because Wall teaches wearable monitoring devices and sensor based therapeutic inputs, while Pierce teaches wearable wristbands and ankle bracelets configured for attachment around a patient’s limb. A person of ordinary skill in the art would have been motivated to utilize such wearable peripheral devices in order to improve patient monitoring, collect additional physiological data, and provide portable patient interaction interfaces, yielding predictable results through the use of known wearable medical monitoring and communication devices.
Regarding claim 30, Wall, Ungureanu, Harris, and Leuthardt teach the invention in claim 21, as discussed above.
Wall, Ungureanu, Harris, and Leuthardt fail to explicitly teach further comprising a plurality of fixed equipment, wherein-each of the plurality of fixed equipment is fixed to a respective one of a vehicle, a building, a medical transport, and a furniture.
Pierce teaches further comprising a plurality of fixed equipment, wherein-each of the plurality of fixed equipment is fixed to a respective one of a vehicle, a building, a medical transport, and a furniture (Pierce [0045] “FIG. 4 illustrates another embodiment in which camera unit C can be used. In this embodiment, camera unit C is illustrated being used within a patient compartment, generally 110, of an ambulance, generally 112, and is positioned for viewing the healthcare services provided by healthcare provider HCP as patient PT is supported on a stretcher, gurney, or other patient transport device, generally 114.”.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to further configure the medical diagnostic system of Wall to include fixed equipment associated with a vehicle and/or medical transport as taught by Pierce, because Pierce teaches camera equipment fixed within an ambulance patient compartment and positioned relative to a stretcher or patient transport device for monitoring and supporting patient care. A person of ordinary skill in the art would have been motivated to incorporate such fixed medical monitoring equipment into Wall’s wearable diagnostic and therapeutic system in order to facilitate patient monitoring, data acquisition, and diagnostic evaluation during patient transport and emergency care situations, yielding predictable results through the use of known fixed medical equipment configurations in mobile healthcare environments.
Regarding claim 31, Wall, Ungureanu, Harris, Leuthardt, and Pierce teach the invention in claim 30, as discussed above, and further teach wherein at least one of the plurality of fixed equipment is fixed to an ambulance and includes a third person video camera, a video console, one or more speakers, and a microphone (Pierce [0045] “FIG. 4 illustrates another embodiment in which camera unit C can be used. In this embodiment, camera unit C is illustrated being used within a patient compartment, generally 110, of an ambulance, generally 112, and is positioned for viewing the healthcare services provided by healthcare provider HCP as patient PT is supported on a stretcher, gurney, or other patient transport device, generally 114.”, and Pierce [0036] “Camera C may also emit a continuous and/or intermittent tone through speaker 80 when in operation.”, and Pierce [0033] “Camera C includes a receptacle, or slot, 60 for receipt of an electronic storage media device, generally SM, such as a compact disc (CD), digital video disc (DVD), mini DVD, memory card, memory cassette, flash drive, vault, etc., for storing digital data created by camera C. Such digital data could include video images as input from lens 20, digital audio signals as input from microphone 22, input from card reader input from reader 44, input from control input from pendent 30, input from bracelet WB, and/or input from a data entry device, such as a touchpad data entry and an input interface, or storage device, 70 manipulatable by the patient and/or the healthcare provider”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate the ambulance mounted audiovisual monitoring equipment of Pierce into the medical diagnostic system of Wall in order to facilitate real time patient monitoring, recording, and communication during emergency transport situations. Pierce teaches fixed camera equipment within an ambulance patient compartment including a camera, speaker, microphone, and associated recording and interface components for capturing and communicating patient care information. A person of ordinary skill in the art would have been motivated to integrate such known ambulance audiovisual monitoring components into Wall’s wearable diagnostic platform to improve remote observation, emergency assessment, and transport patient diagnostics, yielding predictable results using known medical transport monitoring technology.
Regarding claim 32, Wall, Ungureanu, Harris, Leuthardt, and Pierce teach the invention in claim 30, as discussed above, and further teach wherein at least one of the plurality of fixed equipment is fixed to a medical transport used to move a patient in and out of the ambulance vehicle (Pierce [0045] “FIG. 4 illustrates another embodiment in which camera unit C can be used. In this embodiment, camera unit C is illustrated being used within a patient compartment, generally 110, of an ambulance, generally 112, and is positioned for viewing the healthcare services provided by healthcare provider HCP as patient PT is supported on a stretcher, gurney, or other patient transport device, generally 114.” and Pierce [0046] “In one preferred embodiment, camera C attached to cart 100 and used in patient compartment 110 includes a wide angle and/or a fish eye lens for an increased filed of view for camera C. Additionally, such a wide angle and/or a fish eye lens could also be used when camera C is used for documentation of a relatively large space, such as an entire room, operating room, etc. for simultaneously recording numerous medical care actions.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate the transport mounted monitoring equipment of Pierce into the medical diagnostic system of Wall because Pierce teaches camera equipment attached to a cart, stretcher, gurney, or other patient transport device used in an ambulance environment for monitoring patient care activities. A person of ordinary skill in the art would have been motivated to integrate such transport mounted diagnostic equipment into Wall’s medical diagnostic platform in order to facilitate continuous patient monitoring and data acquisition during patient transport into and out of emergency vehicles, yielding predictable results through the use of known mobile medical transport monitoring systems.
Response to Arguments
Applicant’s arguments and amendments, see Remarks/Amendments submitted on 03/02/2026 with respect to the rejection of the claims have been carefully considered and is addressed below.
Claim Interpretation
The limitations in claims 21, 45, and 48 are interpreted under 35 U.S.C. 112(f) as covering the corresponding structure, algorithms, and acts described in the specification and equivalents thereof. The specification discloses corresponding structure for performing the recited functions, including multiple diagnostic processing modules and integration logic executed by the processor for analyzing patient data, generating diagnostic outputs, and integrating the outputs to determine a unified diagnosis, as described throughout the specification and associated figures.
Claim Rejections - 35 USC § 112(a)
Applicant’s arguments have been considered and are persuasive with respect to claims 21, 24, 28-33, 35, 37, 40-43, and 45-51, as amended, because the cited portions of the specification reasonably convey possession of the amended subject matter, including the voting and agreement criteria, weighted diagnostic processing, condition specific integrators, triggered follow up actions, and machine learning models trained on different databases. Accordingly, the rejection under 35 U.S.C. § 112(a) is withdrawn with respect to claims 21, 24, 28-33, 35, 37, 40-43, and 45-51. However, the rejection is maintained as to claim 44 because the specification does not reasonably convey possession of a “second-level classifier” trained to receive diagnostic outputs as inputs for generation of the unified final diagnosis, as presently claimed.
Claim Rejections - 35 USC § 112(b)
In view of the amendments clarifying the limitations, the rejections under 35 U.S.C. § 112(b) of claims 21, 24, 28-33, 35, 37, and 40-51 are withdrawn.
Claim Rejections - 35 USC § 101
Applicant’s arguments have been fully considered but are not persuasive. Although claim 21 recites a headset and associated hardware components, the claim recites an abstract idea as a mental process, specifically evaluating patient information, generating diagnostic conclusions from the information, comparing diagnostic conclusions, and determining a final diagnosis. The additional recited headset components, including cameras, microphones, speakers, visor, and sensory stimulators, are recited at a high level of generality and merely perform conventional functions of collecting information, outputting information, and providing feedback based on the diagnostic conclusion. The recitation of processor executed “data analytics process modules” also does not remove the claim from the mental processes grouping because the claim does not recite a specific improvement in computer functionality or a particular technological implementation for performing the diagnostic analysis.
Applicant’s arguments under Step 2A, Prong Two are also not persuasive. The claims do not improve the functioning of the headset, processor, sensors, or any other computer technology itself, nor do the claims recite a specific technological solution to a technological problem. Instead, the claims use generic computing and wearable hardware components as tools to implement the abstract idea of evaluating patient data to determine a diagnosis. The “cross-validation” of outputs merely represents comparing and integrating multiple diagnostic conclusions, which is part of the abstract mental evaluation process. Unlike McRO, the present claims do not recite a specific set of rules that improve computer animation technology, and unlike Enfish, the claims do not improve the structure or operation of a computer system itself. Further, CardioNet involved a specific improvement in cardiac monitoring technology for detecting atrial fibrillation, whereas the present claims broadly recite generic diagnostic analysis using multiple analytic modules without reciting a specific technological improvement in signal processing, sensing, or computer operation.
Applicant’s arguments under Step 2B are likewise unpersuasive. The additional elements, individually and in combination, amount to no more than generic wearable input and output components and generic computer processing elements performing their ordinary and conventional functions of data collection, analysis, display, and feedback. The claim does not recite a non-conventional arrangement of hardware or a specific technological mechanism that improves diagnostic hardware or computer functionality. Instead, the claim automates the abstract process of evaluating patient information and determining a diagnosis using conventional computer and wearable device technology. Accordingly, the claims do not recite significantly more than the abstract idea and remain ineligible under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
Applicant’s arguments have been fully considered but are not persuasive. The rejection under 35 U.S.C. § 103 is maintained because the cited references, when considered collectively and in view of the knowledge of one of ordinary skill in the art, teach or suggest the limitations of the pending claims. The rejection relies on the combined teachings of Wall, Ungureanu, Harris, Leuthardt, and Pierce.
Applicant argues that Wall is “silent regarding the headset” and the recited augmented reality visor, speakers, microphones, inward-facing cameras, outward-facing cameras, and sensory stimulation features. However, the rejection specifically relies on the combined teachings of Wall and Ungureanu to address these limitations. Wall teaches digital therapeutic and diagnostic systems implemented in virtual or augmented reality environments using wearable devices having cameras, microphones, displays, speakers, and sensor inputs. Ungureanu further teaches the HoloLens 2 augmented reality headset platform including inward and outward sensor streams, eye tracking, audio/video capture, and wearable augmented reality functionality. A person of ordinary skill in the art would have been motivated to implement Wall’s diagnostic and therapeutic framework on known augmented reality headset platforms such as HoloLens 2 in order to improve portability, multimodal sensor acquisition, and immersive diagnostic feedback, yielding predictable results through the combination of known wearable AR technologies with known digital diagnostic systems.
Applicant further argues that Wall and Harris fail to teach communicating the unified diagnosis through the visor and fail to teach sensory stimulation delivered based on the diagnosis. This argument is unpersuasive because Wall teaches presenting diagnostic and therapeutic outputs through displays, speakers, augmented reality environments, and real-time user feedback interfaces. Additionally, Leuthardt teaches stimulation delivery responsive to detected patient conditions and physiological signals, including stimulation delivered in conjunction with augmented reality headset systems. It would have been obvious to combine these teachings in order to provide responsive therapeutic intervention and diagnostic feedback within an integrated wearable diagnostic platform, particularly where each reference is directed toward improving computerized diagnosis, patient monitoring, and therapeutic delivery.
Applicant’s argument that a person of ordinary skill in the art would not combine Wall’s machine learning diagnostic architecture with Harris’s symptom based diagnostic analysis is likewise not persuasive. Both Wall and Harris are directed to computerized medical diagnostic systems that analyze patien -derived data to generate diagnostic determinations. Wall teaches machine learning and combinatorial diagnostic models, while Harris teaches rule based and symptom based diagnostic evaluation using multiple patient inputs and weighted diagnostic scoring. A person of ordinary skill in the art would have been motivated to combine Wall’s machine learning diagnostic framework with Harris’s structured symptom analysis techniques in order to improve diagnostic reliability, cross validation of diagnostic outputs, and confidence in automated diagnostic determinations. This combination represents the predictable use of known diagnostic analysis techniques according to their established functions. When the references are considered together as proposed in the rejection, the combined teachings render the claimed subject matter obvious to one of ordinary skill in the art at the time of the invention. Accordingly, the rejection under 35 U.S.C. § 103 is maintained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure.
McLaughlin et al. (U.S. Publication 2018/0193589 A1) teaches a health and wellness system that creates and dynamically adjusts an immersive environment for an individual using sensor measured physiological and environmental data and analytics based control logic that are tailored to predefined wellness priorities.
Zhu et al. (CN Publication CN111240874A) teaches data processing method and device that compare a user’s diagnosis result with a machine’s diagnosis result for the same event, determine condition data based on the machine’s analysis when results differ, and output that condition data for user reference.
Hsiao et al. (U.S. Publication 2016/0078622 A1) teaches a wearable disease diagnosis apparatus that captures and analyzes pixel variations in sequential images, magnifies specific frequency variations, and overlays the enhanced output onto the current field of view for real time display.
Rufo et al. (U.S. Publication 2018/0342329 A1) teaches an improved home automation system for senior and dementia care that integrates health equipment, smart home devices, sensors, and distributed CPU-based video and speech processing to enable responsive monitoring, medication management, and peer to peer communication with reduced latency.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYRA R LAGOY whose telephone number is (703)756-1773. The examiner can normally be reached Monday - Friday, 8:00 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kambiz Abdi can be reached at (571)272-6702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.L./Examiner, Art Unit 3685
/KAMBIZ ABDI/Supervisory Patent Examiner, Art Unit 3685