DETAILED ACTION
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
2. Receipt of Applicant’s Amendment filed on 06/23/2026 is acknowledged. The amendment includes the amending of the specification, the amending of claims 1, 12, and 19, the cancellation of claim 10, and the addition of claim 21.
Terminal Disclaimer
3. The terminal disclaimer filed on 06/23/2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of U.S. Patent 12,417,232, U.S. Patent Application 19/075929, and U.S. Patent Application 19/076030 has been reviewed and is accepted. The terminal disclaimer has been recorded.
Specification
4. The objection raised in the Office Action mailed on 03/26/2026 has been overcome by applicant’s amendment received on 06/24/2026.
Double Patenting
5. The rejections raised in the Office Action mailed on 03/26/2026 have been overcome by applicant’s submission of a Terminal Disclaimer received on 06/24/2026.
Claim Rejections - 35 USC § 101
6. The rejections raised in the Office Action mailed on 03/26/2026 has been overcome by applicant’s amendment received on 06/24/2026.
Claim Rejections - 35 USC § 103
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
9. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
10. Claims 1, 5, 9, 11-12, 16, 18-19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Salazar et al. (U.S. PGPUB 2018/0218126), in view of Chen et al. (CN104933839A (Machine Translation Provided)).
11. Regarding claims 1, 12, and 19, Salazar teaches a computer-implemented method, computer system, and tangible, non-transitory computer-readable medium comprising:
A) acquiring at least two data streams of a patient by using one or more sensors (Paragraph 19);
B) generating at least one entity-feature-graph based on the acquired at least two data streams of the patient (Paragraphs 19, 27, 43, and 45);
C) selecting at least one intervention based on the generated entity-feature-graph, a trained graph classification model, and information related to the patient (Paragraphs 19, 40, and 45); and
D) outputting an information of the selected intervention to a user (Paragraphs 19 and 45).
The examiner notes that Salazar teaches “acquiring at least two data streams of a patient by using one or more sensors” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19). The examiner further notes that multiple “streams” of patient data (including transcripts (i.e. text), audio, video, images, etc)) are acquired via the use of one or more “sensors” (such as a camera for a video call for example). The examiner further notes that Salazar teaches “generating at least one entity-feature-graph based on the acquired at least two data streams of the patient” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19), “The training set database 245 stores one or more patient complaint-symptom datasets that are used to generate the knowledge graph 225. These datasets are further described in conjunction with FIG. 6. In some embodiments, the training set database 245 is combined with the patient information database 205” (Paragraph 27), “If the medical triage assistance system 200 receives a correction to the symptoms from the nurse, it can use that feedback to rebalance the connections of the knowledge graph 225 and re-score the multi-class classifier. A nurse can provide a correction by selecting the correct symptom that should have been identified, such as through a multiple-choice interface. The knowledge graph 225 is recomputed based on the correction and the recomputed knowledge graph 225 replaces the current knowledge graph 225 once a threshold improvement in performance is reached. Previous versions of the knowledge graph 225 may be stored to allow for analysis of historical data and models” (Paragraph 43), and “The medical triage assistance system 200 may also select 340 one or more specific medical protocols to recommend based on the patient's symptoms. Each medical protocol is based on one or more symptoms and is made up of a series of questions that are designed to differentiate between life-threatening conditions associated with that symptom and less urgent conditions. The medical triage assistance system 200 maps specific medical protocols to the various medical concepts of the knowledge graph 225. This mapping can be manually created, or learned (i.e., as part of the knowledge graph 225) based on existing patient cases. The medical triage assistance system 200 selects 340 the medical protocols based on confidence scoring. The medical triage assistance system 200 may present the selected 340 protocol(s) to the nurse as a recommendation and wait for approval or correction before proceeding” (Paragraph 45). The examiner further notes that an output recommendation (i.e. intervention) is based on a generated knowledge graph (i.e. the claimed undefined entity-feature graph in the broadest reasonable interpretation) that is generated “based” on received multiple streams of patient data. The examiner further notes that Salazar teaches “selecting at least one intervention based on the generated entity-feature-graph, a trained graph classification model, and an information related to the patient” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19), “The knowledge graph 225 may also be traversed based on probabilistic modeling and detection of anchors and triplets, or deep Kalman filters, including deep learning and probabilistic modeling” (Paragraph 40), “If the medical triage assistance system 200 receives a correction to the symptoms from the nurse, it can use that feedback to rebalance the connections of the knowledge graph 225 and re-score the multi-class classifier. A nurse can provide a correction by selecting the correct symptom that should have been identified, such as through a multiple-choice interface. The knowledge graph 225 is recomputed based on the correction and the recomputed knowledge graph 225 replaces the current knowledge graph 225 once a threshold improvement in performance is reached. Previous versions of the knowledge graph 225 may be stored to allow for analysis of historical data and models” (Paragraph 43), and “The medical triage assistance system 200 may also select 340 one or more specific medical protocols to recommend based on the patient's symptoms. Each medical protocol is based on one or more symptoms and is made up of a series of questions that are designed to differentiate between life-threatening conditions associated with that symptom and less urgent conditions. The medical triage assistance system 200 maps specific medical protocols to the various medical concepts of the knowledge graph 225. This mapping can be manually created, or learned (i.e., as part of the knowledge graph 225) based on existing patient cases. The medical triage assistance system 200 selects 340 the medical protocols based on confidence scoring. The medical triage assistance system 200 may present the selected 340 protocol(s) to the nurse as a recommendation and wait for approval or correction before proceeding” (Paragraph 45). The examiner further notes that a selected intervention is based off of a generated knowledge graph (i.e. the claimed undefined entity feature graph in the broadest reasonable interpretation) that is traversed/mapped via a learned process (i.e. the use of a “trained graph classification model” (which is undefined in the claims) in the broadest reasonable interpretation). Such a generated knowledge graph is also based off of information related to a patient. The examiner further notes that Salazar teaches “outputting an information of the selected intervention to a user” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19) and “The medical triage assistance system 200 may also select 340 one or more specific medical protocols to recommend based on the patient's symptoms. Each medical protocol is based on one or more symptoms and is made up of a series of questions that are designed to differentiate between life-threatening conditions associated with that symptom and less urgent conditions. The medical triage assistance system 200 maps specific medical protocols to the various medical concepts of the knowledge graph 225. This mapping can be manually created, or learned (i.e., as part of the knowledge graph 225) based on existing patient cases. The medical triage assistance system 200 selects 340 the medical protocols based on confidence scoring. The medical triage assistance system 200 may present the selected 340 protocol(s) to the nurse as a recommendation and wait for approval or correction before proceeding” (Paragraph 45). The examiner further notes that a selected intervention is output.
Salazar does not explicitly teach:
E) adapting the one or more sensors based on the selected intervention.
Chen, however, teaches “adapting the one or more sensors based on the selected intervention” as “Step 106: After the caller clicks "Yes" to receive the real-time video request, the receiving police officer sees the scene captured by the built-in camera of the smart phone terminal and transmitted in real time through the data transmission network in the image/video display area of the police software interface video. In addition, the receiving police officer can remotely command the alarm person to adjust the shooting angle and distance and take effective measures to deal with the accident through telephone voice communication. At this time, the smartphone terminal is equivalent to a remote pan-tilt for the receiving police officer to use” (Page 07).
The examiner further notes that the secondary reference of Chen teaches the concept of adjusting a shooting angle and distance (i.e. examples of the claimed adapting) of a camera (i.e. sensor). Such an adjustment is “based” off of instructions from emergency personnel (i.e. an example of the claimed selected intervention in the broadest reasonable interpretation) to an accident. The combination would result in the the selected intervention of Salazar to be able to adjust its sensors.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Chen’s would have allowed Salazar’s to provide a method for authorities to be able to remotely pan/tilt sensors, as noted by Chen (Page 07).
Regarding claims 5 and 16, Salazar further teaches a computer-implemented method and computer system comprising:
A) wherein the at least two data streams include at least one data stream including images and at least one data stream including text (Paragraph 19); and
B) wherein the one or more sensors include at least one of a camera, sound recorder presence sensor, a temperature sensor, a sound-level sensor, and a door sensor (Paragraph 19).
The examiner notes that Salazar teaches “wherein the at least two data streams include at least one data stream including images and at least one data stream including text” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19). The examiner further notes that multiple “streams” of patient data (including images and transcripts (i.e. text)) are acquired via the use of one or more “sensors” (such as a camera for a video call for example). The examiner further notes that Salazar teaches “wherein the one or more sensors include at least one of a camera, sound recorder presence sensor, a temperature sensor, a sound-level sensor, and a door sensor” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19). The examiner further notes that multiple “streams” of patient data (including images and transcripts (i.e. text)) are acquired via the use of one or more “sensors” (such as a camera for a video call for example).
Regarding claims 9 and 18, Salazar further teaches a computer-implemented method and computer system comprising:
A) wherein the graph classification model is learned based on training data extracted from historical data streams and previous intervention selection decisions (Paragraphs 19, 40, and 45).
The examiner notes that Salazar teaches “wherein the graph classification model is learned based on training data extracted from historical data streams and previous intervention selection decisions” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19), “The knowledge graph 225 may also be traversed based on probabilistic modeling and detection of anchors and triplets, or deep Kalman filters, including deep learning and probabilistic modeling” (Paragraph 40), “If the medical triage assistance system 200 receives a correction to the symptoms from the nurse, it can use that feedback to rebalance the connections of the knowledge graph 225 and re-score the multi-class classifier. A nurse can provide a correction by selecting the correct symptom that should have been identified, such as through a multiple-choice interface. The knowledge graph 225 is recomputed based on the correction and the recomputed knowledge graph 225 replaces the current knowledge graph 225 once a threshold improvement in performance is reached. Previous versions of the knowledge graph 225 may be stored to allow for analysis of historical data and models” (Paragraph 43), “The medical triage assistance system 200 may also select 340 one or more specific medical protocols to recommend based on the patient's symptoms. Each medical protocol is based on one or more symptoms and is made up of a series of questions that are designed to differentiate between life-threatening conditions associated with that symptom and less urgent conditions. The medical triage assistance system 200 maps specific medical protocols to the various medical concepts of the knowledge graph 225. This mapping can be manually created, or learned (i.e., as part of the knowledge graph 225) based on existing patient cases. The medical triage assistance system 200 selects 340 the medical protocols based on confidence scoring. The medical triage assistance system 200 may present the selected 340 protocol(s) to the nurse as a recommendation and wait for approval or correction before proceeding” (Paragraph 45), and “FIG. 6 illustrates a training phase 600 of the knowledge graph 225, according to one embodiment. The knowledge graph 225 is generated using a patient complaint-symptom dataset comprised of patient case summaries 640. Patients whose patient case summaries 640 are included in the dataset are those who had both a conversation 610 (e.g., chat-based) with a healthcare professional system 130 and an in-person visit with a medical provider. These patients are chosen because the medical provider is able to verify the patient's symptoms and provide treatment during the in-person visit. Each patient case summary 640 in the patient complaint-symptom dataset includes a record of the patient's conversation 610 with the healthcare professional system 130, one or more triage symptoms 620, and one or more observed symptoms 630 from the in-person visit. The triage symptoms 620 and the observed symptoms 630 are both described in healthcare professional-defined medical language. In some embodiments, this medical language is standardized for better consistency across healthcare professionals. The triage symptoms 620 are determined by the healthcare professional (typically a nurse) operating the healthcare professional system 130 based on their conversation with the patient. The observed symptoms 630 are determined based on the observations of a medical provider who saw the patient during the in-person visit. The observed symptoms 630 are considered to be more accurate than the triage symptoms 620 because they are based on the medical provider's direct observation of the patient's symptoms, rather than the patient's description of them via a remote conversation 610” (Paragraphs 49-50). The examiner further notes that a selected intervention is based off of a generated knowledge graph (i.e. the claimed undefined entity feature graph in the broadest reasonable interpretation) that is traversed/mapped via a learned process (i.e. the use of a “trained graph classification model” (which is undefined in the claims) in the broadest reasonable interpretation). Such a learned process is learned based on historical patient streams and interventions from medical professionals.
Regarding claim 11, Salazar further teaches a computer-implemented method comprising:
A) wherein generating the entity-feature-graph based on the acquired at least two data streams comprises: determining, by processing the at least two data streams, a set of event related entities (Paragraphs 19 and 21-23); and
B) constructing, for each entity of the determined set of entities, an associated feature vector of features of the each entity (Paragraphs 19 and 21-23); and
C) generating, based on the determined entities and their respective feature vector, the entity-feature-graph (Paragraphs 19, 21-23, 43, and 52).
The examiner notes that Salazar teaches “wherein generating the entity-feature-graph based on the acquired at least two data streams comprises: determining, by processing the at least two data streams, a set of event related entities” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place. In other embodiments, the medical triage assistance system 200 may receive conversation records after the fact” (Paragraph 19), “The call-response structuring module 210 organizes unstructured conversations (such as conversation records) into call-response units. Call-response units pair questions with corresponding answers to allow the medical triage assistance system 200 to better process the conversation content. For example, a patient's answer alone may omit relevant information that was posed in the preceding question. Call-response units are further described in conjunction with step 320 of FIG. 3 and with FIG. 4” (Paragraph 21), “The medical relevance detection module 215 identifies medically-relevant phrases by tokenizing call-response units (or in some cases, the unstructured conversation), and identifying medically-relevant tokens, such as “pain” and “cough.” The medically-relevant tokens are then mapped back to the call-response units, where they are expanded to medically-relevant phrases” (Paragraph 22), and “The symptom identification module 220 extracts medically-relevant conversation tokens from conversations and uses them to determine medical symptoms by traversing the knowledge graph 225. These tokens are made up of strings (or vectors) explicitly or implicitly derived from the conversation. The tokens may be identified with a type or class of token, such as patient complaints, duration of the complaint, and severity. Patient complaint tokens are words and phrases from mundane language (i.e., from conversations) that directly correspond to symptoms, while duration tokens indicate the duration of a complaint, and severity tokens indicate the severity of a complaint. Tokenization and traversal of the knowledge graph 225 are further discussed in conjunction with FIG. 3” (Paragraph 23). The examiner further notes that ascertaining tokens from multiple streams of a conversation record (which can include images and transcripts) teaches the claimed determination of event-related entities. The examiner further notes that Salazar teaches “constructing, for each entity of the determined set of entities, an associated feature vector of features of the each entity” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place. In other embodiments, the medical triage assistance system 200 may receive conversation records after the fact” (Paragraph 19), “The call-response structuring module 210 organizes unstructured conversations (such as conversation records) into call-response units. Call-response units pair questions with corresponding answers to allow the medical triage assistance system 200 to better process the conversation content. For example, a patient's answer alone may omit relevant information that was posed in the preceding question. Call-response units are further described in conjunction with step 320 of FIG. 3 and with FIG. 4” (Paragraph 21), “The medical relevance detection module 215 identifies medically-relevant phrases by tokenizing call-response units (or in some cases, the unstructured conversation), and identifying medically-relevant tokens, such as “pain” and “cough.” The medically-relevant tokens are then mapped back to the call-response units, where they are expanded to medically-relevant phrases” (Paragraph 22), and “The symptom identification module 220 extracts medically-relevant conversation tokens from conversations and uses them to determine medical symptoms by traversing the knowledge graph 225. These tokens are made up of strings (or vectors) explicitly or implicitly derived from the conversation. The tokens may be identified with a type or class of token, such as patient complaints, duration of the complaint, and severity. Patient complaint tokens are words and phrases from mundane language (i.e., from conversations) that directly correspond to symptoms, while duration tokens indicate the duration of a complaint, and severity tokens indicate the severity of a complaint. Tokenization and traversal of the knowledge graph 225 are further discussed in conjunction with FIG. 3” (Paragraph 23). The examiner further notes that tokens from multiple streams of a conversation record (which can include images and transcripts) are vectors. The examiner further notes that Salazar teaches “generating, based on the determined entities and their respective feature vector, the entity-feature-graph” as “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place. In other embodiments, the medical triage assistance system 200 may receive conversation records after the fact” (Paragraph 19), “The call-response structuring module 210 organizes unstructured conversations (such as conversation records) into call-response units. Call-response units pair questions with corresponding answers to allow the medical triage assistance system 200 to better process the conversation content. For example, a patient's answer alone may omit relevant information that was posed in the preceding question. Call-response units are further described in conjunction with step 320 of FIG. 3 and with FIG. 4” (Paragraph 21), “The medical relevance detection module 215 identifies medically-relevant phrases by tokenizing call-response units (or in some cases, the unstructured conversation), and identifying medically-relevant tokens, such as “pain” and “cough.” The medically-relevant tokens are then mapped back to the call-response units, where they are expanded to medically-relevant phrases” (Paragraph 22), “The symptom identification module 220 extracts medically-relevant conversation tokens from conversations and uses them to determine medical symptoms by traversing the knowledge graph 225. These tokens are made up of strings (or vectors) explicitly or implicitly derived from the conversation. The tokens may be identified with a type or class of token, such as patient complaints, duration of the complaint, and severity. Patient complaint tokens are words and phrases from mundane language (i.e., from conversations) that directly correspond to symptoms, while duration tokens indicate the duration of a complaint, and severity tokens indicate the severity of a complaint. Tokenization and traversal of the knowledge graph 225 are further discussed in conjunction with FIG. 3” (Paragraph 23), “If the medical triage assistance system 200 receives a correction to the symptoms from the nurse, it can use that feedback to rebalance the connections of the knowledge graph 225 and re-score the multi-class classifier. A nurse can provide a correction by selecting the correct symptom that should have been identified, such as through a multiple-choice interface. The knowledge graph 225 is recomputed based on the correction and the recomputed knowledge graph 225 replaces the current knowledge graph 225 once a threshold improvement in performance is reached. Previous versions of the knowledge graph 225 may be stored to allow for analysis of historical data and models” (Paragraph 43), and “The knowledge graph 225 is generated using machine learning techniques. For each patient case summary 640, the conversation 610 is processed as described above in conjunction with FIG. 3—the conversation 610 is organized into call-response units, and medically-relevant phrases are tokenized into words and phrases. An information metric is applied to the tokens to determine which are the most likely to be medically relevant. For example, term frequency—inverse document frequency (tf-idf) can be applied to determine which tokens are present more frequently in the conversation relative to conversations from other patient case summaries in the dataset. The tokens and the triage symptoms from that conversation 610 are represented as vertices of the knowledge graph, and an edge is created between each token and each of the triage symptoms” (Paragraph 52). The examiner further notes that knowledge graphs can be created based off of tokens (which can be vectors).
Regarding claim 21, Salazar does not explicitly teach a computer-implemented method comprising:
A) wherein adapting the one or more sensors includes adjusting a configuration of a monitoring sensor network that includes the one or more sensors based on the selected intervention.
Chen, however, teaches “wherein adapting the one or more sensors includes adjusting a configuration of a monitoring sensor network that includes the one or more sensors based on the selected intervention” as “Step 106: After the caller clicks "Yes" to receive the real-time video request, the receiving police officer sees the scene captured by the built-in camera of the smart phone terminal and transmitted in real time through the data transmission network in the image/video display area of the police software interface video. In addition, the receiving police officer can remotely command the alarm person to adjust the shooting angle and distance and take effective measures to deal with the accident through telephone voice communication. At this time, the smartphone terminal is equivalent to a remote pan-tilt for the receiving police officer to use” (Page 07).
The examiner further notes that the secondary reference of Chen teaches the concept of adjusting a shooting angle and distance (i.e. examples of the claimed adapting of a configuration) of a camera (i.e. sensor). Such an adjustment is “based” off of instructions from emergency personnel (i.e. an example of the claimed selected intervention in the broadest reasonable interpretation) to an accident. Furthermore, because the claimed monitoring sensor network is undefined in the claims, it is interpreted in the broadest reasonable interpretation as simply a network of one or more sensors (See diction of “a monitoring sensor network that includes the one or more sensors”). Thus, the camera of Chen teaches the claimed undefined monitoring sensor network in the broadest reasonable interpretation. The combination would result in the selected intervention of Salazar to be able to adjust its sensors.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Chen’s would have allowed Salazar’s to provide a method for authorities to be able to remotely pan/tilt sensors, as noted by Chen (Page 07).
11. Claims 2, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Salazar et al. (U.S. PGPUB 2018/0218126), in view of Chen et al. (CN104933839A (Machine Translation Provided)) as applied to claims 1, 5, 9, 11-12, 16, 18-19, and 21 above, and in view of Badr et al. (U.S. PGPUB 2018/0324117).
12. Regarding claims 2, 13, and 20, Salazar further teaches a computer-implemented method, computer system, and tangible, non-transitory computer-readable medium comprising:
A) wherein the trained graph classification model is trained by a learning process that considers confidence values assigned to each entity of a set of entities within the entity-feature-graph (Paragraphs 27, 40, 43, and 45).
The examiner notes that Salazar teaches “wherein the trained graph classification model is trained by a learning process that considers confidence values assigned to each entity of a set of entities within the entity-feature-graph” as “The training set database 245 stores one or more patient complaint-symptom datasets that are used to generate the knowledge graph 225. These datasets are further described in conjunction with FIG. 6. In some embodiments, the training set database 245 is combined with the patient information database 205” (Paragraph 27), “the medical triage assistance system 200 determines 330 the patient's symptoms based on the relevant conversation tokens. The medical triage assistance system 200 traverses the knowledge graph 225 based on the relevant conversation tokens and determines a probability and confidence level that the tokens are associated with specific symptoms. One method for generating the knowledge graph 225 is described in conjunction with FIGS. 7-8. Various complex network metrics, such as adjacency matrices and geodesic paths, may be used to traverse the knowledge graph 225. The knowledge graph 225 may also be traversed based on probabilistic modeling and detection of anchors and triplets, or deep Kalman filters, including deep learning and probabilistic modeling. Multiple symptoms can be presented to the nurse, along with the calculated probabilities and confidence levels” (Paragraph 40), “If the medical triage assistance system 200 receives a correction to the symptoms from the nurse, it can use that feedback to rebalance the connections of the knowledge graph 225 and re-score the multi-class classifier. A nurse can provide a correction by selecting the correct symptom that should have been identified, such as through a multiple-choice interface. The knowledge graph 225 is recomputed based on the correction and the recomputed knowledge graph 225 replaces the current knowledge graph 225 once a threshold improvement in performance is reached. Previous versions of the knowledge graph 225 may be stored to allow for analysis of historical data and models” (Paragraph 43), and “The medical triage assistance system 200 may also select 340 one or more specific medical protocols to recommend based on the patient's symptoms. Each medical protocol is based on one or more symptoms and is made up of a series of questions that are designed to differentiate between life-threatening conditions associated with that symptom and less urgent conditions. The medical triage assistance system 200 maps specific medical protocols to the various medical concepts of the knowledge graph 225. This mapping can be manually created, or learned (i.e., as part of the knowledge graph 225) based on existing patient cases. The medical triage assistance system 200 selects 340 the medical protocols based on confidence scoring. The medical triage assistance system 200 may present the selected 340 protocol(s) to the nurse as a recommendation and wait for approval or correction before proceeding” (Paragraph 45). The examiner further notes that a selected intervention is based off of a generated knowledge graph (i.e. the claimed undefined entity feature graph in the broadest reasonable interpretation) that is traversed/mapped via a learned process (i.e. the use of a “trained graph classification model” (which is undefined in the claims) in the broadest reasonable interpretation). Such a learned process is learned based on probabilities/confidence values of the knowledge graph.
Salazar and Chen do not explicitly teach:
B) the confidence values indicating confidence levels with which respective ones of the entities are extracted from the at least two data streams.
Badr, however, teaches “the confidence values indicating confidence levels with which respective ones of the entities are extracted from the at least two data streams” as “if a digital image received by module 108 includes a particular content item(s) such as the Eiffel tower, and/or a dog standing in front of the Eiffel tower, then module 108 can use logic 204 to extract image features, or pixel data, that correspond to at least one of: a) the Eiffel tower, or b) the dog. Module 108 can then use label generation logic 206 to generate one or more labels (e.g., words, or text phrases) based on extracted features for Eiffel tower and dog” (Paragraph 43) and “labels that are more definitive or descriptive of particular attributes or extracted image features of an item of digital content may be assigned a higher confidence score relative to labels that more generic. For example, referencing the above extracted features for the Eiffel tower and the dog, descriptive labels such as “Eiffel” or “Eiffel tower” may receive higher confidence scores when compared to more generic labels such as “tower” or “Paris.” Likewise, descriptive labels such as “golden retriever” or “cute cocker spaniel” may receive higher confidence scores when compared to more generic labels such as “dog” or “cute dog.” (Paragraph 46)”.
The examiner further notes that although Salazar teaches probabilities/confidence values, there is no explicit teaching that such values are indicative of levels of extracted entities. Nevertheless, Badr teaches the concept of confidence values that are indicative of extracted entities from images. The combination would result in expanding Salazar to also have confidence values that are indicative of levels of extracted entities.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Badr’s would have allowed Salazar’s and Chen’s to provide a method for indicating a relevance level of extracted features, as noted by Badr (Paragraph 45).
13. Claims 6 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Salazar et al. (U.S. PGPUB 2018/0218126), in view of Chen et al. (CN104933839A (Machine Translation Provided)) as applied to claims 1, 5, 9, 11-12, 16, 18-19, and 21 above, and in view of Watanabe (U.S. PGPUB 2022/0165092), and further in view of Gabel et al. (U.S. PGPUB 2016/0182707).
14. Regarding claims 6 and 17, Salazar and Chen do not explicitly teach a computer-implemented method and computer system comprising:
A) determining based on the at least two data streams that the patient has suffered an accident.
Watanabe, however, teaches “determining based on the at least two data streams that the patient has suffered an accident” as “Accident detection apparatus 20 detects occurrence of an accident such as a fall of a person, based on an image captured by surveillance camera 10 (hereinafter referred to as a “captured image”). The captured image may be either of a still image or a moving image” (Paragraph 22).
The examiner further notes that the secondary reference of Watanabe teaches the concept of detecting accidents from an image. The combination would result in the detection of accidents in the images of Salazar.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Watanabe’s would have allowed Salazar’s and Chen’s to provide a method for reducing detection failures of accidents, as noted by Watanabe (Paragraph 04).
Salazar, Chen, and Watanabe do not explicitly teach:
B) executing the selected intervention of establishing a phone connection of an emergency contact associated with the patient.
Gabel, however, teaches “executing the selected intervention of establishing a phone connection of an emergency contact associated with the patient” as “The system 104 may also be configured to communicate with one or more devices 120, 122 (e.g., mobile phones or tablet computers) associated with contacts that a given user has indicated should be notified when certain conditions are detected (e.g., the occurrence of an accident or a call to an emergency service provider), as discussed elsewhere herein in great detail” (Paragraph 37), “the application and/or system may be configured to detect an accident event and to notify contacts specified by the user regarding the accident” (Paragraph 48), and “the notifications may be transmitted to the contact(s). The notifications may be provided via a user interface on an instantiation of the application on a device of a notification recipient, via a text message, and/or via an automated voice call to the recipient as similarly described elsewhere herein” (Paragraph 49).
The examiner further notes that the secondary reference of Gabel teaches the concept of automatically notifying (which can include a telephonic call) associated emergency contacts in response to a detected accident. The combination would result in automatically contacting associated emergency contacts after detecting an accident in the image of Watanabe.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Gabel’s would have allowed Salazar’s, Chen’s, and Watanabe’s to provide a method for overwhelming victims of accidents, as noted by Gabel (Paragraph 04).
15. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Salazar et al. (U.S. PGPUB 2018/0218126), in view of Chen et al. (CN104933839A (Machine Translation Provided)) as applied to claims 1, 5, 9, 11-12, 16, 18-19, and 21 above, and in view of Romeo et al. (U.S. PGPUB 2015/0134088).
16. Regarding claim 7, Salazar and Chen do not explicitly teach a computer-implemented method comprising:
A) determining based on the at least two data streams that the patient has undergone a surgery and is underactive; and
B) executing the selected intervention of adapting a therapy associated with the patient.
Romeo, however, teaches “determining based on the at least two data streams that the patient has undergone a surgery and is underactive” as “Once the patient has completed the registration process, the patient meets with the healthcare practitioner to assess the condition and determine a procedure to treat the condition. Upon determining a procedure appropriate for the patient's condition, the physician coordinates the procedure. For example, surgery can be performed and the patient provided with a post-operative brace to be worn during the healing process. The physician can also provide direction to another healthcare practitioner, such as a trainer or physical therapist, to deliver a physical therapy solution for the patient” (Paragraph 24), “The practitioner can then review the data and check on the patient's performance. If the patient is underperforming, the healthcare practitioner can call or schedule an appointment with the patient to review progress, determine if the patient is having difficulties or problems that need to be addressed, or simply to remind the patient to try harder when performing the routines. Where the patient is performing well, the practitioner may reach out and congratulate the patient, and, in some embodiments, the practitioner may have the ability to award the patient with rewards points or additional rewards points for good performance. Importantly, the practitioner uses the information to determine the progress of the patient and to determine whether a change needs to be made in the prescribed physical therapy regimen or if other follow-up treatments are required” (Paragraph 65), and “the doctor or patient can initiate a video or audio phone conference to discuss topics such as the patient's health and condition, the exercise routines, and so on. The doctor or patient can take advantage of a live video conversation to monitor the patient's performance and provide feedback. Likewise, as described in more detail below, photographs and other information can be captured and provided with the data to provide the healthcare practitioner with additional information regarding the patient's performance” (Paragraph 68) and “executing the selected intervention of adapting a therapy associated with the patient” as “Once the patient has completed the registration process, the patient meets with the healthcare practitioner to assess the condition and determine a procedure to treat the condition. Upon determining a procedure appropriate for the patient's condition, the physician coordinates the procedure. For example, surgery can be performed and the patient provided with a post-operative brace to be worn during the healing process. The physician can also provide direction to another healthcare practitioner, such as a trainer or physical therapist, to deliver a physical therapy solution for the patient” (Paragraph 24), “The practitioner can then review the data and check on the patient's performance. If the patient is underperforming, the healthcare practitioner can call or schedule an appointment with the patient to review progress, determine if the patient is having difficulties or problems that need to be addressed, or simply to remind the patient to try harder when performing the routines. Where the patient is performing well, the practitioner may reach out and congratulate the patient, and, in some embodiments, the practitioner may have the ability to award the patient with rewards points or additional rewards points for good performance. Importantly, the practitioner uses the information to determine the progress of the patient and to determine whether a change needs to be made in the prescribed physical therapy regimen or if other follow-up treatments are required” (Paragraph 65), and “the doctor or patient can initiate a video or audio phone conference to discuss topics such as the patient's health and condition, the exercise routines, and so on. The doctor or patient can take advantage of a live video conversation to monitor the patient's performance and provide feedback. Likewise, as described in more detail below, photographs and other information can be captured and provided with the data to provide the healthcare practitioner with additional information regarding the patient's performance” (Paragraph 68).
The examiner further notes that the secondary reference of Romeo teaches the concept of modifying the rehab (i.e. therapy) of a post-op underperforming patient based off of received data. The combination would result in altering interventions based off of underperformance in Salazar.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Romeo’s would have allowed Salazar’s and Chen’s to provide a method for improving the outcomes of surgery, as noted by Romeo (Paragraph 03).
Allowable Subject Matter
17. Claims 3 and 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Specifically, although the prior art (See Salazar) teaches the processing of multiple streams of patient data for subsequent output recommendations, the detailed claim language directed towards the specific transformation of the defined entity-feature graph into an embedding space via a process that uses distances between entity pairs in that space to encode probabilities that those pairs are the same is not found in the prior art in conjunction with the rest of the limitations of the parent claims.
Dependent claims 4 and 15 are deemed allowable for depending on the deemed allowable subject matter of dependent claims 3 and 13 respectively.
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Specifically, although the prior art (See Romeo) teaches the automated modification of therapy of a patient recovering from surgery, the detailed claim language directed towards the specific determination of a patient watching TV and subsequently intervening to increase the difficulty of sports equipment of that patient is not found in the prior art in conjunction with the rest of the limitations of the parent claims.
Response to Arguments
18. Applicant’s arguments with respect to claims 1-9 and 11-21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument (See newly applied art of Chen).
Applicant's arguments filed on 06/23/2026 have been fully considered but they are not persuasive.
Applicants argue on Page 14 that “While Salazar does describe a database storing conversation records between a patient and healthcare professional, where the conversation records can include text transcriptions and video calls, as well as screen shots from a video call or images submitted by a patient, Salazar is silent as to using sensors. For example, the Examiner appears to map whatever device is being used by the patient to conduct a conversation with the healthcare professional to the one or more sensors of claim 1”. However, the examiner wishes to refer to Salazar which states “The patient information database 205 stores information about patients (i.e., users) of the medical triage assistance system 200. Patient information may include identification information, demographics, conversation records, symptoms, medical history, and health insurance claims data. Identification information may be an identifier within the medical triage assistance system 200 associated with the patient, or an identifier from a more ubiquitous entity, like a driver's license or social security number. Conversation records allow the medical triage assistance system 200 access to conversations between the patient and a healthcare professional or the medical triage assistance system 200. These conversations may take place via chat or text messages, or via audio or video calls. For chat or text messages, the conversation record contains the messages and an indication of who sent the message. For an audio or video call, the conversation record is a transcript and may also include who said what. Screenshots (from a video call) or images submitted by the patient may also be included in conversation records. For example, the patient may submit images of a rash. In some embodiments, a conversation between the patient and the healthcare professionals are routed through the medical triage assistance system 200. In this embodiment, the medical triage assistance system 200 is able to record the conversation while it is taking place” (Paragraph 19). The examiner further notes that a camera and/or microphone for a video call (which is then stored as screenshots and transcripts) teaches the claimed one or more sensors in the broadest reasonable interpretation.
Conclusion
19. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. PGPUB 2019/0251480 issued to Duran et al. on 14 August 2019. The subject matter disclosed therein is pertinent to that of claims 1-9 and 11-21 (e.g., methods to process sensor data).
U.S. PGPUB 2019/0148025 issued to Stone et al. on 16 May 2019. The subject matter disclosed therein is pertinent to that of claims 1-9 and 11-21 (e.g., methods to process sensor data).
20. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
21. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272-2731. The examiner can normally be reached on Monday to Friday 8:20 am – 4:40 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached (571) 272-4085. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Mahesh Dwivedi
Primary Examiner
Art Unit 2168
July 12, 2026
/MAHESH H DWIVEDI/Primary Examiner, Art Unit 2168
Prosecution Insights