Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status of Claims
The present Office Action is pursuant to Applicant’s communication on 06-16-2025; current application filed on 06-16-2025.This application is a 371 of PCT/EP2023/085468 12/13/2023 and claims foreign priority to EP 22214801.7 12/20/2022.
Examiner’s Note
The rejections below group claims that may not be identical, but whose language and scope are so substantively similar as to lend themselves to grouping, in the interests of clarity and conciseness.
Information Disclosure Statement
The information disclosure statements (IDS) filed on 06-16-2025, has been acknowledged. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more.
Claim 1 recites:
1. A healthcare apparatus comprising: a memory comprising instruction data representing a set of instructions; and a processor configured to communicate with the memory and to execute the set of instructions, wherein the set of instructions, when executed by the processor, cause the processor to:
(i) receive a first user input from a user input device, the first user input indicating a first zoom command, the first zoom command indicated with respect to a first portion of a display in which first data from an electronic healthcare record is displayed;
(ii) in response to the first zoom command, automatically determine second data that is semantically linked to the first data; and
(iii) in response to the first zoom command, instruct the display to display the second data in a second portion of the display.
Analysis of Independent Claims (1, 13, and 15)
Step 1: Statutory Category
Claims 1, 13, and 15 are directed to a healthcare apparatus, a computer-implemented method, and a non-transitory computer-readable storage medium respectively. These fall squarely within the statutory categories of machine, process, and manufacture/article of manufacture under 35 U.S.C. § 101.
Step 2A Prong One: Judicial Exception (Law of Nature, Natural Phenomenon, or Abstract Idea) The claims recite receiving a zoom command on displayed EHR data, automatically determining semantically linked second data, and displaying it in a second portion of the display. This core concept is directed to the abstract idea of organizing and presenting information (specifically, semantic linking and progressive disclosure of medical data based on user interaction). The specification explicitly frames this as an information organization technique:
Specification discloses the invention facilitates specialized semantic zooming to retrieve related medical information. [Page 2]: “Thus, the apparatus and methods herein facilitate what may be considered as specialized semantic zooming of medical information, whereby receipt of a zoom command from a user initiates retrieval of further semantically linked information from the EHR... Thus, in this manner, a HCP can zoom-in and out of a medical record to receive further information related to the first data.”
The process of identifying semantically linked data and displaying it in response to a zoom gesture is a mental process/data organization concept that can be performed in the human mind or with pen and paper. It falls under the “Certain methods of organizing human activity” and “Mental processes” groupings.
Step 2A Prong Two: Practical Application
The additional elements recited are generic computing components: a memory, processor, user input device, and display. These are recited at a high level of generality merely as tools to perform the abstract idea. The specification confirms that these are standard, off-the-shelf components:
Specification discloses that the apparatus uses conventional computing hardware.
[Page 3]: “Generally, the apparatus may form part of a computer apparatus or system e.g. such as a laptop, desktop computer or other computing device.”
Merely implementing the abstract idea on a generic computer or using a standard display/input device does not integrate the exception into a practical application. The claims do not improve the functioning of the computer itself, nor do they apply the abstract idea in a particular technological environment that solves a technology-specific problem.
Step 2B: Significantly More
The additional elements do not amount to significantly more than the judicial exception. Using a processor to execute instructions, a memory to store data, and a display to show information are well-understood, routine, and conventional activities in the field of computing. The specification describes these functions as standard computer operations:
Specification discloses routine data storage and retrieval functions.
[Page 4]: “The memory 104 us configured to store program code that can be executed by the processor 102 to perform the method described herein... Memory 104 can be used to store the EHR, or a portion thereof, the zoom command… and/or any other information or data received, calculated or determined by the processor.”
Since the computer components are generic and perform conventional functions, they do not transform the abstract idea into patent-eligible subject matter.
Analysis of Dependent Claims (Claims 2-12, 14 and 16-20)
The dependent claims add further limitations that either merely elaborate on the abstract idea or recite conventional computer functions. They fail to overcome the § 101 rejections for the following reasons:
Claims 2 & 3 (NLP/Machine Learning)
These claims recite executing a computational function using NLP or machine learning to retrieve semantically linked data. While this specifies an algorithm, it merely applies a conventional data processing technique to the abstract idea of organizing information. The specification describes this as a standard analytical approach:
Specification discloses using NLP/ML for semantic analysis.
[Page 8]: "As an example, a computational function as described above may use natural language processing or machine learning to determine the meaning of the first data."
Using generic NLP or ML algorithms to find semantic links is a well-understood, routine technique in data science and does not integrate the abstract idea into a practical application or amount to significantly more.
Claim 4 (Hierarchical Data Structure)
The claim recites that the second data is logically connected as part of a hierarchical data structure.
Organizing data hierarchically is a conventional method of data organization and database management. The specification notes:
Specification discloses hierarchical linking.
[Page 6]: "For example, the second data can be logically connected to the first data as part of a hierarchical data structure in the EHR."
This merely describes how the abstract idea is structured in memory and does not add an inventive concept.
Claims 5, 14, & 17 (Zoom-In/Zoom-Out Detail Levels)
Thes claims recite that zoom-in commands yield more detailed data, while zoom-out commands yield less detailed data. This merely describes conventional UI behavior for progressive disclosure. The specification states:
Specification discloses detail-level mapping to zoom gestures.
[Page 6]: "In some embodiments, the second data is more detailed than the first data if the first user input indicated a zoom-in command. In some embodiments, the second data is less detailed than the first data, if the user input comprises a zoom-out command."
This is a routine GUI interaction pattern and does not amount to significantly more.
Claims 6 & 18 (Overlapping/Replacing Display Portions)
These claims recite displaying the second data in an overlapping portion or in place of the first data. This is a standard screen refresh/redraw function. The specification explains:
Specification discloses replacing displayed data.
[Page 9]: "In some embodiments, the second portion of the display is at least partially overlapping with the first portion of the display. As a result, step 206 may comprise instructing the display to display the second data in place of the first data."
Conventional display updating does not overcome the abstract idea rejection.
Claims 7 (Initiating a Second GUI/Window)
This claim recites causing a second GUI/window to appear. The specification describes this as a standard pop-out window function:
Specification discloses opening new windows.
[Page 9]: "In other embodiments, the zoom command causes a new window, or graphical user interface (GUI) to appear. In such an embodiment, the zoom command may cause the new window to pop-out and display the second data."
Opening a new window is a routine computer function.
Claims 8 & 19 (Reversing Zoom Commands)
These claims recite receiving a second zoom command that reverses the first. This describes standard UI navigation (e.g., undo/back functionality). The specification notes:
Specification discloses reversing zoom actions.
[Page 10]: “The second zoom command may at least partially reverse the first zoom command.”
This is a conventional user interface feature that does not add significantly more.
Claims 9 (Re-displaying First Data)
Merely recites instructing the display to show the original data upon reversal. This is a routine screen update function.
Claims 10 & 20 (Dynamic Third Data Retrieval Based on Changes)
These claims recite retrieving third data if the first/second data changes or new data becomes available, making it more semantically linked. The specification describes this as dynamic data updating:
Specification discloses dynamic semantic linking based on data changes.
[Page 10]: “In such a scenario, the third data may be different to the first data, if… there has been a change in the first data; there has been a change in the second data; or if the third data has become available… and as a result, the third data is more highly semantically linked to the second data compared to the first data.”
Automatically updating displayed information based on changing database inputs is a well-understood, routine computer function.
Claims 11 (Specific Medical Condition/Treatment/Medication Relationships)
This claim merely applies the abstract idea to specific healthcare examples (conditions/treatment/medication types). The specification provides these as illustrative examples:
Specification discloses example medical relationships.
[Page 5]: “As another example, the first data may relate to a medical condition referred to in the EHR and the second data may relate to a treatment given for the medical condition.”
Applying an abstract idea to a particular field of use (healthcare) without improving the underlying technology does not render it patent eligible.
Claims 12 (Different Data Types)
This claim recites that first and second data may be different types (e.g., text vs image). The specification notes:
Specification discloses cross-format display.
[Page 6]: “It is further noted that the first data and the second data may be different types, for example, the first data may be in text format and the second data may be in an image format… or vice versa.”
Converting or displaying different data formats is a routine computer operation.
Conclusion
Claims 1-20 are rejected under 35 U.S.C. § 101 as directed to non-statutory subject matter.
The independent claims (1, 13, 15) are directed to abstract ideas of organizing and presenting information via semantic zooming, implemented on generic computer components that do not integrate the exception into a practical application or amount to significantly more. The dependent claims merely add conventional UI behaviors, standard data organization techniques (hierarchical structures, NLP/ML), routine display updates, or field-of-use limitations that fail to overcome the abstract idea.
Thus, taken alone, the additional elements do not amount to significantly more than the abstract idea identified above. Furthermore, looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually, and there is no indication that the combination of elements improves the functioning of a computer or improves any other technology, and their collective functions merely provide conventional computer implementation.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. § 103 as being unpatentable over Linthicum (US 2010/0131293) in view of Bender (US 2021/0294829).
I. Rejection of Independent Claims 1, 13, and 15 (Apparatus, Method, and Medium)
A. Claim Limitations - Claim 1 (Apparatus)
Limitation
Text
1[a]
A healthcare apparatus comprising: a memory comprising instruction data representing a set of instructions; and a processor configured to communicate with the memory and to execute the set of instructions, wherein the set of instructions, when executed by the processor, cause the processor to:
1[b]
(i) receive a first user input from a user input device, the first user input indicating a first zoom command, the first zoom command indicated with respect to a first portion of a display in which first data from an electronic healthcare record is displayed;
1[c]
(ii) in response to the first zoom command, automatically determine second data that is semantically linked to the first data (i.e., wherein semantic links correspond to hierarchical data structure retrieval associated with a computational NLP algorithm such as BERT or GLOVE); and
1[d]
(iii) in response to the first zoom command, instruct the display to display the second data in a second portion of the display.
B. Limitation-by-Limitation Mapping
1[a] - Memory and processor executing instructions:
Linthicum discloses a processor system with memory and a processor executing instructions for a healthcare record display system. [0165]: "the processor system 1710 includes a processor 1712 that is coupled to an interconnection bus 1714… The processor 1712 of FIG. 17 is coupled to a chipset 1718, which includes a memory controller 1720 and an input/output ('I/O') controller 1722… The memory controller 1720 performs functions that enable the processor 1712… to access a system memory 1724 and a mass storage memory 1725." [0166]: "The system memory 1724 may include any desired type of volatile and/or non-volatile memory such as, for example, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, read-only memory (ROM), etc." [0171]: "Certain embodiments contemplate methods, systems and computer program products on any machine-readable media to implement functionality described above… Certain embodiments may be implemented using an existing computer processor, or by a special purpose computer processor…"
1[b] – Receiving a first user input indicating a zoom command with respect to a first portion of a display showing EHR data:
Linthicum discloses a graphical patient health record timeline in which a user provides zoom/magnification input with respect to a displayed portion of an EHR. [0120]: "At higher magnification, greater details of the patient start becoming clearer. Based on particular events or problems, the user may choose to zoom in further for greater detail." [0119]: "A user may navigate, manipulate and view different information and different levels/granularity of information in the interface by dragging, scrolling and/or otherwise moving a viewpoint via mouse and cursor, keyboard, trackball, touch screen, etc." [0118]: "FIG. 6 shows a high level view of a patient timeline displayed graphically for a user. All information for the patient is contained in one context. Patient data is organized by time and correlated with other patient data. A user can view and edit data within the timeline interface." [0124]: "a complete patient timeline 600 can be viewed from a high level. The timeline 600 can be divided into a plurality of categories, such as encounters 610, results 612, problems 614, procedures 616 and medication 618. Using the timeline 600, a high level visualization of encounters/visits and results/data can be viewed for a patient lifetime." [0129]: "a patient medical record aggregated information from a plurality of information systems… Information systems can include… an electronic medical record ('EMR')…" [0093]: "the ED staff access all of Patricia's medical records via a longitudinal timeline record, for example, and become informed about all of the aspects of her case."
The "first portion of a display" corresponds to the initial high-level/macro view of the patient timeline, and the "zoom command" corresponds to the user's selection to magnify/zoom into a particular event or data element.
1[c] Automatically determining second data semantically linked to the first data via a computational NLP algorithm (BERT/GLOVE) and hierarchical data structure retrieval:
Linthicum discloses the semantic linking and retrieval of related data upon zoom/magnification. [0120]: "Information displayed may have hyperlinks attached to allow the user to navigate to an information system that initially generated the data to drill down on finer details. Alternatively and/or in addition, finer details related to the information may be present in the patient history context and become viewable and reviewable as the user drills down into the timeline." [0126]: "a user can further magnify a particular event to view greater detail regarding an event 820 and surrounding data. A user can select and/or further magnify information displayed to access additional detail and/or connect to an information system including additional detail regarding the selected data point, for example." [0144]: "By clicking on and/or otherwise selecting a data point 1130 on the timeline 1110, the system 1100 can display an original data source complete with a full data context, for example." [0145]: "A user can select the element 1130 (e.g., via a mouse click) to display a source document, a list of clinical documents related to the element or issue, and supplemental research material, for example." [0142]: "In certain embodiments, a user can select a certain data element 1130, and corresponding data elements 1130 are highlighted through the patient record timeline 1110. Highlighting similar data elements 1130 across time via the timeline 1110 helps identify and accentuate a frequency of similar out-of-range data elements… Rather than focusing only on event or encounter based organization, the record 1100 facilitates navigation of a patient record by clinical element and/or by patient encounter, for example."
However, Linthicum does not explicitly disclose that the semantic linking is performed via a computational NLP algorithm such as BERT or GLOVE in the context of hierarchical data structure retrieval.
Bender discloses the use of NLP algorithms (BERT, GLOVE, ELMo) to establish semantic links between n-grams through hierarchical ontology graph structures. [0037]: "The machine learning models may include a transformer neural network model such as Elmo, BERT, or the like. In some embodiments, the machine learning model may improve data ingestion accuracy by generating attention vectors for n-grams of an ingested document when performing document analysis or text summarization operations." [0065]: "Alternatively, or in addition, context-independent embedding operations other than neural-network-based operations may be used, such as a matrix factorization method. For example, some embodiments may use a Global Vector ('GloVe') model to determine an embedding vector for an n-gram, where using a GloVe model may include using a matrix model trained a global word to word co-occurrence count." [0201]: "some embodiments may use XLNet or another autoregressive transformer model to generate word embeddings, where XLNet is described in Yang et al.… For example, some embodiments may [use] to generate embeddings for a word of a document, where the word (or another n-gram) may be assigned an embedding vector based on both the word itself, its position with respect to other words surrounding the word, and the embedding vectors of the other words." [0080]: "some embodiments may determine a hierarchy of ontologies based on one or more vertex groups associated with a vector cluster via an edge connection between the vertex group and one or more more shared connections." [0006]: "determining, with the computer system, a first set of embedding vectors in an embedding space based on a first set of n-grams of the natural-language text document; retrieving, with the computer system, a first set of vertices of the first ontology graph based on the first set of embedding vectors, wherein each respective vertex of the first set of vertices is identified by a respective embedding vector of the first set of vertices; selecting, with the computer system, a second set of vertices of the second ontology graph, wherein the second set of vertices is adjacent to the first set of vertices via the set of graph edges associating the first ontology graph and the second ontology graph; generating, with the computer system, a sequence of n-grams based on a second set of embedding vectors identified by the second set of vertices."
It would have been obvious to one of ordinary skill in the art before the effective filing date to combine Linthicum's zoom-based EHR navigation with Bender's NLP-based (BERT/GLOVE) semantic linking through hierarchical ontology graphs, because Linthicum already teaches retrieving "finer details related to the information" upon magnification [0120] and Bender provides the well-known mechanism of using BERT/GLOVE embedding vectors to identify semantically related terms across hierarchical ontology graphs. The motivation to combine is the well-established desire to improve the relevance and accuracy of retrieved information in healthcare systems, as recognized in Linthicum at [0007]: "A variety of clinical data and medical documentation is available throughout various clinical information systems, but it is currently difficult to find, organize, and effectively present the information to physicians and other healthcare providers at a point of care."
1[d] Instructing the display to display the second data in a second portion of the display:
Linthicum discloses displaying the retrieved/linked data in a different portion or view of the display upon zoom. [0126]: "a user can further magnify a particular event to view greater detail regarding an event 820 and surrounding data." [0127]: "FIG. 9 illustrates a further magnification of a patient record 900… Further magnification allows a user to view finer detail in conjunction with a problem or intervention. For example, a user can view test(s), procedure(s), and/or examination(s) 930 saved with respect to a particular patient problem 920, such as atrial fibrillation." [0128]: "further magnification of a patient record timeline 1000 allows a user to review and edit one or more data points 1040 in the record 1000." [0144]: "By clicking on and/or otherwise selecting a data point 1130 on the timeline 1110, the system 1100 can display an original data source complete with a full data context." [0143]: "Certain embodiments allow the user to view data from a macro view (e.g., across an entire patient record) to a micro view (e.g., focusing on an individual data element)." [0139]: "a user can quickly scan a macro view of a patient history and then dive deep into a specific encounter efficiently."
The "second portion of the display" corresponds to the magnified/zoomed view that replaces or supplements the initial macro view, as described in the progression from FIG. 6 (high level) through FIGS. 7–10 (progressive magnification).
C. Claim 13 (Method) and Claim 15 (Medium)
Claims 13 and 15 are rejected under 35 U.S.C. § 103 for the same reasons as Claim 1.
The limitations of Claim 13 mirror those of Claim 1 in method form:
13(i): "receiving a first user input from a user input device, the first user input indicating a first zoom command, the first zoom command being indicated with respect to a first portion of a display in which first data from an electronic healthcare record, (EHR) is displayed" - Linthicum [0120]: “the user may chose to zoom in further for greater detail” and [0119]: "A user may navigate, manipulate and view different information and different levels/granularity of information in the interface by dragging, scrolling and/or otherwise moving a viewpoint via mouse and cursor, keyboard, trackball, touch screen, etc."
13(ii): "in response to the first zoom command, automatically determining second data that is semantically linked to the first data " - Linthicum [0120]: "finer details related to the information may be present in the patient history context and become viewable and reviewable as the user drills down into the timeline"; Bender [0006]: "determining, with the computer system, a first set of embedding vectors in an embedding space based on a first set of n-grams… selecting, with the computer system, a second set of vertices of the second ontology graph, wherein the second set of vertices is adjacent to the first set of vertices via the set of graph edges."
13(iii): "in response to the first zoom command , instructing the display to display the second data in a second portion of the display " - Linthicum [0127]: "Further magnification allows a user to view finer detail in conjunction with a problem or intervention."
The limitations of Claim 15 mirror those of Claim 1 in medium form and are rejected for the same reasons, with the additional support from Linthicum [0173]: "Certain embodiments include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon… such computer-readable media may comprise RAM, ROM, PROM, EPROM, EEPROM, Flash, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices."
II. Rejection of Claims 2, 3, and 16 (Computational Function / NLP / ML for Semantic Linking)
A. Claim Limitations
Limitation
Text
2[a]
(dependent on Claim 1) the set of instructions, when executed by the processor, further cause the processor: execute a computational function to retrieve the second data that is semantically linked to the first data
3[a]
(dependent on Claim 2) the computational function uses natural language processing, NLP, or machine learning to retrieve the second data that is semantically linked to the first data
16[a]
(dependent on Claim 15) a computational function uses natural language processing (NLP) or machine learning to retrieve the second data that is semantically linked to the first data
B. Limitation-by-Limitation Mapping
2[a] / 3[a] / 16[a] – Computational function using NLP/ML to retrieve semantically linked data:
Bender discloses a computational function using NLP and machine learning to retrieve semantically linked data. [0037]: "Some embodiments may use one or more machine learning models to determine learned representations, such as categories, scalar values, or embedding vectors, for the documents. The machine learning models may include a transformer neural network model such as Elmo, BERT, or the like." [0065]: "some embodiments may use a Global Vector ('GloVe') model to determine an embedding vector for an n-gram, where using a GloVe model may include using a matrix model trained a global word to word co-occurrence count." [0102]: "some embodiments may use one or more machine learning models to retrieve documents, summarizations based on documents, or the like as part of providing semantic search results after receiving a query." [0104]: "some embodiments can train a BERT-based machine learning model to predict answers based on training queries from a stored library of queries and answers, where the answers for the queries may include semantic search results." [0201]: "some embodiments may use XLNet or another autoregressive transformer model to generate word embeddings… some embodiments may [use] to generate embeddings for a word of a document, where the word (or another n-gram) may be assigned an embedding vector based on both the word itself, its position with respect to other words surrounding the word, and the embedding vectors of the other words." [0160]: "some embodiments may use a transformer neural network to determine a set of embedding vectors for a set of n-grams of a query using a set of encoder neural network layers of the transformer neural network." [0162]: "some embodiments may use a transformer neural network model that includes one or more attention mechanisms to generate a query based on n-grams from the sentence… Some embodiments may determine a set of attention values based on an attention query value and a attention key value."
Linthicum discloses the context in which this computational function operates — the EHR zoom interface. [0120]: "Information displayed may have hyperlinks attached to allow the user to navigate to an information system that initially generated the data to drill down on finer details." [0145]: "A user can select the element 1130 (e.g., via a mouse click) to display a source document, a list of clinical documents related to the element or issue, and supplemental research material."
It would have been obvious to combine Linthicum's EHR zoom-based retrieval with Bender's NLP/ML-based semantic retrieval (using BERT, GLOVE, or similar) because both references are directed to improving the retrieval and presentation of healthcare/clinical information, and the use of NLP algorithms such as BERT and GLOVE for semantic text analysis was well-known in the art at the time of filing.
III. Rejection of Claim 4 (Hierarchical Data Structure / Neighboring Level)
Limitation
Text
4[a]
the second data is logically connected to the first data as part of a hierarchical data structure in the EHR
4[b]
the second data is in a neighboring level of the hierarchical structure to the first data
B. Limitation-by-Limitation Mapping
4[a]
Bender discloses a hierarchical data structure (hierarchical ontology graphs) in which data elements (n-grams) are logically connected. [0080]: "some embodiments may determine a hierarchy of ontologies based on one or more vertex groups associated with a vector cluster via an edge connection between the vertex group and one or more more shared connections." [0081]: "some embodiments may determine that a first vertex group of a first ontology graph may be associated with a vertex of a second ontology graph via one or more shared vertices." [0006]: "vertices of the first ontology graph is connected to vertices of the second ontology graph via a set of graph edges." [0216]: "some embodiments may determine that the term 'nursing mother' is a type of 'nursing woman' based on a relationship type associated with a graph edge connecting the vertices corresponding with 'nursing mother' and 'nursing woman,' where the relationship type may be 'subset'." [0093] (Bender): "some embodiments may update a set of ontology graphs to indicate new associations between different concepts encoded in the set of ontology graphs, add new vertices representing a new concept with its corresponding n-gram, or otherwise update the set of ontology graphs."
Linthicum discloses the EHR context. [0117]: "FIG. 6 depicts a complete visualization of an exemplary 44 year old male's complete medical record… At a high level, a user can see each clinical encounter, lab result, report, etc., that exists for the patient." [0125]: "a magnification of all or part of a patient record timeline 700 provides additional information regarding patient data points, such as events, problems, reports, etc. For example, in the interface 700 shown in FIG. 7, patient data 720, such as gout, atrial fibrillation, high cholesterol, etc., become legible and/or otherwise visible on the patient record at a point or point(s) in time at which the event or condition occurred."
4[b]
Bender discloses that semantically linked data resides in adjacent/neighboring vertices or graphs in the hierarchy. [0006]: "selecting, with the computer system, a second set of vertices of the second ontology graph, wherein the second set of vertices is adjacent to the first set of vertices via the set of graph edges associating the first ontology graph and the second ontology graph." [0142] (Bender): "some embodiments may determine a set of indirectly associated vertices of the first vertex by crawling through the edges associated with the vertices, where the edges may include ontology graph edges of a shared ontology graph or ontology graph edges that cross ontology graphs and connect vertices from different ontology graphs. For example, a first vertex of a first ontology graph may be associated with a second vertex of the first ontology graph, and the second vertex may be associated with a third vertex of a second ontology graph via a cross-graph edge, where the third vertex may be indirectly associated with the first vertex with an ontology graph edge distance equal to two." [0215] (Bender): "some embodiments may recognize the n-gram 'nursing mother' as a named entity associated with a first vertex of a first ontology by directly identifying the vertex by the named entity or by an embedding vector representing the named entity. Some embodiments may then detect an association between 'nursing mother' and the second n-gram sequence 'nursing woman' via an edge connecting the first vertex and a second vertex of a second ontology graph."
Linthicum discloses the neighboring-level navigation in the EHR timeline. [0143]: "Certain embodiments allow the user to view data from a macro view (e.g., across an entire patient record) to a micro view (e.g., focusing on an individual data element)." [0120]: "At higher magnification, greater details of the patient start becoming clearer. Based on particular events or problems, the user may choose to zoom in further for greater detail. Further magnification allows greater detail for a particular patient event or source of information."
Before the effective filing date of the claimed invention it would have been obvious to combine these references because Linthicum already teaches multi-level navigation (macro to micro) in an EHR, and Bender provides the well-known mechanism of hierarchical ontology graphs with adjacent vertices representing neighboring levels of a data hierarchy.
IV. Rejection of Claims 5, 14, and 17 (Zoom-In = More Detailed; Zoom-Out = Less Detailed)
A. Claim Limitations
Limitation
Text
5[a] / 14[a] / 17[a]
at least one of: the second data is more detailed than the first data if the first user input comprises a zoom-in command/or the second data is less detailed than the first data, if the first user input comprises a zoom-out command
B. Limitation-by-Limitation Mapping
5[a] / 14[a] / 17[a] – Zoom-in yields more detailed data; zoom-out yields less detailed data:
Linthicum directly and explicitly discloses this limitation. [0120]: "At higher magnification, greater details of the patient start becoming clearer. Based on particular events or problems, the user may choose to zoom in further for greater detail." [0121]: "In certain embodiments, at higher levels of magnification, additional text becomes more legible and allows a user to view finer detail regarding a particular problem, intervention, report, etc." [0143]: "Certain embodiments allow the user to view data from a macro view (e.g., across an entire patient record) to a micro view (e.g., focusing on an individual data element)." [0125]: "a magnification of all or part of a patient record timeline 700 provides additional information regarding patient data points, such as events, problems, reports, etc." [0127]: "Further magnification allows a user to view finer detail in conjunction with a problem or intervention." [0139]: "a user can quickly scan a macro view of a patient history and then dive deep into a specific encounter efficiently." [0126]: "a user can further magnify a particular event to view greater detail regarding an event 820 and surrounding data. A user can select and/or further magnify information displayed to access additional detail."
The "zoom-in command" producing "more detailed" data is directly supported by the progressive magnification from FIG. 6 → FIG. 7 → FIG. 8 → FIG. 9 → FIG. 10, where each successive magnification reveals finer-grained clinical data. The "zoom-out" (reverse) is supported by [0143]: "the user can view data from a macro view… to a micro view" — the reverse direction (micro to macro) yields less detailed data. Bender further supports the concept of hierarchical levels of detail through its ontology graph hierarchy. [0083] (Bender): "some embodiments may then associate the first ontology graph with the category 'expert' and the second ontology graph with the category 'layman' based on their associated class categories and update a hierarchy of the two ontology graphs such that the first ontology graph has a higher hierarchy value than the second ontology graph."
V. Rejection of Claims 6 & 18 (Overlapping Display or Display in Place)
Limitation
Text
6[a]
least one of: the second portion of the display is at least partially overlapping with the first portion of the display or the processor is configured to instruct the display to display the second data in place of the first data
B. Limitation-by-Limitation Mapping
6[a] Second portion overlapping or replacing the first portion:
Linthicum discloses that the magnified/zoomed view replaces or overlaps the initial view. [0120]: "At higher magnification, greater details of the patient start becoming clearer… Further magnification allows greater detail for a particular patient event or source of information. Information displayed may have hyperlinks attached to allow the user to navigate to an information system that initially generated the data to drill down on finer details. Alternatively and/or in addition, finer details related to the information may be present in the patient history context and become viewable and reviewable as the user drills down into the timeline." [0121]: "In certain embodiments, at higher levels of magnification, additional text becomes more legible and allows a user to view finer detail regarding a particular problem, intervention, report, etc." [0126]: "a user can further magnify a particular event to view greater detail regarding an event 820 and surrounding data." [0139]: "the interfaces of FIGS. 6-10 can be provided as a widget via the interface 200, 400. Using the longitudinal health record of FIGS. 6-10, a user can quickly scan a macro view of a patient history and then dive deep into a specific encounter efficiently."
The "dive deep" and "magnify" language in Linthicum inherently contemplates the second (magnified) view overlapping or replacing the first (macro) view within the same display area.
Bender further supports overlapping display through concurrent display of multiple views. [0310] (Bender): "some embodiments may provide a UI that concurrently displays a first and a second natural-language text document." [0230] (Bender): "some embodiments may concurrently display both the first and second text summaries, where a user may select which type of text summary they would prefer to view in a UI." [0353] (Bender): "The text comparison window 2250 displays a set of text sections from a first document and a visual indicator 2251 indicating a text difference between a text section of the first document in contrast with a second document."
VI. Rejection of Claim 7 (First GUI / Second GUI Initiation)
Limitation
Text
7[a]
the first portion of the display is in a first Graphical User Interface, (GUI), the second portion of the display is in a second GUI and wherein the processor is configured to cause the second GUI to be initiated on the display in response to receiving the first user input
B. Limitation-by-Limitation Mapping
7[a] – First GUI and second GUI, with second GUI initiated upon user request:
Linthicum discloses multiple GUI levels and the initiation of a new view upon user interaction. [0139]: "the interfaces of FIGS. 6-10 can be provided as a widget via the interface 200, 400 (e.g., a Web- and/or application-based user interface)." [0076]: "FIG. 2 shows an example adaptive user interface ('UI') 200… The UI 200 includes a login and user identification area 205, a patient identification area 210, an alert 212, and a widget display area 215." [0119]: "The patient timeline may be displayed on a computer monitor, an overhead display, a grease board, a viewing table, etc." [0120]: "Information displayed may have hyperlinks attached to allow the user to navigate to an information system that initially generated the data to drill down on finer details." [0123]: "In certain embodiments, hyperlinks allow users to launch and/or access information systems that have more detailed and/or additional documentation that may include radiology images, waveforms, etc."
Bender discloses the initiation of a new UI element/window upon user interaction. [0307] (Bender): "some embodiments may provide a UI that includes one or more UI elements that may be interacted with to send a set of requests to a server based on an input or configuration of the UI." [0310] (Bender): "some embodiments may provide a UI that concurrently displays a first and a second natural-language text document." [0344] (Bender): "FIG. 21 is a diagram of an example set of user interface elements indicating ontology-linked n-grams… The set of UI elements 2100 includes a UI element 2110 and a second UI element 2130, where the UI element 2110 is shown as a first window that includes text from a retrieved natural-language text document." [0353] (Bender): "The set of UI elements 2200 also includes other buttons or other UI elements that may update the data being displayed in the text comparison window 2250… an interaction with the third button 2271 may display a list of natural-language text documents from a corpus of text data."
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to provide a second GUI that is initiated upon a zoom command in Linthicum's system, because Bender teaches that user interaction with a UI element can cause the display of a new window or view.
VII. Rejection of Claims 8, 9, and 19 (Second Zoom Command / Reversal)
A. Claim Limitations
Limitation
Text
8[a] / 19[a]
receive a second user input from the user input device, the second user input indicating a second zoom command with respect to the first portion of the display; and wherein the second zoom command at least partially reverses the first zoom command
9[a]
(dependent on Claim 8) in response to receiving the second zoom command, the set of instructions, when executed by the processor, further cause the processor to: instruct the display to re-display the first data in the first portion of the display
B. Limitation-by-Limitation Mapping
8[a] / 19[a] – Second zoom command reversing the first:
Linthicum discloses bidirectional navigation (zoom in and zoom out) in the EHR timeline. [0143]: "Certain embodiments allow the user to view data from a macro view (e.g., across an entire patient record) to a micro view (e.g., focusing on an individual data element)." The ability to move from macro to micro (zoom in) inherently implies the ability to move from micro back to macro (zoom out / reverse). [0119]: "A user may navigate, manipulate and view different information and different levels/granularity of information in the interface by dragging, scrolling and/or otherwise moving a viewpoint via mouse and cursor, keyboard, trackball, touch screen, etc." [0139]: "a user can quickly scan a macro view of a patient history and then dive deep into a specific encounter efficiently" — the "scan" (macro) and "dive deep" (micro) are bidirectional. [0031]: "Certain embodiments offer a clinician an easy view of anomalies in clinical data over time. Certain embodiments allow the user to view data from a macro view (e.g., across an entire patient record) to a micro view (e.g., an individual data element)."
Bender discloses the ability to update/reverse UI state. [0307] (Bender): "some embodiments may provide a UI that includes one or more UI elements that may be interacted with to send a set of requests to a server based on an input or configuration of the UI." [0322] (Bender): "some embodiments may update a weight, bias, activation function parameter, or other neural network model based on user interaction with a UI."
9[a] – Re-displaying the first data (zoom back out):
Linthicum discloses the return to a macro/high-level view. [0143]: "Certain embodiments allow the user to view data from a macro view (e.g., across an entire patient record) to a micro view (e.g., focusing on an individual data element)." [0117]: "At a high level, a user can see each clinical encounter, lab result, report, etc., that exists for the patient. From the high level view, an overall health of a patient can be assessed with specific visual queues that indicate specific problems or events that have occurred for the patient." [0118]: "FIG. 6 shows a high level view of a patient timeline displayed graphically for a user. All information for the patient is contained in one context."
The progression from FIG. 6 (macro) → FIG. 7 → FIG. 8 → FIG. 9 → FIG. 10 (micro) and the explicit disclosure of bidirectional navigation (macro ↔ micro) at [0143] and [0031] supports the limitation that a second zoom command reverses the first and re-displays the first data.
VIII. Rejection of Claims 10 & 20 (Third Data / Dynamic Semantic Re-linking)
A. Claim Limitations
Limitation
Text
10[a]
in response to receiving the second zoom command, the set of instructions, when executed by the processor, further cause the processor to: retrieve third data that is semantically linked to the second data
10[b]
instruct the display to display the third data in a third portion of the display
10[c]
the third data is different from the first data, if, since the processor retrieved the second data: there has been a change in the first data
B. Limitation-by-Limitation Mapping
10[a] / 10[b] – Retrieving and displaying third data semantically linked to the second data:
Bender discloses dynamic, evolving semantic links where the system updates its retrieval based on changes in data. [0351] (Bender): "some embodiments may track changes in n-grams over time based on the use of the n-gram in multiple documents over time… After obtaining a second set of documents authored after the first date, some embodiments may determine that the n-gram 'vert1' is associated with an n-gram 'vert5' based on an updated embedding vector corresponding with the n-gram 'vert5'… Some embodiments may then update the set of ontology graphs, such as by appending a subarray associating the pair of vertices to an array of subarrays." [0351] (Bender): "some embodiments may then update additional operations based on a discovered association between a first concept and a second concept, such as by updating text-displaying operations to display the second concept after a user highlights the first concept." [0312] (Bender): "some embodiments may use a set of ontology graphs to determine if a set of expected n-grams are missing from a document… In response to a determination that the first set of n-grams mapped to the first concept is not present in the document, some embodiments may determine that one or more alert criteria have been satisfied." [0314] (Bender): "some embodiments may indicate one or more logical contradictions based on associations between vertices mapped to n-grams of a document."
Linthicum discloses the display of additional/updated data upon further interaction. [0128]: "further magnification of a patient record timeline 1000 allows a user to review and edit one or more data points 1040 in the record 1000… a user can annotate the record 1000 with one or more lines 1050 and/or other indicia to connect problems, issues, important forma tion related information, and/or other data points." [0127]: "a user can view test(s), procedure(s), and/or examination(s) 930 saved with respect to a particular patient problem 920, such as atrial fibrillation."
10[c] - Third data different from first data due to changes or new availability, with higher semantic link to second data:
Bender discloses that semantic links evolve over time as data changes. [0351] (Bender): "some embodiments may track changes in n-grams over time based on the use of the n-gram in multiple documents over time. As described elsewhere in this disclosure, some embodiments may update associations between different concepts or other n-grams over time based on documents authored or otherwise obtained at different times. For example, based on a first set of documents authored before a first date, some embodiments may determine that the n-gram 'vert1' is associated with the n-grams 'vert2,' 'vert3,' and 'vert4'… After obtaining a second set of documents authored after the first date, some embodiments may determine that the n-gram 'vert1' is associated with an n-gram 'vert5' based on an updated embedding vector corresponding with the n-gram 'vert5,' where the n-gram 'vert5' may represent another concept and may be encoded in a same or different ontology graph with respect to the n-gram 'vert1.'" [0189] (Bender): "some embodiments may detect changes in concepts or other categories assigned to a word, name, phrase, or other n-grams across different documents or different versions of a document… some embodiments may then determine that the concept represented by the n-gram 'burger' is associated with another concept represented by the n-gram 'vegan' based on a second set of documents… and update the corresponding ontology graph(s) to indicate the association between the concept represented by 'burger' and the concept represented by 'vegan.'"
Linthicum discloses that data in the EHR can be updated. [0128]: "further magnification of a patient record timeline 1000 allows a user to review and edit one or more data points 1040 in the record 1000." [0122]: "Users may also review and/or edit specific lab results, childhood immunizations, specific treatment plans, etc."
It would have been obvious to combine these teachings because Bender explicitly teaches that semantic links between data elements evolve as new data becomes available or existing data changes, and Linthicum provides the EHR zoom context in which such dynamic re-linking would be applied.
IX. Rejection of Claim 11 (Medical Condition → Treatment / Further Info / Medication Type → Specific Medications)
A. Claim Limitations
Limitation
Text
11[a]
the first data relates to a medical condition referred to in the EHR and the second data
relates to a treatment given for the medical condition
11[b]
the first data relates to a medical condition referred to in the EHR and the second data relates to further information related to said medical condition; or
11[c]
the first data relates to a type of medication referred to in the EHR and the second data relates to specific medications within said type
B. Limitation-by-Limitation Mapping
11[a] / 11[b] - Medical condition → treatment /further information:
Linthicum discloses navigation from a medical condition/problem to related treatment and additional information upon zoom. [0127]: "Further magnification allows a user to view finer detail in conjunction with a problem or intervention. For example, a user can view test(s), procedure(s), and/or examination(s) 930 saved with respect to a particular patient problem 920, such as atrial fibrillation." [0125]: "patient data 720, such as gout, atrial fibrillation, high cholesterol, etc., become legible and/or otherwise visible on the patient record at a point or point(s) in time at which the event or condition occurred." [0145]: "the view 1110 allows a user to visually correlate chronic issues directly with data elements 1130 along a clinical data elements axis 1110. A user can position a cursor over an element 1130 (e.g., a mouse over) to be display a summary view of that element 1130 or issue to date… A user can select the element 1130 (e.g., via a mouse click) to display a source document, a list of clinical documents related to the element or issue, and supplemental research material." [0141]: "Data types that can be displayed using this type of visualization include lab values, medications, vital signs, episodic care events, problems, immunizations, allergies, and procedures that span individual patient encounters in a longitudinal format." [0093]: "With Patricia's blood pressure ('BP') skyrocketing at 200/130, the ED doc orders a series of tests—UA, EKG, Chem Panel, and a Head CT… She is administered Hydralazine (through her IV) to control the hypertension and Tylenol 3 for her headache."
Bender discloses the linking of a medical condition (symptom/problem) to a treatment/diagnosis via ontology graph edges. [0359] (Bender): "a decision system that uses the decision tree 2300 may recommend that a patient record be labeled with 'gout' in response to a determination that the n-gram 'pain' is detected from the list of n-grams 2310 and that the n-gram 'swelling' is detected from the list n-grams 2320." [0329] (Bender): "some embodiments may update a decision tree by updating a set of n-grams used to make a labeling decision or associations between the n-grams used to make the labeling decision… some embodiments may replace one or more previous n-grams in a set of n-grams used at an operation represented by a decision tree node to label a record, such as by replacing the n-gram 'hypertension' with the n-gram 'hypertensive crisis' in response to a user's interaction with a UI." [0216] (Bender): "some embodiments may determine that the term 'nursing mother' is a type of 'nursing woman' based on a relationship type associated with a graph edge connecting the vertices."
11[c] – Medical type → specific medications:
Linthicum discloses the display of medication data in the EHR timeline. [0141]: "Data types that can be displayed using this type of visualization include lab values, medications, vital signs, episodic care events, problems, immunizations, allergies, and procedures." [0082]: "Widget 270 provides a listing of medications prescribed to the patient. The medications widget 270 includes a type 272 of medication, a quantity 274 of the medication, and a delivery mechanism 276 for the medication. In certain embodiments, selection of a medication can pull up further detail about the medication and its associated order." [0093]: "After prescribing the Hydralazine and Tylenol 3, Dr. Miller can verify the dosage and delivery methods and modify them following the C-section via a Medications widget 470."
Bender discloses the hierarchical mapping from a general category to specific instances. [0216] (Bender): "some embodiments may determine that the term 'nursing mother' is a type of 'nursing woman' based on a relationship type associated with a graph edge connecting the vertices corresponding with 'nursing mother' and 'nursing woman,' where the relationship type may be 'subset'." [0190] (Bender): "some embodiments may determine that the n-gram 'veggie burger' is a subset of the n-gram 'burger' in an ontology graph based on previous graph edges of the n-gram 'veggie burger' and the n-gram 'burger.'"
X. Rejection of Claim 12 (Different Data Types)
A. Claim Limitations
Limitation
Text
12[a]
the first data is of a first data type and the second data is of a second data type that is different to the first data type
B. Limitation-by-Limitation Mapping
12[a] - First data and second data are of different data types:
Linthicum discloses that the EHR timeline includes multiple different data types, and that zooming from one type reveals another. [0141]: "Data types that can be displayed using this type of visualization include lab values, medications, vital signs, episodic care events, problems, immunizations, allergies, and procedures that span individual patient encounters in a longitudinal format." [0124]: "The timeline 600 can be divided into a plurality of categories, such as encounters 610, results 612, problems 614, procedures 616 and medication 618." [0127]: "a user can view test(s), procedure(s), and/or examination(s) 930 saved with respect to a particular patient problem 920, such as atrial fibrillation" — the first data is a "problem" (atrial fibrillation) and the second data is "tests/procedures/examinations," which are different data types. [0033]: "Certain embodiments provide a three-dimensional timeline view of clinical data. A clinical data elements axis contains individual data elements. The elements are labeled/indicated by color depending on data element type and relativity to normative values."
Bender discloses that semantically linked n-grams can belong to different domains/data types. [0121] (Bender): "Workflows in various domains involve a mixture of general knowledge understanding, domain-specific understanding, and quantitative understanding. A significant challenge to using NLP for domain-specific workflows is the gap between a general knowledge environment and a domain-specific environment. Domain-specific knowledge may include differing interpretations of shared terminology, domain-specific logical relationships that differ between domains." [0128] (Bender): "some embodiments may obtain a second ontology graph in a medical knowledge domain… Using the second ontology graph, some embodiments may convert the word 'client' into 'patient,' convert the word 'after' into the phrase 'due to,' convert the word 'current' into the phrase 'ongoing,' convert the acronym 'CVA' into the word 'stroke'…" - the first data (business/technical terms) and second data (medical terms) are of different data types/domains.
XI. Motive to Combine1
The motivation to combine Linthicum and Bender is grounded in the following:
Linthicum at [0007] explicitly identifies the problem: "A variety of clinical data and medical documentation is available throughout various clinical information systems, but it is currently difficult to find, organize, and effectively present the information to physicians and other healthcare providers at a point of care." This establishes a clear need for improved semantic linking in EHR navigation.
Linthicum at [0120] teaches that "finer details related to the information may be present in the patient history context and become viewable and reviewable as the user drills down into the timeline" — but does not specify the computational mechanism for determining which data is "semantically linked."
Bender at [0037] and [0065] provides the well-known NLP/ML mechanisms (BERT, GLOVE, ELMo) for determining semantic relationships between terms, and at [0080]-[0081] provides the hierarchical ontology graph structure for organizing these relationships.
Before the effective filing date of the claimed inventions, it would have been obvious to one of ordinary skill in the art to apply Bender's NLP-based semantic linking (via BERT/GLOVE embedding vectors and hierarchical ontology graphs) to Linthicum's zoom-based EHR navigation, because both references are directed to the same field of healthcare information presentation and retrieval, and the combination would address the very problem Linthicum identifies at [0007] — the difficulty of finding and organizing clinical data.
Bender at [0102] states: "some embodiments may use one or more machine learning models to retrieve documents, summarizations based on documents, or the like as part of providing semantic search results after receiving a query" - this is directly applicable to the "automatically determine second data that is semantically linked to the first data" limitation.
Conclusion
The prior art made of record2 and NOT relied upon is considered pertinent to applicant's disclosure:
Pittappilly (US 2013/0067398): Semantic zoom techniques are described. In one or more implementations, techniques are described that may be utilized by a user to navigate to content of interest. These techniques may also include a variety of different features, such as to support semantic swaps and zooming “in” and “out.” These techniques may also include a variety of different input features, such as to support gestures, cursor-control device, and keyboard inputs. A variety of other features are also supported as further described in the detailed description and figures.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL EZEWOKO whose telephone number is 571 272 7850. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fonya Long can be reached on 571 270 5096. The fax phone number for the organization where this application or proceeding is assigned is 571-273-7850.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL I EZEWOKO/Primary Examiner, Art Unit 3682
1 That is to say, the overarching motivation to combine Linthicum and Bender
2Please see Form 892 for complete listing