Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The following Office action in response to communications received May 4, 2026. Claims 1, 13, 20, 23 and 29 have been amended. Claims 2, 12 and 24-25 have been deleted. Therefore, claims 1, 3-11, 13-23 and 26-29 are pending and addressed below.
Applicant’s amendments to the claims are sufficient to overcome the 35 USC § 112 rejections set forth in the previous office action dated December 5, 2025.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-11, 13-23 and 26-29 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Based upon consideration of all of the relevant factors with respect to the claims as a whole, the claims are directed to non-statutory subject matter which do not include additional elements that are sufficient to amount to significantly more than the judicial exception because of the following analysis:
The claims are directed to a process and a machine and therefore fall within statutory categories of invention (process and machine) under 35 U.S.C. §101.
Independent Claims 1 and 23 are directed to an abstract idea consisting of collecting, organizing, and comparing medical observation information (abnormalities, positions, evolution, and report content) across studies using tables, data sets, and navigation lists in order to support radiological decision-making.
Independent Claim 1 recites, in substance:
“A computer-implemented method for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion, the method comprises:
‘a step of screening a virtual current presentation of a current study of the body portion for the abnormality;
a step of identifying separate abnormalities in the current presentation;
wherein the step of screening and the step of identifying is carried out with a high degree of automatization or is carried out by a user, wherein the user is supported by a computer
a step of determining, for each identified abnormality, a position information indicating the position of the identified abnormality in the current presentation, wherein the step of determining is an automated or semi-automated step;
a step of generating a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion;
a step of providing the table to a user via a user interface
a step of correcting and/or validating the table, wherein the table is editable by the user via the user interface;
a step of determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, wherein the table indicates whether there is a related abnormality in the previous presentation or not.’”
Independent Claim 23 recites, in substance:
“A system for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion, the system comprises a computer system, said computer system provides a communication unit, a controller and a user interface, wherein:
‘the communication unit is configured to communicate with the controller and the user interface;
the controller is configured for screening in an automated or semi- automated manner a virtual current presentation of a current study of the body portion for the abnormality, for identifying in an automated or semi- automated manner separate abnormalities in the current presentation, and for determining in an automated or semi-automated manner and for each identified abnormality a position information indicating the position of the identified abnormality in the current presentation;
the controller is configured for generating in an automated manner a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion;
the user interface is configured for providing the table to a user, and for correcting and/or validating the table, wherein the table is editable by the user via the user interface;
the system is configured for determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, and wherein the table indicates whether there is a related abnormality in the previous presentation or not;
or a corresponding configuration in which user input identifies the abnormalities and the controller determines position information, generates the table, and performs the same previous-study comparison.”
Under their broadest reasonable interpretation, the limitations of Claims 1, 23, 28 and 29 and the dependent claims therefrom, cover the performance of:
Mental processes, including concepts performed in the human mind such as observation, evaluation, judgment, and opinion (e.g., identifying abnormalities, deciding whether a current abnormality is “related” to a prior abnormality, validating or correcting a table of findings, setting evolution tags, and deciding which probability segmentation map best represents the abnormality).
Certain methods of organizing human activity, including managing clinical workflows and human decision-making (e.g., providing lists of studies and presentations, assigning unique identifiers, navigating between studies, creating summary lists in a radiological report, and organizing how radiologists review and compare current and prior findings).
Mathematical concepts, where the claims include calculations and model-based computations (e.g., running artificial intelligence models to assign probabilities to voxels, generating probability segmentation maps, and adjusting probability thresholds to correct segmentations).
But for the recitation of generic computer components (a “computer system,” “controller,” “communication unit,” “user interface,” “image viewer,” “models of an artificial intelligence”), the claim steps are simply organizing, annotating, and comparing medical image findings and report information across studies, including generating and editing tables of abnormalities, creating lists of study/presentation identifiers, tracking evolution over time, and presenting this information to a user to assist radiological observation and reporting.
The claims recite additional elements such as:
A generic “computer system,” including a “communication unit,” “controller,” and “user interface.”
Generic “virtual presentations,” “virtual medical presentation,” “image viewer,” and “virtual template of the body portion.”
Generic “models of an artificial intelligence” that assign probabilities or labels to voxels or regions.
Tables, data sets, lists of studies, lists of presentations, evolution tags, and radiological reports linked to the tables.
Generic user interface tools, such as “one-click tools,” views, and a “quantitative correction area” for editing entries in the table and switching between different presentations.
These elements are recited at a high level of generality and merely use generic computer components to perform generic computer functions: receiving data (current and prior image presentations, data sets), running models, generating and populating tables, generating and displaying lists, accepting user input, displaying images and reports, and storing and transmitting information.
Looking to the specification (as reflected in the detailed claim language), these components are described at a high level of generality (See Specification page 11 || 12-15: The method is a computer-implemented method or a computer supported method at least. Therefore, it is a further object of the invention to provide a related computer program, a related computer readable medium, a related computer-readable signal, and a related data carrier signal. Page 57 || 17-18: The computer that is caused to carry out the method may be a computer or a computer network of the system in any embodiment disclosed) as conventional computing platforms: standard computing devices with processors and memory executing software instructions to receive imaging data, run AI models, populate tables, generate reports, store data sets, and communicate with external systems and user interfaces.
The additional elements do not integrate the abstract idea into a practical application because:
They do not improve the functioning of a computer, network, model architecture, or image-processing architecture. The claims use known components (controllers, user interfaces, image viewers, models of AI) in their ordinary capacity to process data, generate tables, and display information.
They do not effect a transformation of an article into a different state or thing beyond organizing medical information in tables and reports.
They merely apply the abstract idea of organizing and comparing medical abnormalities and positions across studies in the particular field of radiology, which is a field-of-use limitation.
Steps such as “providing the table to the user,” “providing lists of studies and presentations,” “navigating” by selecting items, generating a radiological report, or allowing the user to correct entries in a “quantitative correction area” amount to extra-solution activity (presentation of results and user interface conventions) around the core abstract idea.
Accordingly, the claims are “directed to” the identified abstract idea under Step 2A and do not integrate the judicial exception into a practical application.
Under Step 2B, the claim elements, individually and in combination, do not amount to significantly more than the abstract idea itself.
The use of generic processors, memory, communication units, controllers, user interfaces, and image viewers is conventional and routine in computing systems, including medical imaging workstations.
The recited “models of an artificial intelligence” that assign probabilities or labels to voxels or regions are described functionally (e.g., assigning probabilities, generating segmentation maps, assigning regions to anatomical locations) without any specific improvement to model architecture or training technique beyond using such models in a straightforward manner to implement the abstract idea.
The generation and use of tables, current and previous data sets, evolution tags, lists of studies and presentations, and summary lists within a radiological report are conventional data organization and reporting operations. They amount to collecting, categorizing, filtering, and presenting information, which courts have held to be forms of abstract data manipulation.
Any storage, display, transmission, or reporting steps (e.g., providing tables and lists to the user, displaying virtual presentations in an image viewer, generating radiological reports) are insignificant extra-solution activity that occur before or after the abstract data analysis and organization, and do not provide an inventive concept.
Considering the claim elements as an ordered combination, the claims simply implement the abstract idea of organizing, tracking, and reporting medical abnormalities across current and prior imaging studies on a generic computer system, using routine AI models and conventional user interface constructs. This ordered combination does not add a “specific limitation beyond the judicial exception” that is more than well-understood, routine, conventional activity in the field, and thus does not supply an inventive concept.
Accordingly, claims 1, 3–11, 13–23 and 26–27 are directed to an abstract idea (mental processes, organizing human clinical activity, and mathematical concepts) without significantly more, and therefore are not patent-eligible under 35 U.S.C. §101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3–11, 13–23 and 26–27 are rejected under 35 U.S.C. 103 as being unpatentable over Pub. No.: US 2017/0337329 A1 to Liu et al. in view of Pub. No.: US 2012/0183188 A1 to Moriya.
As per Claim 1:
“A computer-implemented method for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion, the method comprises:
-- a step of screening a virtual current presentation of a current study of the body portion for the abnormality;
-- a step of identifying separate abnormalities in the current presentation;
-- wherein the step of screening and the step of identifying is carried out with a high degree of automatization or is carried out by a user, wherein the user is supported by a computer
-- a step of determining, for each identified abnormality, a position information indicating the position of the identified abnormality in the current presentation, wherein the step of determining is an automated or semi-automated step;
-- a step of generating a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion;
-- a step of providing the table to a user via a user interface
-- a step of correcting and/or validating the table, wherein the table is editable by the user via the user interface;
-- a step of determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, wherein the table indicates whether there is a related abnormality in the previous presentation or not.”
Liu et al. teaches:
“a step of screening a virtual current presentation of a current study of the body portion for the abnormality; a step of identifying separate abnormalities in the current presentation; wherein the step of screening and the step of identifying is carried out with a high degree of automatization or is carried out by a user, wherein the user is supported by a computer.”
Liu et al. teaches that “a computer-implemented method for automatically generating a radiology report includes a computer receiving an input dataset comprising a plurality of multidimensional patient images and patient information and parsing the input dataset using learned models to determine a clinical domain and relevant image annotations” and that the system uses “discriminative classifiers … hierarchical models … recurrent neural networks … deep Q-learning” to automatically determine findings from images. Liu et al. further explains that “the current standard of practice for reporting in radiology requires clinicians to review each individual slice … and dictate an oral summary of findings,” and the disclosed system replaces this manual workflow with an automatic parsing and report generation process that nevertheless presents a “smart report in natural language with embedded links” to the user for review. See paragraphs [0002–0004], [0031–0038].
Examiner interprets: This disclosure matches and interprets the claimed “screening” and “identifying” as the parsing of the current volumetric images to detect relevant abnormalities (image annotations) with a high degree of automation, while the clinician (user) is supported by the computer via the smart report GUI.
“a step of determining, for each identified abnormality, a position information indicating the position of the identified abnormality in the current presentation, wherein the step of determining is an automated or semi-automated step.”
Liu et al. teaches “annotation specification” and “annotation tables” where each entry is associated with anatomical landmarks and surfaces, such as “Liver dome landmark, 3DPoint,” “Iliac bifurcation landmark, 3DPoint,” “Liver surface, mesh,” “Spleen surface, mesh,” “L. Kidney surface, mesh,” and “R. Kidney surface, mesh.” See FIG. 3E, “CT Abdominal Example modality, CT … Liver dome landmark, 3DPoint … Liver surface, mesh … R. Kidney surface, mesh.” Liu et al. further describes that image parsing models are trained to produce such annotations, and scriptable rules act on them to derive clinical concepts. See paragraphs [0034–0038].
Examiner interprets: These landmarks, surfaces, and meshes provide position information for each identified abnormality in the current presentation, determined in an automated fashion by the image parsing models.
“a step of generating a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion.”
Liu et al. teaches “Annotation Table” and “Report template with clinical concepts and their ranges” and explains that the system “populates an annotation table using the relevant image annotations and applies one or more domain-specific scriptable rules to populate a report template based on the annotation table.” See FIG. 3A (“Annotation Table”) and FIG. 4A (“Filled annotation table … Filled report template … Generate natural language report”). The CT abdominal example shows a structured table with headings “GENERAL – image modality CT – domain Abdomen – contrast Y … FINDINGS – ABDOMEN – LIVER – normal size Y … GALLBLADDER – calcified gallstones Y.” See FIG. 3C/4C and see paragraphs [0002–0004], [0031–0038].
Examiner interprets: In this table, each identified abnormality (e.g., necrotic area in right hepatic dome, gastric wall thickening, lymph node) is associated with an underlying annotation (e.g., liver surface mesh, gastric antrum landmark) indicating the position of the abnormality in the body portion. This matches the claimed “table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion.”
“a step of providing the table to a user via a user interface.”
Liu et al. teaches that “the Clinical Report Module uses the populated clinical report template and Natural Language Generation (NLG) to generate the report,” and that “the output is smart report in natural language with embedded links that navigates back to image coordinates of features which correlate with the findings,” presented in a GUI. FIG. 2 shows a “smart radiology report on the right with embedded links back to image features as described in the findings.” See paragraphs [0033–0034].
This smart report is the user-facing representation of the populated table of abnormalities and their positions, provided via a user interface.
“a step of correcting and/or validating the table, wherein the table is editable by the user via the user interface.”
Liu et al. teaches that the smart report is “presented in an interactive graphical user interface which allows a user to retrieve images depicting the one or more image features via activation of one or more links embedded in the natural language radiology report." See paragraphs [0002–0004], [0031–0038]. Liu et al. further emphasizes that standardized templates and concepts streamline comparison to longitudinal data and similar cases, enabling radiologists to review and adjust findings. See paragraphs [0002–0004], [0031–0038].
One of ordinary skill would interpret this interactive smart report and template system as allowing the user to validate and, as needed, correct the automatically populated table entries.
“a step of determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion … wherein the table indicates whether there is a related abnormality in the previous presentation or not.”
Liu et al. teaches that reports “show morphological progression of primary hepatocellular carcinoma and progressive metastatic disease in the upper abdomen, and retroperitoneal space,” and include “Patient Prior Images” and “This report is generated by referencing the following similar findings,” listing prior CT studies and referencing similar findings. See FIG. 2 and FIG. 4C (“CT volume shows morphological progression … Patient Prior Images … This report is generated by referencing the following similar findings …”).
Liu et al. therefore discloses determining, for each current abnormality, whether corresponding abnormalities are present in prior studies and describing progression vs prior examinations in the structured report. This corresponds to the claimed step of determining whether there is a related abnormality in a previous presentation and indicating that relationship in the report/table.
Liu et al. fails to explicitly teach:
“a computer-implemented method for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion,” insofar as “virtual medical presentation” and “supporting observation” are framed in the claim as a specific image-viewer-centric workflow rather than the more general volumetric image and report generation described by Liu et al.
“wherein the step of screening and the step of identifying is carried out with a high degree of automatization or is carried out by a user, wherein the user is supported by a computer,” to the extent that the claim emphasizes a dual mode (automated or user-driven) for screening/identifying in the image viewer, whereas Liu predominantly focuses on automated parsing and then user review of the generated report, not explicit manual identification of abnormalities in the current image presentation.
“a step of determining, for each identified abnormality, a position information indicating the position of the identified abnormality in the current presentation, wherein the step of determining is an automated or semi-automated step,” at the level of explicitly computing a position information in the current presentation per abnormality that is later used as a key for correspondence to prior presentations, as opposed to using general anatomical annotations and landmarks.
“a step of correcting and/or validating the table, wherein the table is editable by the user via the user interface,” to the extent that Liu does not expressly state that the annotation table itself is directly editable, even though the report and its contents are interactively reviewed.
“a step of determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation … by determining, using the position information … whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, wherein the table indicates whether there is a related abnormality in the previous presentation or not,” because Liu’s prior-comparison is expressed in terms of clinical report templates and longitudinal data, rather than explicitly using per-lesion position information in the current presentation as a key to search per-lesion position information in the previous presentation and then annotating a table with “related/not related” flags.
Moriya teaches:
“a computer-implemented method for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion.”
Moriya teaches “a medical image display apparatus … for displaying, when a lesion character in an image reading report is specified, an image of the lesion area” and describes displaying a “second medical image reconstructed from a plurality of medical images” (e.g., 3D VR/MPR), which is a virtual medical presentation of the body portion. Moriya emphasizes that the apparatus “allows more accurate image interpretation by displaying lesion information described in an image reading report in a medical image.” See paragraphs [0001–0003], [0017–0023].
This directly supports a computer-implemented method that supports observation of abnormalities in or on a body portion displayed in a virtual medical presentation.
“wherein the step of screening and the step of identifying is carried out with a high degree of automatization or is carried out by a user, wherein the user is supported by a computer.”
Moriya teaches that “in image diagnoses using medical images, in general, diagnosticians make diagnoses with reference to image reading reports generated by radiologists” and that the system provides link characters and position indicators allowing the user to specify lesion characters and positions using input devices; the apparatus then automatically stores and displays the corresponding link positions and association indicators. See paragraphs [0004–0008], [0054–0057].
This discloses a user-driven identification of lesions with computer support (e.g., automatic link position computation and display), which complements Liu’s automated parsing.
“a step of determining, for each identified abnormality, a position information indicating the position of the identified abnormality in the current presentation, wherein the step of determining is an automated or semi-automated step.”
Moriya teaches that “a lesion storage unit … stores the lesion character and a position of the lesion area in association with each other” and that “a link position storage unit … calculates a position in the second medical image corresponding to the position of the lesion area and stores the calculated position as a link position.” See paragraphs [0010–0014], [0120–0122].
In Moriya, for each identified lesion (abnormality), the system automatically calculates and stores a position in the reconstructed current presentation (second medical image). This matches the claimed step of determining, for each identified abnormality, a position information indicating the position of the identified abnormality in the current presentation.
“a step of correcting and/or validating the table, wherein the table is editable by the user via the user interface.”
Moriya teaches processes for changing lesion characters and updating lesion information, including “START – SPECIFY LESION CHARACTER – CHANGE LESION CHARACTER – UPDATE LESION INFORMATION – END.” See FIG. 6 and paragraphs [0056–0057].
Although Moriya frames the storage in terms of lesion storage and link position storage rather than an explicit “table,” one of ordinary skill would understand that the stored lesion information (characters and positions) can be represented and edited in a table structure through the user interface.
“a step of determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, wherein the table indicates whether there is a related abnormality in the previous presentation or not.”
Moriya teaches that “the image reading report may include a plurality of image reading reports with respect to past medical images or with respect to a plurality of different lesion positions” and that the system “stores lesion information including lesion positions for past medical images and converts the positions of lesion areas in past medical images to corresponding positions in the reconstructed current medical image.” See paragraphs [0043–0044], [0141–0146].
Moriya further explains that “the link character and link position … are displayed in association with each other,” and multiple link characters and positions for multiple lesions are displayed so that “the positions of a plurality of lesion areas and character strings representing a plurality of corresponding lesions may be recognized at a time by simply observing the medical image, whereby the diagnosis may be made more easily.” See paragraphs [0120–0124].
This teaches using positions in the current virtual presentation and positions corresponding to lesions from past medical images to determine correspondence and display linked characters/positions, which Examiner interpreted as determining whether there is a related abnormality in a previous presentation for each identified abnormality in the current presentation.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to include the method features taught by Moriya within the method taught by Liu et al., with the motivation of improving radiologists’ ability to observe and track medical abnormalities across current and prior imaging studies by combining structured annotation tables and automatic report generation (Liu) with explicit position-based linking and interactive image/report display (Moriya). Liu et al. already emphasizes the importance of managing complex domain knowledge, extracting structured representations of features from images, and comparing current findings to prior reports for longitudinal assessment, while Moriya addresses the need for accurate recognition of lesion positions in reconstructed images and linkage to past reports. See paragraphs [0120–0127], [0141–0146] of Moriya.
As per Claim 3:
“The method according to claim 1, wherein the table comprises entries for identified abnormalities, wherein an entry of the table comprises a sub-entry that relates to a characteristic of the abnormality attributed to the entry.”
Liu et al. teaches:
“the table comprises entries for identified abnormalities.”
Liu et al. teaches an “annotation table” and “report template with clinical concepts and their ranges” and describes that “the computer populates an annotation table using the relevant image annotations and applies one or more domain-specific scriptable rules to populate a report template based on the annotation table.” See FIG. 3A (“Annotation Table”) and paragraph. The CT abdominal example shows a structured table with headings “GENERAL – image modality CT – domain Abdomen – contrast Y” and “FINDINGS – ABDOMEN – LIVER – normal size Y … GALLBLADDER – calcified gallstones Y,” where each row is an entry corresponding to an identified organ condition or abnormality. See FIG. 3C/4C.
Thus, Liu discloses a table comprising entries for identified abnormalities or findings.
“wherein an entry of the table comprises a sub-entry that relates to a characteristic of the abnormality attributed to the entry.”
Liu et al. teaches that each table entry and report template entry includes multiple fields (sub-entries) relating to characteristics of the finding, such as organ size, wall thickness, presence of calcified gallstones, lesion size, SUV values, and other clinical concepts. The CT abdominal template lists, for example, “LIVER – normal size Y,” “GALLBLADDER – calcified gallstones Y,” “ADRENALS – normal size Y,” etc., where “normal size,” “calcified gallstones,” and “Y” are sub-entries relating to characteristics of the abnormality or organ. See FIG. 3C/4C and paragraphs [0032–0035].
This corresponds to an entry comprising sub-entries that relate to characteristics attributed to the abnormality.
Liu et al. fails to explicitly teach:
Using the exact words “sub-entry that relates to a characteristic of the abnormality attributed to the entry,” although Liu’s multiple fields per finding match this structure functionally.
Moriya teaches:
“wherein an entry of the table comprises a sub-entry that relates to a characteristic of the abnormality attributed to the entry.”
Moriya teaches association tables and lesion information where each lesion entry includes lesion characters and additional descriptive text (e.g., phrases before and after the lesion character) that provide characteristics of the lesion. FIG. 13 shows “an association table associating lesion areas with lesion character candidates,” and FIG. 11 shows comments (e.g., “ASCITIC FLUID … IS IN REMISSION, AS TREATMENT”) associated with each lesion character, which function as sub-entries relating to lesion characteristics. See paragraphs [0063–0067], [0126–0128].
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 4:
“The method according to claim 3, wherein, besides the position information, at least one of the following characteristics of the abnormality is listed in the sub-entries: the anatomical extent of the abnormality, the anatomical aspect of the abnormality, the signal characteristic of the abnormality, whether the abnormality is observed for the first time or not, the volume of the abnormality, information concerning contrast uptake, information derived from at least one of the above-listed information.”
Liu et al. teaches:
“wherein, besides the position information, at least one of the following characteristics of the abnormality is listed in the sub-entries: the anatomical extent of the abnormality, the anatomical aspect of the abnormality, the signal characteristic of the abnormality, whether the abnormality is observed for the first time or not, the volume of the abnormality, information concerning contrast uptake.”
Liu et al. teaches that the clinical report template and annotation table include clinical concepts and ranges such as organ size, position, vessel lumen, tissue texture, tissue density, wall thickness, and contrast agent usage. See paragraphs [0032–0034]. The CT abdominal examples show findings like “A 52 x 45 mm necrotic area in the right hepatic dome, increased from 40 mm,” which indicates anatomical extent (“52 x 45 mm”), anatomical aspect (“right hepatic dome”), volume related measurement, and progression (“increased from 40 mm,” indicating that the abnormality is not observed for the first time). See FIG. 4C.
Liu further describes “contrast agent, Y” in annotation tables for CT modalities and uses SUV values as signal characteristics (e.g., “The maximum SUV was 6.0 and is now 9.9”), which reflect signal characteristics and contrast uptake information. See FIG. 3C/4C and paragraphs [0039–0043].
“information derived from at least one of the above-listed information.”
Liu et al. teaches that scriptable rules take annotations and measurements (e.g., liver volume, wall thickness, SUV) and derive higher-level concepts such as “normal size” or “progressive metastatic disease.” For example, RULELIVERNORMALSIZE uses liver volume measurements to determine whether liver size is normal and stores this as a classification. See paragraph and FIG. 3C.
These derived classifications (e.g., “normal size,” “progressive metastatic disease”) are information derived from anatomical extent, volume, and signal characteristics.
Liu et al. fails to explicitly teach:
Using the exact phrase “whether the abnormality is observed for the first time or not,” although progression statements (“increased from 40 mm”) and prior image references demonstrate that abnormalities are tracked over time.
Moriya teaches:
“whether the abnormality is observed for the first time or not,” and “information derived from at least one of the above-listed information.”
Moriya teaches that the image reading report “may include a plurality of image reading reports with respect to past medical images or with respect to a plurality of different lesion positions,” and that link characters and comments (phrases before and after lesion characters) can include information about past states. See paragraphs [0043–0044], [0126–0129].
This shows that the system records whether a lesion was present before (“pointed out last time”) and its current state (“in remission”), providing information about whether the abnormality is observed for the first time or not and derived information based on past and current characteristics.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 5:
“The method according to claim 3, wherein the characteristic of an identified abnormality is determined in an automated or semi- automated manner and wherein the sub-entry of the entry for the identified abnormalities is filled with the characteristic in an automated or semi-automated manner.”
Liu et al. teaches:
“the characteristic of an identified abnormality is determined in an automated or semi- automated manner.”
Liu et al. teaches that “a computer-implemented method … parsing the input dataset using learned models to determine a clinical domain and relevant image annotations,” and that image parsing models (e.g., deep learning, marginal space learning, regression models) are used to derive clinical concepts and measurements (e.g., organ volumes, wall thickness). See paragraphs, [0037–0038].
These models automatically determine characteristics of identified abnormalities (e.g., size, volume, SUV, normal/abnormal classifications).
“wherein the sub-entry of the entry for the identified abnormalities is filled with the characteristic in an automated or semi-automated manner.”
Liu et al. teaches that “given the annotation values, the Rules Module 110B uses the scriptable rules generated offline to select the clinical report template and generate the values for associated clinical report concepts,” and that “the computer populates an annotation table using the relevant image annotations and applies one or more domain-specific scriptable rules to populate a report template based on the annotation table.” FIG. 3E and paragraphs [0037–0038].
This means the sub-entries of the table (clinical concepts and ranges) are filled automatically based on determined characteristics of abnormalities.
Liu et al. fails to explicitly teach:
The exact phrase “semi- automated,” although Liu’s combination of automated parsing and rule application, possibly reviewed by human users, corresponds to semi-automation.
Moriya teaches:
Semi-automation in filling sub-entries (lesion characteristics).
Moriya teaches that lesion characters and positions can be inserted via user input (manual) while the system automatically calculates and stores corresponding link positions and association indicators. See paragraphs [0054–0057], [0120–0122].
This provides a semi-automated workflow in which some characteristics (positions) are automatically derived and stored, and some text is manually entered, aligning with “semi- automated manner.”
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 6:
“The method according to claim 1, wherein the step of screening and the step of identifying are automated or semi-automated steps, wherein the table is generated in an automated manner, and wherein a virtual presentation of the body portion comprised in the current study is provided to the user via the user interface in the step of providing the table, wherein the user interface comprises an image viewer and the virtual presentation is displayed in the image viewer.”
Liu et al. teaches:
“wherein the step of screening and the step of identifying are automated or semi-automated steps, wherein the table is generated in an automated manner.”
Liu et al. teaches automated screening and identification: “parsing the input dataset using learned models to determine a clinical domain and relevant image annotations,” and “Image Processing Module 110D … uses these models to determine domain, modality and annotations.” See paragraphs, [0037–0038]. Liu further teaches that “the computer populates an annotation table using the relevant image annotations and applies … scriptable rules to populate a report template based on the annotation table.” See FIG. 4A and paragraph.
These disclosures correspond to automated screening, identification, and automated table generation.
“wherein a virtual presentation of the body portion comprised in the current study is provided to the user via the user interface … wherein the user interface comprises an image viewer and the virtual presentation is displayed in the image viewer.”
Liu et al. teaches that the system includes “Diagnostic 3D Volumes” and that a smart radiology report with embedded links allows users to retrieve images depicting described features via GUI. FIG. 2 shows images (CT slices/volumes) on the left and the report on the right, with the user able to switch between them. See paragraphs [0029–0033].
The images shown are virtual presentations of the body portion, and the display on the user computer functions as an image viewer within the user interface.
Liu et al. fails to explicitly teach:
The explicit phrase “image viewer,” though the GUI displaying diagnostic volumes serves that function.
Moriya teaches:
“wherein the user interface comprises an image viewer and the virtual presentation is displayed in the image viewer.”
Moriya teaches “medical image display means” and “display section 301” that displays the second medical image reconstructed from multiple medical images (e.g., volume rendering), and FIG. 8 shows a screen with a reconstructed image labeled “REFERENCE IMAGE” that the radiologist uses to observe lesions. See FIGS. 2, 8 and paragraphs [0051–0053], [0058–0060].
This corresponds to an image viewer displaying the virtual presentation.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 7:
“The method according to claim 1, wherein the step of identifying is carried out by the user.”
Liu et al. teaches:
Automated identification but not explicitly “wherein the step of identifying is carried out by the user.”
Liu et al. focuses on automated parsing using learned models to identify relevant annotations and findings. See paragraphs, [0031–0038]. Liu implies user oversight and review via smart report GUI but does not expressly state that identification itself is carried out by the user.
Liu et al. fails to explicitly teach:
“wherein the step of identifying is carried out by the user.”
Moriya teaches:
“wherein the step of identifying is carried out by the user.”
Moriya teaches workflows where the user specifies lesion positions and lesion characters in the image reading report and medical image via input section 303 (mouse, keyboard). FIG. 5 shows the “FINDING FIELD” where the radiologist inputs lesion descriptions, and FIG. 6 shows the flowchart: “START – SPECIFY LESION CHARACTER – CHANGE LESION CHARACTER – UPDATE LESION INFORMATION – END.” See paragraphs [0055–0058], [0130–0133].
Thus, the identification of lesions (abnormalities) is carried out by the user, supported by the system.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 8:
“The method according to claim 7, wherein the step of generating a table is semi-automated by an entry for an identified abnormality being generated by indicating the identified abnormality in a virtual presentation displayed in the image viewer.”
Liu et al. teaches:
Automated table generation but not explicitly semi-automated entry generation by indicating abnormalities in the virtual presentation.
Liu et al. teaches automatic population of the annotation table based on image parsing models, but does not expressly state that individual entries are generated in response to user indication of abnormalities in the image viewer. See paragraphs, [0035–0038].
Liu et al. fails to explicitly teach:
“semi-automated by an entry for an identified abnormality being generated by indicating the identified abnormality in a virtual presentation displayed in the image viewer.”
Moriya teaches:
“semi-automated by an entry for an identified abnormality being generated by indicating the identified abnormality in a virtual presentation displayed in the image viewer.”
Moriya teaches that the radiologist indicates lesion positions in the medical image (virtual presentation) displayed on display section 301 and then inputs lesion characters linked to those positions. The system stores the lesion character and position as lesion information and computes the corresponding link position in the reconstructed image. See FIGS. 5, 8, 9 and paragraphs [0054–0058], [0120–0121].
This corresponds to semi-automation where the user indicates the abnormality in the virtual presentation, and the system automatically generates the associated lesion entry (with position information) in a table of lesion information.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 9:
“The method according to claim 7, comprising the further steps of:
‘an automated step of screening the virtual current presentation of a current study of the body portion for the abnormality;
an automated step of identifying separate abnormalities in the current presentation;
an automated step of determining, for each abnormality identified in the automated step of identifying, a position information indicating the position of the identified abnormality in the current presentation;
an automated step of generating a table listing the abnormalities identified in the automated step of identifying and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion.’”
Liu et al. teaches:
“an automated step of screening the virtual current presentation of a current study of the body portion for the abnormality; an automated step of identifying separate abnormalities in the current presentation.”
Liu et al. teaches automatic screening and identification: “parsing the input dataset using learned models to determine a clinical domain and relevant image annotations.” See paragraph. The Image Processing Module determines domains and annotations from the images, effectively screening and identifying abnormalities. See paragraphs [0037–0038].
“an automated step of determining, for each abnormality identified in the automated step of identifying, a position information indicating the position of the identified abnormality in the current presentation.”
Liu et al. teaches that annotations include anatomical landmarks and surfaces (e.g., “Liver dome landmark, 3DPoint,” “Liver surface, mesh,” “Kidney surface, mesh”), which provide positional information for each abnormality. See FIG. 3E and paragraphs [0039–0043].
These positions are determined automatically by image parsing models.
“an automated step of generating a table listing the abnormalities identified in the automated step of identifying and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion.”
Liu et al. teaches automatically populating an annotation table and report template based on annotations. See FIG. 3A, FIG. 4A and paragraphs [0002–0004], [0031–0043].
The resulting tables list abnormalities and their associated annotations (including positional attributes).
Liu et al. fails to explicitly teach:
Using the exact phrase “virtual current presentation,” though the images and volumes serve as virtual current presentations.
Moriya teaches:
Virtual current presentations in an image viewer.
Moriya teaches reconstructed second medical images (e.g., volume rendering) displayed in a medical image display screen, which act as virtual presentations of the body portion. See paragraphs [0030–0034], [0058–0060], FIG. 8.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 10:
“The method according to claim 9, comprising a step of providing the table generated in the automated step of generating a table or at least an entry thereof to the user.”
Liu et al. teaches:
“a step of providing the table generated in the automated step of generating a table or at least an entry thereof to the user.”
Liu et al. teaches that the filled report template (which reflects the annotation table generated automatically) is used to generate a smart radiology report in natural language, presented in a GUI to the user. See FIG. 2 and paragraphs [0033–0034]. The report entries correspond to table entries and are provided to the user via the GUI.
Thus, Liu discloses providing automatically generated table contents (or entries) to the user.
Liu et al. fails to explicitly teach:
Using the exact phrase “table generated in the automated step of generating a table,” though this is implicit in the populated annotation table.
Moriya teaches:
Providing lesion tables or entries to the user via GUI.
Moriya teaches displaying lesion characters, positions, and comments in the medical image display screen and finding field. See FIG. 8, FIG. 11 and paragraphs [0055–0058], [0120–0127]. The lesion entries form a table of lesions, and the user interface provides these entries to the user.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 11:
“The method according to claim 10, wherein the step of providing the table generated in the automated step of generating a table or at least one entry thereof to the user is carried out if there is a difference between this table and the table generated in the step of generating a table, wherein the abnormalities listed in the latter table have been identified in the step of identifying carried out by the user.”
Liu et al. teaches:
Automated generation of one table and user-involved generation or correction of another table, but not explicitly conditional provision based on differences.
Liu et al. teaches that automated parsing and rule application generate an annotation table and report template, and that the user can interactively review the smart report and associated images. See paragraphs, [0033–0035]. It implies that automated annotations may differ from user interpretations, but does not explicitly state that automated table contents are only provided if differences exist.
Liu et al. fails to explicitly teach:
“is carried out if there is a difference between this table and the table generated in the step of generating a table, wherein the abnormalities listed in the latter table have been identified in the step of identifying carried out by the user.”
Moriya teaches:
Multiple tables/sets of lesion information and their correspondence, which suggests detection of differences between system-generated and user-specified lesion information.
Moriya teaches that lesion storage means holds lesion characters and positions from image reading reports, including multiple reports and lesion positions across past images, and that link position storage means computes positions in reconstructed images. See paragraphs [0043–0044], [0120–0124]. The system can display multiple link characters and positions corresponding to multiple lesion areas, and differentiates them via association indicators.
One of ordinary skill would recognize that, when both system-derived lesion tables and user-entered lesion tables exist, differences between them (e.g., missing lesions or positional discrepancies) can be detected and highlighted, and that it would be beneficial to provide automated table entries to the user specifically when differences exist, in order to prompt reconciliation.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 13:
“The method according to claim1, wherein the step of determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of the body portion comprises:
‘a step of generating a current data set comprising, for each identified abnormality, a current data element, wherein the current data element comprises the position information indicating the position of the identified abnormality in the current presentation, wherein the step of generating a current data set is an automated step;
a step of receiving, in an automated manner, a previous data set of the previous presentation of the body portion, wherein the previous data set comprises, for each abnormality identified in the previous presentation, a previous data element, wherein the previous data element comprises a position information indicating the position of the abnormality identified in the previous presentation;
a step of screening, for each current data element, the previous data set for a previous data element comprising the abnormality at a position in the previous presentation that corresponds to the position of the abnormality according to the current data element, wherein the step of screening the previous data set is an automated step.’”
Liu et al. teaches:
“a step of generating a current data set comprising, for each identified abnormality, a current data element, wherein the current data element comprises the position information indicating the position of the identified abnormality in the current presentation, wherein the step of generating a current data set is an automated step.”
Liu et al. teaches an “annotation specification” and “annotation table” in which clinical report concepts (e.g., organ size, position, wall thickness) are linked to image annotations such as “Liver dome landmark, 3DPoint,” “Iliac bifurcation landmark, 3DPoint,” “Liver surface, mesh,” “Kidney surface, mesh.” See FIG. 3E and paragraphs [0034–0038]. Liu explains that the Image Processing Module “uses these models to determine domain, modality and annotations,” and that the Rules Module populates the annotation table. See paragraphs [0037–0038].
These automatically generated annotation entries are current data elements, each comprising position information indicating the position of the identified abnormality or organ feature in the current presentation. The step of generating the current data set (annotation table) is automated.
“a step of receiving, in an automated manner, a previous data set of the previous presentation of the body portion, wherein the previous data set comprises, for each abnormality identified in the previous presentation, a previous data element, wherein the previous data element comprises a position information indicating the position of the abnormality identified in the previous presentation.”
Liu et al. teaches that the Medical Information Database “comprises diagnostic multidimensional image data, their radiology reports and related non-image patient metadata” and that the Radiology Report Generation Computer retrieves patient-specific input datasets, which may include prior reports and images. See paragraphs [0029–0030]. The CT abdominal example shows prior reports and images being referenced: “Patient Prior Images … This report is generated by referencing the following similar findings,” listing prior CT studies with their findings. See FIG. 4C.
These prior reports and their structured annotations correspond to previous data sets comprising previous data elements with positional annotations for abnormalities.
“a step of screening, for each current data element, the previous data set for a previous data element comprising the abnormality at a position in the previous presentation that corresponds to the position of the abnormality according to the current data element, wherein the step of screening the previous data set is an automated step.”
Liu et al. teaches that rules and templates are used to compare current findings to prior findings and to determine morphological progression (e.g., “A 52 x 45 mm necrotic area … increased from 40 mm measured in [prior study]” and “maximum SUV was 6.0 and is now 9.9”). See FIG. 4C and paragraphs, [0040–0043].
Although Liu does not explicitly say that positional correspondence is used, the combination of anatomical landmarks and prior images implies that the system screens prior structured data for entries that correspond spatially and clinically to current entries.
Liu et al. fails to explicitly teach:
Using the exact language “current data element” and “previous data element” and explicit screening “for a previous data element comprising the abnormality at a position … that corresponds to the position of the abnormality according to the current data element.”
Moriya teaches:
“a step of generating a current data set comprising, for each identified abnormality, a current data element … the current data element comprises the position information indicating the position of the identified abnormality in the current presentation.”
Moriya teaches “lesion storage means” which stores lesion characters and positions, and “link position storage means” which calculates a position in the second medical image (current presentation) corresponding to the lesion area and stores the calculated position as a link position. See paragraphs [0010–0014].
These stored entries (lesion character + position, and link position) form current data elements comprising position information for each identified abnormality in the current virtual presentation.
“a step of receiving, in an automated manner, a previous data set of the previous presentation of the body portion, wherein the previous data set comprises, for each abnormality identified in the previous presentation, a previous data element, wherein the previous data element comprises a position information indicating the position of the abnormality identified in the previous presentation.”
Moriya teaches that “the image reading report may include a plurality of image reading reports with respect to past medical images or with respect to a plurality of different lesion positions,” and that lesion storage means associates lesion characters with positions of lesion areas in these past images. See paragraphs [0043–0044].
These past lesion entries form previous data elements with position information indicating lesion positions in previous presentations.
“a step of screening, for each current data element, the previous data set for a previous data element comprising the abnormality at a position in the previous presentation that corresponds to the position of the abnormality according to the current data element, wherein the step of screening the previous data set is an automated step.”
Moriya teaches that link position storage means “calculates a position in the second medical image corresponding to the position of the lesion area” and that multiple link positions and link characters are displayed, with association indicators, so that the relative positions of lesion areas from past reports can be recognized in the current reconstructed image. See paragraphs [0120–0124], [0141–0146].
From this, one of ordinary skill would understand that the system effectively screens stored past lesion positions for those that correspond (spatially) to positions of current lesion areas, enabling recognition of related abnormalities.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 14:
“The method according to claim 13, comprising a step of setting an evolution tag for each current data element for which a previous data element comprising the abnormality at a position in the previous presentation that corresponds to the position of the abnormality according to the current data element was found, wherein the step of setting an evolution tag is an automated step.”
Liu et al. teaches:
“setting an evolution tag for each current data element for which a previous data element … was found, wherein the step of setting an evolution tag is an automated step.”
Liu et al. teaches morphological progression and longitudinal comparison: “CT volume shows morphological progression of primary hepatocellular carcinoma and progressive metastatic disease,” and reports statements such as “A 52 x 45 mm necrotic area … increased from 40 mm measured in [prior study]” and “The maximum SUV was 6.0 and is now 9.9.” See FIG. 4C and paragraphs, [0040–0043].
These progression statements correspond to evolution tags applied to current abnormalities for which prior abnormalities were found (e.g., “progressive metastatic disease,” “increased from 40 mm”). The system uses scriptable rules to derive these tags automatically based on current and previous data elements.
Liu et al. fails to explicitly teach:
Using the exact phrase “evolution tag,” but the functional equivalent (“progressive,” “increased,” “unchanged”) is present.
Moriya teaches:
Structured representations of lesion states across past and current images, enabling evolution tagging.
Moriya teaches comments associated with lesion characters, including indications such as “ASCITIC FLUID … POINTED OUT LAST TIME IS IN REMISSION, AS TREATMENT,” which show current status relative to past lesion presence. See paragraphs [0126–0129].
These statements effectively tag lesions with evolution information (e.g., “in remission after treatment”), which can be implemented as evolution tags in structured data.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 15:
“The method according to claim 1, comprising:
a step of providing a list of studies comprising a first item that is characteristic for the current study and a second item that is characteristic for a previous study;
a step of providing a list of presentations comprising at least one item that is characteristic for a kind of presentation;
a step of attributing a unique identifier to each of the virtual presentations of the current study or of a selection thereof, and of attributing a unique identifier to each of the virtual presentations of the previous study or of a selection thereof, wherein each unique identifier comprises an identifier of a first kind of identifier and an identifier of a second kind of identifier, wherein each identifier of the first kind of identifier represents one item of the list of studies in a unique manner, and wherein each identifier of the second kind of identifier represents one item of the list of presentations in a unique manner;
a step of providing to the user via the user interface the list of studies or an adapted list of studies and the list of presentations or an adapted list of presentations, wherein the user interface comprises a first view for the list of studies or the adapted list of studies and a second view for the list of presentations or the adapted list of presentations;
a step of navigating through at least one of the virtual presentations of the current study having attributed unique identifiers and the virtual presentations of the previous study having attributed unique identifiers by selecting an item of the list shown in the first view and by selecting an item of the list shown in the second view, wherein the step of navigating comprises displaying a virtual presentation in an image viewer, wherein the displayed virtual presentation has the unique identifier comprising the identifier of the first kind of identifier that represents the item selected in the list of studies and the identifier of the second kind of identifier that represents the item selected in the list of presentations.”
Liu et al. teaches:
“a step of providing a list of studies comprising a first item that is characteristic for the current study and a second item that is characteristic for a previous study.”
Liu et al. teaches “Work List” and lists of prior images in the GUI, such as “Patient Prior Images 1 CT 01/03/2014 2 CT 12/23/2012,” and “This report is generated by referencing the following similar findings 3 CT 01/27/2016 4 CT 05/23/2014.” See FIG. 2. These entries are items characteristic for different studies (current and previous).
“a step of providing a list of presentations comprising at least one item that is characteristic for a kind of presentation.”
Liu et al. teaches that clinical report templates and domain/modality annotations are grouped by “image modality, CT, MR … domain Abdomen, pelvis … contrast Y/N,” and that different domains (cardiac, abdominal) have different report templates. See FIG. 3B–3C and paragraphs [0032–0034].
This corresponds to a list of presentations characteristic for kinds of presentation (e.g., CT cardiac, CT abdomen, CT angiography).
“a step of attributing a unique identifier to each of the virtual presentations of the current study … and … previous study … wherein each unique identifier comprises an identifier of a first kind of identifier and an identifier of a second kind of identifier … each identifier of the first kind of identifier represents one item of the list of studies … each identifier of the second kind of identifier represents one item of the list of presentations.”
Liu et al. teaches that images and reports are indexed by patient ID, exam date, and modality/domain. The Medical Information Database stores diagnostic volumes and metadata (e.g., patient identifier, exam date, imaging modality, clinical domain). See paragraphs [0029–0030].
These metadata fields serve as unique identifiers composed of a first kind (study identifier, such as patient + date) and a second kind (presentation identifier, such as modality/domain).
“a step of providing to the user via the user interface the list of studies or an adapted list of studies and the list of presentations or an adapted list of presentations … first view … second view.”
Liu et al. teaches a GUI with a worklist and prior image lists, where the user can select different studies and domains. See FIG. 2 and paragraphs [0027–0033].
Liu et al. fails to explicitly teach:
The specific arrangement into “first view” and “second view” for lists of studies and presentations, though the worklist and modality/domain selection approximate this.
Moriya teaches:
Explicit list and selection structures for images and presentations.
Moriya teaches “PAST REPORT SELECTION BUTTON,” “REFERENCE IMAGE SELECTION BUTTON,” and display condition tables mapping keywords (e.g., “HEART,” “LIVER”) to types of images (e.g., VR, MPR). See FIGS. 5, 18–20 and paragraphs [0063–0067]. These controls present lists of studies and image types (presentations) that the user can select to display corresponding virtual presentations in the image viewer.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 16:
“The method according to claim 15, wherein the user interface comprises tools allowing the user to at least one of:
‘replacing the presentation displayed in the image viewer with a replacement presentation that corresponds to the presentation assigned to the item arranged in the list shown in the second view precedingly or subsequently to the item to which the presentation displayed in the image viewer is assigned;
replacing the presentation displayed in the image viewer with a replacement presentation that belongs to the study assigned to the item arranged in the list shown in the first view precedingly or subsequently to the item to which the study is assigned to which the presentation displayed in the image viewer belongs;
switching back and forth between displaying in the image viewer a first presentation and a second presentation, wherein the first presentation is a presentation assigned to a first item in the list shown in the second view and the second presentation is a presentation assigned to a second item in the list shown in the second view;
switching back and forth between displaying in the image viewer a first presentation and a second presentation, wherein the first presentation is a presentation of the study assigned to a first item in the list shown in the first view and the second presentation is a presentation of the study assigned to a second item in the list shown in the first view.’”
Liu et al. teaches:
Switching among different studies and presentations via GUI selection.
Liu et al. teaches that the user computer presents a GUI where the user can select different studies (e.g., prior CT exams) from “Patient Prior Images” and different domains/templates from clinical report templates, and that images corresponding to these selections are displayed. See FIG. 2, FIG. 3B–3C and paragraphs [0027–0033], [0032–0034].
The GUI supports replacing the current displayed images with images from selected prior studies or domains.
Liu et al. fails to explicitly teach:
Tools specifically described as “one-click” or “switching back and forth” between first and second presentations in terms of preceding/subsequent list items.
Moriya teaches:
Explicit tools to replace and switch between different presentations in the image viewer.
Moriya teaches that display condition tables map keywords to specific image processing operations (e.g., “EXTRACT HEART,” “EXTRACT CORONARY ARTERY,” “HIDE OTHER THAN EXTRACTED ANATOMICAL STRUCTURE”) and that the system can display different reconstructed images (e.g., VR, MPR, MIP). See FIGS. 19–20 and paragraphs [0168–0171]. The user selects keywords or lesion characters, and the system switches the displayed image accordingly, effectively replacing the current presentation with another corresponding to the selected item.
In modifications, Moriya describes that the user can select different second medical images via menus, enabling switching back and forth between multiple presentations (e.g., different reconstructions or different studies) in the image viewer. See paragraphs [0158–0164].
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 17:
“The method according to claim 1, wherein the step of screening the current presentation for the abnormality, comprises a step of running a model of an artificial intelligence, wherein the model is trained for assigning a probability to each voxel of a presentation of the body portion that the voxel belongs to the abnormality.”
Liu et al. teaches:
“running a model of an artificial intelligence” to screen images for abnormalities.
Liu et al. teaches that image parsing models may include “discriminative classifiers probabilistic boosting trees, marginal space learning, marginal space deep learning, neural networks,” and hierarchical models including reinforcement learning and recurrent neural networks. See paragraphs [0037–0038]. FIG. 6 illustrates “Deep Learning Feature Extraction Input Image … Generate findings in natural language” with deep Q-learning/RNN.
These are AI models used to parse images and generate findings.
Liu et al. fails to explicitly teach:
“wherein the model is trained for assigning a probability to each voxel of a presentation of the body portion that the voxel belongs to the abnormality,” i.e., explicit voxel-wise probability outputs.
Moriya teaches:
wherein the model is trained for assigning a probability to each voxel of a presentation of the body portion that the voxel belongs to the abnormality,” i.e., explicit voxel-wise probability outputs.
Moriya references CAD systems and automatic extraction of organs/lesions, but does not explicitly state that models assign probabilities to each voxel. See FIG. 20 and paragraphs [0168–0171].
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 18:
“The method according to claim 17, wherein the step of running a model of an artificial intelligence comprises running a plurality of models of the artificial intelligence and/or a model of a first artificial intelligence and a model of a second artificial intelligence, wherein the models run are trained for assigning a probability to each voxel of a presentation of the body portion that the voxel belongs to the abnormality, wherein, for each model run, a probability segmentation map is generated, and wherein the user can select a generated probability segmentation map.”
Liu et al. teaches:
“running a plurality of models of the artificial intelligence and/or a model of a first artificial intelligence and a model of a second artificial intelligence.”
Liu et al. teaches that image parsing models may include various types: “discriminative classifiers … marginal space deep learning, neural networks, regression models, hierarchical models, statistical shape models, probabilistic graphical models,” and that different models may be applied depending on domain and modality. See paragraphs [0037–0038].
This corresponds to running multiple models of AI (first and second models) on the image data.
Liu et al. fails to explicitly teach:
That each model is trained for assigning per-voxel probabilities, generating “probability segmentation maps,” and that “the user can select a generated probability segmentation map.”
Moriya teaches:
User selection among different reconstructed images and processing modes, which is analogous to selecting among segmentation maps.
Moriya teaches that the user can select different types of presentations via “REFERENCE IMAGE SELECTION BUTTON” and display condition tables, switching among VR, MPR, and other reconstructions based on keywords. See FIGS. 5, 18–20 and paragraphs [0055–0058], [0168–0171].
This supports a user selecting among multiple image processing outputs.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 19:
“The method according to claim 1, wherein the position information indicating the position of the abnormality in the body portion is provided in an automated manner by at least one of:
‘providing a virtual template of the body portion, wherein anatomic locations are labeled in the virtual template, wherein the virtual template and the virtual current presentation are brought in dense correspondence, and wherein the labels are transferred from the virtual template to the virtual current presentation;
running a model of an artificial intelligence, wherein the model is trained for assigning regions shown in a virtual presentation showing the body portion to anatomic locations.’”
Liu et al. teaches:
“providing a virtual template of the body portion, wherein anatomic locations are labeled in the virtual template, wherein the virtual template and the virtual current presentation are brought in dense correspondence, and wherein the labels are transferred from the virtual template to the virtual current presentation.”
Liu et al. teaches an “annotation specification” where clinical report concepts (e.g., organ size, organ position, vessel lumen) are linked to anatomical annotations such as “Liver dome landmark, 3D point,” “Iliac bifurcation landmark, 3D point,” and “Liver surface, mesh,” “Spleen surface, mesh,” “L. Kidney surface, mesh,” “R. Kidney surface, mesh.” See FIG. 3E and paragraphs [0034–0038]. These annotations effectively form a virtual template of the abdominal body portion with labeled anatomic locations (organs and landmarks). The image parsing models bring the template and the current CT volume into correspondence by registering the test image to the atlas of annotations and then transferring labels (organ surfaces, landmarks) to the current presentation.
“running a model of an artificial intelligence, wherein the model is trained for assigning regions shown in a virtual presentation showing the body portion to anatomic locations.”
Liu et al. teaches using deep learning models and other AI-based image parsing models to determine anatomical structures and segment organs. It notes that “examples of image parsing models … include … neural networks … statistical shape models, probabilistic graphical models,” and that these models determine annotation values for organs and landmarks. See paragraphs [0037–0038].
Thus, Liu discloses AI models trained to assign regions in CT/MR images to anatomical locations (e.g., liver, spleen, kidneys), which are then used as annotated regions.
Liu et al. fails to explicitly teach:
The exact phrase “dense correspondence” and explicit transfer of labels from a “virtual template” to the “virtual current presentation,” although the atlas-based and annotation-based registration implicitly accomplish this.
Moriya teaches:
Use of keyword tables and display conditions to link lesion characters to anatomical regions and specific reconstructed images.
Moriya teaches a “keyword table” where keywords such as “LUNG,” “LIVER,” “BRAIN,” “TUMOR,” “GROUND GLASS OPACITY,” etc., are associated with lesion characters and anatomical regions. See FIG. 18 and paragraphs [0068–0070]. Display condition tables link these keywords to particular second medical images (e.g., VR images of blood vessels, liver, spleen) and image processing operations (e.g., “EXTRACT LIVER,” “HIDE OTHER THAN EXTRACTED ANATOMICAL STRUCTURE”). See FIG. 20 and paragraphs [0168–0171].
This shows that anatomical locations are labeled in templates and that labels are used to derive anatomical regions in virtual presentations.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 20:
“The method according to claim 1, wherein at least one of the following applies:
‘the method comprises a step of correcting by the user the extend extent and/or position of an identified abnormality by at least one of the following:
i. by modifying at least one of a minimal probability-value that is considered in the step of screening the current presentation for the abnormality and a maximal probability-value that is considered in the step of screening the current presentation for the abnormality;
ii. by directly adding voxels to or removing voxels from a set of voxels representing an abnormality listed in the table;
iii. by indicating a corrected extension and/or corrected position in the presentation displayed in the image viewer;
iv. by selecting one of a plurality of probability segmentation maps that is generated in the step of screening the current presentation for the abnormality;
the method comprises a step of generating at least one of the following lists for at least one abnormality listed in the table:
i. a list comprising a plurality of position information indicating the position of the abnormality in the body portion;
ii. a list comprising a plurality of evolution information indicating the presence of the abnormality at a specific date;
iii. a list comprising a plurality of contrast uptake information of the abnormality;
a new entry for an abnormality not listed in the table can be generated in the table by the user and an extension and/or position of the abnormality can be defined by the user by indicating the extension and/or position in the presentation displayed in the image viewer.
an entry for an abnormality can be removed from the table.’”
Liu et al. teaches:
Lists of positions, evolution information, and contrast uptake information.
Liu et al. teaches that clinical report templates include longitudinal measurements and contrast information. For example, “A 52 x 45 mm necrotic area in the right hepatic dome, increased from 40 mm measured in [prior study]” includes multiple position-related measurements over time (sizes at different dates) and evolution information (progression). See FIG. 4C and paragraphs, [0040–0043].
Liu also uses SUV values (e.g., “maximum SUV was 6.0 and is now 9.9”) as signal/contrast uptake information, which can be stored in lists over time. See FIG. 4C.
By storing these repeated measurements and values, Liu effectively generates lists of position information, evolution information, and contrast uptake information for each abnormality.
Adding/removing entries and defining extension/position via user input.
Liu’s smart report GUI allows the user to review findings and modify interpretations, and the underlying annotation system supports “indexed diagnoses by findings” and comparison of findings across images. See paragraphs [0026–0034]. It is natural for one of ordinary skill to interpret this as allowing users to add new findings (abnormality entries) not identified automatically, define their extension/position, and remove entries from the table if incorrect.
Liu et al. fails to explicitly teach:
“modifying at least one of a minimal probability-value … and a maximal probability-value” and “selecting one of a plurality of probability segmentation maps,” since Liu does not explicitly discuss probability thresholds or multiple segmentation maps.
Moriya teaches:
“by indicating a corrected extension and/or corrected position in the presentation displayed in the image viewer; a new entry … can be generated … and an extension and/or position … can be defined … by indicating the extension and/or position in the presentation displayed in the image viewer; an entry … can be removed from the table.”
Moriya teaches that the user specifies lesion characters and positions in the medical image, and can change lesion characters and update lesion information. See FIG. 6 (“SPECIFY LESION CHARACTER – CHANGE LESION CHARACTER – UPDATE LESION INFORMATION”) and paragraphs [0056–0057], [0130–0137]. The user selects lesion characters and corresponding positions in the image, effectively indicating extension and position, and can update or remove entries from lesion storage (the table of lesions).
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 21:
“The method according to claim 1, comprising a step of providing a radiological report, wherein the report comprises information given in or linked to the table.”
Liu et al. teaches:
“a step of providing a radiological report, wherein the report comprises information given in or linked to the table.”
Liu et al. teaches that “the computer populates an annotation table using the relevant image annotations and applies one or more domain-specific scriptable rules to populate a report template based on the annotation table. The computer may then generate a natural language radiology report based on the report template.” See paragraph 4 and Abstract.
Further, Liu describes smart reports in natural language with embedded links back to image coordinates of features which correlate with the findings. See paragraphs [0033–0034] and FIG. 2. These reports comprise information given in the annotation table and are linked to the table entries and their positional annotations.
Liu et al. fails to explicitly teach:
The exact claim wording, though the functionality is directly disclosed.
Moriya teaches:
Lesion information in reports linked to positions in images.
Moriya teaches image reading reports that include lesion characters representing lesion contents of lesion areas, and the medical image display apparatus links these lesion characters to positions in the medical image via link characters and position indicators. See paragraphs [0011–0015], [0120–0122]. Thus, radiological reports comprising lesion information are linked to table-like lesion storage and image positions.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 22:
“The method according to claim 21, wherein the report comprises at least one of the following information: the abnormalities identified in the current study, the position of each identified abnormality in the body portion, a hint whether the abnormality was identified in a previous study or not, the extend of each identified abnormality, the abnormalities identified in a previous study but not in the current study, the position and/or extend of the abnormalities identified in a previous study but not in the current study, a presentation of each abnormality identified in the current study, a presentation that is representative for the medical condition according to the current study, a summary list.”
Liu et al. teaches:
“the abnormalities identified in the current study.”
Liu et al. teaches reports listing findings, such as “Liver – A 52 x 45 mm necrotic area in the right hepatic dome” and “Abdomen and pelvis – Thickening of the gastric antral wall … A new lymph node was noticed … Other findings are normal.” See FIG. 4C. These are abnormalities identified in the current study.
“the position of each identified abnormality in the body portion.”
The findings include anatomical positions, e.g., “right hepatic dome,” “gastric antral wall,” “adjacent to the inferior vena cava,” which indicate positions of abnormalities in the body. See FIG. 4C and paragraphs [0041–0043].
“a hint whether the abnormality was identified in a previous study or not.”
Liu et al. teaches statements such as “increased from 40 mm measured in [prior study],” and lists “Patient Prior Images” and “This report is generated by referencing the following similar findings,” indicating whether abnormalities were present in prior studies and how they changed. See FIG. 4C and FIG. 2. These provide hints as to whether the abnormality was identified previously.
“the extend of each identified abnormality.”
Liu et al. reports sizes and extents (e.g., “52 x 45 mm necrotic area,” “thickening … measuring 12.8 mm,” “new lymph node … measuring 3.8 cm”). See FIG. 4C. These describe the extent of each abnormality.
“the abnormalities identified in a previous study but not in the current study, the position and/or extend of the abnormalities identified in a previous study but not in the current study.”
By comparing current and prior findings (e.g., progression vs prior), Liu’s system can identify abnormalities present previously but absent in the current study and report them (e.g., resolved lesions or regressed disease), though specific examples may focus on progression. The framework supports listing previous findings and their positions/extents from earlier reports even if they are no longer present.
“a presentation of each abnormality identified in the current study, a presentation that is representative for the medical condition according to the current study.”
Liu’s smart report with embedded links allows the user to retrieve images depicting each abnormality by clicking on report elements. See FIG. 2 and paragraphs [0033–0034]. This gives a presentation of each abnormality (linked image slice/volume) and representative images of the medical condition according to the current study.
“a summary list.”
Liu’s report templates include structured “FINDINGS” and “Impressions,” which act as summary lists of abnormalities and conclusions. See FIG. 4C and paragraphs [0033–0035].
Moriya teaches:
Linkage between lesion text and positions, and past vs current lesion information.
Moriya teaches displaying lesion characters and associated positions in the medical image, with comments showing past presence and current remission or progression (e.g., “ASCITIC FLUID … POINTED OUT LAST TIME IS IN REMISSION, AS TREATMENT”). See FIG. 11 and paragraphs [0126–0129]. This supports hints about whether abnormalities were identified in previous studies and their status.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 1, and incorporated herein.
As per Claim 23:
“A system for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion, the system comprises a computer system, said computer system provides a communication unit, a controller and a user interface, wherein:
‘the communication unit is configured to communicate with the controller and the user interface;
the controller is configured for screening in an automated or semi- automated manner a virtual current presentation of a current study of the body portion for the abnormality, for identifying in an automated or semi- automated manner separate abnormalities in the current presentation, and for determining in an automated or semi-automated manner and for each identified abnormality a position information indicating the position of the identified abnormality in the current presentation;
the controller is configured for generating in an automated manner a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion;
the user interface is configured for providing the table to a user, and for correcting and/or validating the table, wherein the table is editable by the user via the user interface;
the system is configured for determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, and wherein the table indicates whether there is a related abnormality in the previous presentation or not;
or wherein
the communication unit is configured to communicate with the controller and the user interface;
the user interface is configured to provide a virtual current presentation of a current study of the body portion to a user and to receive a user-input identifying separate abnormalities in the current presentation;
the controller is configured for determining in an automated or semi- automated manner and for each identified abnormality a position information indicating the position of the identified abnormality in the current presentation;
the controller is configured for generating a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion;
the user interface is configured for providing the table to a user, and for correcting and/or validating the table, wherein the table is editable by the user via the user interface;
the system is configured for determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, and wherein the table indicates whether there is a related abnormality in the previous presentation or not.’”
Liu et al. teaches:
“A system for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion, the system comprises a computer system, said computer system provides a communication unit, a controller and a user interface.”
Liu et al. teaches “a system 100 for automatically generating radiology reports from images” that includes “a User Computer 115, a Medical Information Database 120, and a Radiology Report Generation Computer 110, all connected via a Network 125.” See FIG. 1 and paragraphs [0027–0029]. Liu et al. further explains that the Radiology Report Generation Computer 110 includes modules (Clinical Report Module 110A, Rules Module 110B, Image Annotation Module 110C, Image Processing Module 110D) which are “configured to generate radiology reports from patient data,” and that the User Computer 115 provides a graphical user interface (GUI) where a smart report is displayed. See paragraphs [0031–0033].
The Network 125 corresponds to a communication unit configured to communicate with the Radiology Report Generation Computer (controller) and the User Computer (user interface), and the GUI on the User Computer provides the virtual medical presentations (images) and reports supporting observation of abnormalities.
“the communication unit is configured to communicate with the controller and the user interface.”
Liu et al. teaches that “the Radiology Report Generation Computer 110 communicates with the Medical Information Database 120 over the Network 125 to retrieve information to generate an input dataset” and that the User Computer 115 also communicates with the Radiology Report Generation Computer 110 via the Network 125. See paragraphs [0028–0030].
This network communication between computers and GUI corresponds to a communication unit configured to communicate with the controller (Radiology Report Generation Computer 110) and the user interface (on User Computer 115).
“the controller is configured for screening in an automated or semi- automated manner a virtual current presentation of a current study of the body portion for the abnormality, for identifying in an automated or semi- automated manner separate abnormalities in the current presentation, and for determining in an automated or semi-automated manner and for each identified abnormality a position information indicating the position of the identified abnormality in the current presentation.”
Liu et al. teaches that “a computer-implemented method for automatically generating a radiology report includes a computer receiving an input dataset comprising a plurality of multidimensional patient images and patient information and parsing the input dataset using learned models to determine a clinical domain and relevant image annotations.” See paragraph. Liu et al. further discloses that the Image Processing Module 110D “training image parsing models and uses these models to determine domain, modality and annotations,” and that the annotations include anatomical surfaces, landmarks, and organ boundaries, such as “Liver dome landmark, 3DPoint … Liver surface, mesh … L. Kidney surface, mesh.” See FIG. 3E and paragraphs [0037–0038].
Thus, the controller (Radiology Report Generation Computer 110 with Image Processing Module 110D) is configured to automatically screen the current image data (virtual current presentation) for abnormalities, identify abnormalities via annotations, and determine position information for each abnormality in the current presentation.
“the controller is configured for generating in an automated manner a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion.”
Liu et al. teaches that during offline processing, “the Clinical Report Module 110A applies domain knowledge and references for each clinical domain to create the basic clinical report template and determine the clinical report concepts … the Rules Module 110B determines the image annotations and the corresponding rules which are necessary to generate the values for each of the clinical report concepts … the Image Annotation Module 110C builds the image annotation system and uses it to annotate sample images.” See paragraphs [0032–0037]. During online processing, “the computer populates an annotation table using the relevant image annotations and applies one or more domain-specific scriptable rules to populate a report template based on the annotation table.” See paragraph and FIG. 4A.
The CT cardiac and CT abdominal examples show “Annotation Table” and “GENERAL – image modality CT … FINDINGS – LIVER – normal size Y,” where each row corresponds to an abnormality or organ feature and is linked to annotations (positions, volumes). See FIGS. 3B–3C, 4B–4C. This matches a controller generating, in an automated manner, a table listing the identified abnormalities and, for each listed abnormality, position information indicating the position in the body portion.
“the user interface is configured for providing the table to a user, and for correcting and/or validating the table, wherein the table is editable by the user via the user interface.”
Liu et al. teaches that “the Clinical Report Module 110A uses the populated clinical report template and Natural Language Generation (NLG) to generate the report,” and that “the output is smart report in natural language with embedded links that navigates back to image coordinates of features which correlate with the findings.” See paragraphs [0033–0034]. FIG. 2 shows a GUI 200 in which the smart radiology report is displayed on the right, with embedded links to image features on the left.
This user interface provides the contents of the table (identified abnormalities and positions) to the user and allows the user to retrieve images, review findings, and, as needed, validate or correct them. One of ordinary skill would understand that the underlying structured data (table) is thereby editable via the user interface, even though Liu discusses it in terms of reports and templates rather than explicitly using the word “table” for the GUI.
“the system is configured for determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, and wherein the table indicates whether there is a related abnormality in the previous presentation or not.”
Liu et al. teaches that “the techniques described herein have the potential to not only automate and streamline what is traditionally a manual task … but also to elevate the quality of reports by substantiating clinical observations directly with their points of reference in relevant images,” and that standardized templates “streamline comparison to longitudinal data and similar cases from past reports.” See paragraph. The CT abdominal example shows “CT volume shows morphological progression of primary hepatocellular carcinoma and progressive metastatic disease” and lists “Patient Prior Images” and “This report is generated by referencing the following similar findings,” which reflect comparison of current abnormalities to prior abnormalities. See FIG. 4C and related text.
Thus, Liu et al. discloses determining whether current abnormalities correspond to prior abnormalities and representing progression or similarity in the report; this is consistent with indicating, in a structured table or report, whether a related abnormality exists in a previous presentation.
“the user interface is configured to provide a virtual current presentation of a current study of the body portion to a user and to receive a user-input identifying separate abnormalities in the current presentation; the controller is configured for determining in an automated or semi- automated manner and for each identified abnormality a position information indicating the position of the identified abnormality in the current presentation; the controller is configured for generating a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion; the user interface is configured for providing the table to a user, and for correcting and/or validating the table, wherein the table is editable by the user via the user interface; the system is configured for determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, and wherein the table indicates whether there is a related abnormality in the previous presentation or not.’”
Liu et al. teaches an interactive GUI where the user can view image volumes (virtual current presentations) and smart reports, and can interact with embedded links to navigate between report findings and images. See FIG. 2 and paragraphs [0033–0034]. The system uses annotations and positions to map report entries to image coordinates. This implies a workflow where user input (e.g., selecting findings, confirming or correcting annotations) is used to refine the table of abnormalities and positions, and where the system can determine and indicate relationships to prior abnormalities.
Liu et al. fails to explicitly teach:
“a system for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion,” to the extent that Liu focuses on automatic report generation, not expressly on a dedicated “system for supporting observation” framed around a virtual medical presentation in an image viewer.
“the communication unit is configured to communicate with the controller and the user interface,” using that exact “communication unit” terminology, although the network and interfaces in Liu perform this function.
“the user interface is configured for providing the table to a user, and for correcting and/or validating the table, wherein the table is editable by the user via the user interface,” in that Liu describes smart reports and templates rather than explicitly calling them “tables” or stating that the table itself is editable, even though functional editing of findings is implied.
“the system is configured for determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation … by determining, using the position information … whether there is an abnormality in the previous presentation having a position information indicating that the abnormality … corresponds,” because Liu discusses longitudinal comparison and progression but does not explicitly state that positional correspondence (position information) is the key used to match current abnormalities to prior abnormalities.
The explicit alternative branch in Claim 23 where “the user interface is configured to provide a virtual current presentation … and to receive a user-input identifying separate abnormalities in the current presentation” and the controller then determines position information, generates the table, etc., where the identification is driven by user input in the image viewer rather than purely automated parsing.
Moriya teaches:
“a system for supporting observation of a medical abnormality in or on a body portion displayed in a virtual medical presentation of the body portion.”
Moriya teaches “a medical image display apparatus, method, and program” where “a medical image display apparatus … allows more accurate image interpretation by displaying lesion information described in an image reading report in a medical image.” See paragraphs [0001–0003]. It stores “a first medical image and a second medical image reconstructed from a plurality of medical images, including the first medical image” and displays the second medical image, which is a virtual medical presentation of the body portion. See paragraphs [0010–0014], [0030–0034].
This apparatus is explicitly designed to support observation of lesions (medical abnormalities) in virtual presentations.
“the communication unit is configured to communicate with the controller and the user interface.”
Moriya teaches a system configuration including “MODALITY,” “IMAGE STORAGE SERVER,” and “IMAGE PROCESSING WS,” connected via a “NETWORK,” and describes an image processing workstation comprising a CPU, memory, display section, input section, and communication port. See FIG. 1 and FIG. 2, paragraphs [0051–0053].
The communication port and network function as a communication unit between the controller (image processing workstation) and the display/input (user interface).
“the user interface is configured for providing the table to a user, and for correcting and/or validating the table, wherein the table is editable by the user via the user interface.”
Moriya teaches screens such as FIG. 5 and FIG. 8, showing “FINDING FIELD,” lesion entries (“LESION 1,” “LESION 2,” etc.), and buttons (“FINALIZE BUTTON,” “TEMPORARY STORAGE BUTTON,” “CANCEL BUTTON”), which the user uses to input and change lesion characters and information. See paragraphs [0055–0058]. FIG. 6 shows a flow diagram: “START – SPECIFY LESION CHARACTER – CHANGE LESION CHARACTER – UPDATE LESION INFORMATION – END.”
These lesion entries and associated information can be understood as a table of lesions, which the user edits and validates via the user interface.
“the system is configured for determining, for each identified abnormality, whether there is a related abnormality in a virtual previous presentation of a previous study of the body portion by determining, using the position information indicating the position of the identified abnormality in the current presentation, whether there is an abnormality in the previous presentation having a position information indicating that the abnormality in the previous presentation corresponds to the abnormality identified in the current presentation, and wherein the table indicates whether there is a related abnormality in the previous presentation or not.”
Moriya teaches that “the image reading report may include a plurality of image reading reports with respect to past medical images or with respect to a plurality of different lesion positions” and that the system stores lesion characters and positions and computes “a position in the second medical image corresponding to the position of the lesion area” (link position). See paragraphs [0043–0044], [0019–0021]. Moriya further explains that multiple link positions and link characters are displayed in the reconstructed image, with association indicators, so that the positions of multiple lesion areas and the corresponding characters can be recognized at once. See paragraphs [0120–0124].
This corresponds to determining, using position information, whether lesions in the current virtual presentation correspond to lesion positions recorded in prior images and reports, and displaying that correspondence in association with the lesions (effectively indicating in the lesion table whether there is a related abnormality in previous presentations).
“the user interface is configured to provide a virtual current presentation of a current study of the body portion to a user and to receive a user-input identifying separate abnormalities in the current presentation.”
Moriya teaches that “link character specifying means 80 … in response to the specification of a link character or a lesion character by the operator via input section 303, inputs the specified link character” and that the user specifies the lesion character/position via mouse or other input devices. See paragraphs [0130–0132]. FIG. 8 shows a reconstructed image with link characters and position indicators that the user can select.
Thus, the user interface provides the virtual current presentation and receives user input identifying separate abnormalities (via specifying lesion characters or positions).
“the controller is configured for determining in an automated or semi- automated manner and for each identified abnormality a position information indicating the position of the identified abnormality in the current presentation.”
Moriya teaches “a link position storage unit for calculating a position in the second medical image corresponding to the position of the lesion area and storing the calculated position as a link position.” See paragraphs [0013–0014], [0120–0121].
This is an automated determination of position information for each lesion (abnormality) in the current virtual presentation.
“the controller is configured for generating a table listing the identified abnormalities and, for each listed abnormality, a position information indicating the position of the abnormality in the body portion.”
While Moriya describes lesion storage means and link position storage means rather than explicitly calling them a “table,” the association table in FIG. 13 (“association table associating lesion areas with lesion character candidates”) and the keyword and display condition tables in FIGS. 18–20 demonstrate that Moriya uses tabular representations linking lesion characters, lesions, and display conditions. See paragraphs [0063–0067], [0168–0171].
It would be understood by one of ordinary skill that the stored lesion information and link positions form a table listing identified abnormalities and their positions.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to include the system features taught by Moriya within the system taught by Liu et al., with the motivation of improving radiology workflow by combining Liu’s structured annotation and automatic report generation (including longitudinal comparison) with Moriya’s explicit per-lesion position mapping, interactive virtual image display, and multi-report linkage. Liu already teaches a controller, communication network, annotation tables, and a smart report GUI, while Moriya teaches a medical image display apparatus that computes link positions and displays lesion characters and positions in reconstructed images, allowing users to specify and edit lesion information and reference past reports and images. See paragraphs [0026–0034] of Liu and [0120–0127], [0141–0146] of Moriya.
As per Claim 26:
“The system according to claim 23, wherein the system comprises an image viewer, wherein the system is configured and the user interface comprises one-click tools for allowing the user to at least one of:
‘initiating a replacement of a presentation displayed in the image viewer with a replacement presentation, wherein the displayed presentation and the replacement presentation are presentations of the same study and of a different kind of presentation;
initiating a replacement of a presentation displayed in the image viewer with a replacement presentation, wherein the displayed presentation and the replacement presentation are presentations of the same kind of presentation and of different studies;
switching back and forth between displaying in the image viewer a first presentation and a second presentation, wherein the first presentation and the second presentation belong to the same study and are of a different kind of presentation;
switching back and forth between displaying in the image viewer a first presentation and a second presentation, wherein the first presentation and the second presentation are of the same kind of presentation and belong to different studies.’”
Liu et al. teaches:
“The system according to claim 23, wherein the system comprises an image viewer.”
Liu et al. teaches a system including “Diagnostic 3D Volumes” and a user computer that displays medical images and a smart radiology report in a graphical user interface. FIG. 2 shows the GUI with images on the left and the report on the right. See paragraphs [0027–0033].
The image panes in FIG. 2 function as an image viewer within the system.
“initiating a replacement of a presentation displayed in the image viewer with a replacement presentation, wherein the displayed presentation and the replacement presentation are presentations of the same study and of a different kind of presentation.”
Liu et al. discusses multiple clinical domains and modalities (e.g., CT Angiography, CT Abdomen, cardiac CT) and report templates for each domain. See FIG. 3B–3C and paragraphs [0032–0034]. The GUI can display different reconstructions or views (e.g., different planes or sequences) of the same CT study, and one of ordinary skill would understand that selecting different domain/presentation settings replaces the current view with another kind of presentation of the same study (e.g., switching between axial slices and 3D renderings).
“initiating a replacement of a presentation displayed in the image viewer with a replacement presentation, wherein the displayed presentation and the replacement presentation are presentations of the same kind of presentation and of different studies.”
Liu et al. teaches “Patient Prior Images” and “This report is generated by referencing the following similar findings,” listing prior CT studies. See FIG. 2. The user can select prior studies, causing the system to display images from those studies in place of the current images. This corresponds to replacing the current presentation with a presentation of the same kind (e.g., CT axial slices) from a different study.
“switching back and forth between displaying in the image viewer a first presentation and a second presentation, wherein the first presentation and the second presentation belong to the same study and are of a different kind of presentation.”
Liu’s GUI and report generation system naturally support switching between different views or reconstructions (e.g., different slice orientations, MIP/3D views) of the same CT volume, although the exact phrase “switching back and forth” is not used. One of ordinary skill would recognize that selecting different views can be repeated to toggle between them.
“switching back and forth between displaying in the image viewer a first presentation and a second presentation, wherein the first presentation and the second presentation are of the same kind of presentation and belong to different studies.”
Liu’s worklist and prior image list allow selecting different studies, and the images display accordingly. The user can switch between current and prior studies for comparison, effectively switching back and forth between first and second presentations of the same kind from different studies. See FIG. 2 and paragraphs [0029–0033].
Liu et al. fails to explicitly teach:
“one-click tools” as explicit UI elements labeled that way, and explicit “switching back and forth” language, though the functional ability to change and compare views is present.
Moriya teaches:
“wherein the system comprises an image viewer.”
Moriya teaches a “medical image display apparatus” with a “display section 301” and “medical image display means 60” that display medical images (first and second medical images) reconstructed from slices. See FIGS. 2–3 and paragraphs [0051–0054].
This is an image viewer within the system.
“wherein the system is configured and the user interface comprises one-click tools for allowing the user to at least one of: initiating a replacement of a presentation displayed in the image viewer with a replacement presentation …”
Moriya teaches user interface elements such as “PAST REPORT SELECTION BUTTON,” “REFERENCE IMAGE SELECTION BUTTON,” and “LINK DISPLAY BUTTON,” as shown in FIG. 5 and FIG. 8. See paragraphs [0055–0058]. These are simple GUI controls (buttons) that the user can click once to initiate display of different images (e.g., past reports’ reference images) and link information.
Moriya further teaches display condition tables where keywords are linked to types of images (e.g., VR of blood vessels, VR of liver). See FIG. 20 and paragraphs [0168–0171]. Selecting a keyword or lesion character triggers display of a corresponding second medical image, effectively acting as a one-click tool to replace the current presentation with a new one.
“initiating a replacement of a presentation displayed in the image viewer with a replacement presentation, wherein the displayed presentation and the replacement presentation are presentations of the same study and of a different kind of presentation.”
Moriya’s display condition table maps keywords (e.g., “BLOOD VESSEL,” “LIVER”) to different image types (e.g., VR of blood vessels, VR of liver) along with position information (“0,0, W512 H512”). See FIG. 20. The system can extract and display different anatomical structures (e.g., heart, liver, spleen) from the same CT study by selecting corresponding keywords. This corresponds to replacing the current presentation with a different kind of presentation (e.g., focusing on a different organ or using a different reconstruction method) from the same study.
“initiating a replacement of a presentation displayed in the image viewer with a replacement presentation, wherein the displayed presentation and the replacement presentation are presentations of the same kind of presentation and of different studies.”
Moriya teaches “PAST REPORT SELECTION BUTTON” and “REFERENCE IMAGE SELECTION BUTTON” that allow selecting past reports and reference images, which are associated with different studies. See FIG. 5 and FIG. 11. When the user selects a past report or reference image button, the system displays the corresponding medical image (second medical image) in the image viewer, replacing the current image. This is a replacement of the same kind of presentation (e.g., VR or MPR) from different studies.
“switching back and forth between displaying in the image viewer a first presentation and a second presentation … belong to the same study and are of a different kind of presentation” and “… are of the same kind of presentation and belong to different studies.”
Moriya’s interface with multiple buttons (e.g., link display, past report selection, reference image selection) and link characters allows the user to repeatedly select and re-select different images and reports, switching between them. For example, selecting different lesion characters in FIG. 8 and FIG. 11 causes different parts of the image and report to be highlighted, and selecting different reference images causes different reconstructed images to be displayed. See paragraphs [0120–0137].
This supports switching back and forth between different presentations (e.g., VR vs slice, current vs past study) via simple clicks.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 23, and incorporated herein.
As per Claim 27:
“The system according to claim 23, wherein the user interface comprises a quantitative correction area allowing the user to at least one of:
‘correcting the extend and/or position of an abnormality listed in the table;
generating, in the table, a new entry for an abnormality not listed in the table and defining the extension and/or position of the abnormality by indicating the extension and/or position in the presentation displayed image viewer;
removing an entry for an abnormality from the table.’”
Liu et al. teaches:
“wherein the user interface comprises a quantitative correction area allowing the user to at least one of: correcting the extend and/or position of an abnormality listed in the table.”
Liu et al. teaches that the system uses structured templates and annotations to represent findings (organ sizes, lesion sizes, SUV values) and that reports can be edited and reviewed by radiologists via a GUI. See paragraphs [0026–0034]. The “Deep Learning Feature Extraction … Generate findings in natural language” and the structured field entries imply that radiologists can modify quantitative values (e.g., lesion size, organ volume) in the templates. While Liu does not explicitly label a “quantitative correction area,” the natural interpretation of the report template GUI is that it includes areas where quantitative fields (extent, position, measurements) can be corrected.
“generating, in the table, a new entry for an abnormality not listed in the table and defining the extension and/or position of the abnormality by indicating the extension and/or position in the presentation displayed image viewer.”
Liu’s structured template system supports adding new findings by entering template fields (e.g., additional lesions or organ conditions) that were not automatically detected by models. See paragraphs [0032–0035]. Radiologists can indicate extension/position via text (e.g., “right hepatic dome,” “adjacent to the inferior vena cava”) and measurements (e.g., sizes in mm or cm), thereby defining new entries in the table.
“removing an entry for an abnormality from the table.”
Liu’s emphasis on quality and consistency of reporting implies that radiologists can correct erroneous findings, which may include removing entries from the template or annotation table when abnormalities are not actually present. See paragraphs [0026–0034].
Liu et al. fails to explicitly teach:
An explicitly named “quantitative correction area” and direct image-based correction (e.g., by indicating extension/position in the image viewer) rather than in text fields.
Moriya teaches:
“wherein the user interface comprises a quantitative correction area allowing the user to at least one of: correcting the extend and/or position of an abnormality listed in the table.”
Moriya teaches that lesion storage means stores lesion characters and positions, and link position storage means calculates positions. FIG. 6 shows a process: “START – SPECIFY LESION CHARACTER – CHANGE LESION CHARACTER – UPDATE LESION INFORMATION – END,” indicating that the user can change lesion information. See paragraphs [0056–0057].
Moreover, by specifying a different lesion area or adjusting the lesion character associated with a position, the user corrects the extension and/or position of an abnormality.
“generating, in the table, a new entry for an abnormality not listed in the table and defining the extension and/or position of the abnormality by indicating the extension and/or position in the presentation displayed image viewer.”
Moriya teaches that the image reading report may include multiple lesion characters and lesion areas; the user can insert lesion characters into the report by specifying positions with the mouse and storing lesion information. See FIG. 4, FIG. 5 and paragraphs [0054–0056]. This process generates new entries (lesion character + position) in lesion storage (the table) for abnormalities not previously listed, and the extension/position is defined by indicating the lesion area in the displayed medical image (image viewer).
“removing an entry for an abnormality from the table.”
Moriya describes modifying lesion characters and updating lesion information; one of ordinary skill would understand that this includes removing lesion entries when they are no longer relevant or were erroneously entered. The flow for updating lesion information can encompass deletion of lesion entries in lesion storage. See paragraphs [0056–0057], [0128–0129].
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to include, in the user interface of the system taught by Liu, a quantitative correction area as taught by Moriya, allowing the user to correct the extent and/or position of abnormalities listed in the table, generate new entries for abnormalities not listed by indicating extension/position in the image viewer, and remove entries from the table. Liu already provides structured quantitative fields and reports that can be edited; Moriya provides explicit GUI workflows for editing lesion characters and positions directly on images. Combining these yields the claimed functionality of claim 27, with motivation to empower radiologists to refine AI-generated tables and ensure accurate quantitative representation of abnormalities. See paragraphs [0054–0057], [0126–0129] of Moriya.
The obviousness of combining the teachings of Liu et al. and Moriya are discussed in the rejection of claim 23, and incorporated herein.
As per Claim 28, Claim 28 is directed to a computer program comprising instructions which, when
the program is executed by a computer, cause the computer to execute the steps of the method
according to claim 1. Claim 28 recites the same or substantially similar limitations as those
addressed above for Claim 1 as taught by Liu et al. and Moriya. Claim 28 is therefore
rejected for the same reasons as set forth above for Claim 1 respectively.
As per Claim 29, Claim 29 is directed to a reproducible non-transitory computer-readable signal encoding the computer program according to claim 28. Claim 29 recites the same or substantially similar limitations as those addressed above for Claim 28 as taught by Liu et al. and Moriya. Claim 29 is therefore rejected for the same reasons as set forth above for Claim 28 respectively.
Response to Arguments
Applicant's arguments, filed on May 4, 2026 with respect to arguments in the remarks, have been considered but are moot in view of the new ground(s) of rejection necessitated by the new limitations added to Claims 1, 13, 20, 23 and 29.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
WO 2020106631 A1; The listing system (1100) has non-transitory processor-readable storage medium that stores processor-executable instructions or data. The processor is communicably coupled to non-transitory processor-readable storage medium. The processor is configured to receive a list of radiological studies from a database. The radiological studies are included with image data. The image data associated with the radiological studies is received and the image data is automatically processed in order to determine which images are likely to contain abnormalities. The list of radiological studies or associated images is displayed.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD B WINSTON III whose telephone number is (571)270-7780. The examiner can normally be reached M-F 1030 to 1830.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Robert Morgan can be reached at (571) 272-6773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.B.W/ Examiner, Art Unit 3683
/ROBERT W MORGAN/ Supervisory Patent Examiner, Art Unit 3683
Prosecution Insights