DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page 8, filed 8/27/2025, with respect to the specification objection have been fully considered and are persuasive. The objection of the specification has been withdrawn.
Applicant’s arguments, see page 8, filed 8/27/2025, with respect to claim objections have been fully considered and are persuasive. The objections of the claims has been withdrawn.
Applicant’s arguments, see page 8, filed 8/27/2025, with respect to 112 (b) rejection have been fully considered and are persuasive. The 112 (b) rejection of the claims has been withdrawn.
Applicant’s arguments, see page 9, filed 8/27/2025, with respect to the rejection(s) of claim(s) 1-8, 15-18 and 20 under 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Conjeti in view of Lyman. In particular, the Conjeti reference is still applied for most of the claims including the feature of input of a plurality of medical scan images. In particular, the plurality of medical scan images can be input, in a single input, a determination unit in order to determine the difference in the abnormality changes, which is taught in ¶ [39] and shown in step III in figure 2. The claims are not specific as to whether the images come from a single scan, which is why the primary reference is maintained on this feature. However, the output of scan medical images in a sorted order within a workflow is not disclosed. This deficiency is cured by the reference of Lyman.
Regarding the Lyman reference, the secondary reference teaches queuing patient scans in an automated order based on ranking that can take into consideration the malignancy score. The system that coordinates the queue and preview of medical scans can be performed by a neural network. The medical scan images can include nodules or disease areas on an area of the body. This is taught in ¶ [152], [156]-[160], [259] and [260]. These features combined with the primary reference disclose a system of having a machine learning model that determines characteristics within a medical scan image and providing a workflow of scans that are ordered in a specific manner in order to be seen by a medical professional in a queued order. Therefore, based on the above, the features of the claims are disclosed.
Thus, based on the above, the features of the claims are disclosed below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The phrase “showing the order” in line 9 is considered indefinite. There is lack of antecedent basis for this term. Clarity is needed regarding this phrase. Dependent claims 2-14 are rejected based on their dependency.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 15-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Conjeti (EP 4020492 A1 (Pub Date: 6/29/2022)) in view of Lyman (US Pub 2020/0111561).
Re claim 1: A computer implemented method for analysing medical scan images from a single patient comprising the steps of:
receiving an input of a plurality of medical scan images for the single patient (e.g. the invention discloses acquiring multiple X-ray images of a patient, which is taught in ¶ [39].):
[0039] For example, a preferred method comprises the following steps:
acquire a first image, e.g. an X-ray image
optionally: send the first image to a determination system, especially via DICOM communication,
optionally: pre-processing of the first image, especially normalization or de-noising,
segmentation of the first image (e.g. of the lung area), especially using a (deep) machine learning algorithm,
examining the first image for abnormalities (e.g. a pneumothorax), especially using a (deep) machine learning algorithm, and in the case an abnormality is found
a) read patient identifier, e.g. from a DICOM header of the first image,
b) query an image archive (such as e.g. PACS) whether a prior image of the same patient in a given time frame (e.g. same day, same week, etc.) is available,
c) if yes, fetch this prior image as second image,
optionally: send the second image to the determination system, especially via DICOM communication,
optionally: pre-processing of the second image, especially normalization or de-noising,
segmentation of the second image (the same area as segmented in the first image), especially using a (deep) machine learning algorithm, preferably using the same algorithm as for the first image,
examining the second image for abnormalities (the same as for the first image), especially using a (deep) machine learning algorithm,
optional: registering both images,
compare abnormalities of the first image with corresponding abnormalities or sections (if there is no abnormality) of the second image, e.g. a pneumothorax relative to the ratio of the lung volume,
determining a value for the change of the abnormality between the images (assessment),
in the case the change exceeds a predefined threshold, e.g. if the percentage affected area on the first image is bigger (by a given offset) than in the second image, flag the images of the patient and prioritize the patient in a worklist.
[0050] Preferred technical components for the invention are: an image viewer such as PACS, a worklist (e.g. PACS- or RIS-driven), AI software (software for a machine learning algorithm) to perform computation, computational hardware for AI software (either cloud-based or on-premise).
[0059] The imaging system (here the CT system 1) records images I1, 12 that are processed by the determination system 6 according to the invention. The determination system 6 comprises a data interface 8, an examination unit 9 and a determination unit 10 to perform the steps of the method according to the invention.
[0060] The data interface 8 is designed for receiving images I1, I2 (see e.g. figure 2 or 3) both showing the same predefined anatomical region TH, e.g. the thorax TH (see e.g. figure 3) of the patient P. The examination unit 9 is designed for examining the received images I1, I2 for abnormalities A of a predetermined type. The determination unit 10 is designed for different purposes of the method.
providing an input of one or more characterised features from the medical scan images, in addition to the plurality of medical scan images to an optimisation model (e.g. a machine learning algorithm is used to evaluate first and second images to determine the abnormalities present within the images. The abnormalities found within the deep machine learning model can be considered as input into the device, which is taught in ¶ [39] above. In addition, the abnormalities are passed to the determination unit with the images in order to determine changes of the abnormality. The determination unit uses a decision algorithm to perform this feature, which is taught in ¶ [60]-[62] and [69].),
[0060] The data interface 8 is designed for receiving images I1, I2 (see e.g. figure 2 or 3) both showing the same predefined anatomical region TH, e.g. the thorax TH (see e.g. figure 3) of the patient P. The examination unit 9 is designed for examining the received images I1, I2 for abnormalities A of a predetermined type. The determination unit 10 is designed for different purposes of the method.
[0061] First, the determination unit 10 is designed for determining changes C of an abnormality A between the first image I1 and the second image 12. Then, the determination unit 10 is designed for assessing the determined changes C and furthermore, the determination unit 10 is designed for determining whether the assessed changes C exceed a predefined threshold T by using a decision algorithm D.
[0062] The output unit 11 is designed for triggering an alert notification N and/or to change a worklist prioritization concerning the respective patient P in the case the assessed changes C exceed the predefined threshold T.
[0069] In step IV, it is determined whether the assessed changes C exceed a predefined threshold T. This is done by using a decision algorithm D, especially a trained machine learning algorithm.
where the optimisation model outputs the characterised features and medical scan images (e.g. the invention is directed towards organizing the reading worklist of the chest x-rays of a patient over time and changing this patient’s images in the workflow order for review based on the detected abnormalities, which is taught in ¶ [39] above and [09].); and
[0009] A method according to the invention for automatically determining (alertable) changes of a condition of a patient, especially generating or organizing a medical reading worklist, preferably for longitudinal triaging of chest x-rays, comprises the following steps:
Receiving at least a first image and a second image, both showing the same predefined anatomical region of the patient, wherein the images have been recorded during different examinations of the patient.
providing an output to a user, related to the medical scan images according to the result of the predefined relevance criteria, that comprises a workflow showing the order (e.g. a worklist in a RIS system shows patients in an order and a worklist in the PACS system shows the order of medical data for review. With the above invention changing the priority of the order of a patient’s x-rays for review, this performs the feature of showing a radiologist patient scans reading order that are impacted by the detected patient abnormalities, which is taught in ¶ [09], [39] and [50] above.).
However, Conjeti fails to specifically teach the features of outputs the characterised features and medical scan images sorted according to a predefined relevance criteria, providing an output to a user that comprises a workflow showing the order in which the plurality of medical scan images should be reviewed.
However, this is well known in the art as evidenced by Lyman. Similar to the primary reference, Lyman discloses performing a medical scan of a user (same field of endeavor or reasonably pertinent to the problem).
Lyman discloses outputs the characterised features and medical scan images sorted according to a predefined relevance criteria, providing an output to a user that comprises a workflow showing the order in which the plurality of medical scan images should be reviewed (e.g. a subsystem can be used to assign scores regarding priority or malignancy to a medical scan. The medical scans seen within a queue can be automatically ordered based on the priority score that can be based on a malignancy score, which is taught in ¶ [156]-[160]. The scans can be seen be a reviewer that shows a nodule or disease, which the nodule is taught in ¶ [152]. The system used to present the queue for reviewing scans in a particular order is a part of a machine learning model, which is taught in ¶ [259] and [260]. This reference in combination with the prior reference discloses showing an order of scans that comprised a disease performed automatically to show the reviewer a list for review in an automated manner.).
[0152] In FIG. 8N, the interactive interface enters the new abnormality mode in response to the scan review data 810 indicating that the user elects to enter a new finding. The user can type the report entry corresponding to the new finding in a text window as shown in FIG. 8O. In FIG. 8P, the user elects to select the region of interest corresponding to the new finding, such as a nodule that was overlooked by the medical scan assisted review system, and the interactive interface 275 prompts the user to indicate the region of interest by selecting points on the image slice corresponding to vertices of a polygon that surrounds the nodule. In FIG. 8Q, the interactive interface displays five vertices 850 selected by the user, and the interactive interface 275 prompts the user to indicate when they have finished by double clicking. The medical scan assisted review system 102 automatically generates scan review data 810 corresponding to the new finding based on the text entered by the user in the text window and the polygon indicating the region of interest, automatically determined based on the five vertices 850 indicated by the user, as presented in the interactive interface 275 of FIG. 8R in response to the user electing to approve the new finding, and this scan review data 810 corresponding to the new finding can be added to the medical scan database and/or utilized in training sets used to improve the performance of the medical scan assisted review system 102 or other subsystems in subsequent uses.
[0156] A plurality of triaged and/or uploaded scans can be queued for review by a selected user, and the medical scan assisted review system 102 can present a listed queue of scans for review via another view of the interactive interface 275 presented by the display device. This can be based on a plurality of medical scans triaged to the user and/or in response to receiving a plurality of medical scans uploaded to the medical scan processing system 100 by user. The queue of scans can be displayed as line item data in a row, and corresponding data can be displayed such as include patient data, risk factor data, priority score data, a diagnosis summary, abnormality classifier data, confidence score data, or other data retrieved from the medical scan database 342 or generated by one or more subsystems 101.
[0157] The queue of scans can be displayed via the interactive interface 275 in an order based on an automatically generated priority score or a priority score retrieved from scan priority data 427 of the medical scan database 342. For example, the priority score can be based on a manually assigned score for an incoming scan, based on the date of the scan, based on a severity of patient symptoms, severity of previous diagnosis data of the scan, or severity of the diagnosis data 440 automatically generated for the medical scan, for example, based on a malignancy score indicated in diagnosis data 440. The user can re-sort and/or filter the queue based on one or more selected criteria, selected via user input to the interactive interface and/or determined based on queue criteria preferences associated with the user, for example, mapped to user profile entry 354. The selected criteria can include such as selecting to view the filtered list of scans based on criteria for one or more selected abnormality pattern types. For example, the user can select to view the filtered list of scans where an abnormality pattern corresponding to cardiomegaly was detected, and can further select to sort the list of scans in reverse order by a confidence score corresponding to detection of cardiomegaly of confidence score data 460. As another example, consider the case where diagnosis data 440 has multiple entries corresponding to multiple diagnosis authors, for example with a first entry was generated based on user input by another user of the system and a second entry generated automatically by utilizing a medical scan image analysis function. The user can select to view the filtered list of scans where an abnormality was detected by the first diagnosis author that was not reported by the second diagnosis author, or vice versa. This can be used to quickly find discrepancies between known diagnosis data and other diagnosis data generated by a subsystem, for example, by utilizing a medical scan analysis function. The user can continue to re-sort and/or further filter or un-filter the queue by adding or removing sorting and/or filter criteria.
[0158] The user can be automatically presented a medical scan for review from the top of a sorted and/or filtered queue, or the original queue, by the medical scan assisted review system 102 as described herein. Alternatively or in addition, the user can select a scan from the original queue of scans, or the re-sorted and/or filtered queue of scans, based on clicking the corresponding row of the selected scan or other input to the interactive interface, and the selected medical scan will be presented by the medical scan assisted review system 102 as described herein. The medical scan review system can return to the displayed queue of scans after a user has completed review of the current scan. The user can elect to re-order and/or re-filter the queue by providing new criteria, such as selecting a new confidence score threshold or selecting new abnormality pattern type. The user can also elect to select a new medical scan for review from the displayed queue. In other embodiments, medical scan review system can automatically display the next scan in the queue once the review of the current scan is complete, without returning the view of listed scans.
[0159] In some embodiments, the user can elect to confirm the diagnosis data 440 for a selected medical scan without viewing the medical scan. For example, where the user can elect to confirm the diagnosis data 440 by selecting a menu option presented for each scan in the list of scans via the interactive interface 275. This can be based on confidence score data 460 displayed as a line item in conjunction with the medical scan, for example, where the user elects to confirm diagnosis data 440 without viewing the scan because the displayed confidence score is 99%. The user can select auto-confirm criteria, such as “automatically confirm all normal scans” or “automatically confirm all diagnosis data that indicates cardiomegaly with a confidence score that is greater than 90%”. In such embodiments, the queue can be automatically filtered based on the auto-confirm criteria, where automatically confirmed medical scans are not listed. The auto-confirm criteria can be selected via user input to the interactive interface 275 and/or determined based on queue criteria preferences associated with the user, for example, mapped to user profile entry 354. In some embodiments, the auto-confirm criteria will be learned and selected by a subsystem and/or the auto-confirm criteria will be entered by an administrator.
[0160] In various embodiments, the medical scan assisted review system 102 will only display medical scans with detected abnormalities, for example, where normal scans are included in the auto-confirm criteria. For example, medical scans automatically determined to be normal with at least a threshold confidence score will not be presented to the user for review, and will automatically be filtered from the queue. Diagnosis data, report data, and/or a flag indicating the scan is normal will automatically be mapped to the medical scan in the medical scan database and/or transmitted to the responsible medical entity without user intervention. For quality control, a threshold proportion of normal scans, or other auto-confirmed scans based on other criteria, can be randomly or psuedo-randomly selected and presented to the user for review.
[0259] In embodiments where the medical scan image analysis system is used in conjunction with the medical scan diagnosing system, each of the medical scan image analysis functions associated with each neural network model can correspond to one of the plurality of neural network models generated by the medical scan image analysis system. For example, each of the plurality of neural network models can be trained on a training set classified on scan type, scan human body location, hospital or other originating entity data, machine model data, machine calibration data, contrast agent data, geographic region data, and/or other scan classifying data as discussed in conjunction with the medical scan diagnosing system. In embodiments where the training set classifiers are learned, the medical scan diagnosing system can determine which of the medical scan image analysis functions should be applied based on the learned classifying criteria used to segregate the original training set.
[0260] A computer vision-based learning algorithm used to create each neural network model can include selecting a three-dimensional subregion 1310 for each medical scan in the training set. This three-dimensional subregion 1310 can correspond to a region that is “sampled” from the entire scan that may represent a small fraction of the entire scan. Recall that a medical scan can include a plurality of ordered cross-sectional image slices. Selecting a three-dimensional subregion 1310 can be accomplished by selecting a proper image slice subset 1320 of the plurality of cross-sectional image slices from each of the plurality of medical scans, and by further selecting a two-dimensional subregion 1330 from each of the selected subset of cross-sectional image slices of the each of the medical scans. In some embodiments, the selected image slices can include one or more non-consecutive image slices and thus a plurality of disconnected three-dimensional subregions will be created. In other embodiments, the selected proper subset of the plurality of image slices correspond to a set of consecutive image slices, as to ensure that a single, connected three-dimensional subregion is selected.
Therefore, in view of Lyman, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of outputs the characterised features and medical scan images sorted according to a predefined relevance criteria, providing an output to a user that comprises a workflow showing the order in which the plurality of medical scan images should be reviewed, incorporated in the device of Conjeti, in order to sort scans with signs of severe disease for review in a queue for a reviewer in a specific order, which can aid the medical professional in reviewing medical scans (as stated in Lyman ¶ [100]).
Re claim 2: However, Conjeti fails to specifically teach the features of the method of claim 1, wherein the output is at least one of: a display of medical images sorted according to the predefined relevance criteria, a report containing information about the medical images sorted according to the predefined relevance criteria.
However, this is well known in the art as evidenced by Lyman. Similar to the primary reference, Lyman discloses performing a medical scan of a user (same field of endeavor or reasonably pertinent to the problem).
Lyman discloses wherein the output is at least one of: a display of medical images sorted according to the predefined relevance criteria, a report containing information about the medical images sorted according to the predefined relevance criteria (e.g. an output of the system can consist of a display of medical images that can be previewed or displayed for review or a queue showing the potential reports for review, which is taught in ¶ [152] and [156]-[160] above.).
Therefore, in view of Lyman, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein the output is at least one of: a display of medical images sorted according to the predefined relevance criteria, a report containing information about the medical images sorted according to the predefined relevance criteria, incorporated in the device of Conjeti, in order to sort scans with signs of severe disease for review in a queue for a reviewer in a specific order, which can aid the medical professional in reviewing medical scans (as stated in Lyman ¶ [100]).
Re claim 3: Conjeti discloses the method of claim 1, further comprising the steps of:
detecting one or more features of the plurality of medical scan images (e.g. an abnormality is detected in the scan in order to compare this abnormality to another scan, which is taught in ¶ [39] above); and
characterising the one or more detected features from the plurality of medical scan images; before providing the input to the optimisation model (e.g. the abnormalities can be determined by examination unit before the abnormalities are sent to the determination unit, which is performed by a machine learning algorithm. This is taught in ¶ [60]-[62] and [69] above.).
Re claim 4: Conjeti discloses the method of claim 3, wherein the optimisation model outputs the N most relevant features according to the predefined relevance criteria, where N is a configurable parameter of the method (e.g. a threshold is used to determine if an abnormality should trigger an alert. This threshold used can be defined for actionable changes and can be defined in different ways, which is taught in ¶ [18]-[21]. The optimisation model allows for a flag of the images to be performed associated with the abnormalities, which is taught in ¶ [39] above.).
[0018] The threshold should be defined such that an actionable change that is complying with Standard of Care (or Clinical Standard Operating Procedures) is indicated. The Standard of Care as well as an actionable change is well known in the art concerning a predefined disease. It should be noted that an actionable change is not any change per say, but only ones that trigger changes to patient management. It should also be noted that the method according to the invention is not anticipating the examination of a physician, but may use the thresholds defined for actionable changes.
[0019] For automatically determining changes, said decision algorithm is used, that is designed to calculate whether changes, e.g. in the spatial extent (dimensions) and/or density, texture, shape of the abnormality manifest as an actionable change that is complying with Standard of care. This decision algorithm is necessary. It can be designed such that it is calculating whether a numerical value of a change exceeds a threshold in a very simple embodiment. However, it also could be a machine learning (especially deep learning) algorithm that is specially trained to measure changes of abnormalities in images. Such algorithm facilitates assessment of series of images that are indicative of clinically actionable changes. It should be noted that this decision algorithm should preferably be invoked automatically before a clinician reviews the case, especially in the case of triage.
[0020] The threshold may be a value (e.g. a volume, measurement) or a predefined region that should not be exceeded. However, the threshold may also be a limit for the measure of the change itself, e.g. a threshold for the first derivative of the change with respect to time.
[0021] In the case the assessed changes exceed the predefined threshold:
triggering an alert notification and/or change a worklist prioritization concerning the respective patient.
Re claim 5: Conjeti discloses the method of claim 1, wherein the plurality of medical scan images is one of: CT scan, PET scan, MRI scan, SPECT scan (e.g. CT images can be captured, which is taught in ¶ [10].).
[0010] It is clear that both images should show the same region of interest, e.g. the same organ, since this region is examined for changes. It is also clear that the images should be recorded during different examinations of the patient, since a longitudinal examination is intended, i.e. an examination over two or more time points, e.g. at different hours, days, weeks or months, possibly at a different (physical) location. The more images are processed, the more timepoints are available. Any further image can be processed with this method with respect to the temporally adjacent image. Although it is preferred that both images are of the same type, e.g. both images are X-ray images or CT-images (CT: computer tomography), this is not in every case necessary. Instead of the term "image" also the term "medical image" can be used. For example, the image can be an object according to the DICOM (acronym for "Digital Imaging and Communications in Medicine") standard. examining the images for abnormalities of a predetermined type.
Re claim 6: Conjeti discloses the method of claim 5, wherein the plurality of medical scan images are 2D images or 3D images (e.g. 2D or 3D images can be captured, which is taught in ¶ [55].).
[0055] It should be noted that the images could be 2D-images or 3D-images. By using deep learning algorithms, the images do not have to be reconstructed images, but could also comprise raw data. In general, it is preferred that the images are DICOM-datasets.
Re claim 7: Conjeti discloses the method of claim 5, wherein the one or more medical scan images are images showing all or part of a lung (e.g. a lung area is captured with the scan, which is taught in ¶ [39] above.), and
the characterised features are one or more of: lung nodule, features indicating the presence of at least one of emphysema, fibrotic tissue, consolidation and scarring (e.g. a lung lesion or evidence of emphysema can be detected as an abnormality, which is taught in ¶ [46].).
[0046] According to a preferred embodiment, an abnormality is a (radiographic) finding of the group comprising malpositioned lines or tubes of immediate clinical concern, tension pneumothoraces, pulmonary emboli, lung lesions with high possibility of being active tuberculosis, superior vena cava occlusion, large pericardial effusion and/or suspected tamponade of any cause, active post-traumatic hemorrhage, tracheal obstruction, lobar or lung collapse, pneumomediastinum, interstitial emphysema, extensive subcutaneous emphysema, pleural effusion, significant vena cava compression and pneumonia, or a non-thoracic finding from the group comprising suspected nonaccidental trauma, malpositioned line or tube of immediate clinical concern (e.g., ET tube or enteric tube in bronchus), allergic reaction or other adverse event, foreign body with potential immediate and/or severe consequences, intracranial or spinal hemorrhage (parenchymal, subarachnoid, subdural, epidural), non-hemorrhagic stroke or suspected stroke, thrombolytic candidate, intracranial mass with significant mass effect (midline shift/herniation/hydrocephalus), brain herniation, symptomatic hydrocephalus (malfunctioning shunt or new diagnosis of any cause), depressed skull fracture, post-traumatic pneumocephalus, arterial dissection, brain death (nuclear study or other), severe spinal cord compression of any cause, unstable spine fracture, cord hemorrhage or infarct, airway obstruction, unexplained pneumoperitoneum, closed loop intestinal obstruction, intestinal ischemia and/or portal/mesenteric, venous gas, pseudoaneurysm or active hemorrhage (post-trauma, GI bleed, other), high-grade intra-abdominal organ injury, testicular torsion, ovarian torsion, ectopic pregnancy (high likelihood), placental abruption, uterine rupture, high grade kidney injury and/or ureteral or bladder injury post-trauma, non-spinal fracture and/or dislocation with risk of vascular compromise, necrotizing fasciitis, Ruptured/leaking arterial aneurysm (thoracic or abdominal aortic or other), limb-threatening arterial or venous occlusion or high-grade stenosis, arterial dissection and intramural hematoma (aortic or other).
Re claim 8: Conjeti discloses the method of claim 3, wherein the detected features are characterised according to one or more of: a malignancy score, an invasiveness score, a feature size (e.g. the size of area is considered for the abnormality detection, which is taught in ¶ [13].).
[0013] These changes could be changes of dimensions, e.g. the progression of the collapse of the lung concerning a pneumothorax or the expansion of a haemorrhage. For example, dimensional changes could be computed with area-comparison or apex to cupola distance comparison. However, changes could also be changes of opacity, texture and/or shape of an anatomical region. Concerning digital images, a "change" is a change in digital image information of the predefined region. Since the image information is usually the grayscale value of the pixels of the image, the change is the change of grayscale and/or size and/or form of image-elements in the predefined region.
Re claim 15: Conjeti discloses a device for analysing medical scan images from a single patient comprising:
one or more processors (e.g. processors are utilized, which is taught in ¶ [30] and [31].) configured to:
[0030] Some units or modules of the determination system mentioned above can be completely or partially realized as software modules running on a processor of a computing system. A realization largely in the form of software modules can have the advantage that applications already installed on an existing computing system can be updated, with relatively little effort, to install and run these units of the present application. The object of the invention is also achieved by a computer program product with a computer program that is directly loadable into the memory of a computing system, and which comprises program units to perform the steps of the inventive method when the program is executed by the computing system. In addition to the computer program, such a computer program product can also comprise further parts such as documentation and/or additional components, also hardware components such as a hardware key (dongle etc.) to facilitate access to the software.
[0031] A computer readable medium such as a memory stick, a hard-disk or other transportable or permanently-installed carrier can serve to transport and/or to store the executable parts of the computer program product so that these can be read from a processor unit of a computing system. A processor unit can comprise one or more microprocessors or their equivalents.
receive an input of a plurality of medical scan images for the single patient (e.g. the invention discloses acquiring multiple X-ray images of a patient, which is taught in ¶ [39].):
provide an input of one or more characterised features from the plurality of medical scan images, in addition to the medical scan images to an optimisation model (e.g. a machine learning algorithm is used to evaluate first and second images to determine the abnormalities present within the images. The abnormalities found within the deep machine learning model can be considered as input into the device, which is taught in ¶ [39] above. In addition, the abnormalities are passed to the determination unit with the images in order to determine changes of the abnormality. The determination unit uses a decision algorithm to perform this feature, which is taught in ¶ [60]-[62] and [69] above.),
where the optimisation model outputs the features and images (e.g. the invention is directed towards organizing the reading worklist of the chest x-rays of a patient over time and changing this patient’s examination order in the workflow based on the detected abnormalities, which is taught in ¶ [09] and [39] above.); and
providing an output related to the medical images to a user according to a result of the predefined relevance criteria, that comprises a workflow showing the order (e.g. a worklist in a RIS system shows patients in an order and a worklist in the PACS system shows the order of medical data for review. With the above invention changing the priority of the order of a patient’s x-rays for review, this performs the feature of showing a radiologist patient scans reading order that are impacted by the detected patient abnormalities, which is taught in ¶ [09], [39] and [50] above.).
However, Conjeti fails to specifically teach the features of outputs the characterised features and images sorted according to a predefined relevance criteria, providing an output related to the medical scan images to a user that comprises a workflow showing an order in which the plurality of medical scan images should be reviewed.
However, this is well known in the art as evidenced by Lyman. Similar to the primary reference, Lyman discloses performing a medical scan of a user (same field of endeavor or reasonably pertinent to the problem).
Lyman discloses outputs the characterised features and images sorted according to a predefined relevance criteria, providing an output related to the medical scan images to a user that comprises a workflow showing an order in which the plurality of medical scan images should be reviewed (e.g. a subsystem can be used to assign scores regarding priority or malignancy to a medical scan. The medical scans seen within a queue can be automatically ordered based on the priority score that can be based on a malignancy score, which is taught in ¶ [156]-[160]. The scans can be seen be a reviewer that shows a nodule or disease, which the nodule is taught in ¶ [152]. The system used to present the queue for reviewing scans in a particular order is a part of a machine learning model, which is taught in ¶ [259] and [260]. This reference in combination with the prior reference discloses showing an order of scans that comprised a disease performed automatically to show the reviewer a list for review in an automated manner.).
Therefore, in view of Lyman, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of outputs the characterised features and images sorted according to a predefined relevance criteria, providing an output related to the medical scan images to a user that comprises a workflow showing an order in which the plurality of medical scan images should be reviewed, incorporated in the device of Conjeti, in order to sort scans with signs of severe disease for review in a queue for a reviewer in a specific order, which can aid the medical professional in reviewing medical scans (as stated in Lyman ¶ [100]).
Re claim 16: Conjeti discloses the device for analyzing medical scan images from a single patient of claim 15, wherein the plurality of medical scan images is one of: CT scan, PET scan, MRI scan or SPECT scan (e.g. CT images can be captured, which is taught in ¶ [10] above.), and the one or more medical scan images are images showing all or part of a lung (e.g. a lung area is captured with the scan, which is taught in ¶ [39] above.).
Re claim 17: Conjeti discloses the device for analyzing medical scan images from a single patient of claim 15, wherein the one or more processors are further configured to detect multiple classes of features in the plurality of input medical scan images (e.g. the features detected can be lines sensed within an image or a size of an area that is compared to other scans, which are separate classes and explained in ¶ [46]-[48] above.).
Re claim 18: Conjeti discloses the device for analyzing medical scan images from a single patient of claim 17, wherein the one or more processors are further configured to characterise the one or more detected features from the plurality of medical scan image, before the detected features are provided to the optimisation model (e.g. the abnormalities can be determined by examination unit before the abnormalities are sent to the determination unit, which is performed by a machine learning algorithm. This is taught in ¶ [60]-[62] and [69] above.).
Re claim 20: However, Conjeti fails to specifically teach the features of the device for analyzing medical scan images from a single patient of claim 15, wherein the output is at least one of: a display of medical scan images sorted according to the relevance criteria, a report containing information about the medical images sorted according to the predefined relevance criteria.
However, this is well known in the art as evidenced by Lyman. Similar to the primary reference, Lyman discloses performing a medical scan of a user (same field of endeavor or reasonably pertinent to the problem).
Lyman discloses wherein the output is at least one of: a display of medical scan images sorted according to the relevance criteria, a report containing information about the medical images sorted according to the predefined relevance criteria (e.g. an output of the system can consist of a display of medical images that can be previewed or displayed for review or a queue showing the potential reports for review, which is taught in ¶ [152] and [156]-[160] above.).
Therefore, in view of Lyman, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention was made to have the feature of wherein the output is at least one of: a display of medical scan images sorted according to the relevance criteria, a report containing information about the medical images sorted according to the predefined relevance criteria, incorporated in the device of Conjeti, in order to sort scans with signs of severe disease for review in a queue for a reviewer in a specific order, which can aid the medical professional in reviewing medical scans (as stated in Lyman ¶ [100]).
Claim(s) 9-14 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Conjeti, as modified by Lyman, as applied to claims 3 and 17 above, and further in view of Allen (US Pub 2020/0160032).
Re claim 9: However, Conjeti fails to specifically teach the features of the method of claim 3, wherein detecting the features is done with a first machine learning model, and characterising the features is done with a second machine learning model.
However, this is well known in the art as evidenced by Allen. Similar to the primary reference, Allen discloses a machine learning model that determines features in an image (same field of endeavor or reasonably pertinent to the problem).
Allen discloses wherein detecting the features is done with a first machine learning model, and characterising the features is done with a second machine learning model (e.g. the reference discloses using two machine learning models to perform different parts of the analysis. A first tier can be used to determine the features while a second tier is used to characterize what the scores associated with the features represent, which is taught in ¶ [47] and [55].).
[0047] In some embodiments, the methods or processes described herein generally involve whole slide classification using clinically-relevant cellular features, such as cytomorphologic or histologic criteria. The methods use deep learning to predict whole slide-level pathology diagnoses based on aggregated cellular features chosen from, e.g., established guidelines or clinical practice. Deep learning is often considered an uninterpretable black box process. There are a number of approaches for explaining the behavior of deep learning-based machine learning models, but these are nearly all retrospective and limited. In order to take advantage of deep learning while maintaining interpretability, the approach described herein includes a separate deep learning-based sub-model trained for each cellular feature deemed relevant for a particular interpretation by clinical guidelines, providing a first tier of analysis. The results of these sub-models are then aggregated for all cells or tissue across the whole slide image using statistical summary measures (e.g. mean, standard deviation, skew). These statistical measures, as well as, in some embodiments, other additional data, can then be used in a second tier of machine learning analysis to predict a whole slide-level diagnosis, such as the presence or absence of a disease or disease type. This method is not only accurate, but also enables display of the reasoning for the diagnosis at the cellular and whole slide levels, and allows for optimization of outputs as well as optimization of the system models.
[0055] At stage 191, the system can generate an array of feature scores. The scores can be based on training received by the system. For example, a system can be trained against a pathologist's scores for each selected features, based on the particular representative feature and disease, condition, or process being examined. For example, a series of trained models can be used to produce an array of feature scores including scores for each of the selected features. One skilled in the art will appreciate that any of the various methods available for training one or more models for the systems and methods described herein may be used. In some embodiments, the systems, methods, and devices described herein can use deep learning. In some embodiments, the systems, methods, and devices described herein do not use simple end-to-end learning to analyze a slide image. In some embodiments, the systems, methods, and devices described herein use a first tier of trained models (e.g., a series of models including one or more models for each of two or more features), combined with a second tier trained model that processes an array of feature scores obtained from the first tier of models. In some embodiments, the systems, methods, and devices described herein use machine learning to perform one or more of classification, regression, clustering, and association. In some embodiments, the systems, methods, and devices described herein can employ one or more of dynamic time warping (DTW), decision trees, linear regression, neural networks, multinomial LR, Naive Bayes (NB), trained Gaussian NB, NB with dynamic time warping, MLR, Shannon entropy, support vector machine (SVM), one versus one support vector machine, k-means clustering, Q-learning, temporal difference (TD), deep adversarial networks, and the like. In some embodiments, the systems and methods described herein use one or more multiple instance learning models.
Therefore, in view of Allen, it would have been obvious to one of ordinary skill at the time the invention was made to have the feature of wherein detecting the features is done with a first machine learning model, and characterising the features is done with a second machine learning model, incorporated in the device of Conjeti, in order to utilize models to perform the detection of features and characterizing of the features, which using automated analyses of biologic specimens can decrease time and costs of analyses when compared to traditional methods (as stated in Allen ¶ [14]).
Re claim 10: Conjeti discloses the method of claim 9, wherein the first and second machine learning model uses a neural network, a support vector machine or a random forest algorithm (e.g. leaning models are used for segmenting and determining abnormalities. A neural network is used for the models, which is taught in ¶ [39] above and [42].).
[0042] According to a preferred embodiment, a machine learning algorithm is used for examining the images and/or for determining changes of an abnormality and/or for determining whether the assessed changes exceed a predefined threshold. This machine learning algorithm is preferably a deep-learning network, preferably a deep belief network, ResNet, DenseNet, Autoencoder, capsule network, generative adversarial network, Siamese network, convolutional neural network, deep reinforcement learning algorithm, or preferably based on a machine learning technique of the group support vector machine, Bayesian model, decision tree and k-means clustering.
Re claim 11: Conjeti discloses the method of claim 10, wherein the optimization model uses a machine learning model such as a neural network (e.g. a trained machine learning algorithm is used to perform the feature of the optimization model, which is taught in ¶ [69] above. In addition other models can be used to serve as the optimization model, which is taught in ¶ [42] above.).
Re claim 12: Conjeti discloses the method of claim 9, wherein the feature detection model is a single model that detects multiple feature classes simultaneously (e.g. the features detected can be lines sensed within an image or a size of an area that is compared to other scans, which are separate classes and explained in ¶ [46] above, [47] and [48].).
[0047] According to a preferred embodiment, in the course of comparing the areas of an abnormality in the images, the speed of growth is determined, preferably based on the time difference between the recording of images, especially by determining the difference of the ar