DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are pending under this Office action.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 and 11-20 are rejected under 35 U.S.C. 103 as being unpatentable over Anand, etc. (US 20250336523 A1) in view of Sjolund, etc. (US 20190332900 A1), further in view of Veidman, etc. (US 20200372635 A1).
Regarding claim 1, Anand teaches that a model generation system (See Anand: Fig. 1, and [0023], “Referring now to FIG. 1, an exemplary embodiment of an apparatus 100 for generating diagnostic hypotheses based on electrocardiogram (ECG) data is illustrated. Apparatus 100 includes a computing device. Computing device includes a processor 104 communicatively connected to a memory 108. As used in this disclosure, “communicatively connected” means connected by way of a connection, attachment, or linkage between two or more relata which allows for reception and/or transmittance of information therebetween”) comprising:
memory circuitry (See Anand: Fig. 1, and [0023], “Referring now to FIG. 1, an exemplary embodiment of an apparatus 100 for generating diagnostic hypotheses based on electrocardiogram (ECG) data is illustrated. Apparatus 100 includes a computing device. Computing device includes a processor 104 communicatively connected to a memory 108. As used in this disclosure, “communicatively connected” means connected by way of a connection, attachment, or linkage between two or more relata which allows for reception and/or transmittance of information therebetween. For example, and without limitation, this connection may be wired or wireless, direct, or indirect, and between two or more components, circuits, devices, systems, and the like, which allows for reception and/or transmittance of data and/or signal(s) therebetween”);
instructions in the memory circuitry (See Anand: Fig. 1, and [0125], “Such software may be a computer program product that employs a machine-readable storage medium. A machine-readable storage medium may be any medium that is capable of storing and/or encoding a sequence of instructions for execution by a machine (e.g., a computing device) and that causes the machine to perform any one of the methodologies and/or embodiments described herein”); and
processor circuitry to execute the instructions to at least (See Anand: Fig. 1, and [0125], “Such software may be a computer program product that employs a machine-readable storage medium. A machine-readable storage medium may be any medium that is capable of storing and/or encoding a sequence of instructions for execution by a machine (e.g., a computing device) and that causes the machine to perform any one of the methodologies and/or embodiments described herein”):
train a first diffusion model using a first set of images without contours (See Anand: Fig. 1, and [0027], “With continued reference to FIG. 1, as a non-limiting example, processor may be configured to implement a large language model (LLM). A “large language model,” as used herein, is a deep learning data structure that can recognize, summarize, translate, predict and/or generate text and/or other content based on knowledge gained from massive datasets. LLM may be trained on large sets of data. In one embodiment, generative model 112 is trained on a corpus 116. As used in this disclosure, a “corpus” is a large set of data. Corpus data may include text, images, videos, audio, or the like. Corpus data may be structured, semi-structure, and/or unstructured. In some cases, corpus 116 may include a collection of sufficiently diverse and comprehensive texts, covering desired breadth and depth of knowledge to one or more domains (e.g., medicine including cardiology, pharmacology, epidemiology, and the like), that is used to train LLM, allowing LLM to understand, interpret, and/or generate language-based outputs that are relevant to the model's intended applications as described herein. In some cases, corpus 116 may include a set of medical literatures encompassing research findings, clinical studies, reviews, case reports, scholarly articles, and any other written material related to the field of medicine and healthcare. As a non-limiting example, corpus 116 may include a collection of peer-reviewed medical research papers, reviewed, articles from reputable journals, official clinical guidelines, treatment protocols, best practice documents from recognized medical associations and/or organizations, medical textbooks, reference materials covering explanation of medical conditions, treatments, health maintenance strategies, online medical forums from online medical communities including discussions and Q&A sessions, among others. In some cases, corpus 116 may include information from one or more public or private databases. As a non-limiting example, corpus may include a PubMed database or any other repository of knowledge within medical community”; and [0041], “Other exemplary embodiment of generative model 112 may include, without limitation, an autoencoder for dimensionality reduction and feature learning, a diffusion model for generating image or audio data, among others”. Note that the diffusion mode is a LLM, and it is trained with a corpus of data, which includes medical literature, reviews, images, etc., and this is mapped to train a first diffusion model using a first set of images without contours because medical images are normally captured without contours, the boundary of the objects are not considered as a contour in this Office action for this cited limitation, that is, the contours are added manually or through image processing like a segmentation process, and another art is searched for the image with contour for the next cited limitation);
fine-tune the first diffusion model using a second set of images with contours to form a second diffusion model, wherein the second set of images is smaller than the first set of images (See Anand: Fig. 1, and [0029], “As a non-limiting example, generative model 112 such as, without limitation, a LLM may be pre-training on a general set of medical literatures (i.e., a wide range of medical literatures covering the vast field of medicine) and fine-tuning on a specific set of medical literatures (i.e., texts focused on one or more specific areas), wherein the general set of medical literatures and the specific set of medical literatures are subsets of the set of medical literatures contained in corpus 116. In some cases, majority of medical literatures within the specific set may be more detailed, advanced, or specialized then medical literatures in the general set. For example, general set of medical literatures may include introductory texts and reviews that summarize basic concepts in physiology while specific set of medical literatures may include a plurality of citations to peer-reviewed research papers related to cardiology”. Note that the LLM (diffusion) model is fine-tuning on a specific sub-set of data is mapped to fine-tune the first diffusion model using a second set of images with contours to form a second diffusion model, wherein the second set of images is smaller than the first set of images, however, Anand does not explicitly disclose the second set of images with contour, and a secondary art will be searched);
generate synthetic image patches with contours using the second diffusion model and at least one contour (See Anand: Fig. 8, and [0101], “Further referring to FIG. 8, computing device, processor, and/or module may be configured to perform one or more processes of data augmentation. “Data augmentation” as used in this disclosure is addition of data to a training set using elements and/or entries already in the dataset. Data augmentation may be accomplished, without limitation, using interpolation, generation of modified copies of existing entries and/or examples, and/or one or more generative AI processes, for instance using deep neural networks and/or generative adversarial networks; generative processes may be referred to alternatively in this context as “data synthesis” and as creating “synthetic data.” Augmentation may include performing one or more transformations on data, such as geometric, color space, affine, brightness, cropping, and/or contrast transformations of images”. Note that creating the synthetic data including the images is mapped to generate synthetic images, however, Anand does not explicitly disclose that the synthetic images are patches, and a secondary art will be searched);
train a segmentation model using the synthetic image patches; and
deploy the segmentation model (See Anand: Fig. 8, and [0110], “Still referring to FIG. 8, a machine-learning model and/or process may be deployed or instantiated by incorporation into a program, apparatus, system and/or module. For instance, and without limitation, a machine-learning model, neural network, and/or some or all parameters thereof may be stored and/or deployed in any memory or circuitry. Parameters such as coefficients, weights, and/or biases may be stored as circuit-based constants, such as arrays of wires and/or binary inputs and/or outputs set at logic “1” and “0” voltage levels in a logic circuit to represent a number according to any suitable encoding system including twos complement or the like or may be stored in any volatile and/or non-volatile memory. Similarly, mathematical operations and input and/or output of data to or from models, neural network layers, or the like may be instantiated in hardware circuitry and/or in the form of instructions in firmware, machine-code such as binary operation code instructions, assembly language, or any higher-order programming language”; and [0086], “A “machine learning process,” as used in this disclosure, is a process that automatedly uses training data 804 to generate an algorithm instantiated in hardware or software logic, data structures, and/or functions that will be performed by a computing device/module to produce outputs 808 given data provided as inputs 812; this is in contrast to a non-machine learning software program where the commands to be executed are determined in advance by a user and written in a programming language”. Note that the trained model is deployed to the users to generate output from user input, and this is mapped to deploy the segmentation model) to inference on a third set of images.
However, Anand fails to explicitly disclose that using a second set of images with contours; generate synthetic image patches; train a segmentation model using the synthetic image patches; and to inference on a third set of images.
However, Sjolund teaches that using a second set of images with contours (See Sjolund: Fig. 1, and [0050], “In an example, the image data 152 may include one or more MRI image (e.g., 2D MRI, 3D MRI, 2D streaming MRI, 4D MRI, 4D volumetric MRI, 4D cine MRI, etc.), functional MRI images (e.g., fMRI, DCE-MRI, diffusion MRI), Computed Tomography (CT) images (e.g., 2D CT, Cone beam CT, 3D CT, 4D CT), ultrasound images (e.g., 2D ultrasound, 3D ultrasound, 4D ultrasound), Positron Emission Tomography (PET) images, PET-CT images, X-ray images, fluoroscopic images, radiotherapy portal images, Single-Photo Emission Computed Tomography (SPECT) images, Elastography images, Photoacoustic images, Magnetoencephalography (MEG) images, Electroencephalography (EEG) images, or computer generated synthetic images (e.g., pseudo-CT images) and the like. Further, the image data 152 may also include or be associated with medical image processing data, for instance, training images, and ground truth images, contoured images, and dose images. In other examples, an equivalent representation of an anatomical area may be represented in non-image formats (e.g., coordinates, mappings, etc.)”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was effectively filed to modify Anand to have using a second set of images with contours as taught by Sjolund in order to improve the performance of the neural network by leveraging information from the unified representation (See Sjolund: Fig. 1, and [0037], “As a result, even when only small amounts of data are available for the auxiliary task, the presently disclosed techniques can help improve the performance of the neural network by leveraging information from the unified representation that has been learned on other tasks. In other words, the method can be used for semi-supervised learning of the neural network”). Anand teaches a method and system that may generate diagnostic hypotheses based on electrocardiogram (ECG) data using the LLM algorithms with medical images to train the diffusion model and subset specific images to fine-tuning the diffusion model; while Sjolund teaches a system and method that may use the annotated contoured images to train the artificial intelligent models. Therefore, it is obvious to one of ordinary skill in the art to modify Anand by Sjolund to use the contoured images as the specific subset images to fine-tuning the diffusion model in order to improve the performance of neural network with a smaller amount of data available to train the neural network models. The motivation to modify Anand by Sjolund is “Use of known technique to improve similar devices (methods, or products) in the same way”.
However, Anand, modified by Sjolund, fails to explicitly disclose that generate synthetic image patches; train a segmentation model using the synthetic image patches; and to inference on a third set of images.
However, Veidman teaches that generate synthetic image patches (See Veidman: Fig. 1, and [0180], “Referring now back to FIG. 1, at 112, tissue image patches (also referred to herein as patches) are created from the tissue image. It is noted that patches may be created as part of the segmentation process described herein. Alternatively, the patches for segmentation are different than the patches created for further analysis”. Note that the image may be divided into patches, and thus, the synthetic images can be divided into patches as well, therefore, combing the Anand with Veidman, the synthetic image patches are generated);
train a segmentation model using the synthetic image patches (See Veidman: Fig. 1, and [0186], “The patch-level segmentation code may be implemented as a segmentation CNN (e.g., based on Unet). The patch-level segmentation may output the segmentation as a mask. The patch-level segmentation code may be trained according to patches that are annotated with segmentation markings”. Note that the image patches are used to train the segmentation CNN model, which is mapped to the segmentation model); and
to inference on a third set of images (See Veidman: Fig. 11, and [0271], “Interference flow 1116 is for inferring the image type to compute the slide-level tissue type(s), for example, overall diagnosis. Inference flow 1116 includes a feature 1118 for patch level detection and/or segmentation (e.g., as described with reference to act 110,112, and/or 114 of FIG. 1), a feature 1120 for patch level classification (e.g., as described with reference to act 114 of FIG. 1), a feature 1122 for computational of the slide-level tissue type(s) such as diagnosis (e.g., as described with reference to act 118 of FIG. 1), a feature 1124 for comparison of the slide-level tissue type(s) (e.g., diagnosis) to a manual physician diagnosis (e.g., as described with reference to act 406 of FIG. 4), and a feature 1126 for presenting the results in the GUI (e.g., as described with reference to act 122 of FIG. 1)”. Note that the inference flow for the patient image analysis is mapped to inference on a third set of images).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was effectively filed to modify Anand to have generate synthetic image patches; train a segmentation model using the synthetic image patches; and to inference on a third set of images as taught by Veidman in order to enable computing the slide-level tissue type of the image of the tissue extracted from the patient in an accurate and rapid manner and providing a pathologist, a surgeon or radiologist with control of final diagnosis results using a graphical user interface (GUI) (See Veidman: Fig. 8, and [0229], “The GUI may be designed to provide the user (e.g., pathologist, surgeon, radiologist) with control of the final results (e.g., diagnosis), where at least some of the systems, methods, apparatus, and/or code instructions described herein provide automated assistance to improve the work of the pathologist, to be more accurate and/or more efficient, and/or improve the user of the user performing the medical procedure by providing real time instructions (e.g., to make sure adequate tissue samples are obtained), as described herein. The GUI may be presented on a display, for example, located within the procedure room (e.g., clinic, operating room) such as for real time guiding of the procedure (e.g., biopsy), and/or in the pathology lab such as for automated assistance in analyzing tissue samples”). Anand teaches a method and system that may generate diagnostic hypotheses based on electrocardiogram (ECG) data using the LLM algorithms with medical images to train the diffusion model and subset specific images to fine-tuning the diffusion model; while Veidman teaches a system and method that may compute the slide-level tissue type by the patch-level segmentation CNN model to analyze the patches obtained from the input images. Therefore, it is obvious to one of ordinary skill in the art to modify Anand by Veidman to train the segmentation model using the image patches obtained by the previous diffusion model and compute the slide-level tissue type based on the patch-level segmentation model inference results. The motivation to modify Anand by Veidman is “Use of known technique to improve similar devices (methods, or products) in the same way”.
Regarding claim 2, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Anand teaches that the model generation system of claim 1, wherein the first diffusion model is an unconditioned diffusion model, and wherein the second diffusion model is a fine-tuned, conditioned diffusion model (See Anand: Fig. 8, and [0107], “Further referring to FIG. 8, machine learning processes may include at least an unsupervised machine-learning processes 832. An unsupervised machine-learning process, as used herein, is a process that derives inferences in datasets without regard to labels; as a result, an unsupervised machine-learning process may be free to discover any structure, relationship, and/or correlation provided in the data. Unsupervised processes 832 may not require a response variable; unsupervised processes 832 may be used to find interesting patterns and/or inferences between variables, to determine a degree of correlation between two or more variables, or the like”; and [0104], “Still referring to FIG. 8, machine-learning algorithms may include at least a supervised machine-learning process 828. At least a supervised machine-learning process 828, as defined herein, include algorithms that receive a training set relating a number of inputs to a number of outputs, and seek to generate one or more data structures representing and/or instantiating one or more mathematical relations relating inputs to outputs, where each of the one or more mathematical relations is optimal according to some criterion specified to the algorithm using some scoring function”. Note that an unsupervised machine-learning processes 832 is mapped to an unconditioned diffusion model because un-labeled images are used to train the model, and a supervised machine-learning process 828is mapped to a fine-tuned, conditioned diffusion model because the labeled (ground truth) images are used to train the model).
Regarding claim 3, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Veidman teaches that the model generation system of claim 1, wherein the synthetic image patches include synthetic three-dimensional image patches (See Veidman: Fig. 1, and [0210], “At 119, a multi-slide level tissue type(s) is computed for multiple tissue images, and/or for a 3D tissue image. The multiple slides and/or 3D tissue image may be produced, for example, from the same biopsy, and/or same target tissue (e.g., slices of frozen section), and/or from the same sample of body fluid. The slides may be created with the same stain, or from different stain types, where different processes are used to analyzing the tissue images created from different stain types. In such case a slide-level tissue type (e.g., per-slide diagnosis) may be computed as described with reference to act 118. The multi-slide level diagnosis (e.g., biopsy level analysis, frozen section level analysis, body fluid level analysis) may be performed according to the slide-level analysis and/or slide-level tissue type(s), using multiple slides. Alternatively or additionally, the multi-slide level analysis is performed according to the patch-level tissue type(s) (e.g., as described with reference to act 114) and/or analysis of patches (e.g., as described with reference to act 116)”. Note that the 3D tissue images are divided into patches, and these patches are mapped to the synthetic 3D patches).
Regarding claim 4, Anand, Sjolund, and Veidman teach all the features with respect to claim 3 as outlined above. Further, Veidman teaches that the model generation system of claim 3, wherein the synthetic image patches are 1-2 orders of magnitude less in size than full images (See Veidman: Fig. 1, and [0173], “An exemplary method for segmentation and/or detection of out-of-focus regions is now described: [0174] Optionally, the tissue image is divided into patches. Each patch may be according to a predefined size (e.g., 256×256 pixels or other values) and/or predefined resolution and/or predefined overlap (e.g., 10%, 25%, 50%, or other overlap values). Alternatively, the tissue image as a whole is analyzed”; and [0150], “Optionally, the tissue slides are imaged at high magnification, for example, between about X200-X400, or about X100-400, or about X100-X200, or about X100, or about X200, or about X400, or other values. Such high magnification imaging may create very large images, for example, on the order of Giga Pixel sizes. Such large tissue images of the entire slide may be referred to herein as Whole Slide Images (WSI)”. Note that the whole slide image having size in giga pixels, several orders larger than the patch size of 256x256 pixels, and this is mapped to the synthetic image patches are 1-2 orders of magnitude less in size than full images).
Regarding claim 5, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Sjolund teaches that the model generation system of claim 1, wherein the first set of images is obtained using at least a first modality, and wherein the second set of images is obtained using at least a second modality (See Sjolund: Fig. 1, and [0033], “Given the many combinations of imaging types (e.g., MR, CT, PET, etc.), and different operational modes or parameters within a single imaging type (such as T1, T2, T1 with contrast, and Flair as different MR operational modes), techniques are needed to efficiently consider the information from all available imaging types without needing an exponential amount of training data. Patient artifacts, modality variations, and other factors make automatic segmentation and other image processing functions difficult to be used in many clinical settings from accurate training. In particular, one of the significant challenges is to obtain data for training all paths of a trained model. Missing training data from one or more modality types or modality modes is common, and because the very first layer in neural network considers values from all channels, missing models may result in a bias in the computation in the network”. Note that the image types (MR, CT, PET, etc.) are mapped to the modalities).
Regarding claim 6, Anand, Sjolund, and Veidman teach all the features with respect to claim 5 as outlined above. Further, Sjolund teaches that the model generation system of claim 5, wherein the first modality and the second modality include magnetic resonance imaging and computed tomography imaging (See Sjolund: Fig. 1, and [0033], “Given the many combinations of imaging types (e.g., MR, CT, PET, etc.), and different operational modes or parameters within a single imaging type (such as T1, T2, T1 with contrast, and Flair as different MR operational modes), techniques are needed to efficiently consider the information from all available imaging types without needing an exponential amount of training data. Patient artifacts, modality variations, and other factors make automatic segmentation and other image processing functions difficult to be used in many clinical settings from accurate training. In particular, one of the significant challenges is to obtain data for training all paths of a trained model. Missing training data from one or more modality types or modality modes is common, and because the very first layer in neural network considers values from all channels, missing models may result in a bias in the computation in the network”. Note that the MR and CT images are mapped to the magnetic resonance imaging and computed tomography imaging).
Regarding claim 7, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Veidman teaches that the model generation system of claim 1, wherein the first set of images includes a first set of image patches (See Veidman: Fig. 1, and [0173], “Optionally, the tissue image is divided into patches. Each patch may be according to a predefined size (e.g., 256×256 pixels or other values) and/or predefined resolution and/or predefined overlap (e.g., 10%, 25%, 50%, or other overlap values). Alternatively, the tissue image as a whole is analyzed”. Note that the training tissue images are divided into patches to train the AI model, and this is mapped to a first set of image patches).
Regarding claim 8, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Anand teaches that the model generation system of claim 1, wherein the second diffusion model is to include an abnormality in the synthetic image patches (See Anand: Fig. 4, and [0081], “Now referring to FIG. 4, an exemplary embodiment 400 of a user interface 180 is illustrated. In some cases, interface element displaying set of diagnostic hypotheses 124 may be interactive; for instance, and without limitation, user may be able to click any list entry within list of identified abnormalities 320 and/or list of identified disease 324 (not shown). As a non-limiting example, when a user selects an identified abnormality from the list 320, processor 104 may be configured to visually highlight related segments on ECG data 308 that pertain to the selected abnormality. If “left ventricular hypertrophy” is clicked. Processor 104 may highlight the areas on ECG waveforms typically associated with left ventricular hypertrophy, such as high amplitude QRS complexes in certain leads. In one embodiment, user interface 180 may show a color overlay 404 or other graphical element over the waveform to indicate which parts of the ECG contributed to such diagnostic hypothesis. As a non-limiting example, portions of image box 204 representing areas of interest may be modified to thickened color bands shown on the ECG tracing”. Note that selecting and visually displaying the LLM model identified abnormality ECG signal sections and location in the timeseries ECG signals is mapped to an abnormality in the synthetic image patches).
Regarding claim 9, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Sjolund teaches that the model generation system of claim 1, wherein at least one of the contours in the second set of images is obtained using augmentation (See Sjolund: Fig. 5, and [0131], “The operations of the flowchart 500 continue with data augmentation (operation 520). Data augmentation may be performed to increase the variety of data such that the classifier learned can be more generalized and lead to less overfitting. Data augmentation may include, for instance, image rotation, left-right flipping, or elastic transformation”).
Regarding claim 11, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Anand, Sjolund, and Veidman teach that at least one tangible computer-readable storage medium comprising instructions that, when executed, cause at least one processor to at least (See Anand: Fig. 1, and [0023], “Referring now to FIG. 1, an exemplary embodiment of an apparatus 100 for generating diagnostic hypotheses based on electrocardiogram (ECG) data is illustrated. Apparatus 100 includes a computing device. Computing device includes a processor 104 communicatively connected to a memory 108. As used in this disclosure, “communicatively connected” means connected by way of a connection, attachment, or linkage between two or more relata which allows for reception and/or transmittance of information therebetween”):
train a first diffusion model using a first set of images without contours (See Anand: Fig. 1, and [0027], “With continued reference to FIG. 1, as a non-limiting example, processor may be configured to implement a large language model (LLM). A “large language model,” as used herein, is a deep learning data structure that can recognize, summarize, translate, predict and/or generate text and/or other content based on knowledge gained from massive datasets. LLM may be trained on large sets of data. In one embodiment, generative model 112 is trained on a corpus 116. As used in this disclosure, a “corpus” is a large set of data. Corpus data may include text, images, videos, audio, or the like. Corpus data may be structured, semi-structure, and/or unstructured. In some cases, corpus 116 may include a collection of sufficiently diverse and comprehensive texts, covering desired breadth and depth of knowledge to one or more domains (e.g., medicine including cardiology, pharmacology, epidemiology, and the like), that is used to train LLM, allowing LLM to understand, interpret, and/or generate language-based outputs that are relevant to the model's intended applications as described herein. In some cases, corpus 116 may include a set of medical literatures encompassing research findings, clinical studies, reviews, case reports, scholarly articles, and any other written material related to the field of medicine and healthcare. As a non-limiting example, corpus 116 may include a collection of peer-reviewed medical research papers, reviewed, articles from reputable journals, official clinical guidelines, treatment protocols, best practice documents from recognized medical associations and/or organizations, medical textbooks, reference materials covering explanation of medical conditions, treatments, health maintenance strategies, online medical forums from online medical communities including discussions and Q&A sessions, among others. In some cases, corpus 116 may include information from one or more public or private databases. As a non-limiting example, corpus may include a PubMed database or any other repository of knowledge within medical community”; and [0041], “Other exemplary embodiment of generative model 112 may include, without limitation, an autoencoder for dimensionality reduction and feature learning, a diffusion model for generating image or audio data, among others”. Note that the diffusion mode is a LLM, and it is trained with a corpus of data, which includes medical literature, reviews, images, etc., and this is mapped to train a first diffusion model using a first set of images without contours);
fine-tune the first diffusion model using a second set of images with contours (See Sjolund: Fig. 1, and [0050], “In an example, the image data 152 may include one or more MRI image (e.g., 2D MRI, 3D MRI, 2D streaming MRI, 4D MRI, 4D volumetric MRI, 4D cine MRI, etc.), functional MRI images (e.g., fMRI, DCE-MRI, diffusion MRI), Computed Tomography (CT) images (e.g., 2D CT, Cone beam CT, 3D CT, 4D CT), ultrasound images (e.g., 2D ultrasound, 3D ultrasound, 4D ultrasound), Positron Emission Tomography (PET) images, PET-CT images, X-ray images, fluoroscopic images, radiotherapy portal images, Single-Photo Emission Computed Tomography (SPECT) images, Elastography images, Photoacoustic images, Magnetoencephalography (MEG) images, Electroencephalography (EEG) images, or computer generated synthetic images (e.g., pseudo-CT images) and the like. Further, the image data 152 may also include or be associated with medical image processing data, for instance, training images, and ground truth images, contoured images, and dose images. In other examples, an equivalent representation of an anatomical area may be represented in non-image formats (e.g., coordinates, mappings, etc.)”) to form a second diffusion model (See Anand: Fig. 1, and [0029], “As a non-limiting example, generative model 112 such as, without limitation, a LLM may be pre-training on a general set of medical literatures (i.e., a wide range of medical literatures covering the vast field of medicine) and fine-tuning on a specific set of medical literatures (i.e., texts focused on one or more specific areas), wherein the general set of medical literatures and the specific set of medical literatures are subsets of the set of medical literatures contained in corpus 116. In some cases, majority of medical literatures within the specific set may be more detailed, advanced, or specialized then medical literatures in the general set. For example, general set of medical literatures may include introductory texts and reviews that summarize basic concepts in physiology while specific set of medical literatures may include a plurality of citations to peer-reviewed research papers related to cardiology”. Note that the LLM (diffusion) model is fine-tuning on a specific sub-set of data is mapped to fine-tune the first diffusion model using a second set of images with contours to form a second diffusion model, wherein the second set of images is smaller than the first set of images, however, Anand does not explicitly disclose the second set of images with contour, and a secondary art will be searched);
generate synthetic image patches with contours (See Veidman: Fig. 1, and [0180], “Referring now back to FIG. 1, at 112, tissue image patches (also referred to herein as patches) are created from the tissue image. It is noted that patches may be created as part of the segmentation process described herein. Alternatively, the patches for segmentation are different than the patches created for further analysis”. Note that the image may be divided into patches, and thus, the synthetic images can be divided into patches as well, therefore, combing the Anand with Veidman, the synthetic image patches are generated) using the second diffusion model and at least one contour (See Anand: Fig. 8, and [0101], “Further referring to FIG. 8, computing device, processor, and/or module may be configured to perform one or more processes of data augmentation. “Data augmentation” as used in this disclosure is addition of data to a training set using elements and/or entries already in the dataset. Data augmentation may be accomplished, without limitation, using interpolation, generation of modified copies of existing entries and/or examples, and/or one or more generative AI processes, for instance using deep neural networks and/or generative adversarial networks; generative processes may be referred to alternatively in this context as “data synthesis” and as creating “synthetic data.” Augmentation may include performing one or more transformations on data, such as geometric, color space, affine, brightness, cropping, and/or contrast transformations of images”. Note that creating the synthetic data including the images is mapped to generate synthetic images, however, Anand does not explicitly disclose that the synthetic images are patches, and a secondary art will be searched);
train a segmentation model using the synthetic image patches (See Veidman: Fig. 1, and [0186], “The patch-level segmentation code may be implemented as a segmentation CNN (e.g., based on Unet). The patch-level segmentation may output the segmentation as a mask. The patch-level segmentation code may be trained according to patches that are annotated with segmentation markings”. Note that the image patches are used to train the segmentation CNN model, which is mapped to the segmentation model); and
deploy the segmentation model (See Anand: Fig. 8, and [0110], “Still referring to FIG. 8, a machine-learning model and/or process may be deployed or instantiated by incorporation into a program, apparatus, system and/or module. For instance, and without limitation, a machine-learning model, neural network, and/or some or all parameters thereof may be stored and/or deployed in any memory or circuitry. Parameters such as coefficients, weights, and/or biases may be stored as circuit-based constants, such as arrays of wires and/or binary inputs and/or outputs set at logic “1” and “0” voltage levels in a logic circuit to represent a number according to any suitable encoding system including twos complement or the like or may be stored in any volatile and/or non-volatile memory. Similarly, mathematical operations and input and/or output of data to or from models, neural network layers, or the like may be instantiated in hardware circuitry and/or in the form of instructions in firmware, machine-code such as binary operation code instructions, assembly language, or any higher-order programming language”; and [0086], “A “machine learning process,” as used in this disclosure, is a process that automatedly uses training data 804 to generate an algorithm instantiated in hardware or software logic, data structures, and/or functions that will be performed by a computing device/module to produce outputs 808 given data provided as inputs 812; this is in contrast to a non-machine learning software program where the commands to be executed are determined in advance by a user and written in a programming language”. Note that the trained model is deployed to the users to generate output from user input, and this is mapped to deploy the segmentation model) to inference on a third set of images (See Veidman: Fig. 11, and [0271], “Interference flow 1116 is for inferring the image type to compute the slide-level tissue type(s), for example, overall diagnosis. Inference flow 1116 includes a feature 1118 for patch level detection and/or segmentation (e.g., as described with reference to act 110,112, and/or 114 of FIG. 1), a feature 1120 for patch level classification (e.g., as described with reference to act 114 of FIG. 1), a feature 1122 for computational of the slide-level tissue type(s) such as diagnosis (e.g., as described with reference to act 118 of FIG. 1), a feature 1124 for comparison of the slide-level tissue type(s) (e.g., diagnosis) to a manual physician diagnosis (e.g., as described with reference to act 406 of FIG. 4), and a feature 1126 for presenting the results in the GUI (e.g., as described with reference to act 122 of FIG. 1)”. Note that the inference flow for the patient image analysis is mapped to inference on a third set of images).
Regarding claim 12, Anand, Sjolund, and Veidman teach all the features with respect to claim 11 as outlined above. Further, Anand teaches that the at least one tangible computer-readable storage medium of claim 11, wherein the first diffusion model is an unconditioned diffusion model, and wherein the second diffusion model is a fine-tuned, conditioned diffusion model (See Anand: Fig. 8, and [0107], “Further referring to FIG. 8, machine learning processes may include at least an unsupervised machine-learning processes 832. An unsupervised machine-learning process, as used herein, is a process that derives inferences in datasets without regard to labels; as a result, an unsupervised machine-learning process may be free to discover any structure, relationship, and/or correlation provided in the data. Unsupervised processes 832 may not require a response variable; unsupervised processes 832 may be used to find interesting patterns and/or inferences between variables, to determine a degree of correlation between two or more variables, or the like”; and [0104], “Still referring to FIG. 8, machine-learning algorithms may include at least a supervised machine-learning process 828. At least a supervised machine-learning process 828, as defined herein, include algorithms that receive a training set relating a number of inputs to a number of outputs, and seek to generate one or more data structures representing and/or instantiating one or more mathematical relations relating inputs to outputs, where each of the one or more mathematical relations is optimal according to some criterion specified to the algorithm using some scoring function”. Note that an unsupervised machine-learning processes 832 is mapped to an unconditioned diffusion model because un-labeled images are used to train the model, and a supervised machine-learning process 828is mapped to a fine-tuned, conditioned diffusion model because the labeled (ground truth) images are used to train the model).
Regarding claim 13, Anand, Sjolund, and Veidman teach all the features with respect to claim 11 as outlined above. Further, Veidman teaches that the at least one tangible computer-readable storage medium of claim 11, wherein the synthetic image patches include synthetic three-dimensional image patches (See Veidman: Fig. 1, and [0210], “At 119, a multi-slide level tissue type(s) is computed for multiple tissue images, and/or for a 3D tissue image. The multiple slides and/or 3D tissue image may be produced, for example, from the same biopsy, and/or same target tissue (e.g., slices of frozen section), and/or from the same sample of body fluid. The slides may be created with the same stain, or from different stain types, where different processes are used to analyzing the tissue images created from different stain types. In such case a slide-level tissue type (e.g., per-slide diagnosis) may be computed as described with reference to act 118. The multi-slide level diagnosis (e.g., biopsy level analysis, frozen section level analysis, body fluid level analysis) may be performed according to the slide-level analysis and/or slide-level tissue type(s), using multiple slides. Alternatively or additionally, the multi-slide level analysis is performed according to the patch-level tissue type(s) (e.g., as described with reference to act 114) and/or analysis of patches (e.g., as described with reference to act 116)”. Note that the 3D tissue images are divided into patches, and these patches are mapped to the synthetic 3D patches).
Regarding claim 14, Anand, Sjolund, and Veidman teach all the features with respect to claim 11 as outlined above. Further, Sjolund teaches that the at least one tangible computer-readable storage medium of claim 11, wherein the first set of images is obtained using at least a first modality, and wherein the second set of images with contours is obtained using at least a second modality (See Sjolund: Fig. 1, and [0033], “Given the many combinations of imaging types (e.g., MR, CT, PET, etc.), and different operational modes or parameters within a single imaging type (such as T1, T2, T1 with contrast, and Flair as different MR operational modes), techniques are needed to efficiently consider the information from all available imaging types without needing an exponential amount of training data. Patient artifacts, modality variations, and other factors make automatic segmentation and other image processing functions difficult to be used in many clinical settings from accurate training. In particular, one of the significant challenges is to obtain data for training all paths of a trained model. Missing training data from one or more modality types or modality modes is common, and because the very first layer in neural network considers values from all channels, missing models may result in a bias in the computation in the network”. Note that the image types (MR, CT, PET, etc.) are mapped to the modalities).
Regarding claim 15, Anand, Sjolund, and Veidman teach all the features with respect to claim 11 as outlined above. Further, Veidman teaches that the at least one tangible computer-readable storage medium of claim 11, wherein the first set of images includes a first set of image patches (See Veidman: Fig. 1, and [0173], “Optionally, the tissue image is divided into patches. Each patch may be according to a predefined size (e.g., 256×256 pixels or other values) and/or predefined resolution and/or predefined overlap (e.g., 10%, 25%, 50%, or other overlap values). Alternatively, the tissue image as a whole is analyzed”. Note that the training tissue images are divided into patches to train the AI model, and this is mapped to a first set of image patches).
Regarding claim 16, Anand, Sjolund, and Veidman teach all the features with respect to claim 11 as outlined above. Further, Anand teaches that the at least one tangible computer-readable storage medium of claim 11, wherein the second diffusion model is to include an abnormality in the synthetic image patches (See Anand: Fig. 4, and [0081], “Now referring to FIG. 4, an exemplary embodiment 400 of a user interface 180 is illustrated. In some cases, interface element displaying set of diagnostic hypotheses 124 may be interactive; for instance, and without limitation, user may be able to click any list entry within list of identified abnormalities 320 and/or list of identified disease 324 (not shown). As a non-limiting example, when a user selects an identified abnormality from the list 320, processor 104 may be configured to visually highlight related segments on ECG data 308 that pertain to the selected abnormality. If “left ventricular hypertrophy” is clicked. Processor 104 may highlight the areas on ECG waveforms typically associated with left ventricular hypertrophy, such as high amplitude QRS complexes in certain leads. In one embodiment, user interface 180 may show a color overlay 404 or other graphical element over the waveform to indicate which parts of the ECG contributed to such diagnostic hypothesis. As a non-limiting example, portions of image box 204 representing areas of interest may be modified to thickened color bands shown on the ECG tracing”. Note that selecting and visually displaying the LLM model identified abnormality ECG signal sections and location in the timeseries ECG signals is mapped to an abnormality in the synthetic image patches).
Regarding claim 17, Anand, Sjolund, and Veidman teach all the features with respect to claim 1 as outlined above. Further, Anand, Sjolund, and Veidman teach that a segmentation apparatus (See Anand: Fig. 1, and [0023], “Referring now to FIG. 1, an exemplary embodiment of an apparatus 100 for generating diagnostic hypotheses based on electrocardiogram (ECG) data is illustrated. Apparatus 100 includes a computing device. Computing device includes a processor 104 communicatively connected to a memory 108. As used in this disclosure, “communicatively connected” means connected by way of a connection, attachment, or linkage between two or more relata which allows for reception and/or transmittance of information therebetween”)comprising:
a first diffusion model trained using a first set of images without contours (See Anand: Fig. 1, and [0027], “With continued reference to FIG. 1, as a non-limiting example, processor may be configured to implement a large language model (LLM). A “large language model,” as used herein, is a deep learning data structure that can recognize, summarize, translate, predict and/or generate text and/or other content based on knowledge gained from massive datasets. LLM may be trained on large sets of data. In one embodiment, generative model 112 is trained on a corpus 116. As used in this disclosure, a “corpus” is a large set of data. Corpus data may include text, images, videos, audio, or the like. Corpus data may be structured, semi-structure, and/or unstructured. In some cases, corpus 116 may include a collection of sufficiently diverse and comprehensive texts, covering desired breadth and depth of knowledge to one or more domains (e.g., medicine including cardiology, pharmacology, epidemiology, and the like), that is used to train LLM, allowing LLM to understand, interpret, and/or generate language-based outputs that are relevant to the model's intended applications as described herein. In some cases, corpus 116 may include a set of medical literatures encompassing research findings, clinical studies, reviews, case reports, scholarly articles, and any other written material related to the field of medicine and healthcare. As a non-limiting example, corpus 116 may include a collection of peer-reviewed medical research papers, reviewed, articles from reputable journals, official clinical guidelines, treatment protocols, best practice documents from recognized medical associations and/or organizations, medical textbooks, reference materials covering explanation of medical conditions, treatments, health maintenance strategies, online medical forums from online medical communities including discussions and Q&A sessions, among others. In some cases, corpus 116 may include information from one or more public or private databases. As a non-limiting example, corpus may include a PubMed database or any other repository of knowledge within medical community”; and [0041], “Other exemplary embodiment of generative model 112 may include, without limitation, an autoencoder for dimensionality reduction and feature learning, a diffusion model for generating image or audio data, among others”. Note that the diffusion mode is a LLM, and it is trained with a corpus of data, which includes medical literature, reviews, images, etc., and this is mapped to train a first diffusion model using a first set of images without contours);
a second diffusion model formed from the first diffusion model tuned (See Anand: Fig. 1, and [0029], “As a non-limiting example, generative model 112 such as, without limitation, a LLM may be pre-training on a general set of medical literatures (i.e., a wide range of medical literatures covering the vast field of medicine) and fine-tuning on a specific set of medical literatures (i.e., texts focused on one or more specific areas), wherein the general set of medical literatures and the specific set of medical literatures are subsets of the set of medical literatures contained in corpus 116. In some cases, majority of medical literatures within the specific set may be more detailed, advanced, or specialized then medical literatures in the general set. For example, general set of medical literatures may include introductory texts and reviews that summarize basic concepts in physiology while specific set of medical literatures may include a plurality of citations to peer-reviewed research papers related to cardiology”. Note that the LLM (diffusion) model is fine-tuning on a specific sub-set of data is mapped to fine-tune the first diffusion model using a second set of images with contours to form a second diffusion model, wherein the second set of images is smaller than the first set of images, however, Anand does not explicitly disclose the second set of images with contour, and a secondary art will be searched) using a second set of images with contours (See Sjolund: Fig. 1, and [0050], “In an example, the image data 152 may include one or more MRI image (e.g., 2D MRI, 3D MRI, 2D streaming MRI, 4D MRI, 4D volumetric MRI, 4D cine MRI, etc.), functional MRI images (e.g., fMRI, DCE-MRI, diffusion MRI), Computed Tomography (CT) images (e.g., 2D CT, Cone beam CT, 3D CT, 4D CT), ultrasound images (e.g., 2D ultrasound, 3D ultrasound, 4D ultrasound), Positron Emission Tomography (PET) images, PET-CT images, X-ray images, fluoroscopic images, radiotherapy portal images, Single-Photo Emission Computed Tomography (SPECT) images, Elastography images, Photoacoustic images, Magnetoencephalography (MEG) images, Electroencephalography (EEG) images, or computer generated synthetic images (e.g., pseudo-CT images) and the like. Further, the image data 152 may also include or be associated with medical image processing data, for instance, training images, and ground truth images, contoured images, and dose images. In other examples, an equivalent representation of an anatomical area may be represented in non-image formats (e.g., coordinates, mappings, etc.)”),
the second diffusion model to generate synthetic image patches with contours (See Veidman: Fig. 1, and [0180], “Referring now back to FIG. 1, at 112, tissue image patches (also referred to herein as patches) are created from the tissue image. It is noted that patches may be created as part of the segmentation process described herein. Alternatively, the patches for segmentation are different than the patches created for further analysis”. Note that the image may be divided into patches, and thus, the synthetic images can be divided into patches as well, therefore, combing the Anand with Veidman, the synthetic image patches are generated) using at least one contour (See Anand: Fig. 8, and [0101], “Further referring to FIG. 8, computing device, processor, and/or module may be configured to perform one or more processes of data augmentation. “Data augmentation” as used in this disclosure is addition of data to a training set using elements and/or entries already in the dataset. Data augmentation may be accomplished, without limitation, using interpolation, generation of modified copies of existing entries and/or examples, and/or one or more generative AI processes, for instance using deep neural networks and/or generative adversarial networks; generative processes may be referred to alternatively in this context as “data synthesis” and as creating “synthetic data.” Augmentation may include performing one or more transformations on data, such as geometric, color space, affine, brightness, cropping, and/or contrast transformations of images”. Note that creating the synthetic data including the images is mapped to generate synthetic images, however, Anand does not explicitly disclose that the synthetic images are patches, and a secondary art will be searched); and
a segmentation model trained using the synthetic image patches (See Veidman: Fig. 1, and [0186], “The patch-level segmentation code may be implemented as a segmentation CNN (e.g., based on Unet). The patch-level segmentation may output the segmentation as a mask. The patch-level segmentation code may be trained according to patches that are annotated with segmentation markings”. Note that the image patches are used to train the segmentation CNN model, which is mapped to the segmentation model) and deployed (See Anand: Fig. 8, and [0110], “Still referring to FIG. 8, a machine-learning model and/or process may be deployed or instantiated by incorporation into a program, apparatus, system and/or module. For instance, and without limitation, a machine-learning model, neural network, and/or some or all parameters thereof may be stored and/or deployed in any memory or circuitry. Parameters such as coefficients, weights, and/or biases may be stored as circuit-based constants, such as arrays of wires and/or binary inputs and/or outputs set at logic “1” and “0” voltage levels in a logic circuit to represent a number according to any suitable encoding system including twos complement or the like or may be stored in any volatile and/or non-volatile memory. Similarly, mathematical operations and input and/or output of data to or from models, neural network layers, or the like may be instantiated in hardware circuitry and/or in the form of instructions in firmware, machine-code such as binary operation code instructions, assembly language, or any higher-order programming language”; and [0086], “A “machine learning process,” as used in this disclosure, is a process that automatedly uses training data 804 to generate an algorithm instantiated in hardware or software logic, data structures, and/or functions that will be performed by a computing device/module to produce outputs 808 given data provided as inputs 812; this is in contrast to a non-machine learning software program where the commands to be executed are determined in advance by a user and written in a programming language”. Note that the trained model is deployed to the users to generate output from user input, and this is mapped to deploy the segmentation model) to inference on a third set of images (See Veidman: Fig. 11, and [0271], “Interference flow 1116 is for inferring the image type to compute the slide-level tissue type(s), for example, overall diagnosis. Inference flow 1116 includes a feature 1118 for patch level detection and/or segmentation (e.g., as described with reference to act 110,112, and/or 114 of FIG. 1), a feature 1120 for patch level classification (e.g., as described with reference to act 114 of FIG. 1), a feature 1122 for computational of the slide-level tissue type(s) such as diagnosis (e.g., as described with reference to act 118 of FIG. 1), a feature 1124 for comparison of the slide-level tissue type(s) (e.g., diagnosis) to a manual physician diagnosis (e.g., as described with reference to act 406 of FIG. 4), and a feature 1126 for presenting the results in the GUI (e.g., as described with reference to act 122 of FIG. 1)”. Note that the inference flow for the patient image analysis is mapped to inference on a third set of images).
Regarding claim 18, Anand, Sjolund, and Veidman teach all the features with respect to claim 17 as outlined above. Further, Anand teaches that the segmentation apparatus of claim 17, wherein the first diffusion model is an unconditioned diffusion model, and wherein the second diffusion model is a fine-tuned, conditioned diffusion model (See Anand: Fig. 8, and [0107], “Further referring to FIG. 8, machine learning processes may include at least an unsupervised machine-learning processes 832. An unsupervised machine-learning process, as used herein, is a process that derives inferences in datasets without regard to labels; as a result, an unsupervised machine-learning process may be free to discover any structure, relationship, and/or correlation provided in the data. Unsupervised processes 832 may not require a response variable; unsupervised processes 832 may be used to find interesting patterns and/or inferences between variables, to determine a degree of correlation between two or more variables, or the like”; and [0104], “Still referring to FIG. 8, machine-learning algorithms may include at least a supervised machine-learning process 828. At least a supervised machine-learning process 828, as defined herein, include algorithms that receive a training set relating a number of inputs to a number of outputs, and seek to generate one or more data structures representing and/or instantiating one or more mathematical relations relating inputs to outputs, where each of the one or more mathematical relations is optimal according to some criterion specified to the algorithm using some scoring function”. Note that an unsupervised machine-learning processes 832 is mapped to an unconditioned diffusion model because un-labeled images are used to train the model, and a supervised machine-learning process 828is mapped to a fine-tuned, conditioned diffusion model because the labeled (ground truth) images are used to train the model).
Regarding claim 19, Anand, Sjolund, and Veidman teach all the features with respect to claim 17 as outlined above. Further, Veidman teaches that the segmentation apparatus of claim 17, wherein the synthetic image patches include synthetic three-dimensional image patches (See Veidman: Fig. 1, and [0210], “At 119, a multi-slide level tissue type(s) is computed for multiple tissue images, and/or for a 3D tissue image. The multiple slides and/or 3D tissue image may be produced, for example, from the same biopsy, and/or same target tissue (e.g., slices of frozen section), and/or from the same sample of body fluid. The slides may be created with the same stain, or from different stain types, where different processes are used to analyzing the tissue images created from different stain types. In such case a slide-level tissue type (e.g., per-slide diagnosis) may be computed as described with reference to act 118. The multi-slide level diagnosis (e.g., biopsy level analysis, frozen section level analysis, body fluid level analysis) may be performed according to the slide-level analysis and/or slide-level tissue type(s), using multiple slides. Alternatively or additionally, the multi-slide level analysis is performed according to the patch-level tissue type(s) (e.g., as described with reference to act 114) and/or analysis of patches (e.g., as described with reference to act 116)”. Note that the 3D tissue images are divided into patches, and these patches are mapped to the synthetic 3D patches).
Regarding claim 20, Anand, Sjolund, and Veidman teach all the features with respect to claim 17 as outlined above. Further, Sjolund teaches that the segmentation apparatus of claim 17, wherein the first set of images is obtained using at least a first modality, and wherein the second set of images with contours is obtained using at least a second modality (See Sjolund: Fig. 1, and [0033], “Given the many combinations of imaging types (e.g., MR, CT, PET, etc.), and different operational modes or parameters within a single imaging type (such as T1, T2, T1 with contrast, and Flair as different MR operational modes), techniques are needed to efficiently consider the information from all available imaging types without needing an exponential amount of training data. Patient artifacts, modality variations, and other factors make automatic segmentation and other image processing functions difficult to be used in many clinical settings from accurate training. In particular, one of the significant challenges is to obtain data for training all paths of a trained model. Missing training data from one or more modality types or modality modes is common, and because the very first layer in neural network considers values from all channels, missing models may result in a bias in the computation in the network”. Note that the image types (MR, CT, PET, etc.) are mapped to the modalities).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Anand, etc. (US 20250336523 A1) in view of Sjolund, etc. (US 20190332900 A1), further in view of Veidman, etc. (US 20200372635 A1), and Rappaport, etc. (US 20080075348 A1).
Regarding claim 10, Anand, Sjolund, and Veidman teach all the features with respect to claim 9 as outlined above. However, Anand, modified by Sjolund and Veidman, fails to explicitly disclose that the model generation system of claim 9, wherein the augmentation includes at least one of a normal contour or an abnormal contour.
However, Rappaport teaches that the model generation system of claim 9, wherein the augmentation includes at least one of a normal contour or an abnormal contour (See Rappaport: Fig. 4, and [0193], “For example, osteoporosis is typically characterized by a reduced bone mineral density (a parameter P) from it earliest stages but may not cause any significant change in bone contours C until very advanced stages. As a result, a hip X-ray from an osteoporotic individual can include a femur with a normal contour C and an abnormal parameter map P.sub.M which shows reduced trabecular density. Optionally, angle of incidence .theta. for an osteoporotic bone is determined first by comparison to a series of normal 2D angle specific models, and parameters P indicative of osteoporosis are then evaluated with respect to an osteoporosis pathology mode”; and [0194], “In another example a hip X-ray from an individual recovering from a femoral fracture can include a femur with an abnormal contour C and/or one or more abnormal features F. In cases of fracture, angle of incidence .theta. for is determined first by comparison to a series of normal 2D angle specific models, and irregularities in C and/or F are considered indicative of fracture pathology. The same X-ray from the individual recovering from the femoral fracture may have large portions of a parameter map P.sub.M which correspond to a normal parameter map P.sub.M with the same angle of incidence .theta.. This situation arises because a fracture will typically disrupt inner bone parameters only in close proximity to the fracture. Optionally, definition of an edge or gap between normal portions of parameter map P.sub.M can help to define or identify the fracture”. Note that the normal contour and abnormal contour analysis in the medical image analysis are common operations and techniques to identify diseases detected by various imaging systems like MRI and CT, thus, it is mapped to the cited limitation pf “the augmentation includes at least one of a normal contour or an abnormal contour”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was effectively filed to modify Anand to have the model generation system of claim 9, wherein the augmentation includes at least one of a normal contour or an abnormal contour as taught by Rappaport in order to enable improvement of interpretation of medical images, and improvement of automated analysis of the image (See Rappaport: Fig. 1, and [0013], “A broad aspect of some embodiments of the invention relates to improving interpretation of medical images by estimating an angle of incidence between an imaging instrument and an imaged organ when performing an image analysis”). Anand teaches a method and system that may generate diagnostic hypotheses based on electrocardiogram (ECG) data using the LLM algorithms with medical images to train the diffusion model and subset specific images to fine-tuning the diffusion model; while Rappaport teaches a system and method that may analyze the medical images with contour measurements and mark the normal contour with parameters and the abnormal contour with abnormal contour parameters. Therefore, it is obvious to one of ordinary skill in the art to modify Anand by Rappaport to augment the input image patches with normal and abnormal contour marks and parameters associated with them to improve the interpretation of the medical image analysis results. The motivation to modify Anand by Rappaport is “Use of known technique to improve similar devices (methods, or products) in the same way”.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GORDON G LIU whose telephone number is (571)270-0382. The examiner can normally be reached Monday - Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona E Faulk can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GORDON G LIU/Primary Examiner, Art Unit 2618